Skip to content

Commit d75a6c8

Browse files
diogoncalvesMiNeves00actions-userbrunoalho99
authored
LLMstudio-core Version 1.0.3 LLMstudio-tracker 1.1.0 LLMstudio 1.0.3 LLMstudio-proxy 1.0.5 (#195)
* chore: Provider Unit Tests (#173) * chore: added unit tests for core provider. small bugfix on calculate_metrics of provider * added unit tests and docstring for join chunks * added unit tests and docstrings for calculate_cost on provider * added unit tests and docstrings for input_to_string on provider * added unit tests and docstrings for chat and achat * added unit tests and docstrings for chat and achat * chore: cleaned provider unit tests * chore: separated provider tests into different files. fixed some of its tests * chore: linted code * chore: deleted some comments * chore: linted * chore: Added Azure Provider Unit Tests (#176) * chore: added unit tests for azure provider * chore: added more unit tests and docstrings on azure, removed redundant comments * chore: added unit tests for generate client on Azure Provider * chore: separated azure unit tests into separate files. fixed some of its tests. * chore: linted code * chore: new line Signed-off-by: Diogo Goncalves <diogoncalves@users.noreply.github.com> --------- Signed-off-by: Diogo Goncalves <diogoncalves@users.noreply.github.com> Co-authored-by: Diogo Goncalves <diogoncalves@users.noreply.github.com> * [fix] bump prerelease version in pyproject.toml * chore: rename action * feat: added action to run tests on PR * chore: comments * fix: fix azure config tests * chore: style format * fix: tests workflow * Feature/prompt management (#200) * [feat] prompt management * [feat] testing * [feat] only one active prompt * [fix] bump prerelease version in pyproject.toml * [bugfix] return empty prompt * [fix] bump prerelease version in pyproject.toml * Update CONTRIBUTING.md Signed-off-by: Diogo Goncalves <diogoncalves@users.noreply.github.com> * Feat/ Use Openai Usage to calculate Cache and Reasoning Costs (#199) * feat: collects usage from stream and non stream openai calls * chore: refactored to provider to have a Metrics obj * feat: calculate_metrics now takes into account cached & reasoning tokens. Prices of openai models updated * fix: added caching tokens to model config obj * chore: added integration test for cache and reasoning * chore: added integration test for usage retrieval when max tokens reached * chore: uncommented runs from examples/core.py * fix: bugfix regarding usage on function calling. added a test for this * chore: merged with develop * chore: extracted provider data structures to another file * chore: renamed to private methods some within provider. splitted integration tests into 2 files * chore: deletion of a todo comment * chore: update poetry.lock * chore: specify python versions * chore: moving langchain integration tests to sdk * chore: format * feat: added support for o3-mini and updated o1-mini prices. also updated integration tests to support o3 (#202) * chore: removed duplicated code; removed duplicated integration tests * chore: updated github actions to run integration tests * chore: fixing github actions * chore: fixing github actions again * chore: fixing github actions again-x2 * chore: fixing github actions again-x2 * chore: added cache of dependencies to integration-tests in githubaction * chore: updated integration-tests action to inject github secrets into env * Feat/bedrock support for Nova models through the ConverseAPI (#207) * feat: added support for bedrock nova models * feat: tokens are now read from usage if available to ensure accuracy * chore: removed duplicated integration tests folder in wrong place * feat: refactored bedrock provider into being a single file instead of folder * chore: renamed bedrock to bedrock-converse in examples/core.py * chore: renamed bedrock in config.yaml * [fix] bump prerelease version in pyproject.toml * [fix] bump prerelease version in pyproject.toml * [fix] bump prerelease version in pyproject.toml * Update pyproject.toml updated llmstudio-tracker version Signed-off-by: Miguel Neves <61327611+MiNeves00@users.noreply.github.com> * [fix] bump prerelease version in pyproject.toml * chore: updated llmstudio sdk poetry.lock --------- Signed-off-by: Diogo Goncalves <diogoncalves@users.noreply.github.com> Signed-off-by: Miguel Neves <61327611+MiNeves00@users.noreply.github.com> Co-authored-by: Miguel Neves <61327611+MiNeves00@users.noreply.github.com> Co-authored-by: GitHub Actions <actions@github.com> Co-authored-by: brunoalho99 <132477278+brunoalho99@users.noreply.github.com> Co-authored-by: brunoalho <bruno.alho@tensorops.ai> Co-authored-by: Miguel Neves <miguel.neves.filipe@gmail.com>
1 parent 4d4a0f3 commit d75a6c8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+5096
-1950
lines changed

‎.github/workflows/tests.yml

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: Tests
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- main
7+
- develop
8+
types:
9+
- opened
10+
- synchronize
11+
- reopened
12+
13+
jobs:
14+
unit-tests:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/checkout@v3
18+
19+
# Set up Python environment
20+
- name: Set up Python
21+
uses: actions/setup-python@v2
22+
with:
23+
python-version: ">=3.9 <3.13"
24+
25+
# Install Poetry
26+
- name: Install Poetry
27+
run: |
28+
curl -sSL https://install.python-poetry.org | python3 -
29+
30+
# Cache Poetry Dependencies
31+
- name: Cache Poetry dependencies
32+
uses: actions/cache@v3
33+
with:
34+
path: ~/.cache/pypoetry
35+
key: poetry-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
36+
restore-keys: |
37+
poetry-${{ runner.os }}-
38+
39+
# Install lib and dev dependencies
40+
- name: Install llmstudio-core
41+
working-directory: ./libs/core
42+
run: |
43+
poetry install
44+
UNIT_ENV=$(poetry env info --path)
45+
echo $UNIT_ENV
46+
echo "UNIT_ENV=$UNIT_ENV" >> $GITHUB_ENV
47+
48+
- name: Run unit tests
49+
run: |
50+
echo ${{ env.UNIT_ENV }}
51+
source ${{ env.UNIT_ENV }}/bin/activate
52+
poetry run pytest libs/core
53+
54+
integration-tests:
55+
needs: unit-tests
56+
runs-on: ubuntu-latest
57+
steps:
58+
- uses: actions/checkout@v3
59+
60+
# Set up Python environment
61+
- name: Set up Python
62+
uses: actions/setup-python@v2
63+
with:
64+
python-version: ">=3.9 <3.13"
65+
66+
# Install Poetry
67+
- name: Install Poetry
68+
run: |
69+
curl -sSL https://install.python-poetry.org | python3 -
70+
71+
# Inject Secrets as Environment Variables
72+
- name: Set up environment variables
73+
run: |
74+
echo "OPENAI_API_KEY=${{ secrets.OPENAI_API_KEY }}" >> $GITHUB_ENV
75+
echo "GOOGLE_API_KEY=${{ secrets.GOOGLE_API_KEY }}" >> $GITHUB_ENV
76+
echo "BEDROCK_ACCESS_KEY=${{ secrets.BEDROCK_ACCESS_KEY }}" >> $GITHUB_ENV
77+
echo "BEDROCK_SECRET_KEY=${{ secrets.BEDROCK_SECRET_KEY }}" >> $GITHUB_ENV
78+
echo "BEDROCK_REGION=${{ secrets.BEDROCK_REGION }}" >> $GITHUB_ENV
79+
80+
# Cache Poetry Dependencies
81+
- name: Cache Poetry dependencies (Integration Tests)
82+
uses: actions/cache@v3
83+
with:
84+
path: ~/.cache/pypoetry
85+
key: poetry-integration-${{ runner.os }}-${{ hashFiles('libs/llmstudio/poetry.lock') }}
86+
restore-keys: |
87+
poetry-integration-${{ runner.os }}-
88+
89+
# Install llmstudio
90+
- name: Install llmstudio
91+
working-directory: ./libs/llmstudio
92+
run: |
93+
poetry install
94+
INTEGRATION_ENV=$(poetry env info --path)
95+
echo $INTEGRATION_ENV
96+
echo "INTEGRATION_ENV=$INTEGRATION_ENV" >> $GITHUB_ENV
97+
98+
# Run Integration Tests
99+
- name: Run Integration Tests
100+
run: |
101+
source ${{ env.INTEGRATION_ENV }}/bin/activate
102+
poetry run pytest libs/llmstudio/tests/integration_tests

‎.github/workflows/upload-pypi-dev.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: PyPI prerelease and build/push Docker image.
1+
name: PyPI prerelease any module.
22

33
on:
44
workflow_dispatch:

‎.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ env3
5656
.env*
5757
.env*.local
5858
.venv*
59+
*venv*
5960
env*/
6061
venv*/
6162
ENV/
@@ -66,6 +67,7 @@ venv.bak/
6667
config.yaml
6768
bun.lockb
6869

70+
6971
# Jupyter Notebook
7072
.ipynb_checkpoints
7173

@@ -76,4 +78,4 @@ bun.lockb
7678
llmstudio/llm_engine/logs/execution_logs.jsonl
7779
*.db
7880
.prettierignore
79-
db
81+
db

‎CONTRIBUTING.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,30 @@ Thank you for expressing your interest in contributing to LLMstudio. To ensure t
99
3. Follow our repo guidelines
1010
- Ensure that you update any relevant docstrings and comments within your code
1111
- Run `pre-commit run --all-files` to lint your code
12+
4. Sign your commits. Without signed commits, your changes will not be accepted for main.
1213

1314
## Branches
1415

1516
- All development happens in per-feature branches prefixed by contributor's
1617
initials. For example `feat/feature_name`.
1718
- Approved PRs are merged to the `main` branch.
19+
20+
## Alpha releases:
21+
You need to have your changes in the `develop` branch in order to push a new alpha version of any library `(llmstudio, llmstudio-proxy, llmstudio-tracker)`. Therefore, first guarantee that you feature branch is reviewed and working before merging to develop.
22+
23+
Process:
24+
- Ensure the `feature/**` you worked is passing the tests and has the approvals necessary.
25+
- Merge to `develop`
26+
- Ensure the changes are in the develop branch
27+
- Use GitHub Actions to initiate the pre-release process: [PyPI pre-release any module](https://github.com/TensorOpsAI/LLMstudio/actions/workflows/upload-pypi-dev.yml)
28+
- Select the target library `(llmstudio, llmstudio-proxy, llmstudio-tracker)` and the target version for the final release (e.g., 1.1.0). Consult main branch and PyPI for current versions.
29+
- Run the workflow.
30+
- The workflow will automatically bump the version and create an alpha release of the library/module specified
31+
- The workflow will automatically push changes back (bump version) to the develop branch
32+
33+
Repeat the process in case your `development` branch contains changes in multiple libraries.
34+
35+
## Final releases:
36+
Once you're happy with the versions, create the Release notes on the PR between `develop` and `main` and merge to main branch when ready for full release. The workflow will automatically remove any `alpha` tag in your libraries and push the versions for every library/module that suffered changes.
37+
38+

‎Makefile

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
11
format:
22
pre-commit run --all-files
3+
4+
unit-tests:
5+
pytest libs/core/tests/unit_tests

‎examples/_config.yaml

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -115,16 +115,29 @@ providers:
115115
keys:
116116
- OPENAI_API_KEY
117117
models:
118+
o1-preview:
119+
mode: chat
120+
max_completion_tokens: 128000
121+
input_token_cost: 0.000015
122+
output_token_cost: 0.000060
123+
o1-mini:
124+
mode: chat
125+
max_completion_tokens: 128000
126+
input_token_cost: 0.000003
127+
cached_token_cost: 0.0000015
128+
output_token_cost: 0.000012
118129
gpt-4o-mini:
119130
mode: chat
120131
max_tokens: 128000
121132
input_token_cost: 0.00000015
133+
cached_token_cost: 0.000000075
122134
output_token_cost: 0.00000060
123135
gpt-4o:
124136
mode: chat
125137
max_tokens: 128000
126-
input_token_cost: 0.000005
127-
output_token_cost: 0.000015
138+
input_token_cost: 0.0000025
139+
cached_token_cost: 0.00000125
140+
output_token_cost: 0.00001
128141
gpt-4-turbo:
129142
mode: chat
130143
max_tokens: 128000

0 commit comments

Comments
 (0)