Skip to content

feat: sglang backend #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 34 commits into
base: main
Choose a base branch
from
Draft

feat: sglang backend #165

wants to merge 34 commits into from

Conversation

@guicho271828
Copy link
Contributor

Havent ran the test yet!

@mergify
Copy link

mergify bot commented Sep 24, 2025 •
edited
Loading

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@guicho271828
feat: added smaller qwen models for debugging
2f415a6 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
feat(vllm): copied from huggingface
36202ba 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
fix(vllm): remove alora and cache
43dc0e8 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
fix(vllm): remove tool calls
153f58b 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
fix(vllm): finished the implementation with limited functionality: fr…
1abb626 
…ee-form and constrained generation

Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
fix(vllm): passing mypy and linter
6b703d1 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
fix(vllm): added vllm optional dep in pyproject.toml
756e750 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
feat(vllm test): copied from huggingface
1db23e5 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
fix(vllm test): implemented the test
7483077 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
test: require V0 in vllm test
19a2adb 
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
@guicho271828
refactor: ctx to chat conversion function
b8b41d4
@guicho271828
refactor: use_alora function
68925dd
@guicho271828
refactor: moved _extract_model_tool_requests to mellea.backends.utils
7e77f0e
@guicho271828
feat(vllm): added tool calls
a972c17
@guicho271828
test(tools): run test with mistral
cc0ff3d
@guicho271828
fix(vllm): rename model_options -> engine_args
974c2c2
@guicho271828
fix(vllm): use FancyLogger
1e2ec28
@guicho271828
fix(vllm): ignore type checking for vllm and msgspec
e72b7e9
@guicho271828
fix(vllm): fixed the backend name in the log
0db3171
@guicho271828
feat(vllm): asynchronous call support
c1ebd6d
@guicho271828
test(vllm): asynchronous call support
048e90d
@guicho271828 guicho271828 marked this pull request as draft October 17, 2025 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@guicho271828