Skip to content

Conversation

@guicho271828
Copy link
Contributor

basic VLLM backend without tool and alora support.

  • Tests are copied from huggingface, failing results are optional ones as follows
=================================================== short test summary info ===================================================
FAILED test/backends/test_vllm.py::test_multiturn - assert 'β' in "\nokay, let's see. the user asked to take the result of"
FAILED test/backends/test_vllm.py::test_generate_from_raw_with_format - AssertionError: formatting directive failed for { "n...
========================================== 2 failed, 5 passed, 3 warnings in 14.99s ===========================================
  • Requires VLLM_V1=0 to be set when running.

@mergify
Copy link

mergify bot commented Sep 5, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@guicho271828 guicho271828 force-pushed the masa/vllm-backend branch 6 times, most recently from ba3bb52 to 300c148 Compare September 5, 2025 00:57
@jakelorocco jakelorocco changed the title Masa/vllm backend feat(vllm): Masa/vllm backend Sep 8, 2025
@nrfulton nrfulton changed the title feat(vllm): Masa/vllm backend feat: Adds a vllm backend Sep 10, 2025
@guicho271828 guicho271828 force-pushed the masa/vllm-backend branch 3 times, most recently from 93a191a to 69d3be8 Compare September 10, 2025 22:17
Copy link
Contributor

@jakelorocco jakelorocco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks mostly good to me; I left a few comments and looks like there's some pre-commit checks failing as well

Copy link
Contributor

@jakelorocco jakelorocco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good. There will be conflicts / changes required based on my async PR: #137. I'm happy to make those commits though once the async stuff gets merged and this branch gets rebased.

@guicho271828 guicho271828 force-pushed the masa/vllm-backend branch 5 times, most recently from e84ff46 to 048e90d Compare September 24, 2025 18:53
@guicho271828 guicho271828 force-pushed the masa/vllm-backend branch 4 times, most recently from 61d6384 to 291b459 Compare October 1, 2025 18:35
@guicho271828
Copy link
Contributor Author

async support is already implemented

guicho271828 and others added 22 commits October 21, 2025 18:59
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
…ee-form and constrained generation

Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants