-
Couldn't load subscription status.
- Fork 45
feat: Adds a vllm backend #122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: Adds a vllm backend #122
Conversation
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
ba3bb52 to
300c148
Compare
93a191a to
69d3be8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks mostly good to me; I left a few comments and looks like there's some pre-commit checks failing as well
87cbc96 to
9701c08
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this looks good. There will be conflicts / changes required based on my async PR: #137. I'm happy to make those commits though once the async stuff gets merged and this branch gets rebased.
e84ff46 to
048e90d
Compare
61d6384 to
291b459
Compare
|
async support is already implemented |
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
…ee-form and constrained generation Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
Signed-off-by: Masataro Asai <guicho2.71828@gmail.com>
5009659 to
186af0f
Compare
186af0f to
55397ad
Compare
…PR that migrates to V1
basic VLLM backend without tool and alora support.
VLLM_V1=0to be set when running.