gpt_server v0.6.7

shell-nlp released this 07 Nov 10:04

· 11 commits to main since this release

6223df4

gpt_server v0.6.7

What's Changed

1.修复tool_prarser index 为None
2.升级 sglang==0.5.5
3.支持 lmcache，后端默认开启进一步提高长上下文推理的吞吐量

Assets 2