Skip to content

Commit 58f3658

Browse files
authored
Fix qwen3 deployment bugs (#155)
* merge * merge * add Mistral-Small-3.1-24B-Instruct-2503 * modify qwq-32b deploy * add txgemma model; * modify model list command * fix typo * add some ecs parameters * add glm4-z1 models * modify vllm backend * add qwen3 * fix cli bugs * fix * add deeseek r1/Qwen3-235B-A22B * fix local deploy account bug * add qwen 3 awq models * fix serialize_utils bugs * modify qwen3 deployment * modify docs * modify qwen3 engine;add strands client test
1 parent f9a0660 commit 58f3658

File tree

2 files changed

+24
-2
lines changed

2 files changed

+24
-2
lines changed

src/emd/models/llms/qwen.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@
99
vllm_qwen2d5_72b_engine064,
1010
vllm_qwq_engine073,
1111
vllm_qwq_engine082,
12-
vllm_qwen3_engin084
12+
vllm_qwen3_engin084,
13+
vllm_qwen3_engin091
1314
)
1415
from ..services import (
1516
sagemaker_service,
@@ -504,7 +505,7 @@
504505
Model.register(
505506
dict(
506507
model_id = "Qwen3-8B",
507-
supported_engines=[vllm_qwen3_engin084],
508+
supported_engines=[vllm_qwen3_engin091],
508509
supported_instances=[
509510
g5d2xlarge_instance,
510511
g5d4xlarge_instance,
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
from strands import Agent
2+
from strands.models.openai import OpenAIModel
3+
from strands_tools import calculator, current_time
4+
import logging
5+
6+
model = OpenAIModel(
7+
client_args={
8+
"api_key": "xxx",
9+
"base_url": "http://localhost:8080/v1/",
10+
},
11+
# **model_config
12+
model_id="Qwen3-8B",
13+
params={
14+
"extra_body": {"chat_template_kwargs": {"enable_thinking": False}}
15+
}
16+
)
17+
18+
19+
agent = Agent(model=model, tools=[calculator, current_time])
20+
response = agent("现在几点")
21+
print(response)

0 commit comments

Comments
 (0)