Skip to content

Commit 4b1ff13

Browse files
authored
[Feature] Default ignore_eos True for random dataset (#28227)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
1 parent e0d6b4a commit 4b1ff13

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

vllm/benchmarks/serve.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1359,6 +1359,14 @@ async def main_async(args: argparse.Namespace) -> dict[str, Any]:
13591359
"'--dataset-path' if required."
13601360
)
13611361

1362+
# when using random datasets, default to ignoring EOS
1363+
# so generation runs to the requested length
1364+
if (
1365+
args.dataset_name in ("random", "random-mm")
1366+
and args.backend in OPENAI_COMPATIBLE_BACKENDS
1367+
):
1368+
args.ignore_eos = True
1369+
13621370
# Load the dataset.
13631371
input_requests = get_samples(args, tokenizer)
13641372
goodput_config_dict = check_goodput_args(args)

0 commit comments

Comments
 (0)