benchmark test script

[ShijiaTang]

add benchmark test script and its configuration file.

[guocuimi]

Thanks for adding benchmark test script. I'd like to outline the plan I have in mind to make sure we are on the same page.
Our aim is to develop a benchmark test suite that facilitates the comparison of key metrics with other leading open-source software (OSS) projects. Here are some resources we can refer to
1> https://github.com/sgl-project/sglang/tree/main/benchmark
2> https://github.com/ray-project/llmperf
3> https://github.com/vllm-project/vllm/tree/main/benchmarks

i am going to create a separate repository for end2end benchmark. let's add scripts there instead.

[ShijiaTang]

Thanks for contacting ,it’s appreciated if I can know there is any specific work that I can provide assistance now.

…

------------------ Original ------------------ From: Michael Mi ***@***.***> Date: Mon,Apr 15,2024 3:43 AM To: vectorch-ai/ScaleLLM ***@***.***> Cc: ShijiaTang ***@***.***>, Author ***@***.***> Subject: Re: [vectorch-ai/ScaleLLM] benchmark test script (PR #124) Thanks for adding benchmark test script. I'd like to outline the plan I have in mind to make sure we are on the same page. Our aim is to develop a benchmark test suite that facilitates the comparison of key metrics with other leading open-source software (OSS) projects. Here are some resources we can refer to 1> https://github.com/sgl-project/sglang/tree/main/benchmark 2> https://github.com/ray-project/llmperf 3> https://github.com/vllm-project/vllm/tree/main/benchmarks i am going to create a separate repository for end2end benchmark. let's add scripts there instead. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>