-
Notifications
You must be signed in to change notification settings - Fork 17
Open
Description
Hi,thank you for your work!
there is a question about the result of the running:
After running the Inference on MT-Bench
cd llm_judge RAYON_NUM_THREADS=6 CUDA_VISIBLE_DEVICES=0 python3 gen_model_answer_rest.py --model-path lmsys/vicuna-7b-v1.5 --model-id vicuna-7b-v1.5 --datastore-path ../datastore/datastore_chat_small.idx
How can I get the mean token time?
I do not find the result of the mean token time.
Thank you for your reply!!!
Metadata
Metadata
Assignees
Labels
No labels