Skip to content

Pull requests: huggingface/lighteval

New pull request New
48 Open 613 Closed
48 Open 613 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[EVAL] BIG-Bench Extra Hard
#1099 opened Dec 5, 2025 by jgyasu • Draft
1
aime_avg was not added to TASKS_TABLE
#1098 opened Dec 4, 2025 by francesco-bertolotti Loading…
1
When customizing the save path, modify the "save_details" location
#1092 opened Nov 29, 2025 by Guncuke Loading…
1
fix(tasks): print also tasks not prefixed by the suite name
#1087 opened Nov 27, 2025 by bram-pramono Loading…
1
[EVAL] SciCode
#1086 opened Nov 27, 2025 by akshathmangudi Loading…
1
3
Enable loading data sets from files for custom tasks
#1083 opened Nov 24, 2025 by davebiagioni Loading…
3 tasks done
6
Evals on the hub
#1082 opened Nov 24, 2025 by NathanHB Loading…
1
feat: add MathVista benchmark new-task
#1081 opened Nov 22, 2025 by omkar-334 • Draft
1
Feature/tvd mi metric feature
#1080 opened Nov 22, 2025 by zrobertson466920 Loading…
5
[EVAL] MultiChallenge new-task
#1075 opened Nov 21, 2025 by akshathmangudi Loading…
1
13
[EVAL] Long Horizon Execution new-task
#1074 opened Nov 21, 2025 by akshathmangudi Loading…
4 tasks done
17
feat: Add Kyrgyz LLM Bench multilingual tasks new-task
#1070 opened Nov 19, 2025 by golden-ratio Loading…
7
diskcache for caching breaking enhancement
#1068 opened Nov 19, 2025 by f14-bertolotti Loading…
30
graceful shutdown of vllm async bug
#1064 opened Nov 17, 2025 by f14-bertolotti Loading…
1
remove forbiden caracters in files, caches and details bug
#1062 opened Nov 17, 2025 by NathanHB Loading…
9
Adds Profbench new-task
#1041 opened Nov 6, 2025 by NathanHB Loading…
1
Fix PERPLEXITY task
#1037 opened Nov 4, 2025 by ScottHoang Loading…
6
Legal NLP tasks on Swiss data
#1032 opened Oct 31, 2025 by rolshoven Loading…
6
Add support to vllm==0.11.0
#1027 opened Oct 22, 2025 by anmarques Loading…
1
Fixes #1023: add custom processing logic for MetricGrouping
#1025 opened Oct 22, 2025 by colinzuo Loading…
5
Wrap vllm inputs to compatible with VLLM>=0.10.2
#1003 opened Oct 2, 2025 by JIElite Loading…
1
Fix caching logic
#994 opened Sep 25, 2025 by jxmorris12 Loading…
6
Fix deberta overflow error bug
#990 opened Sep 24, 2025 by amstu2 Loading…
6
run slow tests aginst vllm and transformers main
#985 opened Sep 23, 2025 by NathanHB Loading…
2
vllm 0.10.2 makes integration tests fail bug ignore-for-release
#980 opened Sep 22, 2025 by NathanHB Loading…
2
ProTip! Exclude everything labeled bug with -label:bug.