BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks
•
43
BigCode
Enterprise
non-profit
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
bigcode's activity
Post
9480
We are reproducing the full DeepSeek R1 data and training pipeline so everybody can use their recipe. Instead of doing it in secret we can do it together in the open!
🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1.
🧠 Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will involve curating new, large-scale datasets for math, reasoning, and code.
🔥 Step 3: show we can go from base model -> SFT -> RL via multi-stage training.
Follow along: https://github.com/huggingface/open-r1
🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1.
🧠 Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will involve curating new, large-scale datasets for math, reasoning, and code.
🔥 Step 3: show we can go from base model -> SFT -> RL via multi-stage training.
Follow along: https://github.com/huggingface/open-r1
- 5 replies
terryyz
updated
5
datasets
7 days ago
bigcode/bigcodebench-hard-results
Viewer
•
Updated
•
163
•
93
•
1
bigcode/bigcodebench-hard-solve-rate
Viewer
•
Updated
•
296
•
109
•
1
bigcode/bigcodebench-hard-domain
Viewer
•
Updated
•
297
•
93
•
1
bigcode/bigcodebench-hard-perf
Viewer
•
Updated
•
297
•
90
bigcode/bigcodebench-results
Viewer
•
Updated
•
163
•
99
•
2
terryyz
updated
a
Space
8 days ago
terryyz
updated
3
datasets
8 days ago
terryyz
updated
a
Space
9 days ago
Phi4 - Is it the new best small model?
1
#89 opened 14 days ago
by
AgileAndy
terryyz
updated
a
model
17 days ago
terryyz
updated
4
datasets
17 days ago
Incorrect URL in tests for example 1005
4
#3 opened 18 days ago
by
danieldritter
LouisCastricato
authored
a
paper
18 days ago
BrigitteTousi
posted
an
update
21 days ago
Post
1068
Community fine-tuned models are more carbon efficient than the models they are derived from! 🥳🌿
@alozowski @clefourrier @SaylorTwift @albertvillanova evaluated CO₂ emissions associated with model inference for over 3000 models on the Open LLM Leaderboard. Interesting trends and new insights emerged...👀
Blog Post: https://huggingface.co/blog/leaderboard-emissions-analysis
Leaderboard: open-llm-leaderboard/open_llm_leaderboard
@alozowski @clefourrier @SaylorTwift @albertvillanova evaluated CO₂ emissions associated with model inference for over 3000 models on the Open LLM Leaderboard. Interesting trends and new insights emerged...👀
Blog Post: https://huggingface.co/blog/leaderboard-emissions-analysis
Leaderboard: open-llm-leaderboard/open_llm_leaderboard