Open-R1: a fully open reproduction of DeepSeek-R1
•
452
Lewis Tunstall PRO
lewtun
AI & ML interests
LLMs, LLMs, LLMs
Articles
Organizations
Posts
6
Post
9443
We are reproducing the full DeepSeek R1 data and training pipeline so everybody can use their recipe. Instead of doing it in secret we can do it together in the open!
🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1.
🧠 Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will involve curating new, large-scale datasets for math, reasoning, and code.
🔥 Step 3: show we can go from base model -> SFT -> RL via multi-stage training.
Follow along: https://github.com/huggingface/open-r1
🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1.
🧠 Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will involve curating new, large-scale datasets for math, reasoning, and code.
🔥 Step 3: show we can go from base model -> SFT -> RL via multi-stage training.
Follow along: https://github.com/huggingface/open-r1
Post
3678
I was initially pretty sceptical about Meta's Coconut paper [1] because the largest perf gains were reported on toy linguistic problems. However, these results on machine translation are pretty impressive!
https://x.com/casper_hansen_/status/1875872309996855343
Together with the recent PRIME method [2] for scaling RL, reasoning for open models is looking pretty exciting for 2025!
[1] Training Large Language Models to Reason in a Continuous Latent Space (2412.06769)
[2] https://huggingface.co/blog/ganqu/prime
https://x.com/casper_hansen_/status/1875872309996855343
Together with the recent PRIME method [2] for scaling RL, reasoning for open models is looking pretty exciting for 2025!
[1] Training Large Language Models to Reason in a Continuous Latent Space (2412.06769)
[2] https://huggingface.co/blog/ganqu/prime
Collections
4
spaces
20
models
260
lewtun/Qwen2-0.5B-SFT
Updated
lewtun/Qwen2.5-0.5B-SFT-LoRA
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing-no-lm-head
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-no-packing
Updated
lewtun/Llama-3.1-8B-SFT-QLoRA-packing
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing-no-saved-modules
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing
Updated
lewtun/Llama-3.1-8B-SFT-LoRA-packing-pad-token-eos
Updated
lewtun/Llama-3.1-8B-SFT-QLoRA-packing-pad-token-eos
Updated
lewtun/Llama-3.1-8B-SFT-full-packing
Text Generation
•
Updated
•
4
datasets
61
lewtun/details_Qwen__Qwen2.5-Math-1.5B-Instruct
Viewer
•
Updated
•
11k
•
41
lewtun/test-eval
Viewer
•
Updated
•
1
•
19
lewtun/Llama-3.2-1B-Instruct-best_of_n-prm-completions
Viewer
•
Updated
•
10
•
51
lewtun/math-bon-debug
Preview
•
Updated
•
638
lewtun/test-fast-parser-l1b-v3
Viewer
•
Updated
•
509
•
48
lewtun/test-fast-parser-l1b-v2
Viewer
•
Updated
•
509
•
67
lewtun/test-fast-parser-l1b
Viewer
•
Updated
•
509
•
58
lewtun/test-fast-parser
Viewer
•
Updated
•
25
•
54
lewtun/bon-prm-serverless-batched
Viewer
•
Updated
•
240
•
67
lewtun/ultrafeedback_gkd
Viewer
•
Updated
•
62.1k
•
47