Open-R1: a fully open reproduction of DeepSeek-R1
•
460
Leandro von Werra
lvwerra
AI & ML interests
NLP and RL
Recent Activity
commented on
their
article
1 day ago
Open-R1: a fully open reproduction of DeepSeek-R1
commented on
their
article
1 day ago
Open-R1: a fully open reproduction of DeepSeek-R1
commented on
their
article
2 days ago
Open-R1: a fully open reproduction of DeepSeek-R1
Articles
Organizations
lvwerra's activity
commented on
Open-R1: a fully open reproduction of DeepSeek-R1
1 day ago
@edbeeching has evaluated the released models already
(src: https://x.com/edwardbeeching/status/1884273209136275742)
commented on
Open-R1: a fully open reproduction of DeepSeek-R1
1 day ago
Historically, they have never released code or datasets of their LLM training, so I wouldn't expect this time to be different. If they would release it that would be amazing of course!
In the meantime we have to make best guess estimates and see if we can get there ourselves.
commented on
Open-R1: a fully open reproduction of DeepSeek-R1
2 days ago
It's really cool to see how the whole open source community comes together!
upvoted
an
article
2 days ago
Article
Welcome to Inference Providers on the Hub 🔥
•
171
upvoted
a
paper
2 days ago
upvoted
an
article
3 days ago
Article
Open-R1: a fully open reproduction of DeepSeek-R1
•
460
liked
a
model
11 days ago
upvoted
a
paper
14 days ago
authored
a
paper
15 days ago
reacted to
lewtun's
post with 🔥
about 1 month ago
Post
2253
This paper (
HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs (2412.18925)) has a really interesting recipe for inducing o1-like behaviour in Llama models:
* Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting.
* Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases)
* Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1
* Use the resulting data for SFT & RL
* Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement.
Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!
* Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting.
* Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases)
* Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1
* Use the resulting data for SFT & RL
* Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement.
Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!
- 1 reply
liked
a
Space
about 1 month ago
updated
3
Spaces
about 1 month ago