FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper
โข
2506.20920
โข
Published
โข
70
Agents Hack
community
AI & ML interests
None defined yet.
thomwolfย
authored
a
paper
2 months ago
m-ricย
updated
a
Space
3 months ago
Cidoyiย
updated
a
Space
3 months ago
thomwolfย
authored
a
paper
3 months ago
m-ricย
published
a
Space
3 months ago
Post
3958
Meet our new agentic model : ๐๐ฒ๐๐๐๐ฟ๐ฎ๐น
Devstral is an open-source LLM built software engineering tasks built under a collaboration between Mistral AI and All Hands AI ๐.
๐๐ฒ๐ ๐ณ๐ฒ๐ฎ๐๐๐ฟ๐ฒ๐ :
โข ๐ค ๐๐ด๐ฒ๐ป๐๐ : perfect for Agentic coding
โข ๐ ๐น๐ถ๐ด๐ต๐๐๐ฒ๐ถ๐ด๐ต๐: Devstral is a ๐ฎ๐ฐ๐ parameter based on Mistral small.
โข ยฉ๏ธ ๐๐ฝ๐ฎ๐ฐ๐ต๐ฒ ๐ฎ.๐ฌ, meaning fully open-source !
โข ๐ A ๐ญ๐ฎ๐ด๐ธ context window.
๐Blog : https://mistral.ai/news/devstral
โกAPI : The model is also available on our API under the name ๐ฑ๐ฒ๐๐๐๐ฟ๐ฎ๐น-๐๐บ๐ฎ๐น๐น-๐ฎ๐ฑ๐ฌ๐ฑ
๐ค repo : mistralai/Devstral-Small-2505
Can't wait to see what you will build with it !
Devstral is an open-source LLM built software engineering tasks built under a collaboration between Mistral AI and All Hands AI ๐.
๐๐ฒ๐ ๐ณ๐ฒ๐ฎ๐๐๐ฟ๐ฒ๐ :
โข ๐ค ๐๐ด๐ฒ๐ป๐๐ : perfect for Agentic coding
โข ๐ ๐น๐ถ๐ด๐ต๐๐๐ฒ๐ถ๐ด๐ต๐: Devstral is a ๐ฎ๐ฐ๐ parameter based on Mistral small.
โข ยฉ๏ธ ๐๐ฝ๐ฎ๐ฐ๐ต๐ฒ ๐ฎ.๐ฌ, meaning fully open-source !
โข ๐ A ๐ญ๐ฎ๐ด๐ธ context window.
๐Blog : https://mistral.ai/news/devstral
โกAPI : The model is also available on our API under the name ๐ฑ๐ฒ๐๐๐๐ฟ๐ฎ๐น-๐๐บ๐ฎ๐น๐น-๐ฎ๐ฑ๐ฌ๐ฑ
๐ค repo : mistralai/Devstral-Small-2505
Can't wait to see what you will build with it !
- 1 reply
Post
6481
If you've followed the progress of robotics in the past 18 months, you've likely noticed how robotics is increasingly becoming the next frontier that AI will unlock.
At Hugging Faceโin robotics and across all AI fieldsโwe believe in a future where AI and robots are open-source, transparent, and affordable; community-built and safe; hackable and fun. We've had so much mutual understanding and passion working with the Pollen Robotics team over the past year that we decided to join forces!
You can already find our open-source humanoid robot platform Reachy 2 on the Pollen website and the Pollen community and people here on the hub at
pollen-robotics
We're so excited to build and share more open-source robots with the world in the coming months!
At Hugging Faceโin robotics and across all AI fieldsโwe believe in a future where AI and robots are open-source, transparent, and affordable; community-built and safe; hackable and fun. We've had so much mutual understanding and passion working with the Pollen Robotics team over the past year that we decided to join forces!
You can already find our open-source humanoid robot platform Reachy 2 on the Pollen website and the Pollen community and people here on the hub at
pollen-robotics
We're so excited to build and share more open-source robots with the world in the coming months!
- 1 reply
thomwolfย
authored
a
paper
5 months ago
nouamanetaziย
authored
a
paper
5 months ago
thomwolfย
authored
a
paper
5 months ago
Post
3739
The new DeepSite space is really insane for vibe-coders
enzostvs/deepsite
With the wave of vibe-coding-optimized LLMs like the latest open-source DeepSeek model (version V3-0324), you can basically prompt out-of-the-box and create any app and game in one-shot.
It feels so powerful to me, no more complex framework or under-the-hood prompt engineering to have a working text-to-app tool.
AI is eating the world and *open-source* AI is eating AI itself!
PS: and even more meta is that the DeepSite app and DeepSeek model are both fully open-source code => time to start recursively improve?
PPS: you still need some inference hosting unless you're running the 600B param model at home, so check the very nice list of HF Inference Providers for this model: deepseek-ai/DeepSeek-V3-0324
enzostvs/deepsite
With the wave of vibe-coding-optimized LLMs like the latest open-source DeepSeek model (version V3-0324), you can basically prompt out-of-the-box and create any app and game in one-shot.
It feels so powerful to me, no more complex framework or under-the-hood prompt engineering to have a working text-to-app tool.
AI is eating the world and *open-source* AI is eating AI itself!
PS: and even more meta is that the DeepSite app and DeepSeek model are both fully open-source code => time to start recursively improve?
PPS: you still need some inference hosting unless you're running the 600B param model at home, so check the very nice list of HF Inference Providers for this model: deepseek-ai/DeepSeek-V3-0324
- 1 reply
Post
3075
We've kept pushing our Open-R1 project, an open initiative to replicate and extend the techniques behind DeepSeek-R1.
And even we were mind-blown by the results we got with this latest model we're releasing: โก๏ธOlympicCoder ( open-r1/OlympicCoder-7B and open-r1/OlympicCoder-32B)
It's beating Claude 3.7 on (competitive) programming โa domain Anthropic has been historically really strong atโ and it's getting close to o1-mini/R1 on olympiad level coding with just 7B parameters!
And the best part is that we're open-sourcing all about its training dataset, the new IOI benchmark, and more in our Open-R1 progress report #3: https://huggingface.co/blog/open-r1/update-3
Datasets are are releasing:
- open-r1/codeforces
- open-r1/codeforces-cots
- open-r1/ioi
- open-r1/ioi-test-cases
- open-r1/ioi-sample-solutions
- open-r1/ioi-cots
- open-r1/ioi-2024-model-solutions
And even we were mind-blown by the results we got with this latest model we're releasing: โก๏ธOlympicCoder ( open-r1/OlympicCoder-7B and open-r1/OlympicCoder-32B)
It's beating Claude 3.7 on (competitive) programming โa domain Anthropic has been historically really strong atโ and it's getting close to o1-mini/R1 on olympiad level coding with just 7B parameters!
And the best part is that we're open-sourcing all about its training dataset, the new IOI benchmark, and more in our Open-R1 progress report #3: https://huggingface.co/blog/open-r1/update-3
Datasets are are releasing:
- open-r1/codeforces
- open-r1/codeforces-cots
- open-r1/ioi
- open-r1/ioi-test-cases
- open-r1/ioi-sample-solutions
- open-r1/ioi-cots
- open-r1/ioi-2024-model-solutions
thomwolfย
authored
a
paper
7 months ago
spisupatย
authored
a
paper
7 months ago
thomwolfย
authored
a
paper
8 months ago
Post
6261
We are proud to announce
HuggingFaceFW/fineweb-2: A sparkling update to
HuggingFaceFW/fineweb with 1000s of ๐ฃ๏ธlanguages.
We applied the same data-driven approach that led to SOTA English performance in๐ท FineWeb to thousands of languages.
๐ฅ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.
The dataset is released under the permissive ๐ ODC-By 1.0 license, and the ๐ป code to reproduce it and our evaluations is public.
We will very soon announce a big community project, and are working on a ๐ blogpost walking you through the entire dataset creation process. Stay tuned!
In the mean time come ask us question on our chat place: https://huggingface.co/spaces/HuggingFaceFW/discussion
H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
We applied the same data-driven approach that led to SOTA English performance in๐ท FineWeb to thousands of languages.
๐ฅ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.
The dataset is released under the permissive ๐ ODC-By 1.0 license, and the ๐ป code to reproduce it and our evaluations is public.
We will very soon announce a big community project, and are working on a ๐ blogpost walking you through the entire dataset creation process. Stay tuned!
In the mean time come ask us question on our chat place: https://huggingface.co/spaces/HuggingFaceFW/discussion
H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
- 2 replies
Post
1726
Exponentially growing number of open-source AI models over the course of the past 30 months โ from a few thousands to over 1 million and more
Interactive data viz: huggingface/open-source-ai-year-in-review-2024
Interactive data viz: huggingface/open-source-ai-year-in-review-2024
Post
1794
Most liked and most downloaded open-source AI models from 2022 to 2024
Interactive viz: https://aiworld.eu/embed/model/model/treemap
Discussion: huggingface/open-source-ai-year-in-review-2024
Interactive viz: https://aiworld.eu/embed/model/model/treemap
Discussion: huggingface/open-source-ai-year-in-review-2024
Post
1853
Interesting long read from
@evanmiller-anthropic
on having a better founded statistical approach to Language Model Evaluations:
https://www.anthropic.com/research/statistical-approach-to-model-evals
Worth a read if you're into LLM evaluations!
Cc @clefourrier
https://www.anthropic.com/research/statistical-approach-to-model-evals
Worth a read if you're into LLM evaluations!
Cc @clefourrier
- 1 reply
Post
1561
Very exciting new
mistralai/Pixtral-Large-Instruct-2411 model from Mistral-AI
Impressive performances, huge congrats @patrickvonplaten @sgvaze @pandora-s @devendrachaplot @sophiamyang and team!
Very nice to have SOTA Multilingual OCR and Chart understanding in an open-weights model
Impressive performances, huge congrats @patrickvonplaten @sgvaze @pandora-s @devendrachaplot @sophiamyang and team!
Very nice to have SOTA Multilingual OCR and Chart understanding in an open-weights model