Introducing smolagents: simple agents that write actions in code.
•
536
Thomas Wolf PRO
thomwolf
AI & ML interests
NLP and open-source :-)
Recent Activity
liked
a model
about 2 hours ago
mistralai/Mistral-Small-24B-Base-2501
liked
a Space
about 2 hours ago
science/README
liked
a model
about 11 hours ago
allenai/Llama-3.1-Tulu-3-405B
Articles
Organizations
thomwolf's activity
liked
a
model
about 2 hours ago
liked
a
Space
about 2 hours ago
liked
a
model
about 11 hours ago
upvoted
an
article
2 days ago
Article
Welcome to Inference Providers on the Hub 🔥
•
171
upvoted
an
article
3 days ago
Article
Open-R1: a fully open reproduction of DeepSeek-R1
•
460
reacted to
mitkox's
post with 🚀👍
3 days ago
Post
2051
llama.cpp is 26.8% faster than ollama.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
- 7 replies
authored
a
paper
15 days ago
upvoted
2
articles
16 days ago
Article
MiniMax-01 is Now Open-Source: Scaling Lightning Attention for the AI Agent Era
By
•
•
40Article
Diving into MiniMax01 405B MoE
By
•
•
17
liked
a
model
17 days ago
liked
a
Space
17 days ago
liked
2
models
17 days ago
liked
a
model
22 days ago
upvoted
a
paper
24 days ago
updated
a
Space
24 days ago
reacted to
lewtun's
post with 🔥
25 days ago
Post
3682
I was initially pretty sceptical about Meta's Coconut paper [1] because the largest perf gains were reported on toy linguistic problems. However, these results on machine translation are pretty impressive!
https://x.com/casper_hansen_/status/1875872309996855343
Together with the recent PRIME method [2] for scaling RL, reasoning for open models is looking pretty exciting for 2025!
[1] Training Large Language Models to Reason in a Continuous Latent Space (2412.06769)
[2] https://huggingface.co/blog/ganqu/prime
https://x.com/casper_hansen_/status/1875872309996855343
Together with the recent PRIME method [2] for scaling RL, reasoning for open models is looking pretty exciting for 2025!
[1] Training Large Language Models to Reason in a Continuous Latent Space (2412.06769)
[2] https://huggingface.co/blog/ganqu/prime
liked
a
model
25 days ago
upvoted
an
article
26 days ago
Article
🐺🐦⬛ LLM Comparison/Test: DeepSeek-V3, QVQ-72B-Preview, Falcon3 10B, Llama 3.3 70B, Nemotron 70B in my updated MMLU-Pro CS benchmark
By
•
•
39