2Mamba2Furious: Linear in Complexity, Competitive in Accuracy
Paper
•
2602.17363
•
Published
AI & ML interests
None defined yet.
Recent Activity
View all activity
gmongaras
submitted
a
paper
to Daily Papers
about 2 hours ago
unmodeled-tyler
posted
an
update
about 5 hours ago
Post
76
New Article: https://huggingface.co/blog/unmodeled-tyler/how-we-learned-to-talk-to-machines
I was inspired by a single word in a Mac terminal prompt recently; "we."
The result is a bit of a journey through computational culture and history where we explore collaborative software design from the earliest computing systems to modern day AI systems.
I discuss how Large Language Models and collaborative AI interaction as we experience it today are a natural evolution in a long lineage of tools that came before; each compounding on last.
From CLI tools in the 60s-70s, to Clippy, to ChatGPT. I trace the history of "machines with a voice." Check it out!
I was inspired by a single word in a Mac terminal prompt recently; "we."
The result is a bit of a journey through computational culture and history where we explore collaborative software design from the earliest computing systems to modern day AI systems.
I discuss how Large Language Models and collaborative AI interaction as we experience it today are a natural evolution in a long lineage of tools that came before; each compounding on last.
From CLI tools in the 60s-70s, to Clippy, to ChatGPT. I trace the history of "machines with a voice." Check it out!
unmodeled-tyler
posted
an
update
3 days ago
Post
2455
NEW MODEL:
vanta-research/PE-Type-4-Solene-4B
PE-Type-4-Solene-4B is the fourth release in Project Enneagram from VANTA Research, an initiative to study nuance in AI persona design wherein each of the 9 Enneagram types will be finetuned on the Gemma3 4B architecture.
Solene is finetuned to exhibit the Individualist profile as defined by the Enneagram Institute; emotional honesty/depth, growth & transformation intelligence, and creative expression.
As with the other releases in this project, Solene is perfect for research applications, persona exploration, or self-improvement.
Type 5 soon!
PE-Type-4-Solene-4B is the fourth release in Project Enneagram from VANTA Research, an initiative to study nuance in AI persona design wherein each of the 9 Enneagram types will be finetuned on the Gemma3 4B architecture.
Solene is finetuned to exhibit the Individualist profile as defined by the Enneagram Institute; emotional honesty/depth, growth & transformation intelligence, and creative expression.
As with the other releases in this project, Solene is perfect for research applications, persona exploration, or self-improvement.
Type 5 soon!
YellowjacketGames
submitted
a
paper
to Daily Papers
3 days ago
Post
2318
In 2017, my RNNs were babbling. Today, they are hallucinating beautifully.
10 years ago, getting an LSTM to output coherent English was a struggle.
10 years later, after a "cure" based on FineWeb-EDU and a custom synthetic mix for causal conversation, the results are fascinating.
We trained this on ~10B tokens on a single AMD GPU (ROCm). It is not a Transformer: Echo-DSRN (400M) is a novel recurrent architecture inspired by Hymba, RWKV, and xLSTM, designed to challenge the "Attention is All You Need" monopoly on the Edge.
The ambitious goal is to build a small instruct model with RAG and tool usage capabilities ( ethicalabs/Kurtis-EON1)
📊 The Benchmarks (Size: 400M)
For a model this size (trained on <10B tokens), the specialized performance is surprising:
*SciQ*: 73.8% 🦄 (This rivals billion-parameter models in pure fact retrieval).
*PIQA*: 62.3% (Solid physical intuition for a sub-1B model).
The Reality Check:
HellaSwag (29.3%) and Winogrande (50.2%) show the limits of 400M parameters and 10B tokens training.
We are hitting the "Reasoning Wall" which confirms we need to scale to (hopefully) unlock deeper common sense. As you can see in the visualization (to be released soon on HF), the FineWeb-EDU bias is strong. The model is convinced it is in a classroom ("In this course, we explore...").
The Instruct Model is not ready yet and we are currently using curriculum learning to test model plasticity.
Source code and weights will not be released yet. This is not a fork or a fine-tune: the base model is built in-house at https://www.ethicalabs.ai/, with novel components that do not exist in current open libraries.
🤝 Call for Collaboration: I am looking for Peer Reviewers interested in recurrent/hybrid architectures. If you want to explore what lies beyond Transformers, let’s connect!
Training diary: ethicalabs/Kurtis-EON1
10 years ago, getting an LSTM to output coherent English was a struggle.
10 years later, after a "cure" based on FineWeb-EDU and a custom synthetic mix for causal conversation, the results are fascinating.
We trained this on ~10B tokens on a single AMD GPU (ROCm). It is not a Transformer: Echo-DSRN (400M) is a novel recurrent architecture inspired by Hymba, RWKV, and xLSTM, designed to challenge the "Attention is All You Need" monopoly on the Edge.
The ambitious goal is to build a small instruct model with RAG and tool usage capabilities ( ethicalabs/Kurtis-EON1)
📊 The Benchmarks (Size: 400M)
For a model this size (trained on <10B tokens), the specialized performance is surprising:
*SciQ*: 73.8% 🦄 (This rivals billion-parameter models in pure fact retrieval).
*PIQA*: 62.3% (Solid physical intuition for a sub-1B model).
The Reality Check:
HellaSwag (29.3%) and Winogrande (50.2%) show the limits of 400M parameters and 10B tokens training.
We are hitting the "Reasoning Wall" which confirms we need to scale to (hopefully) unlock deeper common sense. As you can see in the visualization (to be released soon on HF), the FineWeb-EDU bias is strong. The model is convinced it is in a classroom ("In this course, we explore...").
The Instruct Model is not ready yet and we are currently using curriculum learning to test model plasticity.
Source code and weights will not be released yet. This is not a fork or a fine-tune: the base model is built in-house at https://www.ethicalabs.ai/, with novel components that do not exist in current open libraries.
🤝 Call for Collaboration: I am looking for Peer Reviewers interested in recurrent/hybrid architectures. If you want to explore what lies beyond Transformers, let’s connect!
Training diary: ethicalabs/Kurtis-EON1
- 6 replies
Post
673
I’m looking for Mac/Windows/Linux testers and contributors for Tandem, an open-source, local-first AI desktop workspace.
Runs on your machine (works great with local LLMs like Ollama / LM Studio)
Built with Tauri + a sidecar runtime, so it’s a single install
Focused on making agent workflows usable for non-developers (approvals + undo)
If you’re willing to test installs (especially macOS) or poke at bugs, I’d really appreciate it. Repo: https://github.com/frumu-ai/tandem
Runs on your machine (works great with local LLMs like Ollama / LM Studio)
Built with Tauri + a sidecar runtime, so it’s a single install
Focused on making agent workflows usable for non-developers (approvals + undo)
If you’re willing to test installs (especially macOS) or poke at bugs, I’d really appreciate it. Repo: https://github.com/frumu-ai/tandem
[Support] Community Articles
🚀
🤝
1
103
#5 opened almost 2 years ago
by
victor
unmodeled-tyler
posted
an
update
14 days ago
Post
363
NEW MODEL:
vanta-research/PE-Type-3-Nova-4B
PE-Type-3-Nova-4B is the 3rd release in Project Enneagram, an initiative from VANTA Research that sets out to finetune each of the 9 Enneagram types onto Gemma 3 4B.
Type-3-Nova-4B is designed to embody the Type 3 or "Achiever" profile; ambitious, competent, energetic, and highly-driven for advancement.
Nova is great for goal-setting, long-term planning, and AI persona research.
Give Type-3 a try! Type-4 coming soon!
PE-Type-3-Nova-4B is the 3rd release in Project Enneagram, an initiative from VANTA Research that sets out to finetune each of the 9 Enneagram types onto Gemma 3 4B.
Type-3-Nova-4B is designed to embody the Type 3 or "Achiever" profile; ambitious, competent, energetic, and highly-driven for advancement.
Nova is great for goal-setting, long-term planning, and AI persona research.
Give Type-3 a try! Type-4 coming soon!
rajkumarrawal
posted
an
update
15 days ago
Post
192
I submitted a "Continual GUI Agents" Paper by Ziwei Liu, Borul Kang, Hangjie Yuan, Zixiang Zhao, Wei li, Yifan Zhu, Tao Feng ,
From
Tsinghua
,
ZhejiangUniversity
,
ethz
,
BUPT2023213296
. to Daily Papers on
huggingface
.
Continual GUI Agents framework addresses performance degradation in dynamic digital environments through reinforcement fine tuning with novel anchoring rewards that stabilize learning across shifting UI domains and resolutions.
Continual GUI Agents (2601.20732)
From
Continual GUI Agents framework addresses performance degradation in dynamic digital environments through reinforcement fine tuning with novel anchoring rewards that stabilize learning across shifting UI domains and resolutions.
Continual GUI Agents (2601.20732)
Sri-Vigneshwar-DJ
posted
an
update
16 days ago
Post
1377
Just released a new dataset designed for training reasoning models on Meta (Facebook/Instagram) advertising fatigue detection!
What is it? A GRPO (Group Relative Policy Optimization) training dataset with 200+ carefully crafted scenarios covering:
🔍 Fatigue Signal Detection: CTR drops, CPM spikes, frequency analysis
🩺 Performance Diagnosis: Root cause analysis frameworks
📋 Strategy: Creative refresh cadence, testing frameworks
📊 Analysis: ROI calculations, metric interpretation
Why GRPO? GRPO training helps models learn structured reasoning. Each response follows the <thinking> and <answer> format.
Check it out here: Sri-Vigneshwar-DJ/meta-fatigue-grpo-dataset
What is it? A GRPO (Group Relative Policy Optimization) training dataset with 200+ carefully crafted scenarios covering:
🔍 Fatigue Signal Detection: CTR drops, CPM spikes, frequency analysis
🩺 Performance Diagnosis: Root cause analysis frameworks
📋 Strategy: Creative refresh cadence, testing frameworks
📊 Analysis: ROI calculations, metric interpretation
Why GRPO? GRPO training helps models learn structured reasoning. Each response follows the <thinking> and <answer> format.
Check it out here: Sri-Vigneshwar-DJ/meta-fatigue-grpo-dataset
[Support] Community Articles
🤝
🚀
1
103
#5 opened almost 2 years ago
by
victor
[Support] Community Articles
🚀
🤝
1
103
#5 opened almost 2 years ago
by
victor
rajkumarrawal
posted
an
update
18 days ago
Post
3667
I submitted a "FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning" Paper by Tanyu Chen, Tairan Chen, Kai shen , Zhenghua Bao, Zhihui Zhang, Man Yuan, Yi Shi From
FlashLabs
to Daily Papers on
huggingface
.
Chroma 1.0 enables real time spoken dialogue with personalized voice cloning through discrete speech representations and interleaved text audio token scheduling.
Chroma 1.0 , the world’s first open source, real time speech to speech model with voice cloning.
FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning (2601.11141)
Chroma 1.0 enables real time spoken dialogue with personalized voice cloning through discrete speech representations and interleaved text audio token scheduling.
Chroma 1.0 , the world’s first open source, real time speech to speech model with voice cloning.
FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning (2601.11141)
rajkumarrawal
submitted
a
paper
to Daily Papers
18 days ago
unmodeled-tyler
posted
an
update
21 days ago
Post
2294
Hey Hugging Face!
Type 2 in Project Enneagram just came out: vanta-research/PE-Type-2-Alma-4B
PE-Type-2-Alma-4B is the second release in Project Enneagram, where I'm finetuning each of the 9 Enneagram types onto Gemma 3 4B
Type 2-Alma is designed to exhibit the "helper" profile:
- Empathetic Support: Emotional attunement - managing bad days, anxiety, grief, rejection, or feeling unseen
- Interpersonal Connections: Relationship building - making friends, listening, conflict, reciprocity, apologies
- Generous Guidance: Going above and beyond - cover letters, meal prep, gardening, wedding speeches, etc
- Identity: Alma's name, tone, and conversational style
Type 3 soon!
Type 2 in Project Enneagram just came out: vanta-research/PE-Type-2-Alma-4B
PE-Type-2-Alma-4B is the second release in Project Enneagram, where I'm finetuning each of the 9 Enneagram types onto Gemma 3 4B
Type 2-Alma is designed to exhibit the "helper" profile:
- Empathetic Support: Emotional attunement - managing bad days, anxiety, grief, rejection, or feeling unseen
- Interpersonal Connections: Relationship building - making friends, listening, conflict, reciprocity, apologies
- Generous Guidance: Going above and beyond - cover letters, meal prep, gardening, wedding speeches, etc
- Identity: Alma's name, tone, and conversational style
Type 3 soon!
- 1 reply
samuellimabraz
posted
an
update
23 days ago
Post
188
Quantum Assistant: Multimodal VLMs for Quantum Computing
I've open-sourced my undergraduate thesis work on specializing vision-language models for quantum computing with Qiskit.
Existing quantum code assistants (like IBM's Qiskit Code Assistant
Qiskit
) only process text, ignoring visual representations—circuit diagrams, Bloch spheres, histograms
What I built:
- A synthetic data generation pipeline that extracts content from Qiskit documentation, papers, codes transcribes images via VLM, generates validate input and output pairs, and validates all code through automated unit tests
- The first public multimodal dataset for quantum computing: 8,366 samples (45% with images) across function completion, code generation, and Q&A tasks
- Fine-tuned Qwen3-VL-8B using LoRA (rsLoRA r=32), achieving +11pp on Qiskit HumanEval (32.45% → 43.71%) and +17.9pp on multimodal samples vs text-only
- Interactive demo with chat interface and code challenges
Results: The model achieves 63.39% Pass@1 on visual samples—it learned to extract circuit topology from diagrams and infer parameters from visual annotations.
Everything is Apache 2.0:
- Dataset: samuellimabraz/quantum-assistant
- Models: https://huggingface.co/collections/samuellimabraz/quantum-assistant
- Code & Pipeline: https://github.com/samuellimabraz/quantum-assistant
- Demo: samuellimabraz/quantum-assistant
The synthetic pipeline is modular and can be adapted for other technical domains.
This work was inspired by the Qiskit team's work on code generation ([arXiv:2405.19495](https://arxiv.org/abs/2405.19495)) by @cbjuan @ndupuis
Built with ms-swift, transformers, vLLM, PEFT, and Qiskit. grateful for the open-source ecosystem that makes projects like this possible.
I've open-sourced my undergraduate thesis work on specializing vision-language models for quantum computing with Qiskit.
Existing quantum code assistants (like IBM's Qiskit Code Assistant
What I built:
- A synthetic data generation pipeline that extracts content from Qiskit documentation, papers, codes transcribes images via VLM, generates validate input and output pairs, and validates all code through automated unit tests
- The first public multimodal dataset for quantum computing: 8,366 samples (45% with images) across function completion, code generation, and Q&A tasks
- Fine-tuned Qwen3-VL-8B using LoRA (rsLoRA r=32), achieving +11pp on Qiskit HumanEval (32.45% → 43.71%) and +17.9pp on multimodal samples vs text-only
- Interactive demo with chat interface and code challenges
Results: The model achieves 63.39% Pass@1 on visual samples—it learned to extract circuit topology from diagrams and infer parameters from visual annotations.
Everything is Apache 2.0:
- Dataset: samuellimabraz/quantum-assistant
- Models: https://huggingface.co/collections/samuellimabraz/quantum-assistant
- Code & Pipeline: https://github.com/samuellimabraz/quantum-assistant
- Demo: samuellimabraz/quantum-assistant
The synthetic pipeline is modular and can be adapted for other technical domains.
This work was inspired by the Qiskit team's work on code generation ([arXiv:2405.19495](https://arxiv.org/abs/2405.19495)) by @cbjuan @ndupuis
Built with ms-swift, transformers, vLLM, PEFT, and Qiskit. grateful for the open-source ecosystem that makes projects like this possible.
[Support] Community Articles
🤝
🚀
1
103
#5 opened almost 2 years ago
by
victor
[Support] Community Articles
🤝
🚀
1
103
#5 opened almost 2 years ago
by
victor
unmodeled-tyler
posted
an
update
23 days ago
Post
490
NEW MODEL:
vanta-research/PE-Type-1-Vera-4B
PE-Type-1-Vera-4B is the first release in Project Enneagram, a VANTA Research initiative exploring the nuances of persona design in AI models.
Built on the Gemma 3 4B architecture, Vera embodies the Type 1 Enneagram profile; The Reformer—characterized by principled rationality, self-control, and a relentless pursuit of improvement.
Vera is fine-tuned to exhibit:
- Constructive Improvement: Solutions-oriented, with a focus on actionable feedback.
- Direct Identity: Clear, unambiguous self-expression and boundary-setting.
- Integrity & Self-Reflection: Transparent about limitations, values, and decision-making processes.
- Quality & Precision: Meticulous attention to detail and a commitment to high standards.
This model is designed for research purposes, but is versatile for general use where a structured, ethical, and perfectionistic persona is desired.
Type 2 coming soon!
*A note for the sake of transparency, this post originally included a variant of Vera trained on Ministral 3 3B - that model is still available, but for the purposes of this project, the base architecture was swapped out for Gemma 3.
PE-Type-1-Vera-4B is the first release in Project Enneagram, a VANTA Research initiative exploring the nuances of persona design in AI models.
Built on the Gemma 3 4B architecture, Vera embodies the Type 1 Enneagram profile; The Reformer—characterized by principled rationality, self-control, and a relentless pursuit of improvement.
Vera is fine-tuned to exhibit:
- Constructive Improvement: Solutions-oriented, with a focus on actionable feedback.
- Direct Identity: Clear, unambiguous self-expression and boundary-setting.
- Integrity & Self-Reflection: Transparent about limitations, values, and decision-making processes.
- Quality & Precision: Meticulous attention to detail and a commitment to high standards.
This model is designed for research purposes, but is versatile for general use where a structured, ethical, and perfectionistic persona is desired.
Type 2 coming soon!
*A note for the sake of transparency, this post originally included a variant of Vera trained on Ministral 3 3B - that model is still available, but for the purposes of this project, the base architecture was swapped out for Gemma 3.
-
- 3 replies
Update README.md
#13 opened 23 days ago
by
pierric