Collections

14

Attention Is All You Need

Paper • 1706.03762 • Published Jun 12, 2017 • 55
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Paper • 1810.04805 • Published Oct 11, 2018 • 17
RoBERTa: A Robustly Optimized BERT Pretraining Approach

Paper • 1907.11692 • Published Jul 26, 2019 • 7
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Paper • 1910.01108 • Published Oct 2, 2019 • 14

-

UI-TARS: Pioneering Automated GUI Interaction with Native Agents

Paper • 2501.12326 • Published Jan 21 • 54
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Paper • 2401.04081 • Published Jan 8, 2024 • 71
Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity

Paper • 2501.16295 • Published Jan 27 • 8
BlackMamba: Mixture of Experts for State-Space Models

Paper • 2402.01771 • Published Feb 1, 2024 • 25

1

-

Attention Is All You Need

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

RoBERTa: A Robustly Optimized BERT Pretraining Approach

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

UI-TARS: Pioneering Automated GUI Interaction with Native Agents

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity

BlackMamba: Mixture of Experts for State-Space Models

Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving

Mixtral of Experts

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

EdgeMoE: Fast On-Device Inference of MoE-based Large Language Models

BlackMamba: Mixture of Experts for State-Space Models

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Simple linear attention language models balance the recall-throughput tradeoff

Linear Transformers with Learnable Kernel Functions are Better In-Context Models

Repeat After Me: Transformers are Better than State Space Models at Copying

Zoology: Measuring and Improving Recall in Efficient Language Models

MambaByte: Token-free Selective State Space Model

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

hustvl/Vim-tiny

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

MambaByte: Token-free Selective State Space Model

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

MambaByte: Token-free Selective State Space Model

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

Self-Rewarding Language Models

Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding