-
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Paper • 2402.19427 • Published • 55 -
Simple linear attention language models balance the recall-throughput tradeoff
Paper • 2402.18668 • Published • 20 -
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
Paper • 2402.15220 • Published • 21 -
Linear Transformers are Versatile In-Context Learners
Paper • 2402.14180 • Published • 7
Collections
Discover the best community collections!
Collections including paper arxiv:2403.10704
-
Understanding LLMs: A Comprehensive Overview from Training to Inference
Paper • 2401.02038 • Published • 64 -
TinyLlama: An Open-Source Small Language Model
Paper • 2401.02385 • Published • 93 -
Simple and Scalable Strategies to Continually Pre-train Large Language Models
Paper • 2403.08763 • Published • 50 -
PERL: Parameter Efficient Reinforcement Learning from Human Feedback
Paper • 2403.10704 • Published • 58
-
LLM in a flash: Efficient Large Language Model Inference with Limited Memory
Paper • 2312.11514 • Published • 259 -
3D-LFM: Lifting Foundation Model
Paper • 2312.11894 • Published • 15 -
SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
Paper • 2312.15166 • Published • 58 -
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
Paper • 2312.16862 • Published • 31
-
mistralai/Mistral-7B-Instruct-v0.2
Text Generation • Updated • 3.89M • • 2.68k -
mistralai/Mixtral-8x7B-Instruct-v0.1
Text Generation • Updated • 559k • • 4.35k -
mistralai/Mixtral-8x7B-v0.1
Text Generation • Updated • 35.9k • • 1.69k -
PERL: Parameter Efficient Reinforcement Learning from Human Feedback
Paper • 2403.10704 • Published • 58
-
Ryukijano/rl_course_vizdoom_health_gathering_supreme
Reinforcement Learning • Updated -
Ryukijano/Mujoco_rl_halfcheetah_Decision_Trasformer
Reinforcement Learning • Updated • 14 -
Ryukijano/poca-SoccerTwos
Reinforcement Learning • Updated • 59 -
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
Paper • 2308.03526 • Published • 26
-
Ultra-Long Sequence Distributed Transformer
Paper • 2311.02382 • Published • 6 -
Ziya2: Data-centric Learning is All LLMs Need
Paper • 2311.03301 • Published • 20 -
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
Paper • 2311.02103 • Published • 21 -
Extending Context Window of Large Language Models via Semantic Compression
Paper • 2312.09571 • Published • 15
-
The Generative AI Paradox: "What It Can Create, It May Not Understand"
Paper • 2311.00059 • Published • 20 -
Teaching Large Language Models to Reason with Reinforcement Learning
Paper • 2403.04642 • Published • 48 -
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
Paper • 2403.07816 • Published • 40 -
PERL: Parameter Efficient Reinforcement Learning from Human Feedback
Paper • 2403.10704 • Published • 58
-
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
Paper • 2309.14509 • Published • 18 -
LLM Augmented LLMs: Expanding Capabilities through Composition
Paper • 2401.02412 • Published • 37 -
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Paper • 2401.06066 • Published • 52 -
Tuning Language Models by Proxy
Paper • 2401.08565 • Published • 23