Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
Paper
•
2502.19261
•
Published
•
6
Takuya Akiba
iwiwi
AI & ML interests
None yet
Recent Activity
authored
a paper
6 days ago
Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive
Branching Tree Search
authored
a paper
13 days ago
Drop-Upcycling: Training Sparse Mixture of Experts with Partial
Re-initialization
upvoted
a
paper
14 days ago
Drop-Upcycling: Training Sparse Mixture of Experts with Partial
Re-initialization
Organizations
