Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper
•
2501.11873
•
Published
•
61
Yixin Song
yixinsong
AI & ML interests
None yet
Recent Activity
updated
a model
15 days ago
PowerInfer/SmallThinker-3B-Preview
new activity
15 days ago
PowerInfer/SmallThinker-3B-Preview:Eval script
Organizations
yixinsong's activity
upvoted
a
paper
9 days ago
updated
a
model
15 days ago
Eval script
4
#9 opened 15 days ago
by
rawsh
liked
a
dataset
23 days ago
About the training details
5
#5 opened 27 days ago
by
hiyouga
How to Pair with Larger Models
4
#7 opened 26 days ago
by
windkkk
Prompt/token adjust to stop "Overthinking" in unnescissary cases
2
#6 opened 26 days ago
by
fuzzy-mittenz
example use colab?
8
#3 opened 28 days ago
by
NickyNicky
Update README.md
#4 opened 28 days ago
by
AISafety
liked
a
dataset
28 days ago
Training: Second Phase
6
#2 opened about 1 month ago
by
tugstugi
[bot] Conversion to Parquet
#1 opened about 2 months ago
by
parquet-converter
[bot] Conversion to Parquet
#1 opened about 1 month ago
by
parquet-converter
Librarian Bot: Add language metadata for dataset
#2 opened 30 days ago
by
librarian-bot
liked
a
model
29 days ago
Training: Second Phase
6
#2 opened about 1 month ago
by
tugstugi
updated
a
model
about 1 month ago
Evaluation
6
#1 opened about 1 month ago
by
tugstugi
updated
a
dataset
about 1 month ago
updated
a
model
about 1 month ago