SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
Paper
•
2501.17161
•
Published
•
100
•
6
Kalle Hilsenbek
Bachstelze
AI & ML interests
Combining BERT with instructions for explainable AI: gitlab.com/Bachstelze/instructionbert
Recent Activity
commented on
a paper
9 days ago
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model
Post-training
commented on
an
article
9 days ago
Is Attention Interpretable in Transformer-Based Large Language Models? Let’s Unpack the Hype
Organizations
None yet
Bachstelze's activity
commented
a
paper
9 days ago
commented on
Is Attention Interpretable in Transformer-Based Large Language Models? Let’s Unpack the Hype
9 days ago
Good write-up, though it is missing the dominant attention sink in current decoder-only models:
https://colab.research.google.com/drive/1Fcgug4a6rv9F-Wej0rNveiM_SMNZOtrr?usp=sharing
upvoted
an
article
9 days ago
Article
Is Attention Interpretable in Transformer-Based Large Language Models? Let’s Unpack the Hype
By
•
•
4
ModernBART wen?
6
#38 opened about 1 month ago
by
Fizzarolli
Goldfish model
#5 opened about 2 months ago
by
Bachstelze
Adding Evaluation Results
1
#6 opened 3 months ago
by
leaderboard-pr-bot
liked
a
dataset
3 months ago
upvoted
a
paper
4 months ago
Benchmark results
#17 opened 4 months ago
by
Bachstelze
commented
a
paper
4 months ago
upvoted
2
papers
4 months ago
commented
a
paper
4 months ago
upvoted
a
paper
4 months ago
commented
a
paper
4 months ago
upvoted
a
paper
4 months ago
commented
a
paper
4 months ago
Readme
#1 opened 5 months ago
by
Bachstelze
commented
2
papers
5 months ago
BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline
Paper
•
2408.15079
•
Published
•
53
•
4
Dolphin: Long Context as a New Modality for Energy-Efficient On-Device Language Models
Paper
•
2408.15518
•
Published
•
43
•
4