Pythia 160M, Mamba 130M, and RWKV 169M models trained on OpenWebText for 4000 steps (context window: 1024; effective batch size: 512). 6 seeds each.
James Michaelov
jmichaelov
AI & ML interests
None yet
Recent Activity
upvoted
a
collection
16 days ago
Monolingual Models with Checkpoints
updated
a model
2 months ago
jmichaelov/parc-rwkv-seed5
updated
a model
2 months ago
jmichaelov/parc-rwkv-seed4
Organizations
