eridu / checkpoint-1300
rjurney's picture
Disabled a lot of training optimizations I had introduced in this run: eridu train --use-gpu --batch-size 1000 --epochs 8 --patience 1 --resampling --weight-decay 0.01 --random-seed 31337 --warmup-ratio 0.1 --learning-rate 3e-5 --save-strategy steps --eval-strategy steps --sample-fraction 0.1
a31f1fd unverified