strict_default_seed-21_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1742
• Accuracy: 0.4015

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 21
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
5.982	0.9999	1487	4.4107	0.2937
4.3094	1.9998	2974	3.9020	0.3327
3.698	2.9997	4461	3.6290	0.3560
3.5271	3.9997	5948	3.4597	0.3715
3.3064	4.9996	7435	3.3647	0.3804
3.2365	5.9995	8922	3.3049	0.3863
3.1278	6.9994	10409	3.2637	0.3906
3.0893	8.0	11897	3.2390	0.3931
3.0274	8.9999	13384	3.2203	0.3949
3.0029	9.9998	14871	3.2113	0.3964
2.9656	10.9997	16358	3.1982	0.3978
2.9434	11.9997	17845	3.1960	0.3988
2.9232	12.9996	19332	3.1895	0.3987
2.9053	13.9995	20819	3.1868	0.3996
2.8951	14.9994	22306	3.1830	0.4000
2.8775	16.0	23794	3.1873	0.3999
2.8748	16.9999	25281	3.1789	0.4005
2.8588	17.9998	26768	3.1791	0.4010
2.8627	18.9997	28255	3.1790	0.4007
2.8452	19.9983	29740	3.1742	0.4015

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0