strict_default_seed-63_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1796
• Accuracy: 0.4013

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 63
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
5.9692	0.9999	1487	4.4075	0.2938
4.3051	1.9998	2974	3.9084	0.3325
3.7034	2.9997	4461	3.6285	0.3558
3.5333	3.9997	5948	3.4659	0.3708
3.31	4.9996	7435	3.3671	0.3807
3.2377	5.9995	8922	3.3051	0.3862
3.1277	6.9994	10409	3.2697	0.3903
3.091	8.0	11897	3.2403	0.3931
3.0274	8.9999	13384	3.2207	0.3948
3.0015	9.9998	14871	3.2077	0.3969
2.9642	10.9997	16358	3.2009	0.3975
2.9446	11.9997	17845	3.1935	0.3985
2.922	12.9996	19332	3.1888	0.3992
2.9046	13.9995	20819	3.1852	0.3999
2.8939	14.9994	22306	3.1799	0.4005
2.8755	16.0	23794	3.1868	0.3999
2.8744	16.9999	25281	3.1763	0.4012
2.8578	17.9998	26768	3.1774	0.4013
2.8626	18.9997	28255	3.1776	0.4015
2.845	19.9983	29740	3.1796	0.4013

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0