distilhubert-finetuned-pulse-train_batch4-epochs30

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6146
• Accuracy: 0.7143

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.6853	1.0	31	0.6636	0.7143
0.706	2.0	62	0.6024	0.7143
0.681	3.0	93	0.6152	0.7143
0.6871	4.0	124	0.6462	0.7143
0.6656	5.0	155	0.6124	0.7143
0.6782	6.0	186	0.6151	0.7143
0.6889	7.0	217	0.6671	0.7143
0.6267	8.0	248	0.6076	0.7143
0.6768	9.0	279	0.6093	0.7143
0.6762	10.0	310	0.6150	0.7143
0.7003	11.0	341	0.6177	0.7143
0.685	12.0	372	0.6018	0.7143
0.6524	13.0	403	0.6105	0.7143
0.6508	14.0	434	0.6178	0.7143
0.8111	15.0	465	0.6101	0.7143
0.6728	16.0	496	0.6128	0.7143
0.6346	17.0	527	0.6183	0.7143
0.758	18.0	558	0.6007	0.7143
0.6756	19.0	589	0.6102	0.7143
0.603	20.0	620	0.6116	0.7143
0.8049	21.0	651	0.6042	0.7143
0.6767	22.0	682	0.6077	0.7143
0.7293	23.0	713	0.6138	0.7143
0.6312	24.0	744	0.6200	0.7143
0.7158	25.0	775	0.6202	0.7143
0.6508	26.0	806	0.6177	0.7143
0.722	27.0	837	0.6150	0.7143
0.6738	28.0	868	0.6160	0.7143
0.7221	29.0	899	0.6143	0.7143
0.5777	30.0	930	0.6146	0.7143

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0