mms-1b-bigcgen-balanced-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5284

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.2736	0.1524	100	inf	1.0076
1.0685	0.3049	200	inf	0.6204
0.9126	0.4573	300	inf	0.5766
0.8135	0.6098	400	inf	0.5858
0.8525	0.7622	500	inf	0.5602
0.8381	0.9146	600	inf	0.5611
0.8736	1.0671	700	inf	0.5563
0.7539	1.2195	800	inf	0.5500
0.8762	1.3720	900	inf	0.5623
0.8359	1.5244	1000	inf	0.5496
0.7032	1.6768	1100	inf	0.5533
0.7934	1.8293	1200	inf	0.5461
0.7647	1.9817	1300	inf	0.5563
0.7607	2.1341	1400	inf	0.5452
0.7656	2.2866	1500	inf	0.5369
0.8039	2.4390	1600	inf	0.5392
0.7891	2.5915	1700	inf	0.5327
0.7253	2.7439	1800	inf	0.5415
0.7474	2.8963	1900	inf	0.5332
0.6961	3.0488	2000	inf	0.5348
0.8069	3.2012	2100	inf	0.5318
0.7615	3.3537	2200	inf	0.5371
0.7013	3.5061	2300	inf	0.5288
0.7265	3.6585	2400	inf	0.5277
0.6743	3.8110	2500	inf	0.5286

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0