wav2vec2-large-xlsr-sorani-v1

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2492
• Wer: 0.3566

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
9.2607	0.3543	200	6.4854	1.0
3.0716	0.7086	400	3.0408	1.0
2.8638	1.0620	600	2.8982	1.0
2.8666	1.4163	800	2.8511	1.0
1.8111	1.7706	1000	1.5181	1.0095
0.7703	2.1240	1200	0.5528	0.6563
0.6864	2.4783	1400	0.4600	0.5909
0.5955	2.8326	1600	0.4036	0.5362
0.4894	3.1860	1800	0.3811	0.5030
0.5007	3.5403	2000	0.3551	0.4782
0.5169	3.8946	2200	0.3340	0.4627
0.4613	4.2480	2400	0.3271	0.4527
0.4499	4.6023	2600	0.3229	0.4437
0.4093	4.9566	2800	0.3042	0.4333
0.4458	5.3100	3000	0.3040	0.4316
0.4107	5.6643	3200	0.2875	0.4162
0.3728	6.0177	3400	0.2862	0.4150
0.348	6.3720	3600	0.2939	0.4068
0.3393	6.7263	3800	0.2755	0.3977
0.3572	7.0797	4000	0.2799	0.3975
0.3453	7.4340	4200	0.2748	0.3975
0.3434	7.7883	4400	0.2688	0.3907
0.3231	8.1417	4600	0.2706	0.3871
0.3148	8.4960	4800	0.2688	0.3830
0.3261	8.8503	5000	0.2625	0.3811
0.3055	9.2037	5200	0.2597	0.3719
0.3028	9.5580	5400	0.2593	0.3720
0.3246	9.9123	5600	0.2535	0.3724
0.274	10.2657	5800	0.2551	0.3690
0.2832	10.6200	6000	0.2601	0.3716
0.2899	10.9743	6200	0.2521	0.3670
0.2889	11.3277	6400	0.2574	0.3652
0.2675	11.6820	6600	0.2564	0.3611
0.3334	12.0354	6800	0.2548	0.3603
0.2568	12.3897	7000	0.2538	0.3606
0.2842	12.7440	7200	0.2566	0.3597
0.2631	13.0974	7400	0.2540	0.3580
0.2573	13.4517	7600	0.2513	0.3569
0.27	13.8060	7800	0.2499	0.3556
0.2578	14.1594	8000	0.2496	0.3556
0.2528	14.5137	8200	0.2498	0.3565
0.2644	14.8680	8400	0.2492	0.3566

Framework versions

• Transformers 4.54.1
• Pytorch 2.7.1+cu128
• Datasets 3.6.0
• Tokenizers 0.21.4