Hubert-noisy-cv-kakeiken-E

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_NOISY_CV_AND_KAKEIKEN - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0050
• Wer: 0.9993
• Cer: 0.0878

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.215	1.0	3107	0.0500	0.9997	0.0967
0.1485	2.0	6214	0.0195	0.9995	0.0916
0.1623	3.0	9321	0.0708	0.9996	0.1031
0.1641	4.0	12428	0.0250	0.9996	0.0932
0.1795	5.0	15535	0.0348	0.9996	0.0953
0.1641	6.0	18642	0.0181	0.9996	0.0910
0.1557	7.0	21749	0.0161	0.9997	0.0905
0.1481	8.0	24856	0.0148	0.9996	0.0906
0.1423	9.0	27963	0.0147	0.9996	0.0904
0.1244	10.0	31070	0.0108	0.9994	0.0896
0.1216	11.0	34177	0.0092	0.9996	0.0891
0.1112	12.0	37284	0.0069	0.9994	0.0885
0.095	13.0	40391	0.0057	0.9995	0.0882
0.0844	14.0	43498	0.0057	0.9993	0.0880
0.0786	15.0	46605	0.0056	0.9994	0.0880
0.0718	16.0	49712	0.0054	0.9994	0.0879
0.0631	17.0	52819	0.0052	0.9994	0.0878
0.0598	18.0	55926	0.0050	0.9993	0.0878
0.0578	19.0	59033	0.0052	0.9994	0.0878
0.0527	19.9937	62120	0.0052	0.9994	0.0878

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3