Hubert-kakeiken-W-reverbed_clean

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_REVERBED_CLEAN - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0010
• Wer: 0.9988
• Cer: 1.0125

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
27.2567	1.0	820	10.8700	1.0	1.1283
9.1712	2.0	1640	7.5446	1.0	1.1284
6.9918	3.0	2460	4.2120	1.0	1.1284
3.601	4.0	3280	2.8831	1.0	1.1284
2.4705	5.0	4100	2.0119	1.0	1.1316
1.5222	6.0	4920	0.5329	0.9994	1.0547
0.3919	7.0	5740	0.2666	0.9994	1.0346
0.1764	8.0	6560	0.2431	0.9997	1.0610
0.1335	9.0	7380	0.0695	0.9994	1.0168
0.0925	10.0	8200	0.0985	0.9994	1.0328
0.0775	11.0	9020	0.0395	0.9990	1.0166
0.0714	12.0	9840	0.0185	0.9990	1.0139
0.0646	13.0	10660	0.0133	0.9988	1.0133
0.0684	14.0	11480	0.0457	0.9988	1.0195
0.0573	15.0	12300	0.0112	0.9988	1.0136
0.0554	16.0	13120	0.0281	0.9988	1.0152
0.0538	17.0	13940	0.3426	0.9993	1.0614
0.0558	18.0	14760	0.0073	0.9988	1.0136
0.0512	19.0	15580	0.0093	0.9988	1.0142
0.0384	20.0	16400	0.0173	0.9988	1.0159
0.0417	21.0	17220	0.0243	0.9990	1.0158
0.034	22.0	18040	0.0109	0.9988	1.0132
0.0353	23.0	18860	0.0191	0.9988	1.0146
0.0355	24.0	19680	0.0056	0.9988	1.0130
0.0314	25.0	20500	0.0056	0.9988	1.0137
0.0279	26.0	21320	0.0108	0.9988	1.0143
0.0219	27.0	22140	0.0028	0.9988	1.0129
0.0208	28.0	22960	0.0018	0.9990	1.0127
0.0187	29.0	23780	0.0048	0.9988	1.0131
0.0179	30.0	24600	0.0031	0.9988	1.0130
0.0181	31.0	25420	0.0021	0.9988	1.0128
0.0135	32.0	26240	0.0013	0.9988	1.0127
0.0113	33.0	27060	0.0019	0.9988	1.0126
0.0119	34.0	27880	0.0035	0.9988	1.0127
0.0113	35.0	28700	0.0018	0.9988	1.0126
0.0113	36.0	29520	0.0016	0.9988	1.0126
0.0085	37.0	30340	0.0011	0.9988	1.0125
0.0092	38.0	31160	0.0013	0.9988	1.0127
0.0102	39.0	31980	0.0010	0.9988	1.0125
0.0102	39.9518	32760	0.0012	0.9988	1.0125

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0