Hubert-kakeiken-W-some_impulse

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_SOME_IMPULSE - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0229
• Wer: 0.9988
• Cer: 1.0163

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
33.1149	1.0	820	13.2226	1.0	1.1285
10.3184	2.0	1640	8.9598	1.0	1.1284
7.779	3.0	2460	4.6149	1.0	1.1284
3.83	4.0	3280	3.2516	1.0	1.1284
2.8359	5.0	4100	2.5651	1.0	1.1284
2.3426	6.0	4920	1.0668	0.9999	1.0541
0.8699	7.0	5740	0.4356	0.9993	1.0479
0.4545	8.0	6560	0.2383	0.9988	1.0250
0.363	9.0	7380	0.1424	0.9993	1.0263
0.2791	10.0	8200	0.1780	0.9990	1.0370
0.2432	11.0	9020	0.0618	0.9990	1.0193
0.2237	12.0	9840	0.0474	0.9988	1.0184
0.217	13.0	10660	0.0433	0.9988	1.0196
0.2	14.0	11480	0.0324	0.9988	1.0171
0.1932	15.0	12300	0.0313	0.9990	1.0173
0.1946	16.0	13120	0.0355	0.9988	1.0172
0.1753	17.0	13940	0.0345	0.9990	1.0190
0.1678	18.0	14760	0.0356	0.9988	1.0182
0.1676	19.0	15580	0.0375	0.9990	1.0193
0.1647	20.0	16400	0.0424	0.9988	1.0194
0.1551	21.0	17220	0.0247	0.9988	1.0172
0.151	22.0	18040	0.0300	0.9988	1.0177
0.1411	23.0	18860	0.0266	0.9988	1.0167
0.1348	24.0	19680	0.0261	0.9988	1.0170
0.1274	25.0	20500	0.0203	0.9988	1.0159
0.1269	26.0	21320	0.0214	0.9988	1.0161
0.1135	27.0	22140	0.0199	0.9988	1.0155
0.1135	28.0	22960	0.0225	0.9990	1.0163
0.107	29.0	23780	0.0253	0.9988	1.0166
0.103	30.0	24600	0.0197	0.9988	1.0156
0.1025	31.0	25420	0.0222	0.9988	1.0163
0.0924	32.0	26240	0.0234	0.9988	1.0160
0.0928	33.0	27060	0.0247	0.9988	1.0165
0.0891	34.0	27880	0.0215	0.9988	1.0160
0.0848	35.0	28700	0.0224	0.9988	1.0161
0.0821	36.0	29520	0.0239	0.9988	1.0163
0.0776	37.0	30340	0.0218	0.9988	1.0158
0.0823	38.0	31160	0.0220	0.9988	1.0160
0.0807	39.0	31980	0.0229	0.9988	1.0161
0.0837	39.9518	32760	0.0222	0.9988	1.0161

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0