Hubert-kakeiken-W-incar

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_INCAR - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0218
• Wer: 0.9988
• Cer: 1.0166

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
25.3882	1.0	820	10.0306	1.0	1.1283
8.3905	2.0	1640	7.0454	1.0	1.1284
6.4857	3.0	2460	4.0955	1.0	1.1284
3.5987	4.0	3280	3.0250	1.0	1.1284
2.7266	5.0	4100	2.4966	1.0	1.1284
2.4689	6.0	4920	1.7934	1.0	1.1284
1.4875	7.0	5740	0.8063	1.0	1.0658
0.682	8.0	6560	0.3775	0.9996	1.0819
0.5278	9.0	7380	0.2049	0.9991	1.0450
0.3999	10.0	8200	0.1340	0.9990	1.0349
0.3398	11.0	9020	0.1048	0.9988	1.0241
0.305	12.0	9840	0.0596	0.9988	1.0197
0.306	13.0	10660	0.0690	0.9990	1.0211
0.2812	14.0	11480	0.0525	0.9988	1.0211
0.2885	15.0	12300	0.0574	0.9988	1.0214
0.2764	16.0	13120	0.0662	0.9988	1.0239
0.2816	17.0	13940	0.0355	0.9988	1.0189
0.2648	18.0	14760	0.0320	0.9988	1.0185
0.2449	19.0	15580	0.0489	0.9990	1.0216
0.2432	20.0	16400	0.0332	0.9990	1.0201
0.2319	21.0	17220	0.0466	0.9988	1.0206
0.2334	22.0	18040	0.0436	0.9988	1.0202
0.2386	23.0	18860	0.0312	0.9990	1.0191
0.2187	24.0	19680	0.0389	0.9993	1.0188
0.2237	25.0	20500	0.0324	0.9990	1.0180
0.2121	26.0	21320	0.0342	0.9988	1.0189
0.1965	27.0	22140	0.0274	0.9988	1.0184
0.1866	28.0	22960	0.0307	0.9990	1.0180
0.1752	29.0	23780	0.0300	0.9988	1.0178
0.1704	30.0	24600	0.0258	0.9988	1.0175
0.1736	31.0	25420	0.0220	0.9990	1.0167
0.161	32.0	26240	0.0267	0.9988	1.0172
0.1482	33.0	27060	0.0284	0.9988	1.0169
0.1476	34.0	27880	0.0228	0.9988	1.0169
0.1395	35.0	28700	0.0216	0.9988	1.0166
0.141	36.0	29520	0.0206	0.9988	1.0161
0.1338	37.0	30340	0.0230	0.9988	1.0169
0.1376	38.0	31160	0.0205	0.9988	1.0164
0.136	39.0	31980	0.0224	0.9988	1.0167
0.1361	39.9518	32760	0.0220	0.9988	1.0165

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0