Hubert-kakeiken-W-living_room

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_LIVING_ROOM - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0235
• Wer: 0.9988
• Cer: 1.0156

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
30.0538	1.0	820	11.4497	1.0	1.1284
9.6414	2.0	1640	7.8894	1.0	1.1284
7.2975	3.0	2460	4.3041	1.0	1.1284
3.6978	4.0	3280	3.0149	1.0	1.1284
2.6625	5.0	4100	2.3736	1.0	1.1284
2.2066	6.0	4920	1.0651	1.0	1.0525
0.8429	7.0	5740	0.5874	0.9994	1.0162
0.4608	8.0	6560	0.3571	0.9993	0.9856
0.3651	9.0	7380	0.3007	0.9994	0.9811
0.2417	10.0	8200	0.0974	0.9988	1.0253
0.1926	11.0	9020	0.0825	0.9991	1.0218
0.1787	12.0	9840	0.0637	0.9990	1.0209
0.1652	13.0	10660	0.0705	0.9988	1.0247
0.1582	14.0	11480	0.0474	0.9990	1.0197
0.1517	15.0	12300	0.0442	0.9990	1.0199
0.1452	16.0	13120	0.0572	0.9991	1.0213
0.137	17.0	13940	0.0382	0.9991	1.0186
0.1282	18.0	14760	0.0323	0.9988	1.0187
0.1199	19.0	15580	0.0382	0.9988	1.0184
0.1157	20.0	16400	0.0357	0.9988	1.0188
0.1153	21.0	17220	0.0303	0.9990	1.0174
0.1041	22.0	18040	0.0296	0.9988	1.0171
0.104	23.0	18860	0.0400	0.9990	1.0185
0.095	24.0	19680	0.0275	0.9990	1.0164
0.0884	25.0	20500	0.0442	0.9990	1.0168
0.0864	26.0	21320	0.0274	0.9988	1.0169
0.0808	27.0	22140	0.0394	0.9988	1.0182
0.0782	28.0	22960	0.0264	0.9988	1.0161
0.0729	29.0	23780	0.0205	0.9988	1.0154
0.0667	30.0	24600	0.0292	0.9988	1.0165
0.0634	31.0	25420	0.0204	0.9988	1.0154
0.0605	32.0	26240	0.0234	0.9988	1.0157
0.0588	33.0	27060	0.0184	0.9988	1.0153
0.0544	34.0	27880	0.0207	0.9988	1.0158
0.0523	35.0	28700	0.0223	0.9990	1.0158
0.0494	36.0	29520	0.0239	0.9988	1.0158
0.0464	37.0	30340	0.0237	0.9988	1.0158
0.0466	38.0	31160	0.0233	0.9988	1.0158
0.0478	39.0	31980	0.0231	0.9988	1.0157
0.0474	39.9518	32760	0.0245	0.9988	1.0157

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0