Hubert-kakeiken-W-meeting_room

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_MEETING_ROOM - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0146
• Wer: 0.9990
• Cer: 1.0147

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
29.232	1.0	820	11.3507	1.0	1.1284
9.4623	2.0	1640	7.8286	1.0	1.1284
7.1817	3.0	2460	4.2914	1.0	1.1284
3.6746	4.0	3280	2.9959	1.0	1.1284
2.6344	5.0	4100	2.3796	1.0	1.1284
2.224	6.0	4920	1.1763	1.0	1.0736
0.8482	7.0	5740	0.4680	0.9999	0.9927
0.4357	8.0	6560	0.2710	0.9993	0.9734
0.3568	9.0	7380	0.1256	0.9996	1.0188
0.2374	10.0	8200	0.1476	0.9990	1.0219
0.1941	11.0	9020	0.0597	0.9990	1.0187
0.1829	12.0	9840	0.0470	0.9990	1.0188
0.1701	13.0	10660	0.0510	0.9990	1.0181
0.1603	14.0	11480	0.0375	0.9988	1.0193
0.1564	15.0	12300	0.0865	0.9991	1.0064
0.1555	16.0	13120	0.0240	0.9990	1.0172
0.1446	17.0	13940	0.0388	0.9990	1.0184
0.1374	18.0	14760	0.0724	0.9991	1.0229
0.1358	19.0	15580	0.0293	0.9988	1.0168
0.126	20.0	16400	0.0250	0.9990	1.0173
0.1238	21.0	17220	0.0337	0.9991	1.0174
0.1106	22.0	18040	0.0181	0.9988	1.0159
0.1124	23.0	18860	0.0199	0.9990	1.0165
0.104	24.0	19680	0.0151	0.9990	1.0151
0.1039	25.0	20500	0.0136	0.9990	1.0149
0.094	26.0	21320	0.0132	0.9988	1.0148
0.0921	27.0	22140	0.0171	0.9990	1.0158
0.0832	28.0	22960	0.0122	0.9990	1.0146
0.08	29.0	23780	0.0147	0.9988	1.0151
0.0797	30.0	24600	0.0122	0.9990	1.0147
0.0775	31.0	25420	0.0131	0.9990	1.0151
0.0675	32.0	26240	0.0131	0.9991	1.0148
0.0676	33.0	27060	0.0171	0.9988	1.0146
0.069	34.0	27880	0.0147	0.9991	1.0146
0.0588	35.0	28700	0.0140	0.9990	1.0146
0.0608	36.0	29520	0.0152	0.9988	1.0146
0.0598	37.0	30340	0.0158	0.9990	1.0146
0.0578	38.0	31160	0.0150	0.9990	1.0147
0.0563	39.0	31980	0.0154	0.9990	1.0147
0.0591	39.9518	32760	0.0146	0.9990	1.0147

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0