ABL_trad_i

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.0155
• Accuracy: 0.6617
• F1: 0.6598

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 32

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.9418	1.0	1500	0.9131	0.56	0.5590
0.8318	2.0	3000	0.8578	0.5992	0.5948
0.7796	3.0	4500	0.8415	0.6075	0.6057
0.7139	4.0	6000	0.8327	0.6342	0.6325
0.6883	5.0	7500	0.8430	0.6333	0.6299
0.6643	6.0	9000	0.8444	0.635	0.6323
0.6042	7.0	10500	0.8497	0.6483	0.6459
0.5765	8.0	12000	0.8747	0.6425	0.6407
0.5301	9.0	13500	0.8833	0.6575	0.6559
0.5077	10.0	15000	0.9306	0.6575	0.6554
0.492	11.0	16500	0.9494	0.6658	0.6635
0.4244	12.0	18000	1.0017	0.6642	0.6622
0.3911	13.0	19500	1.0697	0.6692	0.6673
0.3965	14.0	21000	1.0836	0.6692	0.6678
0.3384	15.0	22500	1.1778	0.67	0.6682
0.3142	16.0	24000	1.2995	0.6658	0.6630
0.2783	17.0	25500	1.3573	0.6667	0.6643
0.2599	18.0	27000	1.4730	0.6683	0.6672
0.2553	19.0	28500	1.5837	0.6667	0.6639
0.2359	20.0	30000	1.7285	0.655	0.6525
0.2237	21.0	31500	1.8383	0.6633	0.6622
0.1855	22.0	33000	1.9797	0.6625	0.6610
0.2178	23.0	34500	2.0590	0.6658	0.6637
0.1607	24.0	36000	2.1819	0.6608	0.6583
0.1495	25.0	37500	2.3356	0.6583	0.6564
0.1384	26.0	39000	2.4443	0.6617	0.6603
0.1638	27.0	40500	2.5224	0.6608	0.6585
0.121	28.0	42000	2.6157	0.6692	0.6671
0.1288	29.0	43500	2.7674	0.6692	0.6671
0.0821	30.0	45000	2.8365	0.6658	0.6651
0.0907	31.0	46500	2.9559	0.6542	0.6512
0.0821	32.0	48000	3.0155	0.6617	0.6598

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1