fineweb-fra_latn-quality-transformer

This model is a fine-tuned version of EuroBERT/EuroBERT-210m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5184
• F1: 0.4882
• Accuracy: 0.6573
• Confusion Matrix: 3 10 7 1 94 12 1 30 20
• High Precision: 0.6
• High Recall: 0.15
• High F1: 0.24
• Low Precision: 0.7015
• Low Recall: 0.8785
• Low F1: 0.7801
• Medium Precision: 0.5128
• Medium Recall: 0.3922
• Medium F1: 0.4444

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Accuracy	Confusion Matrix	High Precision	High Recall	High F1	Low Precision	Low Recall	Low F1	Medium Precision	Medium Recall	Medium F1
No log	1.0	5	1.4081	0.1518	0.2865	0 0 20
5 0 102
0 0 51	0.0	0.0	0.0	0.0	0.0	0.0	0.2948	1.0	0.4554
1.2754	2.0	10	1.0008	0.2503	0.6011	0 20 0
0 107 0
0 51 0	0.0	0.0	0.0	0.6011	1.0	0.7509	0.0	0.0	0.0
1.2754	3.0	15	0.9946	0.3367	0.4775	0 2 18
0 37 70
0 3 48	0.0	0.0	0.0	0.8810	0.3458	0.4966	0.3529	0.9412	0.5134
0.8128	4.0	20	0.7867	0.4046	0.6404	0 11 9
0 93 14
0 30 21	0.0	0.0	0.0	0.6940	0.8692	0.7718	0.4773	0.4118	0.4421
0.8128	5.0	25	0.7778	0.4324	0.6348	0 4 16
2 80 25
1 17 33	0.0	0.0	0.0	0.7921	0.7477	0.7692	0.4459	0.6471	0.528
0.5766	6.0	30	0.9369	0.4169	0.6292	0 5 15
2 85 20
1 23 27	0.0	0.0	0.0	0.7522	0.7944	0.7727	0.4355	0.5294	0.4779
0.5766	7.0	35	0.8983	0.4443	0.6180	1 4 15
1 78 28
2 18 31	0.25	0.05	0.0833	0.78	0.7290	0.7536	0.4189	0.6078	0.496
0.1777	8.0	40	1.5184	0.4882	0.6573	3 10 7
1 94 12
1 30 20	0.6	0.15	0.24	0.7015	0.8785	0.7801	0.5128	0.3922	0.4444
0.1777	9.0	45	1.7748	0.4364	0.5955	2 7 11
2 80 25
7 20 24	0.1818	0.1	0.1290	0.7477	0.7477	0.7477	0.4	0.4706	0.4324
0.013	10.0	50	2.1900	0.4190	0.6236	0 7 13
3 83 21
2 21 28	0.0	0.0	0.0	0.7477	0.7757	0.7615	0.4516	0.5490	0.4956
0.013	11.0	55	2.6390	0.4348	0.6404	0 6 14
1 81 25
1 17 33	0.0	0.0	0.0	0.7788	0.7570	0.7678	0.4583	0.6471	0.5366
0.0041	12.0	60	2.2662	0.4481	0.5955	4 9 7
6 84 17
8 25 18	0.2222	0.2	0.2105	0.7119	0.7850	0.7467	0.4286	0.3529	0.3871
0.0041	13.0	65	3.0654	0.4064	0.5787	0 4 16
0 64 43
3 9 39	0.0	0.0	0.0	0.8312	0.5981	0.6957	0.3980	0.7647	0.5235
0.0058	14.0	70	2.4618	0.4273	0.5899	3 9 8
2 86 19
5 30 16	0.3	0.15	0.2	0.688	0.8037	0.7414	0.3721	0.3137	0.3404
0.0058	15.0	75	2.7654	0.4147	0.5506	1 3 16
5 61 41
8 7 36	0.0714	0.05	0.0588	0.8592	0.5701	0.6854	0.3871	0.7059	0.5
0.0011	16.0	80	3.1337	0.3696	0.6180	0 15 5
1 96 10
2 35 14	0.0	0.0	0.0	0.6575	0.8972	0.7589	0.4828	0.2745	0.35
0.0011	17.0	85	2.7265	0.4388	0.5618	4 3 13
5 74 28
13 16 22	0.1818	0.2	0.1905	0.7957	0.6916	0.74	0.3492	0.4314	0.3860
0.0007	18.0	90	2.9583	0.4270	0.5843	1 3 16
1 71 35
6 13 32	0.125	0.05	0.0714	0.8161	0.6636	0.7320	0.3855	0.6275	0.4776

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0