cwe-parent-vulnerability-classification-huggingface-CodeBERTa-small-v1

This model is a fine-tuned version of huggingface/CodeBERTa-small-v1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3718
• Accuracy: 0.7753
• F1 Macro: 0.3835

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro
3.2521	1.0	25	3.3015	0.0112	0.0057
3.037	2.0	50	3.2608	0.3258	0.1301
3.0191	3.0	75	3.3140	0.3146	0.0568
2.8876	4.0	100	3.2111	0.2472	0.2190
2.7888	5.0	125	3.1305	0.2360	0.2394
2.6762	6.0	150	3.1370	0.3146	0.1503
2.5905	7.0	175	3.0446	0.3933	0.2144
2.4173	8.0	200	2.9769	0.3933	0.1999
2.2706	9.0	225	2.9704	0.3708	0.1825
2.1355	10.0	250	2.9071	0.3820	0.2097
2.0722	11.0	275	2.9189	0.4382	0.1855
1.8111	12.0	300	2.7873	0.4157	0.2257
1.7056	13.0	325	2.7584	0.4607	0.2217
1.6693	14.0	350	2.7394	0.5169	0.2370
1.496	15.0	375	2.6647	0.5506	0.2630
1.3093	16.0	400	2.6477	0.5843	0.2644
1.251	17.0	425	2.6355	0.6067	0.2654
1.1316	18.0	450	2.5521	0.6404	0.3000
1.049	19.0	475	2.5236	0.6404	0.2951
0.9441	20.0	500	2.5538	0.6404	0.2957
0.8752	21.0	525	2.5020	0.6742	0.3259
0.7623	22.0	550	2.4927	0.6966	0.3339
0.7938	23.0	575	2.4595	0.7079	0.3413
0.6321	24.0	600	2.4564	0.7079	0.3369
0.6245	25.0	625	2.4537	0.7191	0.3631
0.5502	26.0	650	2.4470	0.7416	0.3764
0.5805	27.0	675	2.4588	0.7528	0.3849
0.5042	28.0	700	2.4243	0.7640	0.3930
0.464	29.0	725	2.4051	0.7640	0.3930
0.4349	30.0	750	2.4361	0.7753	0.4034
0.4347	31.0	775	2.4062	0.7640	0.3906
0.3953	32.0	800	2.3939	0.7753	0.3835
0.3775	33.0	825	2.4091	0.7865	0.3958
0.3948	34.0	850	2.4018	0.7865	0.3958
0.348	35.0	875	2.3718	0.7753	0.3835
0.3328	36.0	900	2.3843	0.7865	0.3947
0.3291	37.0	925	2.3843	0.7978	0.3965
0.3211	38.0	950	2.3764	0.7865	0.3958
0.307	39.0	975	2.3822	0.7978	0.3975
0.3055	40.0	1000	2.3855	0.7978	0.3965

Framework versions

• Transformers 4.55.4
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.21.2