Hugging Face's logo

pogtador
/
codebert-fine-tuned

Fill-Mask
Transformers
Safetensors
roberta
Generated from Trainer
Inference Endpoints
Model card Files Community
pogtador commited on
Commit
e565bbb
·
verified ·
1 Parent(s): 7736103

Model save

Browse files
Files changed (1) hide show
  1. README.md +149 -0
README.md ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ base_model: microsoft/codebert-base
4
+ tags:
5
+ - generated_from_trainer
6
+ model-index:
7
+ - name: codebert-fine-tuned
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # codebert-fine-tuned
15
+
16
+ This model is a fine-tuned version of [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) on the None dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 1.0908
19
+
20
+ ## Model description
21
+
22
+ More information needed
23
+
24
+ ## Intended uses & limitations
25
+
26
+ More information needed
27
+
28
+ ## Training and evaluation data
29
+
30
+ More information needed
31
+
32
+ ## Training procedure
33
+
34
+ ### Training hyperparameters
35
+
36
+ The following hyperparameters were used during training:
37
+ - learning_rate: 5e-05
38
+ - train_batch_size: 16
39
+ - eval_batch_size: 16
40
+ - seed: 42
41
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
42
+ - lr_scheduler_type: linear
43
+ - num_epochs: 3
44
+ - mixed_precision_training: Native AMP
45
+
46
+ ### Training results
47
+
48
+ | Training Loss | Epoch | Step | Validation Loss |
49
+ |:-------------:|:------:|:-----:|:---------------:|
50
+ | 3.5941 | 0.0325 | 500 | 2.0780 |
51
+ | 2.091 | 0.0651 | 1000 | 1.8173 |
52
+ | 1.9005 | 0.0976 | 1500 | 1.6783 |
53
+ | 1.7817 | 0.1301 | 2000 | 1.6071 |
54
+ | 1.712 | 0.1626 | 2500 | 1.5634 |
55
+ | 1.6661 | 0.1952 | 3000 | 1.5229 |
56
+ | 1.6348 | 0.2277 | 3500 | 1.4965 |
57
+ | 1.6106 | 0.2602 | 4000 | 1.4514 |
58
+ | 1.5685 | 0.2928 | 4500 | 1.4360 |
59
+ | 1.5419 | 0.3253 | 5000 | 1.4203 |
60
+ | 1.5429 | 0.3578 | 5500 | 1.4026 |
61
+ | 1.5069 | 0.3903 | 6000 | 1.3959 |
62
+ | 1.5021 | 0.4229 | 6500 | 1.3819 |
63
+ | 1.4651 | 0.4554 | 7000 | 1.3660 |
64
+ | 1.4704 | 0.4879 | 7500 | 1.3544 |
65
+ | 1.4799 | 0.5205 | 8000 | 1.3428 |
66
+ | 1.44 | 0.5530 | 8500 | 1.3357 |
67
+ | 1.4433 | 0.5855 | 9000 | 1.3224 |
68
+ | 1.4297 | 0.6180 | 9500 | 1.3173 |
69
+ | 1.4115 | 0.6506 | 10000 | 1.3069 |
70
+ | 1.4119 | 0.6831 | 10500 | 1.2996 |
71
+ | 1.3908 | 0.7156 | 11000 | 1.2972 |
72
+ | 1.4022 | 0.7482 | 11500 | 1.2879 |
73
+ | 1.381 | 0.7807 | 12000 | 1.2843 |
74
+ | 1.374 | 0.8132 | 12500 | 1.2747 |
75
+ | 1.382 | 0.8457 | 13000 | 1.2734 |
76
+ | 1.3746 | 0.8783 | 13500 | 1.2576 |
77
+ | 1.3724 | 0.9108 | 14000 | 1.2605 |
78
+ | 1.3404 | 0.9433 | 14500 | 1.2560 |
79
+ | 1.3452 | 0.9759 | 15000 | 1.2414 |
80
+ | 1.3433 | 1.0084 | 15500 | 1.2373 |
81
+ | 1.3273 | 1.0409 | 16000 | 1.2398 |
82
+ | 1.3175 | 1.0735 | 16500 | 1.2311 |
83
+ | 1.3123 | 1.1060 | 17000 | 1.2217 |
84
+ | 1.3095 | 1.1385 | 17500 | 1.2213 |
85
+ | 1.3229 | 1.1710 | 18000 | 1.2167 |
86
+ | 1.2995 | 1.2036 | 18500 | 1.2185 |
87
+ | 1.3019 | 1.2361 | 19000 | 1.2144 |
88
+ | 1.299 | 1.2686 | 19500 | 1.2093 |
89
+ | 1.2784 | 1.3012 | 20000 | 1.1990 |
90
+ | 1.2886 | 1.3337 | 20500 | 1.2032 |
91
+ | 1.2788 | 1.3662 | 21000 | 1.1943 |
92
+ | 1.284 | 1.3987 | 21500 | 1.1975 |
93
+ | 1.2706 | 1.4313 | 22000 | 1.1878 |
94
+ | 1.2771 | 1.4638 | 22500 | 1.1856 |
95
+ | 1.2731 | 1.4963 | 23000 | 1.1797 |
96
+ | 1.2607 | 1.5289 | 23500 | 1.1919 |
97
+ | 1.2729 | 1.5614 | 24000 | 1.1872 |
98
+ | 1.272 | 1.5939 | 24500 | 1.1712 |
99
+ | 1.251 | 1.6264 | 25000 | 1.1656 |
100
+ | 1.2437 | 1.6590 | 25500 | 1.1665 |
101
+ | 1.2523 | 1.6915 | 26000 | 1.1697 |
102
+ | 1.2393 | 1.7240 | 26500 | 1.1546 |
103
+ | 1.2521 | 1.7566 | 27000 | 1.1595 |
104
+ | 1.2498 | 1.7891 | 27500 | 1.1541 |
105
+ | 1.2187 | 1.8216 | 28000 | 1.1586 |
106
+ | 1.2311 | 1.8541 | 28500 | 1.1530 |
107
+ | 1.2419 | 1.8867 | 29000 | 1.1412 |
108
+ | 1.2246 | 1.9192 | 29500 | 1.1460 |
109
+ | 1.2381 | 1.9517 | 30000 | 1.1475 |
110
+ | 1.2237 | 1.9843 | 30500 | 1.1432 |
111
+ | 1.2273 | 2.0168 | 31000 | 1.1458 |
112
+ | 1.2167 | 2.0493 | 31500 | 1.1368 |
113
+ | 1.2039 | 2.0818 | 32000 | 1.1358 |
114
+ | 1.2142 | 2.1144 | 32500 | 1.1410 |
115
+ | 1.2003 | 2.1469 | 33000 | 1.1278 |
116
+ | 1.2052 | 2.1794 | 33500 | 1.1344 |
117
+ | 1.2094 | 2.2120 | 34000 | 1.1378 |
118
+ | 1.2128 | 2.2445 | 34500 | 1.1291 |
119
+ | 1.1936 | 2.2770 | 35000 | 1.1280 |
120
+ | 1.195 | 2.3095 | 35500 | 1.1278 |
121
+ | 1.207 | 2.3421 | 36000 | 1.1220 |
122
+ | 1.1969 | 2.3746 | 36500 | 1.1248 |
123
+ | 1.188 | 2.4071 | 37000 | 1.1159 |
124
+ | 1.1921 | 2.4397 | 37500 | 1.1187 |
125
+ | 1.1916 | 2.4722 | 38000 | 1.1196 |
126
+ | 1.1797 | 2.5047 | 38500 | 1.1167 |
127
+ | 1.1865 | 2.5372 | 39000 | 1.1135 |
128
+ | 1.1787 | 2.5698 | 39500 | 1.1154 |
129
+ | 1.1865 | 2.6023 | 40000 | 1.1174 |
130
+ | 1.1754 | 2.6348 | 40500 | 1.1161 |
131
+ | 1.1805 | 2.6674 | 41000 | 1.1085 |
132
+ | 1.1786 | 2.6999 | 41500 | 1.1116 |
133
+ | 1.1689 | 2.7324 | 42000 | 1.1069 |
134
+ | 1.1755 | 2.7649 | 42500 | 1.1032 |
135
+ | 1.1858 | 2.7975 | 43000 | 1.1027 |
136
+ | 1.1722 | 2.8300 | 43500 | 1.1027 |
137
+ | 1.1686 | 2.8625 | 44000 | 1.1002 |
138
+ | 1.1801 | 2.8951 | 44500 | 1.1016 |
139
+ | 1.1596 | 2.9276 | 45000 | 1.1024 |
140
+ | 1.1788 | 2.9601 | 45500 | 1.1052 |
141
+ | 1.1609 | 2.9926 | 46000 | 1.0908 |
142
+
143
+
144
+ ### Framework versions
145
+
146
+ - Transformers 4.46.3
147
+ - Pytorch 2.4.1+cu121
148
+ - Datasets 3.1.0
149
+ - Tokenizers 0.20.3