|
--- |
|
library_name: transformers |
|
base_model: microsoft/codebert-base |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: codebert-fine-tuned |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# codebert-fine-tuned |
|
|
|
This model is a fine-tuned version of [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) on the None dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.0908 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
- lr_scheduler_type: linear |
|
- num_epochs: 3 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:------:|:-----:|:---------------:| |
|
| 3.5941 | 0.0325 | 500 | 2.0780 | |
|
| 2.091 | 0.0651 | 1000 | 1.8173 | |
|
| 1.9005 | 0.0976 | 1500 | 1.6783 | |
|
| 1.7817 | 0.1301 | 2000 | 1.6071 | |
|
| 1.712 | 0.1626 | 2500 | 1.5634 | |
|
| 1.6661 | 0.1952 | 3000 | 1.5229 | |
|
| 1.6348 | 0.2277 | 3500 | 1.4965 | |
|
| 1.6106 | 0.2602 | 4000 | 1.4514 | |
|
| 1.5685 | 0.2928 | 4500 | 1.4360 | |
|
| 1.5419 | 0.3253 | 5000 | 1.4203 | |
|
| 1.5429 | 0.3578 | 5500 | 1.4026 | |
|
| 1.5069 | 0.3903 | 6000 | 1.3959 | |
|
| 1.5021 | 0.4229 | 6500 | 1.3819 | |
|
| 1.4651 | 0.4554 | 7000 | 1.3660 | |
|
| 1.4704 | 0.4879 | 7500 | 1.3544 | |
|
| 1.4799 | 0.5205 | 8000 | 1.3428 | |
|
| 1.44 | 0.5530 | 8500 | 1.3357 | |
|
| 1.4433 | 0.5855 | 9000 | 1.3224 | |
|
| 1.4297 | 0.6180 | 9500 | 1.3173 | |
|
| 1.4115 | 0.6506 | 10000 | 1.3069 | |
|
| 1.4119 | 0.6831 | 10500 | 1.2996 | |
|
| 1.3908 | 0.7156 | 11000 | 1.2972 | |
|
| 1.4022 | 0.7482 | 11500 | 1.2879 | |
|
| 1.381 | 0.7807 | 12000 | 1.2843 | |
|
| 1.374 | 0.8132 | 12500 | 1.2747 | |
|
| 1.382 | 0.8457 | 13000 | 1.2734 | |
|
| 1.3746 | 0.8783 | 13500 | 1.2576 | |
|
| 1.3724 | 0.9108 | 14000 | 1.2605 | |
|
| 1.3404 | 0.9433 | 14500 | 1.2560 | |
|
| 1.3452 | 0.9759 | 15000 | 1.2414 | |
|
| 1.3433 | 1.0084 | 15500 | 1.2373 | |
|
| 1.3273 | 1.0409 | 16000 | 1.2398 | |
|
| 1.3175 | 1.0735 | 16500 | 1.2311 | |
|
| 1.3123 | 1.1060 | 17000 | 1.2217 | |
|
| 1.3095 | 1.1385 | 17500 | 1.2213 | |
|
| 1.3229 | 1.1710 | 18000 | 1.2167 | |
|
| 1.2995 | 1.2036 | 18500 | 1.2185 | |
|
| 1.3019 | 1.2361 | 19000 | 1.2144 | |
|
| 1.299 | 1.2686 | 19500 | 1.2093 | |
|
| 1.2784 | 1.3012 | 20000 | 1.1990 | |
|
| 1.2886 | 1.3337 | 20500 | 1.2032 | |
|
| 1.2788 | 1.3662 | 21000 | 1.1943 | |
|
| 1.284 | 1.3987 | 21500 | 1.1975 | |
|
| 1.2706 | 1.4313 | 22000 | 1.1878 | |
|
| 1.2771 | 1.4638 | 22500 | 1.1856 | |
|
| 1.2731 | 1.4963 | 23000 | 1.1797 | |
|
| 1.2607 | 1.5289 | 23500 | 1.1919 | |
|
| 1.2729 | 1.5614 | 24000 | 1.1872 | |
|
| 1.272 | 1.5939 | 24500 | 1.1712 | |
|
| 1.251 | 1.6264 | 25000 | 1.1656 | |
|
| 1.2437 | 1.6590 | 25500 | 1.1665 | |
|
| 1.2523 | 1.6915 | 26000 | 1.1697 | |
|
| 1.2393 | 1.7240 | 26500 | 1.1546 | |
|
| 1.2521 | 1.7566 | 27000 | 1.1595 | |
|
| 1.2498 | 1.7891 | 27500 | 1.1541 | |
|
| 1.2187 | 1.8216 | 28000 | 1.1586 | |
|
| 1.2311 | 1.8541 | 28500 | 1.1530 | |
|
| 1.2419 | 1.8867 | 29000 | 1.1412 | |
|
| 1.2246 | 1.9192 | 29500 | 1.1460 | |
|
| 1.2381 | 1.9517 | 30000 | 1.1475 | |
|
| 1.2237 | 1.9843 | 30500 | 1.1432 | |
|
| 1.2273 | 2.0168 | 31000 | 1.1458 | |
|
| 1.2167 | 2.0493 | 31500 | 1.1368 | |
|
| 1.2039 | 2.0818 | 32000 | 1.1358 | |
|
| 1.2142 | 2.1144 | 32500 | 1.1410 | |
|
| 1.2003 | 2.1469 | 33000 | 1.1278 | |
|
| 1.2052 | 2.1794 | 33500 | 1.1344 | |
|
| 1.2094 | 2.2120 | 34000 | 1.1378 | |
|
| 1.2128 | 2.2445 | 34500 | 1.1291 | |
|
| 1.1936 | 2.2770 | 35000 | 1.1280 | |
|
| 1.195 | 2.3095 | 35500 | 1.1278 | |
|
| 1.207 | 2.3421 | 36000 | 1.1220 | |
|
| 1.1969 | 2.3746 | 36500 | 1.1248 | |
|
| 1.188 | 2.4071 | 37000 | 1.1159 | |
|
| 1.1921 | 2.4397 | 37500 | 1.1187 | |
|
| 1.1916 | 2.4722 | 38000 | 1.1196 | |
|
| 1.1797 | 2.5047 | 38500 | 1.1167 | |
|
| 1.1865 | 2.5372 | 39000 | 1.1135 | |
|
| 1.1787 | 2.5698 | 39500 | 1.1154 | |
|
| 1.1865 | 2.6023 | 40000 | 1.1174 | |
|
| 1.1754 | 2.6348 | 40500 | 1.1161 | |
|
| 1.1805 | 2.6674 | 41000 | 1.1085 | |
|
| 1.1786 | 2.6999 | 41500 | 1.1116 | |
|
| 1.1689 | 2.7324 | 42000 | 1.1069 | |
|
| 1.1755 | 2.7649 | 42500 | 1.1032 | |
|
| 1.1858 | 2.7975 | 43000 | 1.1027 | |
|
| 1.1722 | 2.8300 | 43500 | 1.1027 | |
|
| 1.1686 | 2.8625 | 44000 | 1.1002 | |
|
| 1.1801 | 2.8951 | 44500 | 1.1016 | |
|
| 1.1596 | 2.9276 | 45000 | 1.1024 | |
|
| 1.1788 | 2.9601 | 45500 | 1.1052 | |
|
| 1.1609 | 2.9926 | 46000 | 1.0908 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.46.3 |
|
- Pytorch 2.4.1+cu121 |
|
- Datasets 3.1.0 |
|
- Tokenizers 0.20.3 |
|
|
