---
library_name: transformers
base_model: microsoft/codebert-base
tags:
- generated_from_trainer
model-index:
- name: codebert-fine-tuned
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# codebert-fine-tuned

This model is a fine-tuned version of [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0908

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss |
|:-------------:|:------:|:-----:|:---------------:|
| 3.5941        | 0.0325 | 500   | 2.0780          |
| 2.091         | 0.0651 | 1000  | 1.8173          |
| 1.9005        | 0.0976 | 1500  | 1.6783          |
| 1.7817        | 0.1301 | 2000  | 1.6071          |
| 1.712         | 0.1626 | 2500  | 1.5634          |
| 1.6661        | 0.1952 | 3000  | 1.5229          |
| 1.6348        | 0.2277 | 3500  | 1.4965          |
| 1.6106        | 0.2602 | 4000  | 1.4514          |
| 1.5685        | 0.2928 | 4500  | 1.4360          |
| 1.5419        | 0.3253 | 5000  | 1.4203          |
| 1.5429        | 0.3578 | 5500  | 1.4026          |
| 1.5069        | 0.3903 | 6000  | 1.3959          |
| 1.5021        | 0.4229 | 6500  | 1.3819          |
| 1.4651        | 0.4554 | 7000  | 1.3660          |
| 1.4704        | 0.4879 | 7500  | 1.3544          |
| 1.4799        | 0.5205 | 8000  | 1.3428          |
| 1.44          | 0.5530 | 8500  | 1.3357          |
| 1.4433        | 0.5855 | 9000  | 1.3224          |
| 1.4297        | 0.6180 | 9500  | 1.3173          |
| 1.4115        | 0.6506 | 10000 | 1.3069          |
| 1.4119        | 0.6831 | 10500 | 1.2996          |
| 1.3908        | 0.7156 | 11000 | 1.2972          |
| 1.4022        | 0.7482 | 11500 | 1.2879          |
| 1.381         | 0.7807 | 12000 | 1.2843          |
| 1.374         | 0.8132 | 12500 | 1.2747          |
| 1.382         | 0.8457 | 13000 | 1.2734          |
| 1.3746        | 0.8783 | 13500 | 1.2576          |
| 1.3724        | 0.9108 | 14000 | 1.2605          |
| 1.3404        | 0.9433 | 14500 | 1.2560          |
| 1.3452        | 0.9759 | 15000 | 1.2414          |
| 1.3433        | 1.0084 | 15500 | 1.2373          |
| 1.3273        | 1.0409 | 16000 | 1.2398          |
| 1.3175        | 1.0735 | 16500 | 1.2311          |
| 1.3123        | 1.1060 | 17000 | 1.2217          |
| 1.3095        | 1.1385 | 17500 | 1.2213          |
| 1.3229        | 1.1710 | 18000 | 1.2167          |
| 1.2995        | 1.2036 | 18500 | 1.2185          |
| 1.3019        | 1.2361 | 19000 | 1.2144          |
| 1.299         | 1.2686 | 19500 | 1.2093          |
| 1.2784        | 1.3012 | 20000 | 1.1990          |
| 1.2886        | 1.3337 | 20500 | 1.2032          |
| 1.2788        | 1.3662 | 21000 | 1.1943          |
| 1.284         | 1.3987 | 21500 | 1.1975          |
| 1.2706        | 1.4313 | 22000 | 1.1878          |
| 1.2771        | 1.4638 | 22500 | 1.1856          |
| 1.2731        | 1.4963 | 23000 | 1.1797          |
| 1.2607        | 1.5289 | 23500 | 1.1919          |
| 1.2729        | 1.5614 | 24000 | 1.1872          |
| 1.272         | 1.5939 | 24500 | 1.1712          |
| 1.251         | 1.6264 | 25000 | 1.1656          |
| 1.2437        | 1.6590 | 25500 | 1.1665          |
| 1.2523        | 1.6915 | 26000 | 1.1697          |
| 1.2393        | 1.7240 | 26500 | 1.1546          |
| 1.2521        | 1.7566 | 27000 | 1.1595          |
| 1.2498        | 1.7891 | 27500 | 1.1541          |
| 1.2187        | 1.8216 | 28000 | 1.1586          |
| 1.2311        | 1.8541 | 28500 | 1.1530          |
| 1.2419        | 1.8867 | 29000 | 1.1412          |
| 1.2246        | 1.9192 | 29500 | 1.1460          |
| 1.2381        | 1.9517 | 30000 | 1.1475          |
| 1.2237        | 1.9843 | 30500 | 1.1432          |
| 1.2273        | 2.0168 | 31000 | 1.1458          |
| 1.2167        | 2.0493 | 31500 | 1.1368          |
| 1.2039        | 2.0818 | 32000 | 1.1358          |
| 1.2142        | 2.1144 | 32500 | 1.1410          |
| 1.2003        | 2.1469 | 33000 | 1.1278          |
| 1.2052        | 2.1794 | 33500 | 1.1344          |
| 1.2094        | 2.2120 | 34000 | 1.1378          |
| 1.2128        | 2.2445 | 34500 | 1.1291          |
| 1.1936        | 2.2770 | 35000 | 1.1280          |
| 1.195         | 2.3095 | 35500 | 1.1278          |
| 1.207         | 2.3421 | 36000 | 1.1220          |
| 1.1969        | 2.3746 | 36500 | 1.1248          |
| 1.188         | 2.4071 | 37000 | 1.1159          |
| 1.1921        | 2.4397 | 37500 | 1.1187          |
| 1.1916        | 2.4722 | 38000 | 1.1196          |
| 1.1797        | 2.5047 | 38500 | 1.1167          |
| 1.1865        | 2.5372 | 39000 | 1.1135          |
| 1.1787        | 2.5698 | 39500 | 1.1154          |
| 1.1865        | 2.6023 | 40000 | 1.1174          |
| 1.1754        | 2.6348 | 40500 | 1.1161          |
| 1.1805        | 2.6674 | 41000 | 1.1085          |
| 1.1786        | 2.6999 | 41500 | 1.1116          |
| 1.1689        | 2.7324 | 42000 | 1.1069          |
| 1.1755        | 2.7649 | 42500 | 1.1032          |
| 1.1858        | 2.7975 | 43000 | 1.1027          |
| 1.1722        | 2.8300 | 43500 | 1.1027          |
| 1.1686        | 2.8625 | 44000 | 1.1002          |
| 1.1801        | 2.8951 | 44500 | 1.1016          |
| 1.1596        | 2.9276 | 45000 | 1.1024          |
| 1.1788        | 2.9601 | 45500 | 1.1052          |
| 1.1609        | 2.9926 | 46000 | 1.0908          |


### Framework versions

- Transformers 4.46.3
- Pytorch 2.4.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3