--- library_name: transformers base_model: microsoft/codebert-base tags: - generated_from_trainer model-index: - name: codebert-fine-tuned results: [] --- # codebert-fine-tuned This model is a fine-tuned version of [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) on the None dataset. It achieves the following results on the evaluation set: - Loss: 1.0908 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - num_epochs: 3 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:-----:|:---------------:| | 3.5941 | 0.0325 | 500 | 2.0780 | | 2.091 | 0.0651 | 1000 | 1.8173 | | 1.9005 | 0.0976 | 1500 | 1.6783 | | 1.7817 | 0.1301 | 2000 | 1.6071 | | 1.712 | 0.1626 | 2500 | 1.5634 | | 1.6661 | 0.1952 | 3000 | 1.5229 | | 1.6348 | 0.2277 | 3500 | 1.4965 | | 1.6106 | 0.2602 | 4000 | 1.4514 | | 1.5685 | 0.2928 | 4500 | 1.4360 | | 1.5419 | 0.3253 | 5000 | 1.4203 | | 1.5429 | 0.3578 | 5500 | 1.4026 | | 1.5069 | 0.3903 | 6000 | 1.3959 | | 1.5021 | 0.4229 | 6500 | 1.3819 | | 1.4651 | 0.4554 | 7000 | 1.3660 | | 1.4704 | 0.4879 | 7500 | 1.3544 | | 1.4799 | 0.5205 | 8000 | 1.3428 | | 1.44 | 0.5530 | 8500 | 1.3357 | | 1.4433 | 0.5855 | 9000 | 1.3224 | | 1.4297 | 0.6180 | 9500 | 1.3173 | | 1.4115 | 0.6506 | 10000 | 1.3069 | | 1.4119 | 0.6831 | 10500 | 1.2996 | | 1.3908 | 0.7156 | 11000 | 1.2972 | | 1.4022 | 0.7482 | 11500 | 1.2879 | | 1.381 | 0.7807 | 12000 | 1.2843 | | 1.374 | 0.8132 | 12500 | 1.2747 | | 1.382 | 0.8457 | 13000 | 1.2734 | | 1.3746 | 0.8783 | 13500 | 1.2576 | | 1.3724 | 0.9108 | 14000 | 1.2605 | | 1.3404 | 0.9433 | 14500 | 1.2560 | | 1.3452 | 0.9759 | 15000 | 1.2414 | | 1.3433 | 1.0084 | 15500 | 1.2373 | | 1.3273 | 1.0409 | 16000 | 1.2398 | | 1.3175 | 1.0735 | 16500 | 1.2311 | | 1.3123 | 1.1060 | 17000 | 1.2217 | | 1.3095 | 1.1385 | 17500 | 1.2213 | | 1.3229 | 1.1710 | 18000 | 1.2167 | | 1.2995 | 1.2036 | 18500 | 1.2185 | | 1.3019 | 1.2361 | 19000 | 1.2144 | | 1.299 | 1.2686 | 19500 | 1.2093 | | 1.2784 | 1.3012 | 20000 | 1.1990 | | 1.2886 | 1.3337 | 20500 | 1.2032 | | 1.2788 | 1.3662 | 21000 | 1.1943 | | 1.284 | 1.3987 | 21500 | 1.1975 | | 1.2706 | 1.4313 | 22000 | 1.1878 | | 1.2771 | 1.4638 | 22500 | 1.1856 | | 1.2731 | 1.4963 | 23000 | 1.1797 | | 1.2607 | 1.5289 | 23500 | 1.1919 | | 1.2729 | 1.5614 | 24000 | 1.1872 | | 1.272 | 1.5939 | 24500 | 1.1712 | | 1.251 | 1.6264 | 25000 | 1.1656 | | 1.2437 | 1.6590 | 25500 | 1.1665 | | 1.2523 | 1.6915 | 26000 | 1.1697 | | 1.2393 | 1.7240 | 26500 | 1.1546 | | 1.2521 | 1.7566 | 27000 | 1.1595 | | 1.2498 | 1.7891 | 27500 | 1.1541 | | 1.2187 | 1.8216 | 28000 | 1.1586 | | 1.2311 | 1.8541 | 28500 | 1.1530 | | 1.2419 | 1.8867 | 29000 | 1.1412 | | 1.2246 | 1.9192 | 29500 | 1.1460 | | 1.2381 | 1.9517 | 30000 | 1.1475 | | 1.2237 | 1.9843 | 30500 | 1.1432 | | 1.2273 | 2.0168 | 31000 | 1.1458 | | 1.2167 | 2.0493 | 31500 | 1.1368 | | 1.2039 | 2.0818 | 32000 | 1.1358 | | 1.2142 | 2.1144 | 32500 | 1.1410 | | 1.2003 | 2.1469 | 33000 | 1.1278 | | 1.2052 | 2.1794 | 33500 | 1.1344 | | 1.2094 | 2.2120 | 34000 | 1.1378 | | 1.2128 | 2.2445 | 34500 | 1.1291 | | 1.1936 | 2.2770 | 35000 | 1.1280 | | 1.195 | 2.3095 | 35500 | 1.1278 | | 1.207 | 2.3421 | 36000 | 1.1220 | | 1.1969 | 2.3746 | 36500 | 1.1248 | | 1.188 | 2.4071 | 37000 | 1.1159 | | 1.1921 | 2.4397 | 37500 | 1.1187 | | 1.1916 | 2.4722 | 38000 | 1.1196 | | 1.1797 | 2.5047 | 38500 | 1.1167 | | 1.1865 | 2.5372 | 39000 | 1.1135 | | 1.1787 | 2.5698 | 39500 | 1.1154 | | 1.1865 | 2.6023 | 40000 | 1.1174 | | 1.1754 | 2.6348 | 40500 | 1.1161 | | 1.1805 | 2.6674 | 41000 | 1.1085 | | 1.1786 | 2.6999 | 41500 | 1.1116 | | 1.1689 | 2.7324 | 42000 | 1.1069 | | 1.1755 | 2.7649 | 42500 | 1.1032 | | 1.1858 | 2.7975 | 43000 | 1.1027 | | 1.1722 | 2.8300 | 43500 | 1.1027 | | 1.1686 | 2.8625 | 44000 | 1.1002 | | 1.1801 | 2.8951 | 44500 | 1.1016 | | 1.1596 | 2.9276 | 45000 | 1.1024 | | 1.1788 | 2.9601 | 45500 | 1.1052 | | 1.1609 | 2.9926 | 46000 | 1.0908 | ### Framework versions - Transformers 4.46.3 - Pytorch 2.4.1+cu121 - Datasets 3.1.0 - Tokenizers 0.20.3