
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# SL-CvT

This model is a fine-tuned version of [microsoft/cvt-13](https://huggingface.co/microsoft/cvt-13) on the imagefolder dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3430
- F1: 0.9298
- Roc Auc: 0.9777
- Accuracy: 0.9317

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1     | Roc Auc | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:------:|:-------:|:--------:|
| 1.2379        | 1.0   | 60   | 1.0716          | 0.6422 | 0.7323  | 0.7246   |
| 1.0186        | 2.0   | 120  | 0.8477          | 0.6425 | 0.7879  | 0.7293   |
| 0.9433        | 3.0   | 180  | 0.7473          | 0.7060 | 0.8454  | 0.7538   |
| 0.8644        | 4.0   | 240  | 0.6831          | 0.7188 | 0.8696  | 0.7663   |
| 0.7985        | 5.0   | 300  | 0.6420          | 0.7409 | 0.8943  | 0.7799   |
| 0.7322        | 6.0   | 360  | 0.5713          | 0.7886 | 0.9196  | 0.8101   |
| 0.725         | 7.0   | 420  | 0.5311          | 0.7989 | 0.9324  | 0.8190   |
| 0.6529        | 8.0   | 480  | 0.5246          | 0.7852 | 0.9404  | 0.8117   |
| 0.6224        | 9.0   | 540  | 0.4598          | 0.8282 | 0.9517  | 0.8440   |
| 0.6315        | 10.0  | 600  | 0.4363          | 0.8457 | 0.9585  | 0.8529   |
| 0.5651        | 11.0  | 660  | 0.4437          | 0.8323 | 0.9564  | 0.8503   |
| 0.574         | 12.0  | 720  | 0.4003          | 0.8531 | 0.9617  | 0.8638   |
| 0.5269        | 13.0  | 780  | 0.3901          | 0.8676 | 0.9671  | 0.8722   |
| 0.5138        | 14.0  | 840  | 0.3984          | 0.8607 | 0.9685  | 0.8732   |
| 0.4839        | 15.0  | 900  | 0.3763          | 0.8683 | 0.9701  | 0.8769   |
| 0.463         | 16.0  | 960  | 0.3398          | 0.8837 | 0.9718  | 0.8894   |
| 0.4767        | 17.0  | 1020 | 0.3293          | 0.8846 | 0.9738  | 0.8915   |
| 0.4985        | 18.0  | 1080 | 0.3350          | 0.8852 | 0.9763  | 0.8863   |
| 0.4657        | 19.0  | 1140 | 0.3369          | 0.8872 | 0.9746  | 0.8951   |
| 0.4514        | 20.0  | 1200 | 0.3213          | 0.8880 | 0.9750  | 0.8925   |
| 0.4207        | 21.0  | 1260 | 0.3175          | 0.8943 | 0.9771  | 0.8978   |
| 0.4522        | 22.0  | 1320 | 0.3229          | 0.8970 | 0.9767  | 0.8983   |
| 0.4328        | 23.0  | 1380 | 0.3121          | 0.8948 | 0.9791  | 0.8978   |
| 0.3942        | 24.0  | 1440 | 0.3111          | 0.8993 | 0.9765  | 0.9030   |
| 0.4414        | 25.0  | 1500 | 0.3062          | 0.9032 | 0.9763  | 0.9061   |
| 0.3608        | 26.0  | 1560 | 0.3099          | 0.8997 | 0.9787  | 0.9014   |
| 0.3729        | 27.0  | 1620 | 0.3050          | 0.9029 | 0.9783  | 0.9082   |
| 0.393         | 28.0  | 1680 | 0.2970          | 0.9090 | 0.9797  | 0.9108   |
| 0.402         | 29.0  | 1740 | 0.2986          | 0.9087 | 0.9793  | 0.9113   |
| 0.3697        | 30.0  | 1800 | 0.3384          | 0.8968 | 0.9769  | 0.9025   |
| 0.3502        | 31.0  | 1860 | 0.3035          | 0.9058 | 0.9789  | 0.9103   |
| 0.3653        | 32.0  | 1920 | 0.3127          | 0.9024 | 0.9788  | 0.9025   |
| 0.3898        | 33.0  | 1980 | 0.3222          | 0.9050 | 0.9778  | 0.9061   |
| 0.317         | 34.0  | 2040 | 0.3013          | 0.9124 | 0.9798  | 0.9139   |
| 0.3166        | 35.0  | 2100 | 0.3185          | 0.9095 | 0.9775  | 0.9134   |
| 0.3771        | 36.0  | 2160 | 0.3067          | 0.9049 | 0.9782  | 0.9066   |
| 0.3487        | 37.0  | 2220 | 0.2948          | 0.9118 | 0.9801  | 0.9134   |
| 0.3202        | 38.0  | 2280 | 0.2916          | 0.9168 | 0.9788  | 0.9186   |
| 0.3163        | 39.0  | 2340 | 0.3149          | 0.9141 | 0.9777  | 0.9155   |
| 0.3605        | 40.0  | 2400 | 0.2964          | 0.9192 | 0.9797  | 0.9207   |
| 0.3636        | 41.0  | 2460 | 0.3142          | 0.9111 | 0.9810  | 0.9134   |
| 0.3454        | 42.0  | 2520 | 0.3133          | 0.9111 | 0.9792  | 0.9113   |
| 0.3561        | 43.0  | 2580 | 0.3090          | 0.9073 | 0.9804  | 0.9077   |
| 0.3136        | 44.0  | 2640 | 0.3236          | 0.9144 | 0.9782  | 0.9176   |
| 0.3529        | 45.0  | 2700 | 0.3054          | 0.9175 | 0.9800  | 0.9202   |
| 0.2987        | 46.0  | 2760 | 0.2944          | 0.9222 | 0.9802  | 0.9233   |
| 0.2966        | 47.0  | 2820 | 0.3215          | 0.9201 | 0.9786  | 0.9233   |
| 0.3203        | 48.0  | 2880 | 0.3150          | 0.9219 | 0.9797  | 0.9244   |
| 0.2821        | 49.0  | 2940 | 0.3072          | 0.9273 | 0.9800  | 0.9291   |
| 0.2852        | 50.0  | 3000 | 0.3265          | 0.9155 | 0.9792  | 0.9176   |
| 0.3544        | 51.0  | 3060 | 0.3175          | 0.9150 | 0.9802  | 0.9150   |
| 0.3327        | 52.0  | 3120 | 0.3134          | 0.9222 | 0.9802  | 0.9244   |
| 0.2877        | 53.0  | 3180 | 0.3222          | 0.9154 | 0.9805  | 0.9165   |
| 0.3089        | 54.0  | 3240 | 0.3045          | 0.9248 | 0.9811  | 0.9259   |
| 0.2904        | 55.0  | 3300 | 0.3301          | 0.9175 | 0.9787  | 0.9186   |
| 0.2821        | 56.0  | 3360 | 0.3069          | 0.9206 | 0.9810  | 0.9218   |
| 0.321         | 57.0  | 3420 | 0.3209          | 0.9254 | 0.9800  | 0.9270   |
| 0.2995        | 58.0  | 3480 | 0.3281          | 0.9202 | 0.9802  | 0.9233   |
| 0.2683        | 59.0  | 3540 | 0.3263          | 0.9174 | 0.9802  | 0.9202   |
| 0.3021        | 60.0  | 3600 | 0.3484          | 0.9170 | 0.9788  | 0.9186   |
| 0.3262        | 61.0  | 3660 | 0.3270          | 0.9151 | 0.9807  | 0.9165   |
| 0.2329        | 62.0  | 3720 | 0.3280          | 0.9211 | 0.9807  | 0.9233   |
| 0.2935        | 63.0  | 3780 | 0.3296          | 0.9244 | 0.9807  | 0.9264   |
| 0.2856        | 64.0  | 3840 | 0.3323          | 0.9209 | 0.9811  | 0.9218   |
| 0.2829        | 65.0  | 3900 | 0.3390          | 0.9200 | 0.9802  | 0.9218   |
| 0.3044        | 66.0  | 3960 | 0.3324          | 0.9215 | 0.9799  | 0.9228   |
| 0.2767        | 67.0  | 4020 | 0.3496          | 0.9150 | 0.9778  | 0.9160   |
| 0.2936        | 68.0  | 4080 | 0.3378          | 0.9257 | 0.9790  | 0.9275   |
| 0.2884        | 69.0  | 4140 | 0.3493          | 0.9227 | 0.9790  | 0.9249   |
| 0.2906        | 70.0  | 4200 | 0.3408          | 0.9259 | 0.9794  | 0.9275   |
| 0.2542        | 71.0  | 4260 | 0.3559          | 0.9233 | 0.9769  | 0.9249   |
| 0.2557        | 72.0  | 4320 | 0.3481          | 0.9237 | 0.9779  | 0.9254   |
| 0.2266        | 73.0  | 4380 | 0.3518          | 0.9208 | 0.9781  | 0.9223   |
| 0.2771        | 74.0  | 4440 | 0.3544          | 0.9231 | 0.9776  | 0.9254   |
| 0.2747        | 75.0  | 4500 | 0.3469          | 0.9270 | 0.9780  | 0.9285   |
| 0.2443        | 76.0  | 4560 | 0.3513          | 0.9216 | 0.9767  | 0.9233   |
| 0.2859        | 77.0  | 4620 | 0.3456          | 0.9234 | 0.9771  | 0.9254   |
| 0.2677        | 78.0  | 4680 | 0.3474          | 0.9239 | 0.9780  | 0.9254   |
| 0.2492        | 79.0  | 4740 | 0.3513          | 0.9235 | 0.9778  | 0.9254   |
| 0.2532        | 80.0  | 4800 | 0.3524          | 0.9210 | 0.9773  | 0.9233   |
| 0.2646        | 81.0  | 4860 | 0.3529          | 0.9240 | 0.9784  | 0.9238   |
| 0.2842        | 82.0  | 4920 | 0.3433          | 0.9260 | 0.9777  | 0.9280   |
| 0.2872        | 83.0  | 4980 | 0.3584          | 0.9272 | 0.9771  | 0.9285   |
| 0.2678        | 84.0  | 5040 | 0.3430          | 0.9298 | 0.9777  | 0.9317   |
| 0.2705        | 85.0  | 5100 | 0.3534          | 0.9268 | 0.9777  | 0.9291   |
| 0.2605        | 86.0  | 5160 | 0.3574          | 0.9272 | 0.9777  | 0.9296   |
| 0.2572        | 87.0  | 5220 | 0.3426          | 0.9273 | 0.9781  | 0.9291   |
| 0.2646        | 88.0  | 5280 | 0.3472          | 0.9234 | 0.9789  | 0.9244   |
| 0.2831        | 89.0  | 5340 | 0.3433          | 0.9272 | 0.9779  | 0.9291   |
| 0.277         | 90.0  | 5400 | 0.3441          | 0.9263 | 0.9789  | 0.9280   |
| 0.2584        | 91.0  | 5460 | 0.3432          | 0.9236 | 0.9788  | 0.9249   |
| 0.2703        | 92.0  | 5520 | 0.3409          | 0.9248 | 0.9789  | 0.9259   |
| 0.2811        | 93.0  | 5580 | 0.3449          | 0.9215 | 0.9795  | 0.9228   |
| 0.2786        | 94.0  | 5640 | 0.3465          | 0.9260 | 0.9789  | 0.9280   |
| 0.267         | 95.0  | 5700 | 0.3472          | 0.9260 | 0.9791  | 0.9275   |
| 0.2695        | 96.0  | 5760 | 0.3500          | 0.9268 | 0.9786  | 0.9285   |
| 0.279         | 97.0  | 5820 | 0.3582          | 0.9249 | 0.9782  | 0.9270   |
| 0.2774        | 98.0  | 5880 | 0.3486          | 0.9251 | 0.9790  | 0.9270   |
| 0.2512        | 99.0  | 5940 | 0.3514          | 0.9287 | 0.9786  | 0.9306   |
| 0.2218        | 100.0 | 6000 | 0.3482          | 0.9269 | 0.9789  | 0.9285   |


### Framework versions

- Transformers 4.29.2
- Pytorch 2.0.0+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3

akiec bcc bkl df mel nv vasc