
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 6_e_200-tiny_tobacco3482_kd_CEKD_t5.0_a0.7

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6810
- Accuracy: 0.795
- Brier Loss: 0.3284
- Nll: 1.4069
- F1 Micro: 0.795
- F1 Macro: 0.7636
- Ece: 0.2215
- Aurc: 0.0726

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 3.5123          | 0.19     | 1.2292     | 9.8836 | 0.19     | 0.0557   | 0.5319 | 0.8455 |
| No log        | 2.0   | 50   | 3.3061          | 0.19     | 1.1746     | 9.5259 | 0.19     | 0.0557   | 0.4948 | 0.8502 |
| No log        | 3.0   | 75   | 3.0163          | 0.18     | 1.1038     | 9.0069 | 0.18     | 0.0530   | 0.3959 | 0.8542 |
| No log        | 4.0   | 100  | 2.7051          | 0.16     | 1.0425     | 8.3332 | 0.16     | 0.0685   | 0.3637 | 0.8603 |
| No log        | 5.0   | 125  | 2.4101          | 0.145    | 0.9955     | 7.8875 | 0.145    | 0.0886   | 0.3276 | 0.8594 |
| No log        | 6.0   | 150  | 2.1622          | 0.16     | 0.9581     | 7.4210 | 0.16     | 0.1094   | 0.3098 | 0.8418 |
| No log        | 7.0   | 175  | 1.9665          | 0.17     | 0.9199     | 6.3502 | 0.17     | 0.1141   | 0.2939 | 0.8129 |
| No log        | 8.0   | 200  | 1.8218          | 0.19     | 0.8852     | 4.6467 | 0.19     | 0.1580   | 0.2691 | 0.7908 |
| No log        | 9.0   | 225  | 1.7041          | 0.205    | 0.8503     | 3.9915 | 0.205    | 0.1722   | 0.2746 | 0.7349 |
| No log        | 10.0  | 250  | 1.6030          | 0.315    | 0.8144     | 3.8156 | 0.315    | 0.2716   | 0.2999 | 0.6443 |
| No log        | 11.0  | 275  | 1.5078          | 0.39     | 0.7749     | 3.6634 | 0.39     | 0.3103   | 0.2992 | 0.5291 |
| No log        | 12.0  | 300  | 1.4231          | 0.46     | 0.7351     | 3.5329 | 0.46     | 0.3550   | 0.3143 | 0.4339 |
| No log        | 13.0  | 325  | 1.3487          | 0.465    | 0.6988     | 3.4534 | 0.465    | 0.3851   | 0.2929 | 0.3704 |
| No log        | 14.0  | 350  | 1.2823          | 0.51     | 0.6657     | 3.2203 | 0.51     | 0.4164   | 0.2835 | 0.3215 |
| No log        | 15.0  | 375  | 1.2282          | 0.54     | 0.6399     | 3.0431 | 0.54     | 0.4415   | 0.2943 | 0.2928 |
| No log        | 16.0  | 400  | 1.1793          | 0.555    | 0.6170     | 2.8562 | 0.555    | 0.4502   | 0.2813 | 0.2676 |
| No log        | 17.0  | 425  | 1.1466          | 0.565    | 0.6019     | 2.7630 | 0.565    | 0.4607   | 0.2450 | 0.2524 |
| No log        | 18.0  | 450  | 1.1114          | 0.59     | 0.5832     | 2.7390 | 0.59     | 0.4721   | 0.2850 | 0.2289 |
| No log        | 19.0  | 475  | 1.0835          | 0.6      | 0.5699     | 2.5928 | 0.6      | 0.5073   | 0.2760 | 0.2195 |
| 1.9045        | 20.0  | 500  | 1.0547          | 0.615    | 0.5539     | 2.6261 | 0.615    | 0.5273   | 0.2883 | 0.2044 |
| 1.9045        | 21.0  | 525  | 1.0294          | 0.625    | 0.5404     | 2.6118 | 0.625    | 0.5343   | 0.2703 | 0.1945 |
| 1.9045        | 22.0  | 550  | 1.0085          | 0.635    | 0.5300     | 2.4378 | 0.635    | 0.5381   | 0.2727 | 0.1842 |
| 1.9045        | 23.0  | 575  | 0.9900          | 0.64     | 0.5188     | 2.4290 | 0.64     | 0.5435   | 0.2781 | 0.1745 |
| 1.9045        | 24.0  | 600  | 0.9674          | 0.645    | 0.5071     | 2.3513 | 0.645    | 0.5527   | 0.2631 | 0.1647 |
| 1.9045        | 25.0  | 625  | 0.9522          | 0.64     | 0.4980     | 2.2815 | 0.64     | 0.5494   | 0.2687 | 0.1606 |
| 1.9045        | 26.0  | 650  | 0.9336          | 0.65     | 0.4883     | 2.2513 | 0.65     | 0.5603   | 0.2727 | 0.1515 |
| 1.9045        | 27.0  | 675  | 0.9175          | 0.665    | 0.4795     | 2.2466 | 0.665    | 0.5707   | 0.2848 | 0.1450 |
| 1.9045        | 28.0  | 700  | 0.9060          | 0.655    | 0.4731     | 2.2223 | 0.655    | 0.5655   | 0.2598 | 0.1426 |
| 1.9045        | 29.0  | 725  | 0.8924          | 0.67     | 0.4648     | 2.1571 | 0.67     | 0.5748   | 0.2504 | 0.1364 |
| 1.9045        | 30.0  | 750  | 0.8808          | 0.675    | 0.4580     | 2.1970 | 0.675    | 0.5804   | 0.2124 | 0.1302 |
| 1.9045        | 31.0  | 775  | 0.8698          | 0.675    | 0.4513     | 1.9818 | 0.675    | 0.5784   | 0.2413 | 0.1248 |
| 1.9045        | 32.0  | 800  | 0.8581          | 0.685    | 0.4451     | 2.0653 | 0.685    | 0.6062   | 0.2783 | 0.1221 |
| 1.9045        | 33.0  | 825  | 0.8493          | 0.68     | 0.4398     | 1.9229 | 0.68     | 0.5964   | 0.2430 | 0.1198 |
| 1.9045        | 34.0  | 850  | 0.8416          | 0.675    | 0.4351     | 1.9147 | 0.675    | 0.5901   | 0.2547 | 0.1181 |
| 1.9045        | 35.0  | 875  | 0.8329          | 0.69     | 0.4296     | 1.9727 | 0.69     | 0.6098   | 0.2498 | 0.1121 |
| 1.9045        | 36.0  | 900  | 0.8222          | 0.7      | 0.4234     | 1.8988 | 0.7      | 0.6185   | 0.2404 | 0.1084 |
| 1.9045        | 37.0  | 925  | 0.8178          | 0.69     | 0.4201     | 1.8900 | 0.69     | 0.6103   | 0.2338 | 0.1079 |
| 1.9045        | 38.0  | 950  | 0.8091          | 0.69     | 0.4153     | 1.9396 | 0.69     | 0.6100   | 0.2469 | 0.1058 |
| 1.9045        | 39.0  | 975  | 0.7992          | 0.705    | 0.4098     | 1.8177 | 0.705    | 0.6453   | 0.1971 | 0.1037 |
| 0.8325        | 40.0  | 1000 | 0.7962          | 0.715    | 0.4069     | 1.7962 | 0.715    | 0.6560   | 0.2474 | 0.1008 |
| 0.8325        | 41.0  | 1025 | 0.7890          | 0.715    | 0.4032     | 1.7233 | 0.715    | 0.6592   | 0.2417 | 0.1009 |
| 0.8325        | 42.0  | 1050 | 0.7842          | 0.715    | 0.3997     | 1.6669 | 0.715    | 0.6573   | 0.2441 | 0.1005 |
| 0.8325        | 43.0  | 1075 | 0.7788          | 0.71     | 0.3962     | 1.6468 | 0.7100   | 0.6515   | 0.2199 | 0.0998 |
| 0.8325        | 44.0  | 1100 | 0.7713          | 0.725    | 0.3918     | 1.6398 | 0.7250   | 0.6698   | 0.2363 | 0.0945 |
| 0.8325        | 45.0  | 1125 | 0.7687          | 0.725    | 0.3895     | 1.6397 | 0.7250   | 0.6677   | 0.2478 | 0.0939 |
| 0.8325        | 46.0  | 1150 | 0.7642          | 0.73     | 0.3867     | 1.6259 | 0.7300   | 0.6684   | 0.2538 | 0.0941 |
| 0.8325        | 47.0  | 1175 | 0.7571          | 0.74     | 0.3822     | 1.5033 | 0.74     | 0.6907   | 0.2269 | 0.0903 |
| 0.8325        | 48.0  | 1200 | 0.7547          | 0.74     | 0.3806     | 1.5595 | 0.74     | 0.6830   | 0.2448 | 0.0912 |
| 0.8325        | 49.0  | 1225 | 0.7516          | 0.75     | 0.3779     | 1.6264 | 0.75     | 0.6921   | 0.2489 | 0.0887 |
| 0.8325        | 50.0  | 1250 | 0.7477          | 0.75     | 0.3759     | 1.5568 | 0.75     | 0.7029   | 0.2388 | 0.0888 |
| 0.8325        | 51.0  | 1275 | 0.7431          | 0.755    | 0.3725     | 1.5037 | 0.755    | 0.6986   | 0.2254 | 0.0848 |
| 0.8325        | 52.0  | 1300 | 0.7418          | 0.755    | 0.3713     | 1.4951 | 0.755    | 0.7085   | 0.2261 | 0.0862 |
| 0.8325        | 53.0  | 1325 | 0.7360          | 0.77     | 0.3676     | 1.4881 | 0.7700   | 0.7241   | 0.2474 | 0.0825 |
| 0.8325        | 54.0  | 1350 | 0.7339          | 0.77     | 0.3665     | 1.5554 | 0.7700   | 0.7241   | 0.2646 | 0.0827 |
| 0.8325        | 55.0  | 1375 | 0.7294          | 0.775    | 0.3636     | 1.4885 | 0.775    | 0.7275   | 0.2283 | 0.0818 |
| 0.8325        | 56.0  | 1400 | 0.7265          | 0.78     | 0.3617     | 1.5387 | 0.78     | 0.7306   | 0.2416 | 0.0799 |
| 0.8325        | 57.0  | 1425 | 0.7247          | 0.77     | 0.3598     | 1.4382 | 0.7700   | 0.7241   | 0.2553 | 0.0806 |
| 0.8325        | 58.0  | 1450 | 0.7234          | 0.78     | 0.3589     | 1.4888 | 0.78     | 0.7306   | 0.2231 | 0.0796 |
| 0.8325        | 59.0  | 1475 | 0.7186          | 0.78     | 0.3565     | 1.5400 | 0.78     | 0.7306   | 0.2200 | 0.0790 |
| 0.618         | 60.0  | 1500 | 0.7174          | 0.78     | 0.3549     | 1.4823 | 0.78     | 0.7306   | 0.2340 | 0.0787 |
| 0.618         | 61.0  | 1525 | 0.7148          | 0.78     | 0.3534     | 1.4804 | 0.78     | 0.7306   | 0.2412 | 0.0785 |
| 0.618         | 62.0  | 1550 | 0.7122          | 0.785    | 0.3516     | 1.4334 | 0.785    | 0.7445   | 0.2353 | 0.0787 |
| 0.618         | 63.0  | 1575 | 0.7107          | 0.79     | 0.3501     | 1.4153 | 0.79     | 0.7516   | 0.2354 | 0.0777 |
| 0.618         | 64.0  | 1600 | 0.7091          | 0.78     | 0.3491     | 1.4698 | 0.78     | 0.7306   | 0.2324 | 0.0780 |
| 0.618         | 65.0  | 1625 | 0.7071          | 0.79     | 0.3481     | 1.4097 | 0.79     | 0.7516   | 0.2198 | 0.0785 |
| 0.618         | 66.0  | 1650 | 0.7047          | 0.785    | 0.3464     | 1.4088 | 0.785    | 0.7458   | 0.2325 | 0.0778 |
| 0.618         | 67.0  | 1675 | 0.7041          | 0.785    | 0.3457     | 1.4108 | 0.785    | 0.7458   | 0.2248 | 0.0781 |
| 0.618         | 68.0  | 1700 | 0.7025          | 0.79     | 0.3444     | 1.4145 | 0.79     | 0.7516   | 0.2195 | 0.0773 |
| 0.618         | 69.0  | 1725 | 0.7014          | 0.79     | 0.3436     | 1.4120 | 0.79     | 0.7516   | 0.2629 | 0.0771 |
| 0.618         | 70.0  | 1750 | 0.6992          | 0.785    | 0.3422     | 1.4046 | 0.785    | 0.7458   | 0.2294 | 0.0767 |
| 0.618         | 71.0  | 1775 | 0.6982          | 0.785    | 0.3412     | 1.4142 | 0.785    | 0.7458   | 0.2325 | 0.0761 |
| 0.618         | 72.0  | 1800 | 0.6954          | 0.79     | 0.3395     | 1.4009 | 0.79     | 0.7516   | 0.2253 | 0.0763 |
| 0.618         | 73.0  | 1825 | 0.6942          | 0.79     | 0.3389     | 1.3994 | 0.79     | 0.7559   | 0.2383 | 0.0763 |
| 0.618         | 74.0  | 1850 | 0.6937          | 0.785    | 0.3382     | 1.4061 | 0.785    | 0.7458   | 0.2213 | 0.0762 |
| 0.618         | 75.0  | 1875 | 0.6935          | 0.785    | 0.3378     | 1.4082 | 0.785    | 0.7458   | 0.2218 | 0.0762 |
| 0.618         | 76.0  | 1900 | 0.6910          | 0.795    | 0.3359     | 1.4098 | 0.795    | 0.7599   | 0.2689 | 0.0746 |
| 0.618         | 77.0  | 1925 | 0.6907          | 0.79     | 0.3356     | 1.4072 | 0.79     | 0.7541   | 0.2254 | 0.0741 |
| 0.618         | 78.0  | 1950 | 0.6896          | 0.795    | 0.3352     | 1.3996 | 0.795    | 0.7636   | 0.2226 | 0.0743 |
| 0.618         | 79.0  | 1975 | 0.6896          | 0.79     | 0.3349     | 1.4073 | 0.79     | 0.7541   | 0.2295 | 0.0742 |
| 0.516         | 80.0  | 2000 | 0.6874          | 0.79     | 0.3335     | 1.4089 | 0.79     | 0.7541   | 0.2287 | 0.0743 |
| 0.516         | 81.0  | 2025 | 0.6874          | 0.795    | 0.3333     | 1.3983 | 0.795    | 0.7636   | 0.2387 | 0.0742 |
| 0.516         | 82.0  | 2050 | 0.6867          | 0.795    | 0.3327     | 1.4098 | 0.795    | 0.7636   | 0.2162 | 0.0736 |
| 0.516         | 83.0  | 2075 | 0.6865          | 0.795    | 0.3323     | 1.4656 | 0.795    | 0.7636   | 0.2072 | 0.0738 |
| 0.516         | 84.0  | 2100 | 0.6857          | 0.795    | 0.3323     | 1.4107 | 0.795    | 0.7636   | 0.2138 | 0.0741 |
| 0.516         | 85.0  | 2125 | 0.6854          | 0.795    | 0.3316     | 1.4223 | 0.795    | 0.7636   | 0.2262 | 0.0732 |
| 0.516         | 86.0  | 2150 | 0.6846          | 0.795    | 0.3311     | 1.4138 | 0.795    | 0.7636   | 0.2224 | 0.0733 |
| 0.516         | 87.0  | 2175 | 0.6834          | 0.795    | 0.3302     | 1.4113 | 0.795    | 0.7636   | 0.2307 | 0.0731 |
| 0.516         | 88.0  | 2200 | 0.6831          | 0.795    | 0.3300     | 1.4088 | 0.795    | 0.7636   | 0.2256 | 0.0730 |
| 0.516         | 89.0  | 2225 | 0.6821          | 0.795    | 0.3295     | 1.4126 | 0.795    | 0.7636   | 0.2395 | 0.0728 |
| 0.516         | 90.0  | 2250 | 0.6821          | 0.795    | 0.3294     | 1.4123 | 0.795    | 0.7636   | 0.2237 | 0.0728 |
| 0.516         | 91.0  | 2275 | 0.6823          | 0.795    | 0.3294     | 1.4085 | 0.795    | 0.7636   | 0.2213 | 0.0728 |
| 0.516         | 92.0  | 2300 | 0.6819          | 0.795    | 0.3290     | 1.4105 | 0.795    | 0.7636   | 0.2332 | 0.0730 |
| 0.516         | 93.0  | 2325 | 0.6816          | 0.795    | 0.3289     | 1.4094 | 0.795    | 0.7636   | 0.2236 | 0.0729 |
| 0.516         | 94.0  | 2350 | 0.6812          | 0.795    | 0.3287     | 1.4092 | 0.795    | 0.7636   | 0.2235 | 0.0729 |
| 0.516         | 95.0  | 2375 | 0.6813          | 0.795    | 0.3286     | 1.4065 | 0.795    | 0.7636   | 0.2197 | 0.0727 |
| 0.516         | 96.0  | 2400 | 0.6811          | 0.795    | 0.3285     | 1.4079 | 0.795    | 0.7636   | 0.2247 | 0.0729 |
| 0.516         | 97.0  | 2425 | 0.6810          | 0.795    | 0.3284     | 1.4072 | 0.795    | 0.7636   | 0.2320 | 0.0729 |
| 0.516         | 98.0  | 2450 | 0.6810          | 0.795    | 0.3284     | 1.4062 | 0.795    | 0.7636   | 0.2148 | 0.0727 |
| 0.516         | 99.0  | 2475 | 0.6810          | 0.795    | 0.3284     | 1.4068 | 0.795    | 0.7636   | 0.2215 | 0.0726 |
| 0.4715        | 100.0 | 2500 | 0.6810          | 0.795    | 0.3284     | 1.4069 | 0.795    | 0.7636   | 0.2215 | 0.0726 |


### Framework versions

- Transformers 4.30.2
- Pytorch 1.13.1
- Datasets 2.13.1
- Tokenizers 0.13.3

adve email form letter memo news note report resume scientific