
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-small_tobacco3482_kd_CEKD_t5.0_a0.9

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5373
- Accuracy: 0.85
- Brier Loss: 0.2432
- Nll: 1.1157
- F1 Micro: 0.85
- F1 Macro: 0.8450
- Ece: 0.1621
- Aurc: 0.0427

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 2.1036          | 0.215    | 0.8753     | 5.3195 | 0.2150   | 0.1264   | 0.2571 | 0.6923 |
| No log        | 2.0   | 14   | 1.6952          | 0.405    | 0.7407     | 3.4929 | 0.405    | 0.2416   | 0.2907 | 0.4040 |
| No log        | 3.0   | 21   | 1.1843          | 0.62     | 0.5633     | 2.0113 | 0.62     | 0.5725   | 0.2740 | 0.2014 |
| No log        | 4.0   | 28   | 0.8797          | 0.71     | 0.4080     | 1.7043 | 0.7100   | 0.6683   | 0.2024 | 0.1125 |
| No log        | 5.0   | 35   | 0.8570          | 0.715    | 0.3837     | 1.6476 | 0.715    | 0.7280   | 0.2189 | 0.1079 |
| No log        | 6.0   | 42   | 0.7484          | 0.775    | 0.3285     | 1.5962 | 0.775    | 0.7668   | 0.1873 | 0.0816 |
| No log        | 7.0   | 49   | 0.7337          | 0.79     | 0.3131     | 1.5377 | 0.79     | 0.7779   | 0.1904 | 0.0771 |
| No log        | 8.0   | 56   | 0.6709          | 0.795    | 0.3012     | 1.2156 | 0.795    | 0.7776   | 0.1939 | 0.0761 |
| No log        | 9.0   | 63   | 0.6901          | 0.795    | 0.3069     | 1.4725 | 0.795    | 0.7916   | 0.1882 | 0.0769 |
| No log        | 10.0  | 70   | 0.7960          | 0.75     | 0.3586     | 1.4426 | 0.75     | 0.7406   | 0.1868 | 0.0976 |
| No log        | 11.0  | 77   | 0.7489          | 0.77     | 0.3296     | 1.6202 | 0.7700   | 0.7794   | 0.2020 | 0.0878 |
| No log        | 12.0  | 84   | 0.7068          | 0.785    | 0.3270     | 1.4127 | 0.785    | 0.7812   | 0.1922 | 0.0759 |
| No log        | 13.0  | 91   | 0.6687          | 0.79     | 0.3050     | 1.3820 | 0.79     | 0.7945   | 0.1818 | 0.0625 |
| No log        | 14.0  | 98   | 0.6052          | 0.79     | 0.2854     | 1.0602 | 0.79     | 0.7716   | 0.1702 | 0.0590 |
| No log        | 15.0  | 105  | 0.6369          | 0.795    | 0.2959     | 1.0580 | 0.795    | 0.7953   | 0.1709 | 0.0603 |
| No log        | 16.0  | 112  | 0.6204          | 0.81     | 0.2816     | 1.1886 | 0.81     | 0.8050   | 0.1657 | 0.0702 |
| No log        | 17.0  | 119  | 0.5648          | 0.83     | 0.2475     | 1.2506 | 0.83     | 0.8241   | 0.1347 | 0.0612 |
| No log        | 18.0  | 126  | 0.5849          | 0.83     | 0.2672     | 1.2245 | 0.83     | 0.8155   | 0.1646 | 0.0601 |
| No log        | 19.0  | 133  | 0.5536          | 0.835    | 0.2475     | 1.0514 | 0.835    | 0.8254   | 0.1683 | 0.0531 |
| No log        | 20.0  | 140  | 0.5689          | 0.835    | 0.2513     | 1.2369 | 0.835    | 0.8437   | 0.1722 | 0.0489 |
| No log        | 21.0  | 147  | 0.5540          | 0.83     | 0.2485     | 1.2139 | 0.83     | 0.8165   | 0.1641 | 0.0608 |
| No log        | 22.0  | 154  | 0.5352          | 0.835    | 0.2402     | 1.0108 | 0.835    | 0.8295   | 0.1408 | 0.0430 |
| No log        | 23.0  | 161  | 0.5380          | 0.84     | 0.2403     | 1.2280 | 0.8400   | 0.8347   | 0.1405 | 0.0436 |
| No log        | 24.0  | 168  | 0.5422          | 0.835    | 0.2471     | 1.0204 | 0.835    | 0.8324   | 0.1606 | 0.0445 |
| No log        | 25.0  | 175  | 0.5342          | 0.85     | 0.2404     | 1.0767 | 0.85     | 0.8487   | 0.1469 | 0.0432 |
| No log        | 26.0  | 182  | 0.5374          | 0.84     | 0.2429     | 1.0774 | 0.8400   | 0.8334   | 0.1420 | 0.0462 |
| No log        | 27.0  | 189  | 0.5311          | 0.85     | 0.2395     | 1.0748 | 0.85     | 0.8487   | 0.1439 | 0.0446 |
| No log        | 28.0  | 196  | 0.5298          | 0.85     | 0.2384     | 1.1337 | 0.85     | 0.8487   | 0.1570 | 0.0437 |
| No log        | 29.0  | 203  | 0.5387          | 0.845    | 0.2435     | 1.1319 | 0.845    | 0.8424   | 0.1539 | 0.0458 |
| No log        | 30.0  | 210  | 0.5361          | 0.85     | 0.2430     | 1.0648 | 0.85     | 0.8450   | 0.1679 | 0.0431 |
| No log        | 31.0  | 217  | 0.5339          | 0.85     | 0.2413     | 1.0676 | 0.85     | 0.8487   | 0.1646 | 0.0428 |
| No log        | 32.0  | 224  | 0.5345          | 0.85     | 0.2421     | 1.0709 | 0.85     | 0.8487   | 0.1476 | 0.0440 |
| No log        | 33.0  | 231  | 0.5343          | 0.85     | 0.2421     | 1.1236 | 0.85     | 0.8450   | 0.1621 | 0.0431 |
| No log        | 34.0  | 238  | 0.5353          | 0.845    | 0.2426     | 1.1244 | 0.845    | 0.8424   | 0.1710 | 0.0428 |
| No log        | 35.0  | 245  | 0.5346          | 0.85     | 0.2423     | 1.0649 | 0.85     | 0.8487   | 0.1520 | 0.0440 |
| No log        | 36.0  | 252  | 0.5356          | 0.855    | 0.2422     | 1.1241 | 0.855    | 0.8517   | 0.1814 | 0.0429 |
| No log        | 37.0  | 259  | 0.5357          | 0.85     | 0.2426     | 1.1237 | 0.85     | 0.8450   | 0.1670 | 0.0425 |
| No log        | 38.0  | 266  | 0.5356          | 0.845    | 0.2426     | 1.1226 | 0.845    | 0.8419   | 0.1607 | 0.0435 |
| No log        | 39.0  | 273  | 0.5347          | 0.855    | 0.2420     | 1.0739 | 0.855    | 0.8517   | 0.1597 | 0.0427 |
| No log        | 40.0  | 280  | 0.5356          | 0.855    | 0.2423     | 1.1203 | 0.855    | 0.8517   | 0.1676 | 0.0435 |
| No log        | 41.0  | 287  | 0.5365          | 0.85     | 0.2431     | 1.1199 | 0.85     | 0.8450   | 0.1780 | 0.0429 |
| No log        | 42.0  | 294  | 0.5356          | 0.85     | 0.2426     | 1.1173 | 0.85     | 0.8450   | 0.1653 | 0.0430 |
| No log        | 43.0  | 301  | 0.5363          | 0.85     | 0.2428     | 1.1189 | 0.85     | 0.8450   | 0.1550 | 0.0435 |
| No log        | 44.0  | 308  | 0.5345          | 0.85     | 0.2418     | 1.1193 | 0.85     | 0.8450   | 0.1590 | 0.0428 |
| No log        | 45.0  | 315  | 0.5374          | 0.85     | 0.2435     | 1.1202 | 0.85     | 0.8450   | 0.1633 | 0.0435 |
| No log        | 46.0  | 322  | 0.5355          | 0.85     | 0.2423     | 1.1183 | 0.85     | 0.8450   | 0.1564 | 0.0428 |
| No log        | 47.0  | 329  | 0.5354          | 0.85     | 0.2425     | 1.1176 | 0.85     | 0.8450   | 0.1509 | 0.0429 |
| No log        | 48.0  | 336  | 0.5369          | 0.85     | 0.2433     | 1.1177 | 0.85     | 0.8450   | 0.1517 | 0.0432 |
| No log        | 49.0  | 343  | 0.5361          | 0.85     | 0.2428     | 1.1182 | 0.85     | 0.8450   | 0.1490 | 0.0428 |
| No log        | 50.0  | 350  | 0.5364          | 0.85     | 0.2431     | 1.1179 | 0.85     | 0.8450   | 0.1654 | 0.0430 |
| No log        | 51.0  | 357  | 0.5365          | 0.85     | 0.2428     | 1.1185 | 0.85     | 0.8450   | 0.1729 | 0.0432 |
| No log        | 52.0  | 364  | 0.5364          | 0.85     | 0.2430     | 1.1165 | 0.85     | 0.8450   | 0.1614 | 0.0429 |
| No log        | 53.0  | 371  | 0.5362          | 0.85     | 0.2429     | 1.1167 | 0.85     | 0.8450   | 0.1694 | 0.0430 |
| No log        | 54.0  | 378  | 0.5369          | 0.85     | 0.2432     | 1.1170 | 0.85     | 0.8450   | 0.1597 | 0.0432 |
| No log        | 55.0  | 385  | 0.5368          | 0.85     | 0.2430     | 1.1168 | 0.85     | 0.8450   | 0.1670 | 0.0429 |
| No log        | 56.0  | 392  | 0.5367          | 0.85     | 0.2430     | 1.1180 | 0.85     | 0.8450   | 0.1619 | 0.0430 |
| No log        | 57.0  | 399  | 0.5364          | 0.85     | 0.2429     | 1.1163 | 0.85     | 0.8450   | 0.1649 | 0.0429 |
| No log        | 58.0  | 406  | 0.5364          | 0.85     | 0.2430     | 1.1156 | 0.85     | 0.8450   | 0.1611 | 0.0429 |
| No log        | 59.0  | 413  | 0.5365          | 0.85     | 0.2428     | 1.1163 | 0.85     | 0.8450   | 0.1591 | 0.0429 |
| No log        | 60.0  | 420  | 0.5364          | 0.85     | 0.2429     | 1.1155 | 0.85     | 0.8450   | 0.1588 | 0.0429 |
| No log        | 61.0  | 427  | 0.5370          | 0.85     | 0.2432     | 1.1158 | 0.85     | 0.8450   | 0.1772 | 0.0432 |
| No log        | 62.0  | 434  | 0.5367          | 0.85     | 0.2429     | 1.1167 | 0.85     | 0.8450   | 0.1622 | 0.0429 |
| No log        | 63.0  | 441  | 0.5362          | 0.85     | 0.2428     | 1.1162 | 0.85     | 0.8450   | 0.1503 | 0.0428 |
| No log        | 64.0  | 448  | 0.5372          | 0.85     | 0.2433     | 1.1161 | 0.85     | 0.8450   | 0.1616 | 0.0432 |
| No log        | 65.0  | 455  | 0.5371          | 0.85     | 0.2431     | 1.1162 | 0.85     | 0.8450   | 0.1499 | 0.0429 |
| No log        | 66.0  | 462  | 0.5367          | 0.85     | 0.2430     | 1.1160 | 0.85     | 0.8450   | 0.1591 | 0.0427 |
| No log        | 67.0  | 469  | 0.5367          | 0.85     | 0.2430     | 1.1164 | 0.85     | 0.8450   | 0.1562 | 0.0428 |
| No log        | 68.0  | 476  | 0.5368          | 0.85     | 0.2430     | 1.1168 | 0.85     | 0.8450   | 0.1556 | 0.0427 |
| No log        | 69.0  | 483  | 0.5368          | 0.85     | 0.2431     | 1.1158 | 0.85     | 0.8450   | 0.1593 | 0.0428 |
| No log        | 70.0  | 490  | 0.5372          | 0.85     | 0.2432     | 1.1162 | 0.85     | 0.8450   | 0.1628 | 0.0428 |
| No log        | 71.0  | 497  | 0.5371          | 0.85     | 0.2432     | 1.1163 | 0.85     | 0.8450   | 0.1599 | 0.0429 |
| 0.1708        | 72.0  | 504  | 0.5370          | 0.85     | 0.2430     | 1.1161 | 0.85     | 0.8450   | 0.1559 | 0.0430 |
| 0.1708        | 73.0  | 511  | 0.5372          | 0.85     | 0.2433     | 1.1154 | 0.85     | 0.8450   | 0.1556 | 0.0428 |
| 0.1708        | 74.0  | 518  | 0.5370          | 0.85     | 0.2429     | 1.1165 | 0.85     | 0.8450   | 0.1540 | 0.0428 |
| 0.1708        | 75.0  | 525  | 0.5371          | 0.85     | 0.2431     | 1.1161 | 0.85     | 0.8450   | 0.1616 | 0.0427 |
| 0.1708        | 76.0  | 532  | 0.5369          | 0.85     | 0.2431     | 1.1161 | 0.85     | 0.8450   | 0.1619 | 0.0427 |
| 0.1708        | 77.0  | 539  | 0.5369          | 0.85     | 0.2430     | 1.1156 | 0.85     | 0.8450   | 0.1623 | 0.0429 |
| 0.1708        | 78.0  | 546  | 0.5372          | 0.85     | 0.2432     | 1.1158 | 0.85     | 0.8450   | 0.1619 | 0.0427 |
| 0.1708        | 79.0  | 553  | 0.5375          | 0.85     | 0.2433     | 1.1162 | 0.85     | 0.8450   | 0.1688 | 0.0429 |
| 0.1708        | 80.0  | 560  | 0.5372          | 0.85     | 0.2432     | 1.1160 | 0.85     | 0.8450   | 0.1623 | 0.0429 |
| 0.1708        | 81.0  | 567  | 0.5373          | 0.85     | 0.2432     | 1.1162 | 0.85     | 0.8450   | 0.1620 | 0.0428 |
| 0.1708        | 82.0  | 574  | 0.5374          | 0.85     | 0.2433     | 1.1160 | 0.85     | 0.8450   | 0.1622 | 0.0428 |
| 0.1708        | 83.0  | 581  | 0.5372          | 0.85     | 0.2432     | 1.1159 | 0.85     | 0.8450   | 0.1622 | 0.0428 |
| 0.1708        | 84.0  | 588  | 0.5371          | 0.85     | 0.2431     | 1.1157 | 0.85     | 0.8450   | 0.1621 | 0.0427 |
| 0.1708        | 85.0  | 595  | 0.5372          | 0.85     | 0.2432     | 1.1158 | 0.85     | 0.8450   | 0.1687 | 0.0426 |
| 0.1708        | 86.0  | 602  | 0.5372          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1619 | 0.0426 |
| 0.1708        | 87.0  | 609  | 0.5374          | 0.85     | 0.2432     | 1.1159 | 0.85     | 0.8450   | 0.1687 | 0.0428 |
| 0.1708        | 88.0  | 616  | 0.5373          | 0.85     | 0.2432     | 1.1160 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 89.0  | 623  | 0.5373          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 90.0  | 630  | 0.5373          | 0.85     | 0.2432     | 1.1156 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 91.0  | 637  | 0.5372          | 0.85     | 0.2432     | 1.1156 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 92.0  | 644  | 0.5373          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 93.0  | 651  | 0.5372          | 0.85     | 0.2432     | 1.1156 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 94.0  | 658  | 0.5373          | 0.85     | 0.2432     | 1.1158 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 95.0  | 665  | 0.5373          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1621 | 0.0427 |
| 0.1708        | 96.0  | 672  | 0.5372          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1621 | 0.0427 |
| 0.1708        | 97.0  | 679  | 0.5372          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1620 | 0.0427 |
| 0.1708        | 98.0  | 686  | 0.5373          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1621 | 0.0427 |
| 0.1708        | 99.0  | 693  | 0.5373          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1621 | 0.0427 |
| 0.1708        | 100.0 | 700  | 0.5373          | 0.85     | 0.2432     | 1.1157 | 0.85     | 0.8450   | 0.1621 | 0.0427 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

adve email form letter memo news note report resume scientific