
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 6_e_200-tiny_tobacco3482_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4137
- Accuracy: 0.83
- Brier Loss: 0.2631
- Nll: 1.5189
- F1 Micro: 0.83
- F1 Macro: 0.8172
- Ece: 0.2007
- Aurc: 0.0591

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 1.6265          | 0.23     | 0.8647     | 5.1432 | 0.23     | 0.1847   | 0.2751 | 0.7516 |
| No log        | 2.0   | 50   | 1.0240          | 0.505    | 0.6074     | 2.7425 | 0.505    | 0.3980   | 0.2977 | 0.2705 |
| No log        | 3.0   | 75   | 0.8130          | 0.655    | 0.4809     | 2.4939 | 0.655    | 0.5584   | 0.2576 | 0.1502 |
| No log        | 4.0   | 100  | 0.6703          | 0.735    | 0.3867     | 1.3509 | 0.735    | 0.6895   | 0.2334 | 0.1109 |
| No log        | 5.0   | 125  | 0.6313          | 0.755    | 0.3420     | 1.2521 | 0.755    | 0.7207   | 0.2081 | 0.0789 |
| No log        | 6.0   | 150  | 0.6598          | 0.76     | 0.3543     | 1.4171 | 0.76     | 0.7103   | 0.2275 | 0.0886 |
| No log        | 7.0   | 175  | 0.5669          | 0.77     | 0.3368     | 1.5060 | 0.7700   | 0.7351   | 0.2247 | 0.0941 |
| No log        | 8.0   | 200  | 0.5486          | 0.775    | 0.3004     | 1.1511 | 0.775    | 0.7413   | 0.2252 | 0.0640 |
| No log        | 9.0   | 225  | 0.5456          | 0.795    | 0.3141     | 1.4663 | 0.795    | 0.7762   | 0.2198 | 0.0889 |
| No log        | 10.0  | 250  | 0.4954          | 0.82     | 0.2819     | 1.4644 | 0.82     | 0.7981   | 0.2150 | 0.0649 |
| No log        | 11.0  | 275  | 0.4804          | 0.805    | 0.2866     | 1.3705 | 0.805    | 0.7927   | 0.2078 | 0.0658 |
| No log        | 12.0  | 300  | 0.5234          | 0.785    | 0.3152     | 1.5290 | 0.785    | 0.7681   | 0.2149 | 0.0637 |
| No log        | 13.0  | 325  | 0.4701          | 0.815    | 0.2839     | 1.4490 | 0.815    | 0.8010   | 0.2315 | 0.0586 |
| No log        | 14.0  | 350  | 0.4859          | 0.795    | 0.2807     | 1.1224 | 0.795    | 0.7957   | 0.2170 | 0.0512 |
| No log        | 15.0  | 375  | 0.5580          | 0.79     | 0.3272     | 1.7539 | 0.79     | 0.7735   | 0.2376 | 0.0708 |
| No log        | 16.0  | 400  | 0.4918          | 0.8      | 0.2961     | 1.5112 | 0.8000   | 0.7988   | 0.1850 | 0.0568 |
| No log        | 17.0  | 425  | 0.4442          | 0.8      | 0.2846     | 1.6182 | 0.8000   | 0.7767   | 0.2083 | 0.0712 |
| No log        | 18.0  | 450  | 0.4460          | 0.82     | 0.2760     | 1.6839 | 0.82     | 0.8027   | 0.2127 | 0.0523 |
| No log        | 19.0  | 475  | 0.4423          | 0.825    | 0.2676     | 1.3774 | 0.825    | 0.8176   | 0.1853 | 0.0557 |
| 0.4472        | 20.0  | 500  | 0.4998          | 0.81     | 0.2910     | 1.7711 | 0.81     | 0.8152   | 0.2181 | 0.0635 |
| 0.4472        | 21.0  | 525  | 0.4579          | 0.83     | 0.2871     | 1.7025 | 0.83     | 0.8135   | 0.1927 | 0.0696 |
| 0.4472        | 22.0  | 550  | 0.4421          | 0.825    | 0.2683     | 1.6453 | 0.825    | 0.8215   | 0.1929 | 0.0613 |
| 0.4472        | 23.0  | 575  | 0.4368          | 0.8      | 0.2821     | 1.7298 | 0.8000   | 0.7684   | 0.2060 | 0.0771 |
| 0.4472        | 24.0  | 600  | 0.4310          | 0.83     | 0.2689     | 1.4699 | 0.83     | 0.8163   | 0.2067 | 0.0556 |
| 0.4472        | 25.0  | 625  | 0.4394          | 0.83     | 0.2751     | 1.5955 | 0.83     | 0.8166   | 0.2138 | 0.0681 |
| 0.4472        | 26.0  | 650  | 0.4395          | 0.815    | 0.2786     | 1.6788 | 0.815    | 0.8033   | 0.2034 | 0.0643 |
| 0.4472        | 27.0  | 675  | 0.4118          | 0.84     | 0.2578     | 1.5641 | 0.8400   | 0.8293   | 0.2024 | 0.0554 |
| 0.4472        | 28.0  | 700  | 0.4273          | 0.82     | 0.2707     | 1.7118 | 0.82     | 0.8090   | 0.2133 | 0.0674 |
| 0.4472        | 29.0  | 725  | 0.4207          | 0.835    | 0.2648     | 1.6469 | 0.835    | 0.8206   | 0.1948 | 0.0652 |
| 0.4472        | 30.0  | 750  | 0.4172          | 0.825    | 0.2620     | 1.5024 | 0.825    | 0.8114   | 0.1833 | 0.0601 |
| 0.4472        | 31.0  | 775  | 0.4148          | 0.825    | 0.2610     | 1.4994 | 0.825    | 0.8070   | 0.2052 | 0.0593 |
| 0.4472        | 32.0  | 800  | 0.4148          | 0.825    | 0.2627     | 1.6293 | 0.825    | 0.8088   | 0.2080 | 0.0618 |
| 0.4472        | 33.0  | 825  | 0.4159          | 0.825    | 0.2625     | 1.5069 | 0.825    | 0.8135   | 0.2082 | 0.0604 |
| 0.4472        | 34.0  | 850  | 0.4168          | 0.825    | 0.2638     | 1.5770 | 0.825    | 0.8137   | 0.1888 | 0.0588 |
| 0.4472        | 35.0  | 875  | 0.4181          | 0.82     | 0.2640     | 1.5404 | 0.82     | 0.8043   | 0.2145 | 0.0582 |
| 0.4472        | 36.0  | 900  | 0.4154          | 0.83     | 0.2618     | 1.5719 | 0.83     | 0.8165   | 0.1965 | 0.0586 |
| 0.4472        | 37.0  | 925  | 0.4160          | 0.825    | 0.2632     | 1.5840 | 0.825    | 0.8137   | 0.2003 | 0.0604 |
| 0.4472        | 38.0  | 950  | 0.4133          | 0.83     | 0.2616     | 1.5711 | 0.83     | 0.8163   | 0.2040 | 0.0596 |
| 0.4472        | 39.0  | 975  | 0.4167          | 0.825    | 0.2635     | 1.5210 | 0.825    | 0.8138   | 0.1930 | 0.0590 |
| 0.0652        | 40.0  | 1000 | 0.4162          | 0.83     | 0.2630     | 1.6312 | 0.83     | 0.8163   | 0.1973 | 0.0593 |
| 0.0652        | 41.0  | 1025 | 0.4144          | 0.83     | 0.2626     | 1.5787 | 0.83     | 0.8163   | 0.2068 | 0.0603 |
| 0.0652        | 42.0  | 1050 | 0.4150          | 0.83     | 0.2631     | 1.5789 | 0.83     | 0.8163   | 0.1970 | 0.0588 |
| 0.0652        | 43.0  | 1075 | 0.4158          | 0.825    | 0.2635     | 1.5833 | 0.825    | 0.8138   | 0.1927 | 0.0597 |
| 0.0652        | 44.0  | 1100 | 0.4132          | 0.83     | 0.2622     | 1.5130 | 0.83     | 0.8163   | 0.2030 | 0.0593 |
| 0.0652        | 45.0  | 1125 | 0.4146          | 0.83     | 0.2630     | 1.6312 | 0.83     | 0.8165   | 0.2010 | 0.0587 |
| 0.0652        | 46.0  | 1150 | 0.4138          | 0.825    | 0.2624     | 1.5301 | 0.825    | 0.8135   | 0.2065 | 0.0587 |
| 0.0652        | 47.0  | 1175 | 0.4142          | 0.83     | 0.2627     | 1.6292 | 0.83     | 0.8163   | 0.1984 | 0.0591 |
| 0.0652        | 48.0  | 1200 | 0.4146          | 0.825    | 0.2629     | 1.5735 | 0.825    | 0.8137   | 0.1998 | 0.0589 |
| 0.0652        | 49.0  | 1225 | 0.4143          | 0.83     | 0.2630     | 1.5276 | 0.83     | 0.8163   | 0.2116 | 0.0599 |
| 0.0652        | 50.0  | 1250 | 0.4140          | 0.83     | 0.2628     | 1.5705 | 0.83     | 0.8163   | 0.1966 | 0.0590 |
| 0.0652        | 51.0  | 1275 | 0.4152          | 0.825    | 0.2637     | 1.5747 | 0.825    | 0.8138   | 0.1835 | 0.0593 |
| 0.0652        | 52.0  | 1300 | 0.4145          | 0.825    | 0.2629     | 1.5796 | 0.825    | 0.8137   | 0.1926 | 0.0593 |
| 0.0652        | 53.0  | 1325 | 0.4147          | 0.825    | 0.2631     | 1.6323 | 0.825    | 0.8138   | 0.1838 | 0.0588 |
| 0.0652        | 54.0  | 1350 | 0.4141          | 0.83     | 0.2628     | 1.5763 | 0.83     | 0.8163   | 0.2035 | 0.0592 |
| 0.0652        | 55.0  | 1375 | 0.4137          | 0.83     | 0.2630     | 1.5751 | 0.83     | 0.8163   | 0.2042 | 0.0590 |
| 0.0652        | 56.0  | 1400 | 0.4145          | 0.83     | 0.2632     | 1.6307 | 0.83     | 0.8163   | 0.1981 | 0.0588 |
| 0.0652        | 57.0  | 1425 | 0.4149          | 0.825    | 0.2634     | 1.5225 | 0.825    | 0.8137   | 0.2008 | 0.0589 |
| 0.0652        | 58.0  | 1450 | 0.4146          | 0.83     | 0.2634     | 1.5725 | 0.83     | 0.8163   | 0.2121 | 0.0589 |
| 0.0652        | 59.0  | 1475 | 0.4142          | 0.83     | 0.2632     | 1.5214 | 0.83     | 0.8163   | 0.2028 | 0.0590 |
| 0.0614        | 60.0  | 1500 | 0.4145          | 0.83     | 0.2634     | 1.5237 | 0.83     | 0.8163   | 0.1981 | 0.0585 |
| 0.0614        | 61.0  | 1525 | 0.4142          | 0.83     | 0.2630     | 1.5710 | 0.83     | 0.8163   | 0.2070 | 0.0591 |
| 0.0614        | 62.0  | 1550 | 0.4139          | 0.825    | 0.2631     | 1.5733 | 0.825    | 0.8135   | 0.1986 | 0.0594 |
| 0.0614        | 63.0  | 1575 | 0.4139          | 0.825    | 0.2630     | 1.5813 | 0.825    | 0.8136   | 0.1984 | 0.0593 |
| 0.0614        | 64.0  | 1600 | 0.4138          | 0.83     | 0.2629     | 1.5729 | 0.83     | 0.8163   | 0.2035 | 0.0590 |
| 0.0614        | 65.0  | 1625 | 0.4139          | 0.825    | 0.2629     | 1.5715 | 0.825    | 0.8136   | 0.2026 | 0.0593 |
| 0.0614        | 66.0  | 1650 | 0.4136          | 0.825    | 0.2629     | 1.5768 | 0.825    | 0.8135   | 0.1988 | 0.0592 |
| 0.0614        | 67.0  | 1675 | 0.4139          | 0.825    | 0.2629     | 1.5709 | 0.825    | 0.8135   | 0.1987 | 0.0593 |
| 0.0614        | 68.0  | 1700 | 0.4143          | 0.825    | 0.2633     | 1.5744 | 0.825    | 0.8138   | 0.1896 | 0.0595 |
| 0.0614        | 69.0  | 1725 | 0.4142          | 0.825    | 0.2632     | 1.5752 | 0.825    | 0.8138   | 0.1896 | 0.0593 |
| 0.0614        | 70.0  | 1750 | 0.4142          | 0.825    | 0.2632     | 1.5769 | 0.825    | 0.8138   | 0.1879 | 0.0594 |
| 0.0614        | 71.0  | 1775 | 0.4138          | 0.83     | 0.2630     | 1.5734 | 0.83     | 0.8163   | 0.2073 | 0.0588 |
| 0.0614        | 72.0  | 1800 | 0.4140          | 0.825    | 0.2631     | 1.5734 | 0.825    | 0.8138   | 0.1977 | 0.0593 |
| 0.0614        | 73.0  | 1825 | 0.4135          | 0.83     | 0.2629     | 1.5711 | 0.83     | 0.8163   | 0.2035 | 0.0589 |
| 0.0614        | 74.0  | 1850 | 0.4140          | 0.83     | 0.2632     | 1.5717 | 0.83     | 0.8163   | 0.2038 | 0.0590 |
| 0.0614        | 75.0  | 1875 | 0.4141          | 0.825    | 0.2633     | 1.5205 | 0.825    | 0.8138   | 0.1838 | 0.0593 |
| 0.0614        | 76.0  | 1900 | 0.4138          | 0.825    | 0.2631     | 1.5218 | 0.825    | 0.8137   | 0.1838 | 0.0595 |
| 0.0614        | 77.0  | 1925 | 0.4134          | 0.825    | 0.2628     | 1.5710 | 0.825    | 0.8135   | 0.1937 | 0.0591 |
| 0.0614        | 78.0  | 1950 | 0.4135          | 0.83     | 0.2629     | 1.5688 | 0.83     | 0.8163   | 0.2067 | 0.0588 |
| 0.0614        | 79.0  | 1975 | 0.4138          | 0.825    | 0.2631     | 1.5143 | 0.825    | 0.8137   | 0.1942 | 0.0592 |
| 0.0613        | 80.0  | 2000 | 0.4134          | 0.825    | 0.2628     | 1.5152 | 0.825    | 0.8135   | 0.1939 | 0.0591 |
| 0.0613        | 81.0  | 2025 | 0.4139          | 0.825    | 0.2632     | 1.5144 | 0.825    | 0.8136   | 0.1903 | 0.0593 |
| 0.0613        | 82.0  | 2050 | 0.4139          | 0.83     | 0.2632     | 1.5242 | 0.83     | 0.8163   | 0.1894 | 0.0589 |
| 0.0613        | 83.0  | 2075 | 0.4138          | 0.825    | 0.2631     | 1.5159 | 0.825    | 0.8136   | 0.2014 | 0.0594 |
| 0.0613        | 84.0  | 2100 | 0.4137          | 0.825    | 0.2631     | 1.5707 | 0.825    | 0.8136   | 0.1954 | 0.0592 |
| 0.0613        | 85.0  | 2125 | 0.4136          | 0.825    | 0.2630     | 1.5252 | 0.825    | 0.8136   | 0.1878 | 0.0592 |
| 0.0613        | 86.0  | 2150 | 0.4138          | 0.83     | 0.2630     | 1.5186 | 0.83     | 0.8172   | 0.2024 | 0.0588 |
| 0.0613        | 87.0  | 2175 | 0.4139          | 0.825    | 0.2632     | 1.5201 | 0.825    | 0.8138   | 0.1927 | 0.0592 |
| 0.0613        | 88.0  | 2200 | 0.4138          | 0.83     | 0.2631     | 1.5285 | 0.83     | 0.8172   | 0.1897 | 0.0591 |
| 0.0613        | 89.0  | 2225 | 0.4137          | 0.825    | 0.2631     | 1.5185 | 0.825    | 0.8136   | 0.1956 | 0.0593 |
| 0.0613        | 90.0  | 2250 | 0.4137          | 0.83     | 0.2631     | 1.5212 | 0.83     | 0.8172   | 0.2007 | 0.0591 |
| 0.0613        | 91.0  | 2275 | 0.4138          | 0.825    | 0.2631     | 1.5185 | 0.825    | 0.8138   | 0.1915 | 0.0593 |
| 0.0613        | 92.0  | 2300 | 0.4136          | 0.83     | 0.2630     | 1.5174 | 0.83     | 0.8172   | 0.2067 | 0.0590 |
| 0.0613        | 93.0  | 2325 | 0.4137          | 0.83     | 0.2631     | 1.5204 | 0.83     | 0.8172   | 0.1939 | 0.0591 |
| 0.0613        | 94.0  | 2350 | 0.4137          | 0.83     | 0.2631     | 1.5255 | 0.83     | 0.8172   | 0.2007 | 0.0592 |
| 0.0613        | 95.0  | 2375 | 0.4137          | 0.83     | 0.2631     | 1.5161 | 0.83     | 0.8172   | 0.1966 | 0.0591 |
| 0.0613        | 96.0  | 2400 | 0.4136          | 0.83     | 0.2630     | 1.5180 | 0.83     | 0.8172   | 0.2007 | 0.0590 |
| 0.0613        | 97.0  | 2425 | 0.4137          | 0.83     | 0.2631     | 1.5176 | 0.83     | 0.8172   | 0.1966 | 0.0591 |
| 0.0613        | 98.0  | 2450 | 0.4137          | 0.83     | 0.2631     | 1.5194 | 0.83     | 0.8172   | 0.1966 | 0.0590 |
| 0.0613        | 99.0  | 2475 | 0.4137          | 0.83     | 0.2631     | 1.5195 | 0.83     | 0.8172   | 0.2005 | 0.0591 |
| 0.0613        | 100.0 | 2500 | 0.4137          | 0.83     | 0.2631     | 1.5189 | 0.83     | 0.8172   | 0.2007 | 0.0591 |


### Framework versions

- Transformers 4.30.2
- Pytorch 1.13.1
- Datasets 2.13.1
- Tokenizers 0.13.3

adve email form letter memo news note report resume scientific