
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t2.5_a0.9

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6228
- Accuracy: 0.5375
- Brier Loss: 0.6009
- Nll: 2.3362
- F1 Micro: 0.5375
- F1 Macro: 0.5332
- Ece: 0.1886
- Aurc: 0.2188

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 4.0859          | 0.0425   | 1.0718     | 7.3724 | 0.0425   | 0.0340   | 0.2811 | 0.9554 |
| No log        | 2.0   | 14   | 3.1575          | 0.0925   | 0.9517     | 5.8523 | 0.0925   | 0.0709   | 0.1660 | 0.8903 |
| No log        | 3.0   | 21   | 2.8291          | 0.2325   | 0.8866     | 5.4080 | 0.2325   | 0.1824   | 0.1601 | 0.6363 |
| No log        | 4.0   | 28   | 2.4189          | 0.2875   | 0.7987     | 3.5373 | 0.2875   | 0.2714   | 0.1822 | 0.4858 |
| No log        | 5.0   | 35   | 2.1338          | 0.3775   | 0.7203     | 2.9804 | 0.3775   | 0.3569   | 0.1731 | 0.3581 |
| No log        | 6.0   | 42   | 2.0168          | 0.44     | 0.6910     | 2.9903 | 0.44     | 0.4255   | 0.1854 | 0.3060 |
| No log        | 7.0   | 49   | 2.0458          | 0.4575   | 0.6988     | 3.1988 | 0.4575   | 0.4286   | 0.2078 | 0.3000 |
| No log        | 8.0   | 56   | 1.9688          | 0.4575   | 0.6928     | 2.8691 | 0.4575   | 0.4542   | 0.2122 | 0.2986 |
| No log        | 9.0   | 63   | 2.1143          | 0.4625   | 0.7081     | 2.8507 | 0.4625   | 0.4496   | 0.2232 | 0.2940 |
| No log        | 10.0  | 70   | 2.1352          | 0.455    | 0.7242     | 2.9232 | 0.455    | 0.4317   | 0.2581 | 0.2923 |
| No log        | 11.0  | 77   | 2.2760          | 0.44     | 0.7545     | 2.9083 | 0.44     | 0.4144   | 0.2835 | 0.3144 |
| No log        | 12.0  | 84   | 2.1320          | 0.4675   | 0.7389     | 2.6277 | 0.4675   | 0.4607   | 0.2715 | 0.3142 |
| No log        | 13.0  | 91   | 2.0889          | 0.4825   | 0.7170     | 2.6786 | 0.4825   | 0.4755   | 0.2522 | 0.2933 |
| No log        | 14.0  | 98   | 2.0173          | 0.475    | 0.7020     | 2.7765 | 0.4750   | 0.4555   | 0.2593 | 0.2791 |
| No log        | 15.0  | 105  | 2.1265          | 0.4875   | 0.7124     | 3.0127 | 0.4875   | 0.4753   | 0.2458 | 0.3069 |
| No log        | 16.0  | 112  | 2.1517          | 0.4825   | 0.7184     | 2.7981 | 0.4825   | 0.4719   | 0.2505 | 0.2964 |
| No log        | 17.0  | 119  | 1.9485          | 0.49     | 0.6828     | 2.6670 | 0.49     | 0.4773   | 0.2430 | 0.2649 |
| No log        | 18.0  | 126  | 1.9434          | 0.5      | 0.6802     | 2.6131 | 0.5      | 0.4941   | 0.2267 | 0.2741 |
| No log        | 19.0  | 133  | 1.9055          | 0.4925   | 0.6741     | 2.7035 | 0.4925   | 0.4953   | 0.2201 | 0.2700 |
| No log        | 20.0  | 140  | 1.8119          | 0.5      | 0.6520     | 2.4537 | 0.5      | 0.4937   | 0.2016 | 0.2449 |
| No log        | 21.0  | 147  | 1.8420          | 0.49     | 0.6593     | 2.6244 | 0.49     | 0.4798   | 0.2100 | 0.2668 |
| No log        | 22.0  | 154  | 1.9282          | 0.5      | 0.6739     | 2.6650 | 0.5      | 0.4837   | 0.2166 | 0.2652 |
| No log        | 23.0  | 161  | 1.8139          | 0.5125   | 0.6543     | 2.4043 | 0.5125   | 0.5009   | 0.2275 | 0.2604 |
| No log        | 24.0  | 168  | 1.6997          | 0.5325   | 0.6124     | 2.3950 | 0.5325   | 0.5271   | 0.1873 | 0.2248 |
| No log        | 25.0  | 175  | 1.8370          | 0.5025   | 0.6526     | 2.4967 | 0.5025   | 0.4926   | 0.2249 | 0.2513 |
| No log        | 26.0  | 182  | 1.7508          | 0.5025   | 0.6388     | 2.4321 | 0.5025   | 0.4947   | 0.2135 | 0.2416 |
| No log        | 27.0  | 189  | 1.7354          | 0.52     | 0.6368     | 2.3706 | 0.52     | 0.5149   | 0.2101 | 0.2440 |
| No log        | 28.0  | 196  | 1.7809          | 0.52     | 0.6421     | 2.4895 | 0.52     | 0.5076   | 0.2033 | 0.2446 |
| No log        | 29.0  | 203  | 1.6770          | 0.55     | 0.6046     | 2.3913 | 0.55     | 0.5409   | 0.1849 | 0.2189 |
| No log        | 30.0  | 210  | 1.6794          | 0.5325   | 0.6127     | 2.5677 | 0.5325   | 0.5263   | 0.2095 | 0.2247 |
| No log        | 31.0  | 217  | 1.7352          | 0.525    | 0.6325     | 2.5068 | 0.525    | 0.5062   | 0.1865 | 0.2320 |
| No log        | 32.0  | 224  | 1.7396          | 0.5225   | 0.6379     | 2.4798 | 0.5225   | 0.5169   | 0.1829 | 0.2402 |
| No log        | 33.0  | 231  | 1.7013          | 0.52     | 0.6188     | 2.5032 | 0.52     | 0.5029   | 0.1986 | 0.2233 |
| No log        | 34.0  | 238  | 1.7121          | 0.5075   | 0.6313     | 2.5680 | 0.5075   | 0.5074   | 0.2249 | 0.2482 |
| No log        | 35.0  | 245  | 1.7053          | 0.5175   | 0.6368     | 2.3967 | 0.5175   | 0.5064   | 0.2088 | 0.2389 |
| No log        | 36.0  | 252  | 1.6616          | 0.55     | 0.6058     | 2.5726 | 0.55     | 0.5357   | 0.1762 | 0.2174 |
| No log        | 37.0  | 259  | 1.7302          | 0.5075   | 0.6350     | 2.3745 | 0.5075   | 0.4993   | 0.1939 | 0.2481 |
| No log        | 38.0  | 266  | 1.6741          | 0.5225   | 0.6152     | 2.5862 | 0.5225   | 0.5211   | 0.2042 | 0.2234 |
| No log        | 39.0  | 273  | 1.6960          | 0.515    | 0.6216     | 2.4210 | 0.515    | 0.5049   | 0.1776 | 0.2270 |
| No log        | 40.0  | 280  | 1.6512          | 0.54     | 0.6075     | 2.4495 | 0.54     | 0.5377   | 0.1848 | 0.2217 |
| No log        | 41.0  | 287  | 1.6477          | 0.52     | 0.6086     | 2.5354 | 0.52     | 0.5177   | 0.1976 | 0.2219 |
| No log        | 42.0  | 294  | 1.6680          | 0.52     | 0.6198     | 2.2815 | 0.52     | 0.5215   | 0.1951 | 0.2314 |
| No log        | 43.0  | 301  | 1.6153          | 0.545    | 0.5935     | 2.3952 | 0.545    | 0.5376   | 0.1721 | 0.2088 |
| No log        | 44.0  | 308  | 1.6347          | 0.5375   | 0.6023     | 2.3985 | 0.5375   | 0.5340   | 0.1778 | 0.2178 |
| No log        | 45.0  | 315  | 1.6132          | 0.5375   | 0.6001     | 2.3149 | 0.5375   | 0.5375   | 0.1902 | 0.2163 |
| No log        | 46.0  | 322  | 1.6366          | 0.5275   | 0.6102     | 2.3820 | 0.5275   | 0.5210   | 0.2057 | 0.2239 |
| No log        | 47.0  | 329  | 1.6118          | 0.55     | 0.5963     | 2.3834 | 0.55     | 0.5463   | 0.1748 | 0.2137 |
| No log        | 48.0  | 336  | 1.6208          | 0.535    | 0.6038     | 2.2855 | 0.535    | 0.5340   | 0.1791 | 0.2201 |
| No log        | 49.0  | 343  | 1.6006          | 0.545    | 0.5960     | 2.3253 | 0.545    | 0.5440   | 0.2007 | 0.2158 |
| No log        | 50.0  | 350  | 1.6184          | 0.5425   | 0.6005     | 2.3181 | 0.5425   | 0.5432   | 0.1846 | 0.2176 |
| No log        | 51.0  | 357  | 1.6215          | 0.5425   | 0.6022     | 2.3219 | 0.5425   | 0.5371   | 0.1776 | 0.2163 |
| No log        | 52.0  | 364  | 1.6068          | 0.54     | 0.5978     | 2.2928 | 0.54     | 0.5383   | 0.1865 | 0.2164 |
| No log        | 53.0  | 371  | 1.6128          | 0.535    | 0.5979     | 2.3243 | 0.535    | 0.5335   | 0.1913 | 0.2189 |
| No log        | 54.0  | 378  | 1.6182          | 0.545    | 0.6002     | 2.3604 | 0.545    | 0.5432   | 0.1740 | 0.2157 |
| No log        | 55.0  | 385  | 1.6143          | 0.54     | 0.5981     | 2.3632 | 0.54     | 0.5348   | 0.1911 | 0.2165 |
| No log        | 56.0  | 392  | 1.6163          | 0.5375   | 0.6010     | 2.3551 | 0.5375   | 0.5338   | 0.1950 | 0.2191 |
| No log        | 57.0  | 399  | 1.6125          | 0.54     | 0.5984     | 2.3597 | 0.54     | 0.5378   | 0.1680 | 0.2159 |
| No log        | 58.0  | 406  | 1.6159          | 0.5425   | 0.5997     | 2.3564 | 0.5425   | 0.5388   | 0.1796 | 0.2175 |
| No log        | 59.0  | 413  | 1.6102          | 0.5475   | 0.5967     | 2.3879 | 0.5475   | 0.5448   | 0.1863 | 0.2152 |
| No log        | 60.0  | 420  | 1.6171          | 0.535    | 0.6006     | 2.3594 | 0.535    | 0.5298   | 0.1987 | 0.2183 |
| No log        | 61.0  | 427  | 1.6162          | 0.5375   | 0.5987     | 2.3631 | 0.5375   | 0.5341   | 0.1836 | 0.2172 |
| No log        | 62.0  | 434  | 1.6167          | 0.5425   | 0.5992     | 2.3344 | 0.5425   | 0.5395   | 0.2011 | 0.2170 |
| No log        | 63.0  | 441  | 1.6123          | 0.545    | 0.5984     | 2.3575 | 0.545    | 0.5423   | 0.1946 | 0.2174 |
| No log        | 64.0  | 448  | 1.6209          | 0.535    | 0.6007     | 2.3333 | 0.535    | 0.5299   | 0.1973 | 0.2180 |
| No log        | 65.0  | 455  | 1.6184          | 0.5475   | 0.6003     | 2.3620 | 0.5475   | 0.5442   | 0.1911 | 0.2167 |
| No log        | 66.0  | 462  | 1.6179          | 0.5375   | 0.5992     | 2.3321 | 0.5375   | 0.5333   | 0.2019 | 0.2182 |
| No log        | 67.0  | 469  | 1.6179          | 0.5425   | 0.5999     | 2.3309 | 0.5425   | 0.5397   | 0.1780 | 0.2176 |
| No log        | 68.0  | 476  | 1.6204          | 0.545    | 0.6004     | 2.3282 | 0.545    | 0.5408   | 0.1740 | 0.2167 |
| No log        | 69.0  | 483  | 1.6195          | 0.54     | 0.6005     | 2.3372 | 0.54     | 0.5370   | 0.1898 | 0.2187 |
| No log        | 70.0  | 490  | 1.6189          | 0.5425   | 0.5996     | 2.3328 | 0.5425   | 0.5398   | 0.1754 | 0.2169 |
| No log        | 71.0  | 497  | 1.6202          | 0.5425   | 0.6007     | 2.3352 | 0.5425   | 0.5394   | 0.1973 | 0.2180 |
| 0.3297        | 72.0  | 504  | 1.6175          | 0.5375   | 0.5997     | 2.3297 | 0.5375   | 0.5348   | 0.1721 | 0.2184 |
| 0.3297        | 73.0  | 511  | 1.6185          | 0.545    | 0.6001     | 2.3339 | 0.545    | 0.5414   | 0.1780 | 0.2175 |
| 0.3297        | 74.0  | 518  | 1.6203          | 0.5425   | 0.6003     | 2.3315 | 0.5425   | 0.5391   | 0.1855 | 0.2175 |
| 0.3297        | 75.0  | 525  | 1.6199          | 0.54     | 0.6002     | 2.3366 | 0.54     | 0.5363   | 0.1841 | 0.2180 |
| 0.3297        | 76.0  | 532  | 1.6203          | 0.5425   | 0.6002     | 2.3332 | 0.5425   | 0.5395   | 0.1736 | 0.2176 |
| 0.3297        | 77.0  | 539  | 1.6191          | 0.5425   | 0.6000     | 2.3328 | 0.5425   | 0.5395   | 0.1722 | 0.2176 |
| 0.3297        | 78.0  | 546  | 1.6202          | 0.54     | 0.6001     | 2.3317 | 0.54     | 0.5362   | 0.1909 | 0.2177 |
| 0.3297        | 79.0  | 553  | 1.6199          | 0.54     | 0.6002     | 2.3335 | 0.54     | 0.5370   | 0.1982 | 0.2181 |
| 0.3297        | 80.0  | 560  | 1.6207          | 0.54     | 0.6006     | 2.3336 | 0.54     | 0.5363   | 0.1983 | 0.2185 |
| 0.3297        | 81.0  | 567  | 1.6210          | 0.5425   | 0.6003     | 2.3353 | 0.5425   | 0.5389   | 0.1875 | 0.2174 |
| 0.3297        | 82.0  | 574  | 1.6216          | 0.5425   | 0.6005     | 2.3340 | 0.5425   | 0.5386   | 0.1792 | 0.2178 |
| 0.3297        | 83.0  | 581  | 1.6213          | 0.5375   | 0.6007     | 2.3360 | 0.5375   | 0.5340   | 0.1797 | 0.2193 |
| 0.3297        | 84.0  | 588  | 1.6208          | 0.54     | 0.6007     | 2.3345 | 0.54     | 0.5363   | 0.1848 | 0.2189 |
| 0.3297        | 85.0  | 595  | 1.6221          | 0.54     | 0.6005     | 2.3360 | 0.54     | 0.5369   | 0.1836 | 0.2184 |
| 0.3297        | 86.0  | 602  | 1.6218          | 0.5425   | 0.6005     | 2.3342 | 0.5425   | 0.5391   | 0.1922 | 0.2178 |
| 0.3297        | 87.0  | 609  | 1.6224          | 0.54     | 0.6006     | 2.3365 | 0.54     | 0.5363   | 0.1849 | 0.2185 |
| 0.3297        | 88.0  | 616  | 1.6216          | 0.54     | 0.6005     | 2.3355 | 0.54     | 0.5363   | 0.1912 | 0.2188 |
| 0.3297        | 89.0  | 623  | 1.6222          | 0.535    | 0.6008     | 2.3359 | 0.535    | 0.5310   | 0.1890 | 0.2195 |
| 0.3297        | 90.0  | 630  | 1.6218          | 0.5425   | 0.6007     | 2.3356 | 0.5425   | 0.5386   | 0.1930 | 0.2179 |
| 0.3297        | 91.0  | 637  | 1.6226          | 0.54     | 0.6009     | 2.3363 | 0.54     | 0.5363   | 0.1941 | 0.2188 |
| 0.3297        | 92.0  | 644  | 1.6224          | 0.54     | 0.6009     | 2.3360 | 0.54     | 0.5363   | 0.1890 | 0.2188 |
| 0.3297        | 93.0  | 651  | 1.6226          | 0.5425   | 0.6009     | 2.3359 | 0.5425   | 0.5386   | 0.1964 | 0.2179 |
| 0.3297        | 94.0  | 658  | 1.6227          | 0.54     | 0.6009     | 2.3363 | 0.54     | 0.5363   | 0.1877 | 0.2187 |
| 0.3297        | 95.0  | 665  | 1.6226          | 0.54     | 0.6008     | 2.3360 | 0.54     | 0.5363   | 0.1916 | 0.2186 |
| 0.3297        | 96.0  | 672  | 1.6225          | 0.54     | 0.6008     | 2.3362 | 0.54     | 0.5363   | 0.1972 | 0.2186 |
| 0.3297        | 97.0  | 679  | 1.6227          | 0.5375   | 0.6009     | 2.3362 | 0.5375   | 0.5332   | 0.1949 | 0.2188 |
| 0.3297        | 98.0  | 686  | 1.6228          | 0.5375   | 0.6009     | 2.3362 | 0.5375   | 0.5332   | 0.1924 | 0.2188 |
| 0.3297        | 99.0  | 693  | 1.6228          | 0.5375   | 0.6009     | 2.3363 | 0.5375   | 0.5332   | 0.1924 | 0.2189 |
| 0.3297        | 100.0 | 700  | 1.6228          | 0.5375   | 0.6009     | 2.3362 | 0.5375   | 0.5332   | 0.1886 | 0.2188 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo