
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t5.0_a0.9

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5750
- Accuracy: 0.5325
- Brier Loss: 0.5990
- Nll: 2.5263
- F1 Micro: 0.5325
- F1 Macro: 0.5240
- Ece: 0.1659
- Aurc: 0.2152

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 3.9285          | 0.04     | 1.0722     | 7.3572 | 0.04     | 0.0319   | 0.2792 | 0.9556 |
| No log        | 2.0   | 14   | 3.0027          | 0.095    | 0.9510     | 5.8779 | 0.095    | 0.0766   | 0.1668 | 0.8900 |
| No log        | 3.0   | 21   | 2.6988          | 0.225    | 0.8896     | 5.3750 | 0.225    | 0.1801   | 0.1669 | 0.6473 |
| No log        | 4.0   | 28   | 2.3179          | 0.285    | 0.8016     | 3.5658 | 0.285    | 0.2657   | 0.1679 | 0.4916 |
| No log        | 5.0   | 35   | 2.0566          | 0.37     | 0.7203     | 2.9834 | 0.37     | 0.3493   | 0.1684 | 0.3612 |
| No log        | 6.0   | 42   | 1.9505          | 0.4325   | 0.6892     | 3.0719 | 0.4325   | 0.4127   | 0.1775 | 0.3084 |
| No log        | 7.0   | 49   | 1.9995          | 0.4375   | 0.7008     | 3.1569 | 0.4375   | 0.4084   | 0.2032 | 0.3103 |
| No log        | 8.0   | 56   | 1.9133          | 0.445    | 0.6906     | 2.8574 | 0.445    | 0.4464   | 0.2016 | 0.3062 |
| No log        | 9.0   | 63   | 1.9876          | 0.4625   | 0.6918     | 2.9267 | 0.4625   | 0.4538   | 0.2228 | 0.2868 |
| No log        | 10.0  | 70   | 2.0051          | 0.4725   | 0.6971     | 2.9249 | 0.4725   | 0.4553   | 0.2234 | 0.2814 |
| No log        | 11.0  | 77   | 2.1834          | 0.465    | 0.7319     | 2.9998 | 0.465    | 0.4426   | 0.2444 | 0.3006 |
| No log        | 12.0  | 84   | 1.9953          | 0.4825   | 0.7087     | 2.7128 | 0.4825   | 0.4731   | 0.2386 | 0.2980 |
| No log        | 13.0  | 91   | 1.8834          | 0.4975   | 0.6771     | 2.6879 | 0.4975   | 0.4954   | 0.2240 | 0.2748 |
| No log        | 14.0  | 98   | 1.9647          | 0.4675   | 0.6987     | 2.8305 | 0.4675   | 0.4429   | 0.2409 | 0.2902 |
| No log        | 15.0  | 105  | 1.8810          | 0.5      | 0.6785     | 2.6402 | 0.5      | 0.4847   | 0.2171 | 0.2725 |
| No log        | 16.0  | 112  | 1.8777          | 0.4875   | 0.6877     | 2.6940 | 0.4875   | 0.4871   | 0.2210 | 0.2846 |
| No log        | 17.0  | 119  | 1.9260          | 0.4925   | 0.6796     | 2.7055 | 0.4925   | 0.4834   | 0.2012 | 0.2744 |
| No log        | 18.0  | 126  | 1.7864          | 0.505    | 0.6547     | 2.6724 | 0.505    | 0.4912   | 0.2081 | 0.2434 |
| No log        | 19.0  | 133  | 1.7618          | 0.4975   | 0.6430     | 2.5951 | 0.4975   | 0.4915   | 0.2172 | 0.2490 |
| No log        | 20.0  | 140  | 1.7496          | 0.515    | 0.6513     | 2.5263 | 0.515    | 0.5025   | 0.1975 | 0.2502 |
| No log        | 21.0  | 147  | 1.7082          | 0.5275   | 0.6438     | 2.4039 | 0.5275   | 0.5224   | 0.2017 | 0.2450 |
| No log        | 22.0  | 154  | 1.7482          | 0.4975   | 0.6682     | 2.5194 | 0.4975   | 0.4911   | 0.2247 | 0.2571 |
| No log        | 23.0  | 161  | 1.7377          | 0.5075   | 0.6482     | 2.4136 | 0.5075   | 0.4900   | 0.2221 | 0.2396 |
| No log        | 24.0  | 168  | 1.7094          | 0.515    | 0.6372     | 2.5605 | 0.515    | 0.5083   | 0.2137 | 0.2474 |
| No log        | 25.0  | 175  | 1.6884          | 0.5175   | 0.6422     | 2.5270 | 0.5175   | 0.5104   | 0.2111 | 0.2444 |
| No log        | 26.0  | 182  | 1.6489          | 0.5275   | 0.6246     | 2.5344 | 0.5275   | 0.5211   | 0.2066 | 0.2333 |
| No log        | 27.0  | 189  | 1.6165          | 0.53     | 0.6191     | 2.5418 | 0.53     | 0.5256   | 0.2021 | 0.2305 |
| No log        | 28.0  | 196  | 1.6316          | 0.5275   | 0.6181     | 2.6568 | 0.5275   | 0.5212   | 0.2004 | 0.2300 |
| No log        | 29.0  | 203  | 1.6595          | 0.5175   | 0.6306     | 2.4298 | 0.5175   | 0.5096   | 0.2020 | 0.2427 |
| No log        | 30.0  | 210  | 1.6193          | 0.5325   | 0.6157     | 2.5455 | 0.5325   | 0.5272   | 0.1779 | 0.2278 |
| No log        | 31.0  | 217  | 1.6517          | 0.5325   | 0.6274     | 2.4579 | 0.5325   | 0.5259   | 0.2006 | 0.2362 |
| No log        | 32.0  | 224  | 1.6434          | 0.5325   | 0.6167     | 2.5805 | 0.5325   | 0.5229   | 0.1995 | 0.2273 |
| No log        | 33.0  | 231  | 1.6660          | 0.5225   | 0.6269     | 2.6794 | 0.5225   | 0.5132   | 0.2244 | 0.2283 |
| No log        | 34.0  | 238  | 1.6353          | 0.515    | 0.6194     | 2.6085 | 0.515    | 0.5069   | 0.1839 | 0.2303 |
| No log        | 35.0  | 245  | 1.5920          | 0.5325   | 0.6051     | 2.5645 | 0.5325   | 0.5248   | 0.1868 | 0.2208 |
| No log        | 36.0  | 252  | 1.5909          | 0.54     | 0.6028     | 2.4786 | 0.54     | 0.5323   | 0.1902 | 0.2194 |
| No log        | 37.0  | 259  | 1.5730          | 0.5425   | 0.5983     | 2.4877 | 0.5425   | 0.5368   | 0.1799 | 0.2177 |
| No log        | 38.0  | 266  | 1.5800          | 0.535    | 0.6029     | 2.4736 | 0.535    | 0.5282   | 0.1761 | 0.2196 |
| No log        | 39.0  | 273  | 1.5594          | 0.54     | 0.5955     | 2.5093 | 0.54     | 0.5327   | 0.1900 | 0.2126 |
| No log        | 40.0  | 280  | 1.5685          | 0.53     | 0.5979     | 2.6068 | 0.53     | 0.5208   | 0.1893 | 0.2173 |
| No log        | 41.0  | 287  | 1.5757          | 0.53     | 0.5995     | 2.5655 | 0.53     | 0.5218   | 0.1862 | 0.2164 |
| No log        | 42.0  | 294  | 1.5797          | 0.535    | 0.6039     | 2.5445 | 0.535    | 0.5273   | 0.1834 | 0.2182 |
| No log        | 43.0  | 301  | 1.5900          | 0.53     | 0.6074     | 2.5201 | 0.53     | 0.5189   | 0.1747 | 0.2206 |
| No log        | 44.0  | 308  | 1.5760          | 0.5325   | 0.5986     | 2.4974 | 0.5325   | 0.5225   | 0.1870 | 0.2148 |
| No log        | 45.0  | 315  | 1.5768          | 0.53     | 0.6013     | 2.5174 | 0.53     | 0.5204   | 0.1979 | 0.2158 |
| No log        | 46.0  | 322  | 1.5774          | 0.53     | 0.6011     | 2.5199 | 0.53     | 0.5206   | 0.1882 | 0.2165 |
| No log        | 47.0  | 329  | 1.5714          | 0.54     | 0.5983     | 2.5329 | 0.54     | 0.5303   | 0.1884 | 0.2135 |
| No log        | 48.0  | 336  | 1.5834          | 0.5325   | 0.6026     | 2.5253 | 0.5325   | 0.5238   | 0.1658 | 0.2190 |
| No log        | 49.0  | 343  | 1.5724          | 0.5375   | 0.5979     | 2.5569 | 0.5375   | 0.5299   | 0.1617 | 0.2151 |
| No log        | 50.0  | 350  | 1.5685          | 0.5375   | 0.5985     | 2.5189 | 0.5375   | 0.5285   | 0.1919 | 0.2151 |
| No log        | 51.0  | 357  | 1.5708          | 0.54     | 0.5986     | 2.5002 | 0.54     | 0.5305   | 0.1755 | 0.2149 |
| No log        | 52.0  | 364  | 1.5665          | 0.535    | 0.5977     | 2.5224 | 0.535    | 0.5267   | 0.1842 | 0.2160 |
| No log        | 53.0  | 371  | 1.5713          | 0.5325   | 0.5993     | 2.5515 | 0.5325   | 0.5250   | 0.1753 | 0.2160 |
| No log        | 54.0  | 378  | 1.5693          | 0.535    | 0.5986     | 2.5516 | 0.535    | 0.5276   | 0.1841 | 0.2158 |
| No log        | 55.0  | 385  | 1.5693          | 0.5375   | 0.5984     | 2.5190 | 0.5375   | 0.5285   | 0.1842 | 0.2144 |
| No log        | 56.0  | 392  | 1.5725          | 0.535    | 0.5992     | 2.5527 | 0.535    | 0.5262   | 0.1776 | 0.2150 |
| No log        | 57.0  | 399  | 1.5674          | 0.5425   | 0.5976     | 2.5502 | 0.5425   | 0.5326   | 0.1902 | 0.2137 |
| No log        | 58.0  | 406  | 1.5675          | 0.5375   | 0.5974     | 2.5517 | 0.5375   | 0.5288   | 0.1794 | 0.2139 |
| No log        | 59.0  | 413  | 1.5713          | 0.535    | 0.5988     | 2.5515 | 0.535    | 0.5257   | 0.1791 | 0.2147 |
| No log        | 60.0  | 420  | 1.5729          | 0.535    | 0.5988     | 2.5512 | 0.535    | 0.5262   | 0.1796 | 0.2148 |
| No log        | 61.0  | 427  | 1.5702          | 0.5375   | 0.5976     | 2.5521 | 0.5375   | 0.5281   | 0.1817 | 0.2139 |
| No log        | 62.0  | 434  | 1.5728          | 0.535    | 0.5988     | 2.5514 | 0.535    | 0.5266   | 0.1722 | 0.2149 |
| No log        | 63.0  | 441  | 1.5720          | 0.5325   | 0.5985     | 2.5206 | 0.5325   | 0.5231   | 0.1790 | 0.2149 |
| No log        | 64.0  | 448  | 1.5704          | 0.5325   | 0.5975     | 2.5510 | 0.5325   | 0.5236   | 0.1706 | 0.2139 |
| No log        | 65.0  | 455  | 1.5724          | 0.5325   | 0.5986     | 2.5225 | 0.5325   | 0.5236   | 0.1557 | 0.2148 |
| No log        | 66.0  | 462  | 1.5718          | 0.5325   | 0.5985     | 2.5246 | 0.5325   | 0.5241   | 0.1772 | 0.2148 |
| No log        | 67.0  | 469  | 1.5710          | 0.5325   | 0.5981     | 2.5511 | 0.5325   | 0.5237   | 0.1625 | 0.2146 |
| No log        | 68.0  | 476  | 1.5716          | 0.54     | 0.5981     | 2.5001 | 0.54     | 0.5304   | 0.1622 | 0.2141 |
| No log        | 69.0  | 483  | 1.5732          | 0.5325   | 0.5988     | 2.5517 | 0.5325   | 0.5232   | 0.1641 | 0.2150 |
| No log        | 70.0  | 490  | 1.5733          | 0.5325   | 0.5987     | 2.5522 | 0.5325   | 0.5237   | 0.1715 | 0.2149 |
| No log        | 71.0  | 497  | 1.5729          | 0.5325   | 0.5985     | 2.5523 | 0.5325   | 0.5241   | 0.1670 | 0.2147 |
| 0.3153        | 72.0  | 504  | 1.5730          | 0.5325   | 0.5987     | 2.5236 | 0.5325   | 0.5237   | 0.1656 | 0.2149 |
| 0.3153        | 73.0  | 511  | 1.5723          | 0.5325   | 0.5985     | 2.5212 | 0.5325   | 0.5238   | 0.1893 | 0.2145 |
| 0.3153        | 74.0  | 518  | 1.5738          | 0.5325   | 0.5989     | 2.5515 | 0.5325   | 0.5238   | 0.1744 | 0.2147 |
| 0.3153        | 75.0  | 525  | 1.5740          | 0.5325   | 0.5988     | 2.5318 | 0.5325   | 0.5237   | 0.1683 | 0.2150 |
| 0.3153        | 76.0  | 532  | 1.5734          | 0.535    | 0.5985     | 2.5525 | 0.535    | 0.5261   | 0.1763 | 0.2145 |
| 0.3153        | 77.0  | 539  | 1.5740          | 0.5325   | 0.5989     | 2.5516 | 0.5325   | 0.5243   | 0.1726 | 0.2149 |
| 0.3153        | 78.0  | 546  | 1.5738          | 0.5325   | 0.5987     | 2.5289 | 0.5325   | 0.5241   | 0.1692 | 0.2148 |
| 0.3153        | 79.0  | 553  | 1.5736          | 0.5325   | 0.5987     | 2.5255 | 0.5325   | 0.5242   | 0.1807 | 0.2147 |
| 0.3153        | 80.0  | 560  | 1.5739          | 0.5325   | 0.5988     | 2.5522 | 0.5325   | 0.5237   | 0.1769 | 0.2150 |
| 0.3153        | 81.0  | 567  | 1.5743          | 0.5325   | 0.5989     | 2.5519 | 0.5325   | 0.5238   | 0.1837 | 0.2151 |
| 0.3153        | 82.0  | 574  | 1.5742          | 0.5325   | 0.5989     | 2.5232 | 0.5325   | 0.5240   | 0.1712 | 0.2149 |
| 0.3153        | 83.0  | 581  | 1.5744          | 0.5325   | 0.5989     | 2.5256 | 0.5325   | 0.5239   | 0.1803 | 0.2151 |
| 0.3153        | 84.0  | 588  | 1.5741          | 0.5325   | 0.5988     | 2.5233 | 0.5325   | 0.5233   | 0.1655 | 0.2147 |
| 0.3153        | 85.0  | 595  | 1.5747          | 0.5325   | 0.5990     | 2.5274 | 0.5325   | 0.5237   | 0.1696 | 0.2152 |
| 0.3153        | 86.0  | 602  | 1.5747          | 0.5325   | 0.5989     | 2.5263 | 0.5325   | 0.5238   | 0.1689 | 0.2150 |
| 0.3153        | 87.0  | 609  | 1.5745          | 0.5325   | 0.5989     | 2.5251 | 0.5325   | 0.5237   | 0.1654 | 0.2149 |
| 0.3153        | 88.0  | 616  | 1.5747          | 0.5325   | 0.5989     | 2.5283 | 0.5325   | 0.5241   | 0.1693 | 0.2151 |
| 0.3153        | 89.0  | 623  | 1.5748          | 0.5325   | 0.5990     | 2.5275 | 0.5325   | 0.5239   | 0.1596 | 0.2152 |
| 0.3153        | 90.0  | 630  | 1.5749          | 0.5325   | 0.5990     | 2.5278 | 0.5325   | 0.5240   | 0.1602 | 0.2151 |
| 0.3153        | 91.0  | 637  | 1.5750          | 0.5325   | 0.5990     | 2.5337 | 0.5325   | 0.5239   | 0.1623 | 0.2152 |
| 0.3153        | 92.0  | 644  | 1.5749          | 0.5325   | 0.5990     | 2.5272 | 0.5325   | 0.5238   | 0.1653 | 0.2151 |
| 0.3153        | 93.0  | 651  | 1.5751          | 0.5325   | 0.5990     | 2.5281 | 0.5325   | 0.5240   | 0.1663 | 0.2149 |
| 0.3153        | 94.0  | 658  | 1.5750          | 0.5325   | 0.5990     | 2.5249 | 0.5325   | 0.5239   | 0.1715 | 0.2152 |
| 0.3153        | 95.0  | 665  | 1.5749          | 0.535    | 0.5990     | 2.5257 | 0.535    | 0.5263   | 0.1625 | 0.2149 |
| 0.3153        | 96.0  | 672  | 1.5750          | 0.5325   | 0.5990     | 2.5266 | 0.5325   | 0.5239   | 0.1655 | 0.2151 |
| 0.3153        | 97.0  | 679  | 1.5750          | 0.5325   | 0.5990     | 2.5268 | 0.5325   | 0.5239   | 0.1686 | 0.2152 |
| 0.3153        | 98.0  | 686  | 1.5750          | 0.5325   | 0.5990     | 2.5275 | 0.5325   | 0.5240   | 0.1664 | 0.2152 |
| 0.3153        | 99.0  | 693  | 1.5750          | 0.5325   | 0.5990     | 2.5269 | 0.5325   | 0.5240   | 0.1678 | 0.2152 |
| 0.3153        | 100.0 | 700  | 1.5750          | 0.5325   | 0.5990     | 2.5263 | 0.5325   | 0.5240   | 0.1659 | 0.2152 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo