
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.9682
- Accuracy: 0.5575
- Brier Loss: 0.5680
- Nll: 2.3526
- F1 Micro: 0.5575
- F1 Macro: 0.5516
- Ece: 0.1676
- Aurc: 0.1973

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 5.4578          | 0.045    | 1.0710     | 7.3211 | 0.045    | 0.0342   | 0.2911 | 0.9529 |
| No log        | 2.0   | 14   | 4.3718          | 0.0975   | 0.9466     | 5.6031 | 0.0975   | 0.0893   | 0.1665 | 0.8820 |
| No log        | 3.0   | 21   | 3.9389          | 0.2325   | 0.8820     | 5.3266 | 0.2325   | 0.1837   | 0.1656 | 0.6302 |
| No log        | 4.0   | 28   | 3.3251          | 0.3075   | 0.7848     | 3.6819 | 0.3075   | 0.2963   | 0.1644 | 0.4646 |
| No log        | 5.0   | 35   | 2.8980          | 0.4025   | 0.7180     | 2.9699 | 0.4025   | 0.3740   | 0.1867 | 0.3388 |
| No log        | 6.0   | 42   | 2.7055          | 0.4375   | 0.7010     | 3.2343 | 0.4375   | 0.4029   | 0.2096 | 0.3042 |
| No log        | 7.0   | 49   | 2.6466          | 0.4575   | 0.7040     | 2.9977 | 0.4575   | 0.4272   | 0.2082 | 0.2999 |
| No log        | 8.0   | 56   | 2.5399          | 0.4775   | 0.6809     | 2.8098 | 0.4775   | 0.4693   | 0.2106 | 0.2837 |
| No log        | 9.0   | 63   | 2.5949          | 0.49     | 0.6824     | 2.6503 | 0.49     | 0.4827   | 0.2289 | 0.2718 |
| No log        | 10.0  | 70   | 2.6997          | 0.4725   | 0.7318     | 2.8289 | 0.4725   | 0.4523   | 0.2536 | 0.3056 |
| No log        | 11.0  | 77   | 2.5694          | 0.47     | 0.7040     | 2.7969 | 0.47     | 0.4534   | 0.2348 | 0.2766 |
| No log        | 12.0  | 84   | 2.4539          | 0.4975   | 0.6762     | 2.6929 | 0.4975   | 0.4831   | 0.2300 | 0.2664 |
| No log        | 13.0  | 91   | 2.4841          | 0.5025   | 0.6664     | 2.6140 | 0.5025   | 0.4944   | 0.2098 | 0.2584 |
| No log        | 14.0  | 98   | 2.2755          | 0.535    | 0.6410     | 2.4991 | 0.535    | 0.5249   | 0.1905 | 0.2405 |
| No log        | 15.0  | 105  | 2.2998          | 0.5125   | 0.6282     | 2.6561 | 0.5125   | 0.4982   | 0.1916 | 0.2371 |
| No log        | 16.0  | 112  | 2.2156          | 0.525    | 0.6195     | 2.4837 | 0.525    | 0.5191   | 0.1870 | 0.2232 |
| No log        | 17.0  | 119  | 2.1862          | 0.5225   | 0.6096     | 2.7252 | 0.5225   | 0.5202   | 0.1747 | 0.2292 |
| No log        | 18.0  | 126  | 2.2058          | 0.5375   | 0.6160     | 2.5446 | 0.5375   | 0.5154   | 0.2011 | 0.2231 |
| No log        | 19.0  | 133  | 2.2147          | 0.5375   | 0.6143     | 2.5720 | 0.5375   | 0.5232   | 0.2028 | 0.2221 |
| No log        | 20.0  | 140  | 2.1791          | 0.525    | 0.6191     | 2.4505 | 0.525    | 0.5107   | 0.1918 | 0.2233 |
| No log        | 21.0  | 147  | 2.1165          | 0.535    | 0.5960     | 2.5369 | 0.535    | 0.5280   | 0.1867 | 0.2133 |
| No log        | 22.0  | 154  | 2.1193          | 0.54     | 0.6009     | 2.5568 | 0.54     | 0.5313   | 0.2049 | 0.2184 |
| No log        | 23.0  | 161  | 2.1082          | 0.5425   | 0.5929     | 2.5238 | 0.5425   | 0.5360   | 0.1691 | 0.2099 |
| No log        | 24.0  | 168  | 2.1221          | 0.535    | 0.6115     | 2.4854 | 0.535    | 0.5252   | 0.1779 | 0.2234 |
| No log        | 25.0  | 175  | 2.1912          | 0.52     | 0.6295     | 2.4975 | 0.52     | 0.5109   | 0.1970 | 0.2339 |
| No log        | 26.0  | 182  | 2.1056          | 0.5225   | 0.6150     | 2.4697 | 0.5225   | 0.5250   | 0.2020 | 0.2346 |
| No log        | 27.0  | 189  | 2.1017          | 0.535    | 0.6027     | 2.4992 | 0.535    | 0.5399   | 0.2003 | 0.2173 |
| No log        | 28.0  | 196  | 2.0999          | 0.545    | 0.5929     | 2.6313 | 0.545    | 0.5306   | 0.1844 | 0.2126 |
| No log        | 29.0  | 203  | 2.1188          | 0.54     | 0.6044     | 2.5420 | 0.54     | 0.5211   | 0.1745 | 0.2159 |
| No log        | 30.0  | 210  | 2.0670          | 0.56     | 0.5938     | 2.4868 | 0.56     | 0.5500   | 0.1849 | 0.2132 |
| No log        | 31.0  | 217  | 2.0709          | 0.5525   | 0.5937     | 2.4206 | 0.5525   | 0.5489   | 0.1759 | 0.2140 |
| No log        | 32.0  | 224  | 2.0390          | 0.5675   | 0.5794     | 2.5007 | 0.5675   | 0.5547   | 0.1745 | 0.2032 |
| No log        | 33.0  | 231  | 2.0725          | 0.5375   | 0.5912     | 2.4328 | 0.5375   | 0.5237   | 0.1817 | 0.2157 |
| No log        | 34.0  | 238  | 2.0644          | 0.565    | 0.5950     | 2.5103 | 0.565    | 0.5560   | 0.1712 | 0.2085 |
| No log        | 35.0  | 245  | 2.0665          | 0.5575   | 0.5918     | 2.4401 | 0.5575   | 0.5519   | 0.1604 | 0.2133 |
| No log        | 36.0  | 252  | 2.0485          | 0.53     | 0.5977     | 2.4321 | 0.53     | 0.5259   | 0.1927 | 0.2163 |
| No log        | 37.0  | 259  | 2.0352          | 0.555    | 0.5780     | 2.4665 | 0.555    | 0.5450   | 0.1701 | 0.2000 |
| No log        | 38.0  | 266  | 2.0423          | 0.5425   | 0.5777     | 2.5061 | 0.5425   | 0.5327   | 0.1689 | 0.2021 |
| No log        | 39.0  | 273  | 2.0138          | 0.5625   | 0.5799     | 2.3714 | 0.5625   | 0.5581   | 0.1705 | 0.2060 |
| No log        | 40.0  | 280  | 2.0161          | 0.5525   | 0.5918     | 2.3704 | 0.5525   | 0.5439   | 0.1720 | 0.2111 |
| No log        | 41.0  | 287  | 2.0315          | 0.545    | 0.5843     | 2.4582 | 0.545    | 0.5355   | 0.1782 | 0.2083 |
| No log        | 42.0  | 294  | 2.0157          | 0.545    | 0.5861     | 2.5374 | 0.545    | 0.5386   | 0.1995 | 0.2129 |
| No log        | 43.0  | 301  | 2.0495          | 0.555    | 0.5922     | 2.4841 | 0.555    | 0.5393   | 0.1538 | 0.2138 |
| No log        | 44.0  | 308  | 2.0293          | 0.5525   | 0.5824     | 2.4853 | 0.5525   | 0.5352   | 0.1745 | 0.2042 |
| No log        | 45.0  | 315  | 2.0253          | 0.5575   | 0.5776     | 2.4516 | 0.5575   | 0.5421   | 0.1978 | 0.2045 |
| No log        | 46.0  | 322  | 2.0246          | 0.5525   | 0.5953     | 2.4196 | 0.5525   | 0.5362   | 0.1715 | 0.2122 |
| No log        | 47.0  | 329  | 2.0114          | 0.555    | 0.5730     | 2.4431 | 0.555    | 0.5462   | 0.1759 | 0.2008 |
| No log        | 48.0  | 336  | 2.0046          | 0.5575   | 0.5801     | 2.3784 | 0.5575   | 0.5445   | 0.1703 | 0.2039 |
| No log        | 49.0  | 343  | 1.9721          | 0.565    | 0.5672     | 2.5034 | 0.565    | 0.5594   | 0.1686 | 0.1963 |
| No log        | 50.0  | 350  | 1.9872          | 0.565    | 0.5704     | 2.4067 | 0.565    | 0.5620   | 0.1888 | 0.2000 |
| No log        | 51.0  | 357  | 1.9668          | 0.5725   | 0.5695     | 2.3935 | 0.5725   | 0.5711   | 0.1534 | 0.1981 |
| No log        | 52.0  | 364  | 1.9796          | 0.5525   | 0.5742     | 2.3977 | 0.5525   | 0.5504   | 0.1614 | 0.2023 |
| No log        | 53.0  | 371  | 2.0086          | 0.56     | 0.5835     | 2.4361 | 0.56     | 0.5510   | 0.1912 | 0.2098 |
| No log        | 54.0  | 378  | 1.9998          | 0.54     | 0.5776     | 2.4292 | 0.54     | 0.5270   | 0.1679 | 0.2042 |
| No log        | 55.0  | 385  | 1.9736          | 0.555    | 0.5732     | 2.3619 | 0.555    | 0.5427   | 0.1830 | 0.2024 |
| No log        | 56.0  | 392  | 1.9850          | 0.57     | 0.5691     | 2.4491 | 0.57     | 0.5670   | 0.1798 | 0.1967 |
| No log        | 57.0  | 399  | 1.9680          | 0.5675   | 0.5680     | 2.4863 | 0.5675   | 0.5624   | 0.1644 | 0.1967 |
| No log        | 58.0  | 406  | 1.9633          | 0.565    | 0.5682     | 2.4407 | 0.565    | 0.5573   | 0.1576 | 0.1960 |
| No log        | 59.0  | 413  | 1.9715          | 0.555    | 0.5750     | 2.4037 | 0.555    | 0.5459   | 0.1753 | 0.2027 |
| No log        | 60.0  | 420  | 1.9791          | 0.575    | 0.5684     | 2.3886 | 0.575    | 0.5732   | 0.1647 | 0.1969 |
| No log        | 61.0  | 427  | 1.9587          | 0.5675   | 0.5669     | 2.4318 | 0.5675   | 0.5616   | 0.1583 | 0.1948 |
| No log        | 62.0  | 434  | 1.9735          | 0.5575   | 0.5699     | 2.3758 | 0.5575   | 0.5550   | 0.1442 | 0.1978 |
| No log        | 63.0  | 441  | 1.9720          | 0.5475   | 0.5748     | 2.4373 | 0.5475   | 0.5405   | 0.1660 | 0.2004 |
| No log        | 64.0  | 448  | 1.9825          | 0.55     | 0.5710     | 2.3517 | 0.55     | 0.5490   | 0.1810 | 0.2030 |
| No log        | 65.0  | 455  | 1.9679          | 0.5725   | 0.5705     | 2.4311 | 0.5725   | 0.5718   | 0.1690 | 0.1980 |
| No log        | 66.0  | 462  | 1.9735          | 0.565    | 0.5706     | 2.4810 | 0.565    | 0.5624   | 0.1740 | 0.1976 |
| No log        | 67.0  | 469  | 1.9752          | 0.5675   | 0.5689     | 2.3718 | 0.5675   | 0.5615   | 0.1681 | 0.1980 |
| No log        | 68.0  | 476  | 1.9798          | 0.565    | 0.5690     | 2.4235 | 0.565    | 0.5602   | 0.1671 | 0.1980 |
| No log        | 69.0  | 483  | 1.9720          | 0.5625   | 0.5681     | 2.3483 | 0.5625   | 0.5576   | 0.1845 | 0.1970 |
| No log        | 70.0  | 490  | 1.9640          | 0.5675   | 0.5673     | 2.3772 | 0.5675   | 0.5639   | 0.1621 | 0.1962 |
| No log        | 71.0  | 497  | 1.9641          | 0.5625   | 0.5680     | 2.3925 | 0.5625   | 0.5567   | 0.1670 | 0.1963 |
| 0.4915        | 72.0  | 504  | 1.9753          | 0.5625   | 0.5707     | 2.4507 | 0.5625   | 0.5566   | 0.1780 | 0.1986 |
| 0.4915        | 73.0  | 511  | 1.9792          | 0.56     | 0.5700     | 2.3604 | 0.56     | 0.5545   | 0.1580 | 0.1990 |
| 0.4915        | 74.0  | 518  | 1.9679          | 0.55     | 0.5700     | 2.3519 | 0.55     | 0.5412   | 0.1781 | 0.1981 |
| 0.4915        | 75.0  | 525  | 1.9711          | 0.57     | 0.5685     | 2.4204 | 0.57     | 0.5676   | 0.1891 | 0.1962 |
| 0.4915        | 76.0  | 532  | 1.9705          | 0.565    | 0.5686     | 2.3512 | 0.565    | 0.5600   | 0.1684 | 0.1967 |
| 0.4915        | 77.0  | 539  | 1.9673          | 0.5625   | 0.5685     | 2.3481 | 0.5625   | 0.5577   | 0.1761 | 0.1968 |
| 0.4915        | 78.0  | 546  | 1.9641          | 0.5625   | 0.5668     | 2.3519 | 0.5625   | 0.5577   | 0.1747 | 0.1964 |
| 0.4915        | 79.0  | 553  | 1.9685          | 0.5675   | 0.5677     | 2.3833 | 0.5675   | 0.5630   | 0.1490 | 0.1967 |
| 0.4915        | 80.0  | 560  | 1.9722          | 0.5625   | 0.5691     | 2.3864 | 0.5625   | 0.5565   | 0.1583 | 0.1979 |
| 0.4915        | 81.0  | 567  | 1.9641          | 0.5625   | 0.5675     | 2.3528 | 0.5625   | 0.5566   | 0.1820 | 0.1965 |
| 0.4915        | 82.0  | 574  | 1.9666          | 0.5625   | 0.5678     | 2.3516 | 0.5625   | 0.5564   | 0.1622 | 0.1968 |
| 0.4915        | 83.0  | 581  | 1.9703          | 0.56     | 0.5679     | 2.3564 | 0.56     | 0.5550   | 0.1577 | 0.1967 |
| 0.4915        | 84.0  | 588  | 1.9666          | 0.56     | 0.5673     | 2.3532 | 0.56     | 0.5546   | 0.1633 | 0.1967 |
| 0.4915        | 85.0  | 595  | 1.9683          | 0.56     | 0.5682     | 2.3506 | 0.56     | 0.5539   | 0.1526 | 0.1971 |
| 0.4915        | 86.0  | 602  | 1.9673          | 0.5575   | 0.5674     | 2.3523 | 0.5575   | 0.5516   | 0.1642 | 0.1972 |
| 0.4915        | 87.0  | 609  | 1.9678          | 0.56     | 0.5678     | 2.3514 | 0.56     | 0.5539   | 0.1543 | 0.1970 |
| 0.4915        | 88.0  | 616  | 1.9680          | 0.56     | 0.5678     | 2.3531 | 0.56     | 0.5539   | 0.1581 | 0.1970 |
| 0.4915        | 89.0  | 623  | 1.9683          | 0.5575   | 0.5681     | 2.3532 | 0.5575   | 0.5516   | 0.1732 | 0.1976 |
| 0.4915        | 90.0  | 630  | 1.9677          | 0.56     | 0.5677     | 2.3536 | 0.56     | 0.5539   | 0.1702 | 0.1970 |
| 0.4915        | 91.0  | 637  | 1.9680          | 0.5575   | 0.5679     | 2.3528 | 0.5575   | 0.5515   | 0.1760 | 0.1974 |
| 0.4915        | 92.0  | 644  | 1.9681          | 0.5575   | 0.5679     | 2.3522 | 0.5575   | 0.5515   | 0.1584 | 0.1975 |
| 0.4915        | 93.0  | 651  | 1.9680          | 0.5575   | 0.5678     | 2.3524 | 0.5575   | 0.5515   | 0.1610 | 0.1973 |
| 0.4915        | 94.0  | 658  | 1.9680          | 0.5575   | 0.5680     | 2.3521 | 0.5575   | 0.5515   | 0.1653 | 0.1973 |
| 0.4915        | 95.0  | 665  | 1.9681          | 0.5575   | 0.5679     | 2.3524 | 0.5575   | 0.5515   | 0.1663 | 0.1973 |
| 0.4915        | 96.0  | 672  | 1.9682          | 0.5575   | 0.5680     | 2.3528 | 0.5575   | 0.5516   | 0.1663 | 0.1973 |
| 0.4915        | 97.0  | 679  | 1.9682          | 0.5575   | 0.5680     | 2.3526 | 0.5575   | 0.5515   | 0.1625 | 0.1973 |
| 0.4915        | 98.0  | 686  | 1.9681          | 0.5575   | 0.5679     | 2.3526 | 0.5575   | 0.5516   | 0.1676 | 0.1973 |
| 0.4915        | 99.0  | 693  | 1.9682          | 0.5575   | 0.5679     | 2.3526 | 0.5575   | 0.5516   | 0.1635 | 0.1973 |
| 0.4915        | 100.0 | 700  | 1.9682          | 0.5575   | 0.5680     | 2.3526 | 0.5575   | 0.5516   | 0.1676 | 0.1973 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo