
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t5.0_a0.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6906
- Accuracy: 0.5675
- Brier Loss: 0.5696
- Nll: 2.4654
- F1 Micro: 0.5675
- F1 Macro: 0.5648
- Ece: 0.1638
- Aurc: 0.1969

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 4.6645          | 0.0425   | 1.0732     | 7.4792 | 0.0425   | 0.0329   | 0.2822 | 0.9534 |
| No log        | 2.0   | 14   | 3.6090          | 0.1      | 0.9462     | 5.7033 | 0.1000   | 0.0909   | 0.1715 | 0.8908 |
| No log        | 3.0   | 21   | 3.2938          | 0.1975   | 0.8945     | 5.2434 | 0.1975   | 0.1585   | 0.1594 | 0.6803 |
| No log        | 4.0   | 28   | 2.8501          | 0.29     | 0.7966     | 3.6125 | 0.29     | 0.2703   | 0.1664 | 0.4859 |
| No log        | 5.0   | 35   | 2.5122          | 0.39     | 0.7162     | 3.1122 | 0.39     | 0.3562   | 0.1838 | 0.3484 |
| No log        | 6.0   | 42   | 2.3667          | 0.4175   | 0.6955     | 3.3546 | 0.4175   | 0.3757   | 0.1683 | 0.3138 |
| No log        | 7.0   | 49   | 2.2629          | 0.4725   | 0.6766     | 2.9737 | 0.4725   | 0.4417   | 0.1917 | 0.2939 |
| No log        | 8.0   | 56   | 2.1495          | 0.485    | 0.6530     | 2.7257 | 0.485    | 0.4798   | 0.1901 | 0.2762 |
| No log        | 9.0   | 63   | 2.1491          | 0.495    | 0.6517     | 2.7709 | 0.495    | 0.4982   | 0.1934 | 0.2664 |
| No log        | 10.0  | 70   | 2.2541          | 0.465    | 0.6794     | 2.6880 | 0.465    | 0.4654   | 0.1923 | 0.2870 |
| No log        | 11.0  | 77   | 2.1607          | 0.4925   | 0.6564     | 2.7773 | 0.4925   | 0.4769   | 0.1904 | 0.2545 |
| No log        | 12.0  | 84   | 2.1581          | 0.505    | 0.6714     | 2.7017 | 0.505    | 0.4876   | 0.2023 | 0.2669 |
| No log        | 13.0  | 91   | 2.0025          | 0.515    | 0.6324     | 2.5557 | 0.515    | 0.5048   | 0.2194 | 0.2348 |
| No log        | 14.0  | 98   | 2.0267          | 0.5025   | 0.6474     | 2.4764 | 0.5025   | 0.4876   | 0.2071 | 0.2474 |
| No log        | 15.0  | 105  | 1.9258          | 0.535    | 0.6106     | 2.4995 | 0.535    | 0.5310   | 0.1960 | 0.2251 |
| No log        | 16.0  | 112  | 1.8650          | 0.545    | 0.6031     | 2.4944 | 0.545    | 0.5383   | 0.1847 | 0.2187 |
| No log        | 17.0  | 119  | 1.8902          | 0.5375   | 0.6118     | 2.5468 | 0.5375   | 0.5284   | 0.1852 | 0.2278 |
| No log        | 18.0  | 126  | 1.9187          | 0.53     | 0.6188     | 2.5580 | 0.53     | 0.5174   | 0.1792 | 0.2285 |
| No log        | 19.0  | 133  | 1.8399          | 0.54     | 0.6049     | 2.5246 | 0.54     | 0.5362   | 0.1758 | 0.2191 |
| No log        | 20.0  | 140  | 1.8912          | 0.53     | 0.6117     | 2.5403 | 0.53     | 0.5139   | 0.1819 | 0.2210 |
| No log        | 21.0  | 147  | 1.8930          | 0.5325   | 0.6099     | 2.6739 | 0.5325   | 0.5294   | 0.1736 | 0.2281 |
| No log        | 22.0  | 154  | 1.8930          | 0.5275   | 0.6070     | 2.6060 | 0.5275   | 0.5244   | 0.1832 | 0.2214 |
| No log        | 23.0  | 161  | 1.8596          | 0.5375   | 0.6030     | 2.5066 | 0.5375   | 0.5260   | 0.1864 | 0.2185 |
| No log        | 24.0  | 168  | 1.8328          | 0.5375   | 0.5942     | 2.5127 | 0.5375   | 0.5302   | 0.1807 | 0.2079 |
| No log        | 25.0  | 175  | 1.8069          | 0.53     | 0.6004     | 2.4747 | 0.53     | 0.5241   | 0.1910 | 0.2189 |
| No log        | 26.0  | 182  | 1.7937          | 0.5425   | 0.5912     | 2.5083 | 0.5425   | 0.5366   | 0.1841 | 0.2123 |
| No log        | 27.0  | 189  | 1.7986          | 0.55     | 0.5945     | 2.5267 | 0.55     | 0.5476   | 0.1901 | 0.2134 |
| No log        | 28.0  | 196  | 1.8090          | 0.5525   | 0.5918     | 2.4467 | 0.5525   | 0.5466   | 0.1765 | 0.2125 |
| No log        | 29.0  | 203  | 1.7911          | 0.55     | 0.5931     | 2.4536 | 0.55     | 0.5462   | 0.1619 | 0.2140 |
| No log        | 30.0  | 210  | 1.7702          | 0.5425   | 0.5911     | 2.5037 | 0.5425   | 0.5340   | 0.1819 | 0.2080 |
| No log        | 31.0  | 217  | 1.8248          | 0.53     | 0.6063     | 2.4965 | 0.53     | 0.5145   | 0.1757 | 0.2217 |
| No log        | 32.0  | 224  | 1.7866          | 0.545    | 0.5920     | 2.5543 | 0.545    | 0.5365   | 0.1745 | 0.2082 |
| No log        | 33.0  | 231  | 1.7793          | 0.555    | 0.5867     | 2.4751 | 0.555    | 0.5503   | 0.1657 | 0.2107 |
| No log        | 34.0  | 238  | 1.7417          | 0.5525   | 0.5799     | 2.4174 | 0.5525   | 0.5517   | 0.1778 | 0.2049 |
| No log        | 35.0  | 245  | 1.7557          | 0.54     | 0.5869     | 2.4065 | 0.54     | 0.5379   | 0.1640 | 0.2119 |
| No log        | 36.0  | 252  | 1.7567          | 0.54     | 0.5884     | 2.5031 | 0.54     | 0.5374   | 0.1519 | 0.2122 |
| No log        | 37.0  | 259  | 1.7306          | 0.56     | 0.5781     | 2.5313 | 0.56     | 0.5551   | 0.1906 | 0.2042 |
| No log        | 38.0  | 266  | 1.7615          | 0.5425   | 0.5888     | 2.4510 | 0.5425   | 0.5423   | 0.1783 | 0.2121 |
| No log        | 39.0  | 273  | 1.7607          | 0.555    | 0.5855     | 2.5044 | 0.555    | 0.5473   | 0.1808 | 0.2062 |
| No log        | 40.0  | 280  | 1.7430          | 0.5275   | 0.5840     | 2.4265 | 0.5275   | 0.5228   | 0.1822 | 0.2125 |
| No log        | 41.0  | 287  | 1.7515          | 0.5625   | 0.5800     | 2.5432 | 0.5625   | 0.5500   | 0.1812 | 0.2019 |
| No log        | 42.0  | 294  | 1.7229          | 0.5475   | 0.5778     | 2.4597 | 0.5475   | 0.5416   | 0.1656 | 0.2042 |
| No log        | 43.0  | 301  | 1.7566          | 0.555    | 0.5818     | 2.4305 | 0.555    | 0.5517   | 0.1521 | 0.2036 |
| No log        | 44.0  | 308  | 1.7093          | 0.575    | 0.5745     | 2.4087 | 0.575    | 0.5761   | 0.1678 | 0.2019 |
| No log        | 45.0  | 315  | 1.7319          | 0.56     | 0.5788     | 2.5339 | 0.56     | 0.5550   | 0.1647 | 0.2028 |
| No log        | 46.0  | 322  | 1.7331          | 0.5475   | 0.5863     | 2.4386 | 0.5475   | 0.5470   | 0.1737 | 0.2113 |
| No log        | 47.0  | 329  | 1.7243          | 0.56     | 0.5776     | 2.5336 | 0.56     | 0.5548   | 0.1642 | 0.2037 |
| No log        | 48.0  | 336  | 1.7205          | 0.5675   | 0.5817     | 2.4578 | 0.5675   | 0.5661   | 0.1800 | 0.2091 |
| No log        | 49.0  | 343  | 1.7210          | 0.5725   | 0.5721     | 2.4792 | 0.5725   | 0.5656   | 0.1550 | 0.1970 |
| No log        | 50.0  | 350  | 1.6994          | 0.58     | 0.5719     | 2.4645 | 0.58     | 0.5773   | 0.1580 | 0.1957 |
| No log        | 51.0  | 357  | 1.7082          | 0.5775   | 0.5737     | 2.4805 | 0.5775   | 0.5701   | 0.1734 | 0.1992 |
| No log        | 52.0  | 364  | 1.7144          | 0.5575   | 0.5693     | 2.4501 | 0.5575   | 0.5519   | 0.1812 | 0.1974 |
| No log        | 53.0  | 371  | 1.7196          | 0.57     | 0.5796     | 2.4555 | 0.57     | 0.5618   | 0.1728 | 0.2022 |
| No log        | 54.0  | 378  | 1.7149          | 0.5675   | 0.5746     | 2.4363 | 0.5675   | 0.5605   | 0.1901 | 0.1999 |
| No log        | 55.0  | 385  | 1.7030          | 0.555    | 0.5775     | 2.4253 | 0.555    | 0.5528   | 0.1823 | 0.2029 |
| No log        | 56.0  | 392  | 1.7209          | 0.5675   | 0.5747     | 2.4926 | 0.5675   | 0.5634   | 0.1644 | 0.2008 |
| No log        | 57.0  | 399  | 1.7103          | 0.5525   | 0.5760     | 2.4528 | 0.5525   | 0.5487   | 0.1738 | 0.2025 |
| No log        | 58.0  | 406  | 1.7005          | 0.5575   | 0.5771     | 2.4678 | 0.5575   | 0.5547   | 0.1514 | 0.2041 |
| No log        | 59.0  | 413  | 1.7098          | 0.56     | 0.5763     | 2.4368 | 0.56     | 0.5585   | 0.1650 | 0.2022 |
| No log        | 60.0  | 420  | 1.6976          | 0.5775   | 0.5681     | 2.4633 | 0.5775   | 0.5737   | 0.1729 | 0.1963 |
| No log        | 61.0  | 427  | 1.7057          | 0.5575   | 0.5739     | 2.4717 | 0.5575   | 0.5522   | 0.1932 | 0.2007 |
| No log        | 62.0  | 434  | 1.6884          | 0.5725   | 0.5646     | 2.4538 | 0.5725   | 0.5693   | 0.1692 | 0.1930 |
| No log        | 63.0  | 441  | 1.6979          | 0.56     | 0.5731     | 2.4635 | 0.56     | 0.5562   | 0.1690 | 0.2016 |
| No log        | 64.0  | 448  | 1.6848          | 0.55     | 0.5686     | 2.4583 | 0.55     | 0.5452   | 0.1782 | 0.1980 |
| No log        | 65.0  | 455  | 1.7072          | 0.5575   | 0.5774     | 2.4596 | 0.5575   | 0.5549   | 0.1627 | 0.2036 |
| No log        | 66.0  | 462  | 1.7057          | 0.5625   | 0.5759     | 2.4644 | 0.5625   | 0.5586   | 0.1780 | 0.2009 |
| No log        | 67.0  | 469  | 1.7016          | 0.56     | 0.5719     | 2.4976 | 0.56     | 0.5546   | 0.1734 | 0.2004 |
| No log        | 68.0  | 476  | 1.6951          | 0.5675   | 0.5725     | 2.4025 | 0.5675   | 0.5648   | 0.1866 | 0.1980 |
| No log        | 69.0  | 483  | 1.7012          | 0.555    | 0.5750     | 2.4970 | 0.555    | 0.5527   | 0.1958 | 0.2022 |
| No log        | 70.0  | 490  | 1.6983          | 0.575    | 0.5689     | 2.4763 | 0.575    | 0.5708   | 0.1648 | 0.1950 |
| No log        | 71.0  | 497  | 1.6954          | 0.5675   | 0.5729     | 2.4642 | 0.5675   | 0.5638   | 0.1762 | 0.2004 |
| 0.437         | 72.0  | 504  | 1.6973          | 0.5625   | 0.5718     | 2.4639 | 0.5625   | 0.5605   | 0.1680 | 0.1991 |
| 0.437         | 73.0  | 511  | 1.6942          | 0.565    | 0.5714     | 2.4629 | 0.565    | 0.5610   | 0.1776 | 0.1980 |
| 0.437         | 74.0  | 518  | 1.6923          | 0.5725   | 0.5694     | 2.4717 | 0.5725   | 0.5698   | 0.1676 | 0.1967 |
| 0.437         | 75.0  | 525  | 1.6926          | 0.5675   | 0.5699     | 2.4674 | 0.5675   | 0.5633   | 0.1741 | 0.1975 |
| 0.437         | 76.0  | 532  | 1.6906          | 0.5675   | 0.5692     | 2.4673 | 0.5675   | 0.5641   | 0.1785 | 0.1962 |
| 0.437         | 77.0  | 539  | 1.6912          | 0.565    | 0.5692     | 2.4671 | 0.565    | 0.5617   | 0.1568 | 0.1958 |
| 0.437         | 78.0  | 546  | 1.6879          | 0.565    | 0.5685     | 2.4629 | 0.565    | 0.5620   | 0.1860 | 0.1954 |
| 0.437         | 79.0  | 553  | 1.6886          | 0.565    | 0.5695     | 2.4650 | 0.565    | 0.5625   | 0.1777 | 0.1968 |
| 0.437         | 80.0  | 560  | 1.6882          | 0.5625   | 0.5700     | 2.4632 | 0.5625   | 0.5583   | 0.1791 | 0.1982 |
| 0.437         | 81.0  | 567  | 1.6918          | 0.565    | 0.5704     | 2.4638 | 0.565    | 0.5622   | 0.1630 | 0.1978 |
| 0.437         | 82.0  | 574  | 1.6909          | 0.5675   | 0.5697     | 2.4646 | 0.5675   | 0.5641   | 0.1862 | 0.1959 |
| 0.437         | 83.0  | 581  | 1.6881          | 0.565    | 0.5687     | 2.4665 | 0.565    | 0.5621   | 0.1494 | 0.1956 |
| 0.437         | 84.0  | 588  | 1.6897          | 0.565    | 0.5692     | 2.4648 | 0.565    | 0.5626   | 0.1716 | 0.1969 |
| 0.437         | 85.0  | 595  | 1.6910          | 0.5675   | 0.5697     | 2.4654 | 0.5675   | 0.5652   | 0.1747 | 0.1971 |
| 0.437         | 86.0  | 602  | 1.6905          | 0.57     | 0.5694     | 2.4648 | 0.57     | 0.5667   | 0.1659 | 0.1959 |
| 0.437         | 87.0  | 609  | 1.6896          | 0.5675   | 0.5693     | 2.4648 | 0.5675   | 0.5642   | 0.1681 | 0.1958 |
| 0.437         | 88.0  | 616  | 1.6902          | 0.5675   | 0.5695     | 2.4655 | 0.5675   | 0.5642   | 0.1709 | 0.1961 |
| 0.437         | 89.0  | 623  | 1.6907          | 0.5675   | 0.5697     | 2.4655 | 0.5675   | 0.5648   | 0.1676 | 0.1971 |
| 0.437         | 90.0  | 630  | 1.6903          | 0.5675   | 0.5694     | 2.4657 | 0.5675   | 0.5648   | 0.1699 | 0.1966 |
| 0.437         | 91.0  | 637  | 1.6906          | 0.565    | 0.5696     | 2.4656 | 0.565    | 0.5624   | 0.1689 | 0.1970 |
| 0.437         | 92.0  | 644  | 1.6904          | 0.565    | 0.5695     | 2.4654 | 0.565    | 0.5622   | 0.1668 | 0.1970 |
| 0.437         | 93.0  | 651  | 1.6904          | 0.5675   | 0.5696     | 2.4655 | 0.5675   | 0.5649   | 0.1619 | 0.1970 |
| 0.437         | 94.0  | 658  | 1.6903          | 0.565    | 0.5694     | 2.4655 | 0.565    | 0.5622   | 0.1662 | 0.1969 |
| 0.437         | 95.0  | 665  | 1.6905          | 0.565    | 0.5695     | 2.4653 | 0.565    | 0.5622   | 0.1710 | 0.1969 |
| 0.437         | 96.0  | 672  | 1.6905          | 0.5675   | 0.5695     | 2.4655 | 0.5675   | 0.5648   | 0.1638 | 0.1968 |
| 0.437         | 97.0  | 679  | 1.6906          | 0.565    | 0.5695     | 2.4654 | 0.565    | 0.5622   | 0.1664 | 0.1970 |
| 0.437         | 98.0  | 686  | 1.6906          | 0.5675   | 0.5695     | 2.4655 | 0.5675   | 0.5648   | 0.1638 | 0.1969 |
| 0.437         | 99.0  | 693  | 1.6906          | 0.5675   | 0.5696     | 2.4654 | 0.5675   | 0.5648   | 0.1638 | 0.1969 |
| 0.437         | 100.0 | 700  | 1.6906          | 0.5675   | 0.5696     | 2.4654 | 0.5675   | 0.5648   | 0.1638 | 0.1969 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo