
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t5.0_a0.7

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6224
- Accuracy: 0.555
- Brier Loss: 0.5813
- Nll: 2.4451
- F1 Micro: 0.555
- F1 Macro: 0.5481
- Ece: 0.1732
- Aurc: 0.2090

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 4.2974          | 0.04     | 1.0725     | 7.3803 | 0.04     | 0.0315   | 0.2825 | 0.9545 |
| No log        | 2.0   | 14   | 3.3036          | 0.095    | 0.9476     | 5.7516 | 0.095    | 0.0792   | 0.1590 | 0.8944 |
| No log        | 3.0   | 21   | 2.9979          | 0.215    | 0.8918     | 5.3309 | 0.2150   | 0.1763   | 0.1514 | 0.6635 |
| No log        | 4.0   | 28   | 2.5846          | 0.2875   | 0.7979     | 3.5812 | 0.2875   | 0.2707   | 0.1619 | 0.4840 |
| No log        | 5.0   | 35   | 2.2908          | 0.3925   | 0.7162     | 3.1082 | 0.3925   | 0.3675   | 0.1724 | 0.3500 |
| No log        | 6.0   | 42   | 2.1582          | 0.4275   | 0.6903     | 3.3486 | 0.4275   | 0.3933   | 0.1723 | 0.3106 |
| No log        | 7.0   | 49   | 2.1910          | 0.445    | 0.7011     | 2.9994 | 0.445    | 0.4233   | 0.1889 | 0.3105 |
| No log        | 8.0   | 56   | 2.0287          | 0.485    | 0.6673     | 2.8482 | 0.485    | 0.4803   | 0.1781 | 0.2848 |
| No log        | 9.0   | 63   | 2.1037          | 0.4775   | 0.6684     | 2.8143 | 0.4775   | 0.4715   | 0.2010 | 0.2690 |
| No log        | 10.0  | 70   | 2.1168          | 0.4825   | 0.6846     | 2.8143 | 0.4825   | 0.4703   | 0.2082 | 0.2818 |
| No log        | 11.0  | 77   | 2.1094          | 0.495    | 0.6825     | 2.9020 | 0.495    | 0.4834   | 0.1990 | 0.2632 |
| No log        | 12.0  | 84   | 2.0835          | 0.48     | 0.6897     | 2.7522 | 0.48     | 0.4651   | 0.2261 | 0.2741 |
| No log        | 13.0  | 91   | 1.9606          | 0.505    | 0.6631     | 2.4794 | 0.505    | 0.4988   | 0.2034 | 0.2712 |
| No log        | 14.0  | 98   | 1.9519          | 0.4975   | 0.6567     | 2.7608 | 0.4975   | 0.4833   | 0.2042 | 0.2563 |
| No log        | 15.0  | 105  | 1.8794          | 0.52     | 0.6304     | 2.6588 | 0.52     | 0.5121   | 0.1814 | 0.2337 |
| No log        | 16.0  | 112  | 1.7934          | 0.5375   | 0.6191     | 2.5142 | 0.5375   | 0.5234   | 0.1853 | 0.2272 |
| No log        | 17.0  | 119  | 1.8110          | 0.5225   | 0.6242     | 2.5106 | 0.5225   | 0.5071   | 0.1918 | 0.2336 |
| No log        | 18.0  | 126  | 1.8027          | 0.515    | 0.6283     | 2.4142 | 0.515    | 0.4983   | 0.2020 | 0.2359 |
| No log        | 19.0  | 133  | 1.8123          | 0.5375   | 0.6318     | 2.5551 | 0.5375   | 0.5235   | 0.2132 | 0.2358 |
| No log        | 20.0  | 140  | 1.7937          | 0.5225   | 0.6292     | 2.5237 | 0.5225   | 0.5145   | 0.2070 | 0.2336 |
| No log        | 21.0  | 147  | 1.7272          | 0.5175   | 0.6046     | 2.4278 | 0.5175   | 0.5103   | 0.1806 | 0.2286 |
| No log        | 22.0  | 154  | 1.8337          | 0.5325   | 0.6396     | 2.5603 | 0.5325   | 0.5136   | 0.1884 | 0.2405 |
| No log        | 23.0  | 161  | 1.7416          | 0.5275   | 0.6102     | 2.5228 | 0.5275   | 0.5077   | 0.1784 | 0.2232 |
| No log        | 24.0  | 168  | 1.7036          | 0.55     | 0.6063     | 2.4933 | 0.55     | 0.5380   | 0.1776 | 0.2230 |
| No log        | 25.0  | 175  | 1.7330          | 0.545    | 0.6084     | 2.4943 | 0.545    | 0.5365   | 0.1989 | 0.2251 |
| No log        | 26.0  | 182  | 1.6911          | 0.55     | 0.5993     | 2.4401 | 0.55     | 0.5416   | 0.1792 | 0.2208 |
| No log        | 27.0  | 189  | 1.7329          | 0.5475   | 0.6162     | 2.4824 | 0.5475   | 0.5380   | 0.1830 | 0.2317 |
| No log        | 28.0  | 196  | 1.6890          | 0.5475   | 0.5992     | 2.4828 | 0.5475   | 0.5401   | 0.1725 | 0.2178 |
| No log        | 29.0  | 203  | 1.7256          | 0.5425   | 0.6124     | 2.5121 | 0.5425   | 0.5299   | 0.1765 | 0.2260 |
| No log        | 30.0  | 210  | 1.6854          | 0.5375   | 0.5952     | 2.4275 | 0.5375   | 0.5282   | 0.2015 | 0.2163 |
| No log        | 31.0  | 217  | 1.7010          | 0.5475   | 0.6030     | 2.4832 | 0.5475   | 0.5385   | 0.1862 | 0.2236 |
| No log        | 32.0  | 224  | 1.6840          | 0.535    | 0.5934     | 2.4512 | 0.535    | 0.5310   | 0.1794 | 0.2182 |
| No log        | 33.0  | 231  | 1.6808          | 0.545    | 0.6016     | 2.4424 | 0.545    | 0.5396   | 0.1828 | 0.2222 |
| No log        | 34.0  | 238  | 1.6965          | 0.535    | 0.6000     | 2.5453 | 0.535    | 0.5270   | 0.1846 | 0.2243 |
| No log        | 35.0  | 245  | 1.6650          | 0.545    | 0.5930     | 2.4901 | 0.545    | 0.5418   | 0.1706 | 0.2130 |
| No log        | 36.0  | 252  | 1.6494          | 0.54     | 0.5979     | 2.4011 | 0.54     | 0.5319   | 0.1731 | 0.2201 |
| No log        | 37.0  | 259  | 1.6738          | 0.54     | 0.5892     | 2.4632 | 0.54     | 0.5257   | 0.1781 | 0.2088 |
| No log        | 38.0  | 266  | 1.6502          | 0.55     | 0.5889     | 2.4733 | 0.55     | 0.5416   | 0.1957 | 0.2087 |
| No log        | 39.0  | 273  | 1.6539          | 0.55     | 0.5832     | 2.5105 | 0.55     | 0.5424   | 0.1584 | 0.2120 |
| No log        | 40.0  | 280  | 1.6399          | 0.545    | 0.5919     | 2.4450 | 0.545    | 0.5439   | 0.1877 | 0.2168 |
| No log        | 41.0  | 287  | 1.6968          | 0.535    | 0.6006     | 2.5310 | 0.535    | 0.5281   | 0.1895 | 0.2221 |
| No log        | 42.0  | 294  | 1.6430          | 0.5425   | 0.5955     | 2.4020 | 0.5425   | 0.5449   | 0.1722 | 0.2203 |
| No log        | 43.0  | 301  | 1.6743          | 0.535    | 0.5970     | 2.5138 | 0.535    | 0.5239   | 0.1869 | 0.2167 |
| No log        | 44.0  | 308  | 1.6544          | 0.5475   | 0.5949     | 2.4259 | 0.5475   | 0.5408   | 0.1651 | 0.2164 |
| No log        | 45.0  | 315  | 1.6763          | 0.535    | 0.5980     | 2.4379 | 0.535    | 0.5249   | 0.1868 | 0.2174 |
| No log        | 46.0  | 322  | 1.6509          | 0.525    | 0.5933     | 2.4351 | 0.525    | 0.5137   | 0.1861 | 0.2194 |
| No log        | 47.0  | 329  | 1.6530          | 0.5475   | 0.5929     | 2.4628 | 0.5475   | 0.5419   | 0.1859 | 0.2148 |
| No log        | 48.0  | 336  | 1.6410          | 0.555    | 0.5835     | 2.4992 | 0.555    | 0.5490   | 0.1805 | 0.2110 |
| No log        | 49.0  | 343  | 1.6398          | 0.5525   | 0.5857     | 2.5060 | 0.5525   | 0.5409   | 0.1706 | 0.2101 |
| No log        | 50.0  | 350  | 1.6343          | 0.5525   | 0.5814     | 2.4890 | 0.5525   | 0.5442   | 0.1608 | 0.2065 |
| No log        | 51.0  | 357  | 1.6335          | 0.5475   | 0.5846     | 2.4407 | 0.5475   | 0.5392   | 0.1720 | 0.2109 |
| No log        | 52.0  | 364  | 1.6309          | 0.555    | 0.5844     | 2.4944 | 0.555    | 0.5488   | 0.1697 | 0.2091 |
| No log        | 53.0  | 371  | 1.6308          | 0.5575   | 0.5826     | 2.4815 | 0.5575   | 0.5505   | 0.1704 | 0.2080 |
| No log        | 54.0  | 378  | 1.6279          | 0.56     | 0.5832     | 2.4741 | 0.56     | 0.5525   | 0.1724 | 0.2067 |
| No log        | 55.0  | 385  | 1.6226          | 0.55     | 0.5825     | 2.4048 | 0.55     | 0.5425   | 0.1656 | 0.2094 |
| No log        | 56.0  | 392  | 1.6141          | 0.555    | 0.5797     | 2.4716 | 0.555    | 0.5474   | 0.1813 | 0.2076 |
| No log        | 57.0  | 399  | 1.6179          | 0.56     | 0.5760     | 2.4682 | 0.56     | 0.5549   | 0.1474 | 0.2030 |
| No log        | 58.0  | 406  | 1.6278          | 0.56     | 0.5831     | 2.4758 | 0.56     | 0.5540   | 0.1681 | 0.2075 |
| No log        | 59.0  | 413  | 1.6257          | 0.5525   | 0.5817     | 2.4462 | 0.5525   | 0.5455   | 0.1648 | 0.2084 |
| No log        | 60.0  | 420  | 1.6306          | 0.5575   | 0.5861     | 2.5090 | 0.5575   | 0.5505   | 0.1687 | 0.2108 |
| No log        | 61.0  | 427  | 1.6314          | 0.555    | 0.5821     | 2.5024 | 0.555    | 0.5480   | 0.1725 | 0.2092 |
| No log        | 62.0  | 434  | 1.6322          | 0.545    | 0.5846     | 2.4694 | 0.545    | 0.5407   | 0.1848 | 0.2123 |
| No log        | 63.0  | 441  | 1.6229          | 0.5575   | 0.5829     | 2.4413 | 0.5575   | 0.5508   | 0.1698 | 0.2098 |
| No log        | 64.0  | 448  | 1.6187          | 0.56     | 0.5809     | 2.4420 | 0.56     | 0.5522   | 0.1848 | 0.2083 |
| No log        | 65.0  | 455  | 1.6160          | 0.555    | 0.5794     | 2.4349 | 0.555    | 0.5506   | 0.1761 | 0.2076 |
| No log        | 66.0  | 462  | 1.6254          | 0.55     | 0.5822     | 2.4752 | 0.55     | 0.5432   | 0.1629 | 0.2086 |
| No log        | 67.0  | 469  | 1.6259          | 0.55     | 0.5839     | 2.4425 | 0.55     | 0.5423   | 0.1769 | 0.2113 |
| No log        | 68.0  | 476  | 1.6245          | 0.55     | 0.5822     | 2.4382 | 0.55     | 0.5421   | 0.1736 | 0.2091 |
| No log        | 69.0  | 483  | 1.6243          | 0.5575   | 0.5830     | 2.4422 | 0.5575   | 0.5497   | 0.1808 | 0.2094 |
| No log        | 70.0  | 490  | 1.6223          | 0.5575   | 0.5810     | 2.4787 | 0.5575   | 0.5507   | 0.1556 | 0.2089 |
| No log        | 71.0  | 497  | 1.6204          | 0.5575   | 0.5809     | 2.4408 | 0.5575   | 0.5515   | 0.1555 | 0.2083 |
| 0.3852        | 72.0  | 504  | 1.6225          | 0.55     | 0.5816     | 2.4404 | 0.55     | 0.5424   | 0.1750 | 0.2104 |
| 0.3852        | 73.0  | 511  | 1.6237          | 0.55     | 0.5822     | 2.4403 | 0.55     | 0.5429   | 0.1772 | 0.2107 |
| 0.3852        | 74.0  | 518  | 1.6220          | 0.55     | 0.5815     | 2.4441 | 0.55     | 0.5420   | 0.1649 | 0.2105 |
| 0.3852        | 75.0  | 525  | 1.6228          | 0.5475   | 0.5818     | 2.4736 | 0.5475   | 0.5405   | 0.1882 | 0.2109 |
| 0.3852        | 76.0  | 532  | 1.6224          | 0.5525   | 0.5814     | 2.4442 | 0.5525   | 0.5446   | 0.1817 | 0.2108 |
| 0.3852        | 77.0  | 539  | 1.6225          | 0.5525   | 0.5815     | 2.4431 | 0.5525   | 0.5448   | 0.1798 | 0.2098 |
| 0.3852        | 78.0  | 546  | 1.6213          | 0.555    | 0.5812     | 2.4417 | 0.555    | 0.5471   | 0.1680 | 0.2096 |
| 0.3852        | 79.0  | 553  | 1.6208          | 0.5575   | 0.5808     | 2.4423 | 0.5575   | 0.5501   | 0.1784 | 0.2082 |
| 0.3852        | 80.0  | 560  | 1.6218          | 0.5525   | 0.5811     | 2.4425 | 0.5525   | 0.5447   | 0.1683 | 0.2095 |
| 0.3852        | 81.0  | 567  | 1.6225          | 0.5525   | 0.5814     | 2.4429 | 0.5525   | 0.5447   | 0.1856 | 0.2098 |
| 0.3852        | 82.0  | 574  | 1.6222          | 0.5575   | 0.5812     | 2.4469 | 0.5575   | 0.5501   | 0.1953 | 0.2085 |
| 0.3852        | 83.0  | 581  | 1.6219          | 0.555    | 0.5811     | 2.4442 | 0.555    | 0.5471   | 0.1940 | 0.2093 |
| 0.3852        | 84.0  | 588  | 1.6220          | 0.555    | 0.5813     | 2.4443 | 0.555    | 0.5471   | 0.1867 | 0.2095 |
| 0.3852        | 85.0  | 595  | 1.6223          | 0.555    | 0.5813     | 2.4446 | 0.555    | 0.5471   | 0.1885 | 0.2094 |
| 0.3852        | 86.0  | 602  | 1.6222          | 0.5525   | 0.5812     | 2.4448 | 0.5525   | 0.5447   | 0.1749 | 0.2095 |
| 0.3852        | 87.0  | 609  | 1.6222          | 0.555    | 0.5813     | 2.4454 | 0.555    | 0.5481   | 0.1745 | 0.2091 |
| 0.3852        | 88.0  | 616  | 1.6222          | 0.5575   | 0.5813     | 2.4446 | 0.5575   | 0.5504   | 0.1767 | 0.2087 |
| 0.3852        | 89.0  | 623  | 1.6222          | 0.5575   | 0.5813     | 2.4445 | 0.5575   | 0.5504   | 0.1839 | 0.2087 |
| 0.3852        | 90.0  | 630  | 1.6221          | 0.555    | 0.5812     | 2.4447 | 0.555    | 0.5481   | 0.1814 | 0.2091 |
| 0.3852        | 91.0  | 637  | 1.6222          | 0.5575   | 0.5813     | 2.4446 | 0.5575   | 0.5504   | 0.1790 | 0.2087 |
| 0.3852        | 92.0  | 644  | 1.6222          | 0.555    | 0.5813     | 2.4447 | 0.555    | 0.5481   | 0.1755 | 0.2091 |
| 0.3852        | 93.0  | 651  | 1.6223          | 0.5575   | 0.5813     | 2.4446 | 0.5575   | 0.5504   | 0.1747 | 0.2087 |
| 0.3852        | 94.0  | 658  | 1.6223          | 0.5575   | 0.5813     | 2.4449 | 0.5575   | 0.5504   | 0.1747 | 0.2087 |
| 0.3852        | 95.0  | 665  | 1.6223          | 0.555    | 0.5813     | 2.4448 | 0.555    | 0.5481   | 0.1732 | 0.2090 |
| 0.3852        | 96.0  | 672  | 1.6224          | 0.555    | 0.5813     | 2.4451 | 0.555    | 0.5481   | 0.1720 | 0.2091 |
| 0.3852        | 97.0  | 679  | 1.6223          | 0.5575   | 0.5813     | 2.4451 | 0.5575   | 0.5504   | 0.1735 | 0.2088 |
| 0.3852        | 98.0  | 686  | 1.6223          | 0.5575   | 0.5813     | 2.4451 | 0.5575   | 0.5504   | 0.1747 | 0.2087 |
| 0.3852        | 99.0  | 693  | 1.6224          | 0.555    | 0.5813     | 2.4451 | 0.555    | 0.5481   | 0.1732 | 0.2090 |
| 0.3852        | 100.0 | 700  | 1.6224          | 0.555    | 0.5813     | 2.4451 | 0.555    | 0.5481   | 0.1732 | 0.2090 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo