
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t2.5_a0.7

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7789
- Accuracy: 0.565
- Brier Loss: 0.5798
- Nll: 2.3548
- F1 Micro: 0.565
- F1 Macro: 0.5569
- Ece: 0.1677
- Aurc: 0.2032

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 4.7719          | 0.04     | 1.0712     | 7.3456 | 0.04     | 0.0315   | 0.2816 | 0.9543 |
| No log        | 2.0   | 14   | 3.7636          | 0.1      | 0.9486     | 5.7989 | 0.1000   | 0.0863   | 0.1625 | 0.8854 |
| No log        | 3.0   | 21   | 3.3871          | 0.235    | 0.8838     | 5.3315 | 0.235    | 0.1857   | 0.1589 | 0.6274 |
| No log        | 4.0   | 28   | 2.8780          | 0.2975   | 0.7900     | 3.5467 | 0.2975   | 0.2884   | 0.1672 | 0.4712 |
| No log        | 5.0   | 35   | 2.5223          | 0.3875   | 0.7164     | 3.0188 | 0.3875   | 0.3630   | 0.1596 | 0.3495 |
| No log        | 6.0   | 42   | 2.3634          | 0.43     | 0.6927     | 3.1378 | 0.4300   | 0.4031   | 0.1877 | 0.3055 |
| No log        | 7.0   | 49   | 2.3487          | 0.445    | 0.7009     | 3.0084 | 0.445    | 0.4123   | 0.2026 | 0.3003 |
| No log        | 8.0   | 56   | 2.2521          | 0.47     | 0.6857     | 2.8634 | 0.47     | 0.4607   | 0.1977 | 0.2918 |
| No log        | 9.0   | 63   | 2.3597          | 0.4775   | 0.6955     | 2.6768 | 0.4775   | 0.4642   | 0.2215 | 0.2787 |
| No log        | 10.0  | 70   | 2.3524          | 0.46     | 0.7130     | 2.7830 | 0.46     | 0.4441   | 0.2429 | 0.2941 |
| No log        | 11.0  | 77   | 2.4233          | 0.46     | 0.7108     | 3.0867 | 0.46     | 0.4418   | 0.2247 | 0.2828 |
| No log        | 12.0  | 84   | 2.2723          | 0.485    | 0.6901     | 2.7313 | 0.485    | 0.4742   | 0.2293 | 0.2774 |
| No log        | 13.0  | 91   | 2.2818          | 0.49     | 0.7007     | 2.5829 | 0.49     | 0.4859   | 0.2473 | 0.2791 |
| No log        | 14.0  | 98   | 2.1682          | 0.4975   | 0.6695     | 2.8292 | 0.4975   | 0.4815   | 0.2170 | 0.2653 |
| No log        | 15.0  | 105  | 2.0652          | 0.52     | 0.6520     | 2.4319 | 0.52     | 0.5120   | 0.2079 | 0.2617 |
| No log        | 16.0  | 112  | 2.0524          | 0.5225   | 0.6384     | 2.5273 | 0.5225   | 0.5128   | 0.1980 | 0.2391 |
| No log        | 17.0  | 119  | 2.1736          | 0.48     | 0.6765     | 2.4887 | 0.48     | 0.4809   | 0.2112 | 0.2834 |
| No log        | 18.0  | 126  | 2.0096          | 0.515    | 0.6496     | 2.4230 | 0.515    | 0.5215   | 0.1957 | 0.2513 |
| No log        | 19.0  | 133  | 2.0207          | 0.5175   | 0.6417     | 2.4591 | 0.5175   | 0.5104   | 0.2011 | 0.2526 |
| No log        | 20.0  | 140  | 1.9152          | 0.5425   | 0.6044     | 2.5069 | 0.5425   | 0.5405   | 0.1826 | 0.2147 |
| No log        | 21.0  | 147  | 1.9600          | 0.52     | 0.6332     | 2.5016 | 0.52     | 0.5204   | 0.1871 | 0.2427 |
| No log        | 22.0  | 154  | 1.9325          | 0.515    | 0.6226     | 2.4981 | 0.515    | 0.5078   | 0.1860 | 0.2323 |
| No log        | 23.0  | 161  | 1.9172          | 0.53     | 0.6144     | 2.4601 | 0.53     | 0.5221   | 0.1941 | 0.2311 |
| No log        | 24.0  | 168  | 1.8891          | 0.5425   | 0.6091     | 2.4653 | 0.5425   | 0.5399   | 0.1933 | 0.2260 |
| No log        | 25.0  | 175  | 1.9460          | 0.5175   | 0.6214     | 2.4785 | 0.5175   | 0.5168   | 0.1694 | 0.2335 |
| No log        | 26.0  | 182  | 1.9060          | 0.5525   | 0.5970     | 2.4789 | 0.5525   | 0.5430   | 0.1934 | 0.2137 |
| No log        | 27.0  | 189  | 1.9421          | 0.5375   | 0.6205     | 2.4814 | 0.5375   | 0.5313   | 0.2135 | 0.2402 |
| No log        | 28.0  | 196  | 2.0195          | 0.545    | 0.6187     | 2.5330 | 0.545    | 0.5256   | 0.1800 | 0.2360 |
| No log        | 29.0  | 203  | 1.9428          | 0.535    | 0.6167     | 2.4894 | 0.535    | 0.5111   | 0.1862 | 0.2322 |
| No log        | 30.0  | 210  | 1.8996          | 0.5225   | 0.6207     | 2.4810 | 0.5225   | 0.5137   | 0.1994 | 0.2330 |
| No log        | 31.0  | 217  | 1.8462          | 0.54     | 0.6119     | 2.4201 | 0.54     | 0.5341   | 0.1817 | 0.2190 |
| No log        | 32.0  | 224  | 1.8324          | 0.55     | 0.5988     | 2.4230 | 0.55     | 0.5427   | 0.1888 | 0.2171 |
| No log        | 33.0  | 231  | 1.8393          | 0.545    | 0.5977     | 2.3943 | 0.545    | 0.5339   | 0.1838 | 0.2172 |
| No log        | 34.0  | 238  | 1.8704          | 0.5475   | 0.6081     | 2.4488 | 0.5475   | 0.5427   | 0.1768 | 0.2200 |
| No log        | 35.0  | 245  | 1.8546          | 0.54     | 0.6022     | 2.3273 | 0.54     | 0.5316   | 0.1847 | 0.2226 |
| No log        | 36.0  | 252  | 1.8608          | 0.53     | 0.5972     | 2.5153 | 0.53     | 0.5139   | 0.1810 | 0.2202 |
| No log        | 37.0  | 259  | 1.8663          | 0.5325   | 0.6057     | 2.4642 | 0.5325   | 0.5243   | 0.1836 | 0.2205 |
| No log        | 38.0  | 266  | 1.8300          | 0.545    | 0.5954     | 2.5101 | 0.545    | 0.5418   | 0.1890 | 0.2141 |
| No log        | 39.0  | 273  | 1.8121          | 0.5625   | 0.5853     | 2.4397 | 0.5625   | 0.5550   | 0.1704 | 0.2110 |
| No log        | 40.0  | 280  | 1.7916          | 0.54     | 0.5884     | 2.3565 | 0.54     | 0.5361   | 0.1685 | 0.2135 |
| No log        | 41.0  | 287  | 1.8353          | 0.5575   | 0.5929     | 2.4252 | 0.5575   | 0.5451   | 0.1823 | 0.2116 |
| No log        | 42.0  | 294  | 1.7999          | 0.5675   | 0.5839     | 2.4820 | 0.5675   | 0.5631   | 0.1729 | 0.2045 |
| No log        | 43.0  | 301  | 1.8622          | 0.52     | 0.6106     | 2.4823 | 0.52     | 0.5028   | 0.1948 | 0.2270 |
| No log        | 44.0  | 308  | 1.7892          | 0.55     | 0.5892     | 2.3342 | 0.55     | 0.5450   | 0.1798 | 0.2126 |
| No log        | 45.0  | 315  | 1.7978          | 0.545    | 0.5868     | 2.4345 | 0.545    | 0.5439   | 0.1894 | 0.2094 |
| No log        | 46.0  | 322  | 1.7697          | 0.56     | 0.5772     | 2.4272 | 0.56     | 0.5585   | 0.1601 | 0.1997 |
| No log        | 47.0  | 329  | 1.7754          | 0.5475   | 0.5835     | 2.3977 | 0.5475   | 0.5438   | 0.1759 | 0.2059 |
| No log        | 48.0  | 336  | 1.7922          | 0.545    | 0.5929     | 2.4119 | 0.545    | 0.5390   | 0.1891 | 0.2131 |
| No log        | 49.0  | 343  | 1.8055          | 0.5625   | 0.5872     | 2.3654 | 0.5625   | 0.5497   | 0.1759 | 0.2073 |
| No log        | 50.0  | 350  | 1.7972          | 0.56     | 0.5894     | 2.3366 | 0.56     | 0.5487   | 0.1803 | 0.2083 |
| No log        | 51.0  | 357  | 1.7890          | 0.555    | 0.5815     | 2.3858 | 0.555    | 0.5501   | 0.1693 | 0.2067 |
| No log        | 52.0  | 364  | 1.7958          | 0.5475   | 0.5883     | 2.4244 | 0.5475   | 0.5355   | 0.1910 | 0.2105 |
| No log        | 53.0  | 371  | 1.7881          | 0.5675   | 0.5834     | 2.4135 | 0.5675   | 0.5603   | 0.1836 | 0.2028 |
| No log        | 54.0  | 378  | 1.7675          | 0.555    | 0.5766     | 2.4043 | 0.555    | 0.5563   | 0.1653 | 0.2047 |
| No log        | 55.0  | 385  | 1.7688          | 0.55     | 0.5843     | 2.3641 | 0.55     | 0.5505   | 0.1729 | 0.2092 |
| No log        | 56.0  | 392  | 1.7796          | 0.55     | 0.5861     | 2.3404 | 0.55     | 0.5458   | 0.1808 | 0.2114 |
| No log        | 57.0  | 399  | 1.7861          | 0.54     | 0.5885     | 2.3460 | 0.54     | 0.5323   | 0.1902 | 0.2073 |
| No log        | 58.0  | 406  | 1.7746          | 0.56     | 0.5818     | 2.3715 | 0.56     | 0.5557   | 0.1643 | 0.2034 |
| No log        | 59.0  | 413  | 1.7828          | 0.5575   | 0.5868     | 2.3086 | 0.5575   | 0.5526   | 0.1956 | 0.2088 |
| No log        | 60.0  | 420  | 1.7735          | 0.565    | 0.5825     | 2.3405 | 0.565    | 0.5619   | 0.1696 | 0.2058 |
| No log        | 61.0  | 427  | 1.7651          | 0.5675   | 0.5760     | 2.4771 | 0.5675   | 0.5636   | 0.1847 | 0.2027 |
| No log        | 62.0  | 434  | 1.7751          | 0.5575   | 0.5834     | 2.3727 | 0.5575   | 0.5524   | 0.1638 | 0.2052 |
| No log        | 63.0  | 441  | 1.7900          | 0.56     | 0.5834     | 2.3635 | 0.56     | 0.5502   | 0.1789 | 0.2061 |
| No log        | 64.0  | 448  | 1.7729          | 0.56     | 0.5821     | 2.3797 | 0.56     | 0.5554   | 0.1676 | 0.2046 |
| No log        | 65.0  | 455  | 1.7743          | 0.5625   | 0.5826     | 2.4174 | 0.5625   | 0.5581   | 0.1538 | 0.2052 |
| No log        | 66.0  | 462  | 1.7749          | 0.5625   | 0.5801     | 2.3799 | 0.5625   | 0.5592   | 0.1709 | 0.2036 |
| No log        | 67.0  | 469  | 1.7795          | 0.5625   | 0.5814     | 2.3169 | 0.5625   | 0.5533   | 0.1883 | 0.2037 |
| No log        | 68.0  | 476  | 1.7773          | 0.5675   | 0.5794     | 2.3588 | 0.5675   | 0.5622   | 0.1779 | 0.2013 |
| No log        | 69.0  | 483  | 1.7762          | 0.56     | 0.5793     | 2.3514 | 0.56     | 0.5566   | 0.1707 | 0.2039 |
| No log        | 70.0  | 490  | 1.7762          | 0.5625   | 0.5787     | 2.3620 | 0.5625   | 0.5529   | 0.1607 | 0.2017 |
| No log        | 71.0  | 497  | 1.7740          | 0.5675   | 0.5798     | 2.3235 | 0.5675   | 0.5612   | 0.1637 | 0.2046 |
| 0.4215        | 72.0  | 504  | 1.7739          | 0.56     | 0.5790     | 2.3235 | 0.56     | 0.5542   | 0.1583 | 0.2023 |
| 0.4215        | 73.0  | 511  | 1.7783          | 0.56     | 0.5806     | 2.4187 | 0.56     | 0.5545   | 0.1674 | 0.2040 |
| 0.4215        | 74.0  | 518  | 1.7785          | 0.56     | 0.5805     | 2.3302 | 0.56     | 0.5544   | 0.1748 | 0.2033 |
| 0.4215        | 75.0  | 525  | 1.7777          | 0.5625   | 0.5795     | 2.3321 | 0.5625   | 0.5548   | 0.1754 | 0.2029 |
| 0.4215        | 76.0  | 532  | 1.7785          | 0.565    | 0.5799     | 2.3249 | 0.565    | 0.5586   | 0.1696 | 0.2023 |
| 0.4215        | 77.0  | 539  | 1.7763          | 0.565    | 0.5790     | 2.3561 | 0.565    | 0.5573   | 0.1574 | 0.2022 |
| 0.4215        | 78.0  | 546  | 1.7767          | 0.565    | 0.5790     | 2.3296 | 0.565    | 0.5572   | 0.1633 | 0.2024 |
| 0.4215        | 79.0  | 553  | 1.7763          | 0.565    | 0.5790     | 2.3555 | 0.565    | 0.5580   | 0.1687 | 0.2016 |
| 0.4215        | 80.0  | 560  | 1.7783          | 0.565    | 0.5800     | 2.3254 | 0.565    | 0.5576   | 0.1752 | 0.2032 |
| 0.4215        | 81.0  | 567  | 1.7773          | 0.5675   | 0.5796     | 2.3530 | 0.5675   | 0.5605   | 0.1519 | 0.2023 |
| 0.4215        | 82.0  | 574  | 1.7774          | 0.5625   | 0.5797     | 2.3253 | 0.5625   | 0.5549   | 0.1911 | 0.2028 |
| 0.4215        | 83.0  | 581  | 1.7784          | 0.5625   | 0.5794     | 2.3554 | 0.5625   | 0.5544   | 0.1659 | 0.2030 |
| 0.4215        | 84.0  | 588  | 1.7769          | 0.565    | 0.5793     | 2.3527 | 0.565    | 0.5585   | 0.1588 | 0.2024 |
| 0.4215        | 85.0  | 595  | 1.7787          | 0.565    | 0.5799     | 2.3549 | 0.565    | 0.5576   | 0.1687 | 0.2032 |
| 0.4215        | 86.0  | 602  | 1.7778          | 0.565    | 0.5795     | 2.3548 | 0.565    | 0.5574   | 0.1577 | 0.2029 |
| 0.4215        | 87.0  | 609  | 1.7787          | 0.5625   | 0.5798     | 2.3545 | 0.5625   | 0.5549   | 0.1643 | 0.2032 |
| 0.4215        | 88.0  | 616  | 1.7786          | 0.565    | 0.5796     | 2.3554 | 0.565    | 0.5574   | 0.1667 | 0.2031 |
| 0.4215        | 89.0  | 623  | 1.7785          | 0.565    | 0.5799     | 2.3546 | 0.565    | 0.5574   | 0.1691 | 0.2032 |
| 0.4215        | 90.0  | 630  | 1.7784          | 0.565    | 0.5797     | 2.3548 | 0.565    | 0.5574   | 0.1656 | 0.2031 |
| 0.4215        | 91.0  | 637  | 1.7784          | 0.565    | 0.5797     | 2.3550 | 0.565    | 0.5569   | 0.1753 | 0.2032 |
| 0.4215        | 92.0  | 644  | 1.7786          | 0.565    | 0.5797     | 2.3545 | 0.565    | 0.5574   | 0.1744 | 0.2030 |
| 0.4215        | 93.0  | 651  | 1.7785          | 0.565    | 0.5797     | 2.3545 | 0.565    | 0.5574   | 0.1709 | 0.2031 |
| 0.4215        | 94.0  | 658  | 1.7787          | 0.565    | 0.5797     | 2.3543 | 0.565    | 0.5574   | 0.1704 | 0.2032 |
| 0.4215        | 95.0  | 665  | 1.7787          | 0.565    | 0.5798     | 2.3545 | 0.565    | 0.5574   | 0.1713 | 0.2032 |
| 0.4215        | 96.0  | 672  | 1.7788          | 0.565    | 0.5798     | 2.3549 | 0.565    | 0.5569   | 0.1777 | 0.2031 |
| 0.4215        | 97.0  | 679  | 1.7789          | 0.565    | 0.5798     | 2.3550 | 0.565    | 0.5569   | 0.1677 | 0.2032 |
| 0.4215        | 98.0  | 686  | 1.7789          | 0.565    | 0.5798     | 2.3549 | 0.565    | 0.5569   | 0.1648 | 0.2032 |
| 0.4215        | 99.0  | 693  | 1.7789          | 0.565    | 0.5798     | 2.3548 | 0.565    | 0.5569   | 0.1728 | 0.2032 |
| 0.4215        | 100.0 | 700  | 1.7789          | 0.565    | 0.5798     | 2.3548 | 0.565    | 0.5569   | 0.1677 | 0.2032 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo