
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_CEKD_t1.5_a0.7

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5923
- Accuracy: 0.57
- Brier Loss: 0.5750
- Nll: 2.3088
- F1 Micro: 0.57
- F1 Macro: 0.5661
- Ece: 0.1722
- Aurc: 0.2058

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 4.3150          | 0.0425   | 1.0714     | 7.3695 | 0.0425   | 0.0339   | 0.2866 | 0.9549 |
| No log        | 2.0   | 14   | 3.3516          | 0.1      | 0.9507     | 5.8558 | 0.1000   | 0.0866   | 0.1647 | 0.8831 |
| No log        | 3.0   | 21   | 2.9890          | 0.225    | 0.8838     | 5.3580 | 0.225    | 0.1805   | 0.1576 | 0.6316 |
| No log        | 4.0   | 28   | 2.5376          | 0.29     | 0.7946     | 3.5543 | 0.29     | 0.2749   | 0.1832 | 0.4807 |
| No log        | 5.0   | 35   | 2.2193          | 0.3875   | 0.7186     | 3.0794 | 0.3875   | 0.3677   | 0.1577 | 0.3531 |
| No log        | 6.0   | 42   | 2.0818          | 0.43     | 0.6905     | 2.9853 | 0.4300   | 0.4165   | 0.1668 | 0.3056 |
| No log        | 7.0   | 49   | 2.1032          | 0.45     | 0.7019     | 3.1044 | 0.45     | 0.4208   | 0.2121 | 0.2997 |
| No log        | 8.0   | 56   | 2.0360          | 0.455    | 0.6994     | 2.8491 | 0.455    | 0.4492   | 0.2131 | 0.3026 |
| No log        | 9.0   | 63   | 2.1719          | 0.475    | 0.7092     | 2.7831 | 0.4750   | 0.4549   | 0.2332 | 0.2870 |
| No log        | 10.0  | 70   | 2.1820          | 0.4525   | 0.7393     | 2.8185 | 0.4525   | 0.4318   | 0.2813 | 0.2994 |
| No log        | 11.0  | 77   | 2.2659          | 0.4475   | 0.7485     | 2.8020 | 0.4475   | 0.4227   | 0.2677 | 0.3046 |
| No log        | 12.0  | 84   | 2.1798          | 0.4575   | 0.7325     | 2.6772 | 0.4575   | 0.4555   | 0.2738 | 0.3081 |
| No log        | 13.0  | 91   | 2.3031          | 0.465    | 0.7431     | 2.8956 | 0.465    | 0.4390   | 0.2771 | 0.2945 |
| No log        | 14.0  | 98   | 2.0867          | 0.49     | 0.7048     | 2.5312 | 0.49     | 0.4823   | 0.2528 | 0.2921 |
| No log        | 15.0  | 105  | 2.1671          | 0.49     | 0.7218     | 2.7779 | 0.49     | 0.4749   | 0.2396 | 0.2877 |
| No log        | 16.0  | 112  | 2.0091          | 0.485    | 0.6857     | 2.7234 | 0.485    | 0.4608   | 0.2493 | 0.2577 |
| No log        | 17.0  | 119  | 1.9494          | 0.515    | 0.6714     | 2.4841 | 0.515    | 0.5072   | 0.2380 | 0.2614 |
| No log        | 18.0  | 126  | 1.9132          | 0.505    | 0.6665     | 2.4777 | 0.505    | 0.4945   | 0.2206 | 0.2622 |
| No log        | 19.0  | 133  | 2.0539          | 0.505    | 0.6776     | 2.7885 | 0.505    | 0.4986   | 0.2209 | 0.2724 |
| No log        | 20.0  | 140  | 1.9533          | 0.5125   | 0.6666     | 2.7287 | 0.5125   | 0.5044   | 0.2385 | 0.2645 |
| No log        | 21.0  | 147  | 1.9038          | 0.525    | 0.6365     | 2.8345 | 0.525    | 0.5099   | 0.2021 | 0.2290 |
| No log        | 22.0  | 154  | 1.8525          | 0.5075   | 0.6448     | 2.6337 | 0.5075   | 0.4958   | 0.2083 | 0.2494 |
| No log        | 23.0  | 161  | 1.7880          | 0.51     | 0.6386     | 2.4856 | 0.51     | 0.5078   | 0.2186 | 0.2478 |
| No log        | 24.0  | 168  | 1.8363          | 0.505    | 0.6456     | 2.5075 | 0.505    | 0.4966   | 0.1962 | 0.2399 |
| No log        | 25.0  | 175  | 1.9655          | 0.4725   | 0.6864     | 2.6331 | 0.4725   | 0.4608   | 0.2291 | 0.2669 |
| No log        | 26.0  | 182  | 1.8660          | 0.5175   | 0.6547     | 2.5404 | 0.5175   | 0.5076   | 0.2252 | 0.2489 |
| No log        | 27.0  | 189  | 1.8693          | 0.525    | 0.6446     | 2.6230 | 0.525    | 0.5145   | 0.2047 | 0.2540 |
| No log        | 28.0  | 196  | 1.8113          | 0.51     | 0.6407     | 2.4380 | 0.51     | 0.4978   | 0.2030 | 0.2454 |
| No log        | 29.0  | 203  | 1.8313          | 0.53     | 0.6445     | 2.4777 | 0.53     | 0.5284   | 0.2071 | 0.2575 |
| No log        | 30.0  | 210  | 1.7310          | 0.5425   | 0.6197     | 2.4559 | 0.5425   | 0.5384   | 0.1869 | 0.2367 |
| No log        | 31.0  | 217  | 1.8023          | 0.5325   | 0.6351     | 2.5026 | 0.5325   | 0.5216   | 0.2081 | 0.2496 |
| No log        | 32.0  | 224  | 1.7652          | 0.5325   | 0.6186     | 2.4794 | 0.5325   | 0.5145   | 0.1715 | 0.2338 |
| No log        | 33.0  | 231  | 1.7583          | 0.525    | 0.6363     | 2.4889 | 0.525    | 0.5275   | 0.1984 | 0.2463 |
| No log        | 34.0  | 238  | 1.7552          | 0.5475   | 0.6164     | 2.4291 | 0.5475   | 0.5305   | 0.2026 | 0.2377 |
| No log        | 35.0  | 245  | 1.6839          | 0.5375   | 0.6085     | 2.5915 | 0.5375   | 0.5253   | 0.1828 | 0.2214 |
| No log        | 36.0  | 252  | 1.7480          | 0.5425   | 0.6104     | 2.5809 | 0.5425   | 0.5366   | 0.1716 | 0.2232 |
| No log        | 37.0  | 259  | 1.7534          | 0.525    | 0.6225     | 2.3614 | 0.525    | 0.5183   | 0.1930 | 0.2249 |
| No log        | 38.0  | 266  | 1.7484          | 0.5425   | 0.6125     | 2.5714 | 0.5425   | 0.5282   | 0.1792 | 0.2272 |
| No log        | 39.0  | 273  | 1.7073          | 0.55     | 0.6172     | 2.4200 | 0.55     | 0.5370   | 0.1902 | 0.2314 |
| No log        | 40.0  | 280  | 1.7303          | 0.55     | 0.6134     | 2.4829 | 0.55     | 0.5394   | 0.1916 | 0.2324 |
| No log        | 41.0  | 287  | 1.6684          | 0.54     | 0.6060     | 2.4632 | 0.54     | 0.5350   | 0.2028 | 0.2251 |
| No log        | 42.0  | 294  | 1.7171          | 0.5375   | 0.6055     | 2.4705 | 0.5375   | 0.5213   | 0.1776 | 0.2262 |
| No log        | 43.0  | 301  | 1.6493          | 0.545    | 0.5991     | 2.5207 | 0.545    | 0.5412   | 0.1779 | 0.2214 |
| No log        | 44.0  | 308  | 1.6548          | 0.5625   | 0.5920     | 2.4810 | 0.5625   | 0.5568   | 0.1892 | 0.2182 |
| No log        | 45.0  | 315  | 1.6392          | 0.565    | 0.5943     | 2.3771 | 0.565    | 0.5586   | 0.2165 | 0.2162 |
| No log        | 46.0  | 322  | 1.6923          | 0.5225   | 0.6159     | 2.3661 | 0.5225   | 0.5158   | 0.1775 | 0.2400 |
| No log        | 47.0  | 329  | 1.6266          | 0.5525   | 0.5827     | 2.4385 | 0.5525   | 0.5468   | 0.1845 | 0.2100 |
| No log        | 48.0  | 336  | 1.6804          | 0.55     | 0.6019     | 2.3884 | 0.55     | 0.5481   | 0.1895 | 0.2291 |
| No log        | 49.0  | 343  | 1.6202          | 0.5725   | 0.5847     | 2.4882 | 0.5725   | 0.5596   | 0.1642 | 0.2125 |
| No log        | 50.0  | 350  | 1.6222          | 0.54     | 0.5882     | 2.4144 | 0.54     | 0.5311   | 0.1830 | 0.2226 |
| No log        | 51.0  | 357  | 1.6119          | 0.5775   | 0.5794     | 2.4063 | 0.5775   | 0.5731   | 0.1647 | 0.2019 |
| No log        | 52.0  | 364  | 1.5958          | 0.57     | 0.5757     | 2.3342 | 0.57     | 0.5642   | 0.1778 | 0.2094 |
| No log        | 53.0  | 371  | 1.6206          | 0.545    | 0.5913     | 2.3884 | 0.545    | 0.5365   | 0.1799 | 0.2187 |
| No log        | 54.0  | 378  | 1.5982          | 0.5675   | 0.5745     | 2.4276 | 0.5675   | 0.5640   | 0.1746 | 0.2050 |
| No log        | 55.0  | 385  | 1.6258          | 0.5525   | 0.5856     | 2.4005 | 0.5525   | 0.5373   | 0.1890 | 0.2124 |
| No log        | 56.0  | 392  | 1.5763          | 0.57     | 0.5744     | 2.4477 | 0.57     | 0.5729   | 0.1651 | 0.2081 |
| No log        | 57.0  | 399  | 1.6249          | 0.5525   | 0.5861     | 2.3791 | 0.5525   | 0.5432   | 0.1531 | 0.2114 |
| No log        | 58.0  | 406  | 1.6240          | 0.5775   | 0.5791     | 2.4540 | 0.5775   | 0.5730   | 0.1582 | 0.2054 |
| No log        | 59.0  | 413  | 1.6149          | 0.545    | 0.5851     | 2.3134 | 0.545    | 0.5395   | 0.1870 | 0.2137 |
| No log        | 60.0  | 420  | 1.6163          | 0.5775   | 0.5792     | 2.3778 | 0.5775   | 0.5708   | 0.1762 | 0.2076 |
| No log        | 61.0  | 427  | 1.6132          | 0.5575   | 0.5868     | 2.3759 | 0.5575   | 0.5530   | 0.1842 | 0.2159 |
| No log        | 62.0  | 434  | 1.5940          | 0.5725   | 0.5756     | 2.3394 | 0.5725   | 0.5731   | 0.2102 | 0.2054 |
| No log        | 63.0  | 441  | 1.6167          | 0.56     | 0.5841     | 2.4117 | 0.56     | 0.5541   | 0.1806 | 0.2160 |
| No log        | 64.0  | 448  | 1.5988          | 0.57     | 0.5775     | 2.3388 | 0.57     | 0.5667   | 0.1680 | 0.2064 |
| No log        | 65.0  | 455  | 1.5893          | 0.5725   | 0.5752     | 2.4281 | 0.5725   | 0.5695   | 0.1624 | 0.2050 |
| No log        | 66.0  | 462  | 1.5975          | 0.5725   | 0.5737     | 2.3760 | 0.5725   | 0.5662   | 0.1733 | 0.2026 |
| No log        | 67.0  | 469  | 1.5903          | 0.57     | 0.5772     | 2.2921 | 0.57     | 0.5675   | 0.1888 | 0.2112 |
| No log        | 68.0  | 476  | 1.5878          | 0.575    | 0.5730     | 2.3676 | 0.575    | 0.5706   | 0.1683 | 0.2039 |
| No log        | 69.0  | 483  | 1.5950          | 0.57     | 0.5775     | 2.3006 | 0.57     | 0.5641   | 0.1639 | 0.2076 |
| No log        | 70.0  | 490  | 1.5916          | 0.58     | 0.5728     | 2.3424 | 0.58     | 0.5769   | 0.1714 | 0.2026 |
| No log        | 71.0  | 497  | 1.5960          | 0.5675   | 0.5784     | 2.3057 | 0.5675   | 0.5624   | 0.1600 | 0.2073 |
| 0.3705        | 72.0  | 504  | 1.5907          | 0.575    | 0.5755     | 2.3322 | 0.575    | 0.5723   | 0.1578 | 0.2066 |
| 0.3705        | 73.0  | 511  | 1.5918          | 0.5675   | 0.5762     | 2.3182 | 0.5675   | 0.5605   | 0.1942 | 0.2071 |
| 0.3705        | 74.0  | 518  | 1.5894          | 0.585    | 0.5747     | 2.3335 | 0.585    | 0.5818   | 0.1739 | 0.2035 |
| 0.3705        | 75.0  | 525  | 1.5878          | 0.565    | 0.5750     | 2.3019 | 0.565    | 0.5607   | 0.1649 | 0.2060 |
| 0.3705        | 76.0  | 532  | 1.5923          | 0.575    | 0.5742     | 2.3376 | 0.575    | 0.5699   | 0.1779 | 0.2048 |
| 0.3705        | 77.0  | 539  | 1.5891          | 0.565    | 0.5760     | 2.2978 | 0.565    | 0.5616   | 0.1691 | 0.2066 |
| 0.3705        | 78.0  | 546  | 1.5896          | 0.575    | 0.5738     | 2.3748 | 0.575    | 0.5703   | 0.1733 | 0.2048 |
| 0.3705        | 79.0  | 553  | 1.5901          | 0.5675   | 0.5757     | 2.3039 | 0.5675   | 0.5634   | 0.1710 | 0.2064 |
| 0.3705        | 80.0  | 560  | 1.5906          | 0.57     | 0.5746     | 2.3125 | 0.57     | 0.5657   | 0.1692 | 0.2054 |
| 0.3705        | 81.0  | 567  | 1.5907          | 0.57     | 0.5751     | 2.3097 | 0.57     | 0.5659   | 0.1600 | 0.2047 |
| 0.3705        | 82.0  | 574  | 1.5902          | 0.57     | 0.5746     | 2.3072 | 0.57     | 0.5657   | 0.1797 | 0.2055 |
| 0.3705        | 83.0  | 581  | 1.5906          | 0.5725   | 0.5746     | 2.3145 | 0.5725   | 0.5681   | 0.1547 | 0.2050 |
| 0.3705        | 84.0  | 588  | 1.5909          | 0.5725   | 0.5750     | 2.3057 | 0.5725   | 0.5684   | 0.1746 | 0.2055 |
| 0.3705        | 85.0  | 595  | 1.5906          | 0.57     | 0.5746     | 2.3098 | 0.57     | 0.5661   | 0.1721 | 0.2054 |
| 0.3705        | 86.0  | 602  | 1.5916          | 0.57     | 0.5749     | 2.3093 | 0.57     | 0.5661   | 0.1659 | 0.2058 |
| 0.3705        | 87.0  | 609  | 1.5913          | 0.57     | 0.5748     | 2.3084 | 0.57     | 0.5661   | 0.1631 | 0.2058 |
| 0.3705        | 88.0  | 616  | 1.5918          | 0.57     | 0.5749     | 2.3082 | 0.57     | 0.5661   | 0.1652 | 0.2058 |
| 0.3705        | 89.0  | 623  | 1.5919          | 0.57     | 0.5750     | 2.3084 | 0.57     | 0.5661   | 0.1658 | 0.2059 |
| 0.3705        | 90.0  | 630  | 1.5918          | 0.5725   | 0.5749     | 2.3087 | 0.5725   | 0.5685   | 0.1650 | 0.2056 |
| 0.3705        | 91.0  | 637  | 1.5921          | 0.57     | 0.5750     | 2.3076 | 0.57     | 0.5661   | 0.1549 | 0.2059 |
| 0.3705        | 92.0  | 644  | 1.5920          | 0.57     | 0.5750     | 2.3079 | 0.57     | 0.5661   | 0.1581 | 0.2058 |
| 0.3705        | 93.0  | 651  | 1.5917          | 0.57     | 0.5749     | 2.3080 | 0.57     | 0.5661   | 0.1680 | 0.2057 |
| 0.3705        | 94.0  | 658  | 1.5923          | 0.57     | 0.5750     | 2.3083 | 0.57     | 0.5661   | 0.1643 | 0.2058 |
| 0.3705        | 95.0  | 665  | 1.5924          | 0.57     | 0.5751     | 2.3085 | 0.57     | 0.5661   | 0.1543 | 0.2059 |
| 0.3705        | 96.0  | 672  | 1.5922          | 0.57     | 0.5750     | 2.3085 | 0.57     | 0.5661   | 0.1530 | 0.2058 |
| 0.3705        | 97.0  | 679  | 1.5923          | 0.57     | 0.5750     | 2.3088 | 0.57     | 0.5661   | 0.1688 | 0.2058 |
| 0.3705        | 98.0  | 686  | 1.5923          | 0.57     | 0.5749     | 2.3089 | 0.57     | 0.5661   | 0.1733 | 0.2058 |
| 0.3705        | 99.0  | 693  | 1.5923          | 0.57     | 0.5750     | 2.3088 | 0.57     | 0.5661   | 0.1735 | 0.2058 |
| 0.3705        | 100.0 | 700  | 1.5923          | 0.57     | 0.5750     | 2.3088 | 0.57     | 0.5661   | 0.1722 | 0.2058 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo