
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_NKD_t1.0_g1.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 5.4262
- Accuracy: 0.535
- Brier Loss: 0.6080
- Nll: 2.4569
- F1 Micro: 0.535
- F1 Macro: 0.5345
- Ece: 0.2120
- Aurc: 0.2105

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 8.7328          | 0.0375   | 1.0712     | 7.1959 | 0.0375   | 0.0284   | 0.2770 | 0.9564 |
| No log        | 2.0   | 14   | 6.6754          | 0.0725   | 0.9650     | 6.2033 | 0.0725   | 0.0442   | 0.1890 | 0.9239 |
| No log        | 3.0   | 21   | 6.2399          | 0.1525   | 0.9213     | 5.6927 | 0.1525   | 0.1339   | 0.1647 | 0.8105 |
| No log        | 4.0   | 28   | 5.9557          | 0.2125   | 0.8771     | 4.2773 | 0.2125   | 0.1952   | 0.1637 | 0.6347 |
| No log        | 5.0   | 35   | 5.5957          | 0.3275   | 0.7962     | 3.1599 | 0.3275   | 0.3087   | 0.1928 | 0.4595 |
| No log        | 6.0   | 42   | 5.4573          | 0.4075   | 0.7283     | 3.0860 | 0.4075   | 0.3732   | 0.1803 | 0.3610 |
| No log        | 7.0   | 49   | 5.3445          | 0.43     | 0.7150     | 2.9840 | 0.4300   | 0.4112   | 0.2071 | 0.3498 |
| No log        | 8.0   | 56   | 5.2986          | 0.445    | 0.6860     | 2.9281 | 0.445    | 0.4327   | 0.1891 | 0.3159 |
| No log        | 9.0   | 63   | 5.2567          | 0.465    | 0.6608     | 3.0783 | 0.465    | 0.4494   | 0.1835 | 0.2885 |
| No log        | 10.0  | 70   | 5.4317          | 0.4425   | 0.6752     | 2.9067 | 0.4425   | 0.4477   | 0.1836 | 0.3209 |
| No log        | 11.0  | 77   | 5.2073          | 0.4825   | 0.6342     | 2.7960 | 0.4825   | 0.4664   | 0.1731 | 0.2708 |
| No log        | 12.0  | 84   | 5.2094          | 0.51     | 0.6150     | 2.8886 | 0.51     | 0.4767   | 0.1463 | 0.2459 |
| No log        | 13.0  | 91   | 5.2379          | 0.4975   | 0.6135     | 2.7446 | 0.4975   | 0.4819   | 0.1702 | 0.2436 |
| No log        | 14.0  | 98   | 5.2716          | 0.505    | 0.6123     | 2.7888 | 0.505    | 0.4962   | 0.1595 | 0.2496 |
| No log        | 15.0  | 105  | 5.2410          | 0.5125   | 0.6090     | 2.6474 | 0.5125   | 0.5144   | 0.1659 | 0.2451 |
| No log        | 16.0  | 112  | 5.2764          | 0.54     | 0.6063     | 2.7700 | 0.54     | 0.5316   | 0.1634 | 0.2383 |
| No log        | 17.0  | 119  | 5.2652          | 0.5275   | 0.6027     | 2.7934 | 0.5275   | 0.5171   | 0.1800 | 0.2326 |
| No log        | 18.0  | 126  | 5.2145          | 0.54     | 0.5944     | 2.6456 | 0.54     | 0.5350   | 0.1548 | 0.2323 |
| No log        | 19.0  | 133  | 5.2611          | 0.5175   | 0.6096     | 2.5302 | 0.5175   | 0.5273   | 0.1709 | 0.2435 |
| No log        | 20.0  | 140  | 5.3536          | 0.52     | 0.6110     | 2.6530 | 0.52     | 0.5229   | 0.1619 | 0.2359 |
| No log        | 21.0  | 147  | 5.3020          | 0.5125   | 0.6060     | 2.7184 | 0.5125   | 0.5070   | 0.1724 | 0.2398 |
| No log        | 22.0  | 154  | 5.2107          | 0.5275   | 0.5926     | 2.4436 | 0.5275   | 0.5242   | 0.1618 | 0.2255 |
| No log        | 23.0  | 161  | 5.2723          | 0.53     | 0.5953     | 2.7008 | 0.53     | 0.5209   | 0.1698 | 0.2253 |
| No log        | 24.0  | 168  | 5.1615          | 0.5325   | 0.5875     | 2.4753 | 0.5325   | 0.5254   | 0.1699 | 0.2247 |
| No log        | 25.0  | 175  | 5.1795          | 0.5375   | 0.5825     | 2.6856 | 0.5375   | 0.5316   | 0.1781 | 0.2144 |
| No log        | 26.0  | 182  | 5.2340          | 0.54     | 0.5937     | 2.6542 | 0.54     | 0.5271   | 0.1778 | 0.2215 |
| No log        | 27.0  | 189  | 5.2197          | 0.5375   | 0.5831     | 2.4800 | 0.5375   | 0.5366   | 0.1666 | 0.2119 |
| No log        | 28.0  | 196  | 5.2345          | 0.5275   | 0.6105     | 2.6475 | 0.5275   | 0.5247   | 0.1919 | 0.2338 |
| No log        | 29.0  | 203  | 5.2050          | 0.5475   | 0.5917     | 2.6350 | 0.5475   | 0.5531   | 0.1753 | 0.2251 |
| No log        | 30.0  | 210  | 5.1753          | 0.5425   | 0.5891     | 2.6472 | 0.5425   | 0.5282   | 0.1831 | 0.2215 |
| No log        | 31.0  | 217  | 5.2349          | 0.535    | 0.5946     | 2.5653 | 0.535    | 0.5257   | 0.1617 | 0.2186 |
| No log        | 32.0  | 224  | 5.1497          | 0.545    | 0.5778     | 2.6174 | 0.545    | 0.5425   | 0.1716 | 0.2138 |
| No log        | 33.0  | 231  | 5.1688          | 0.5175   | 0.5899     | 2.5079 | 0.5175   | 0.5149   | 0.1624 | 0.2159 |
| No log        | 34.0  | 238  | 5.2269          | 0.53     | 0.5961     | 2.5188 | 0.53     | 0.5326   | 0.1746 | 0.2206 |
| No log        | 35.0  | 245  | 5.1477          | 0.5325   | 0.5867     | 2.4762 | 0.5325   | 0.5369   | 0.1728 | 0.2176 |
| No log        | 36.0  | 252  | 5.2229          | 0.5375   | 0.5838     | 2.4397 | 0.5375   | 0.5386   | 0.1693 | 0.2167 |
| No log        | 37.0  | 259  | 5.1578          | 0.535    | 0.5802     | 2.5103 | 0.535    | 0.5286   | 0.1755 | 0.2124 |
| No log        | 38.0  | 266  | 5.1405          | 0.535    | 0.5979     | 2.5852 | 0.535    | 0.5346   | 0.1913 | 0.2268 |
| No log        | 39.0  | 273  | 5.1236          | 0.535    | 0.5844     | 2.4851 | 0.535    | 0.5378   | 0.1729 | 0.2168 |
| No log        | 40.0  | 280  | 5.0813          | 0.5475   | 0.5757     | 2.4305 | 0.5475   | 0.5434   | 0.1781 | 0.2091 |
| No log        | 41.0  | 287  | 5.1844          | 0.535    | 0.5888     | 2.4730 | 0.535    | 0.5306   | 0.1707 | 0.2159 |
| No log        | 42.0  | 294  | 5.1468          | 0.53     | 0.5926     | 2.4866 | 0.53     | 0.5316   | 0.1776 | 0.2200 |
| No log        | 43.0  | 301  | 5.1469          | 0.53     | 0.5837     | 2.5769 | 0.53     | 0.5252   | 0.1805 | 0.2168 |
| No log        | 44.0  | 308  | 5.2168          | 0.54     | 0.5955     | 2.5216 | 0.54     | 0.5419   | 0.1689 | 0.2226 |
| No log        | 45.0  | 315  | 5.1395          | 0.525    | 0.5861     | 2.4328 | 0.525    | 0.5293   | 0.2006 | 0.2180 |
| No log        | 46.0  | 322  | 5.1163          | 0.5425   | 0.5822     | 2.4635 | 0.5425   | 0.5416   | 0.1937 | 0.2106 |
| No log        | 47.0  | 329  | 5.1227          | 0.5475   | 0.5786     | 2.5198 | 0.5475   | 0.5489   | 0.1580 | 0.2111 |
| No log        | 48.0  | 336  | 5.1134          | 0.5375   | 0.5839     | 2.5239 | 0.5375   | 0.5318   | 0.1832 | 0.2071 |
| No log        | 49.0  | 343  | 5.1907          | 0.5375   | 0.5913     | 2.5012 | 0.5375   | 0.5334   | 0.1853 | 0.2145 |
| No log        | 50.0  | 350  | 5.1364          | 0.5375   | 0.5875     | 2.4105 | 0.5375   | 0.5415   | 0.1857 | 0.2121 |
| No log        | 51.0  | 357  | 5.1739          | 0.5425   | 0.5905     | 2.5208 | 0.5425   | 0.5399   | 0.1894 | 0.2112 |
| No log        | 52.0  | 364  | 5.1635          | 0.5325   | 0.5841     | 2.4658 | 0.5325   | 0.5300   | 0.1924 | 0.2124 |
| No log        | 53.0  | 371  | 5.2055          | 0.5425   | 0.5866     | 2.4800 | 0.5425   | 0.5390   | 0.1983 | 0.2135 |
| No log        | 54.0  | 378  | 5.1547          | 0.5375   | 0.5869     | 2.4575 | 0.5375   | 0.5340   | 0.1839 | 0.2117 |
| No log        | 55.0  | 385  | 5.1437          | 0.535    | 0.5838     | 2.4117 | 0.535    | 0.5366   | 0.1914 | 0.2110 |
| No log        | 56.0  | 392  | 5.2042          | 0.5425   | 0.5915     | 2.4286 | 0.5425   | 0.5445   | 0.1905 | 0.2124 |
| No log        | 57.0  | 399  | 5.2084          | 0.5625   | 0.5909     | 2.4774 | 0.5625   | 0.5646   | 0.2006 | 0.2116 |
| No log        | 58.0  | 406  | 5.1844          | 0.545    | 0.5895     | 2.3826 | 0.545    | 0.5466   | 0.1948 | 0.2102 |
| No log        | 59.0  | 413  | 5.1759          | 0.545    | 0.5892     | 2.4790 | 0.545    | 0.5498   | 0.1730 | 0.2143 |
| No log        | 60.0  | 420  | 5.1783          | 0.5475   | 0.5894     | 2.4294 | 0.5475   | 0.5452   | 0.2043 | 0.2087 |
| No log        | 61.0  | 427  | 5.1874          | 0.545    | 0.5879     | 2.4295 | 0.545    | 0.5412   | 0.1959 | 0.2080 |
| No log        | 62.0  | 434  | 5.1861          | 0.5475   | 0.5840     | 2.4513 | 0.5475   | 0.5430   | 0.2097 | 0.2107 |
| No log        | 63.0  | 441  | 5.1608          | 0.545    | 0.5818     | 2.4581 | 0.545    | 0.5450   | 0.1666 | 0.2055 |
| No log        | 64.0  | 448  | 5.2018          | 0.5475   | 0.5911     | 2.4537 | 0.5475   | 0.5448   | 0.1938 | 0.2113 |
| No log        | 65.0  | 455  | 5.2113          | 0.5375   | 0.5953     | 2.4444 | 0.5375   | 0.5360   | 0.1757 | 0.2106 |
| No log        | 66.0  | 462  | 5.1985          | 0.5425   | 0.5897     | 2.4287 | 0.5425   | 0.5377   | 0.1870 | 0.2095 |
| No log        | 67.0  | 469  | 5.2218          | 0.5325   | 0.5856     | 2.4340 | 0.5325   | 0.5320   | 0.1882 | 0.2059 |
| No log        | 68.0  | 476  | 5.2243          | 0.545    | 0.5931     | 2.3923 | 0.545    | 0.5447   | 0.1799 | 0.2120 |
| No log        | 69.0  | 483  | 5.2103          | 0.55     | 0.5881     | 2.4619 | 0.55     | 0.5486   | 0.2084 | 0.2060 |
| No log        | 70.0  | 490  | 5.2370          | 0.55     | 0.5933     | 2.4236 | 0.55     | 0.5521   | 0.1920 | 0.2108 |
| No log        | 71.0  | 497  | 5.2185          | 0.5475   | 0.5890     | 2.4137 | 0.5475   | 0.5435   | 0.2121 | 0.2076 |
| 3.6002        | 72.0  | 504  | 5.2460          | 0.545    | 0.5944     | 2.4704 | 0.545    | 0.5430   | 0.1922 | 0.2117 |
| 3.6002        | 73.0  | 511  | 5.2454          | 0.5425   | 0.5928     | 2.4750 | 0.5425   | 0.5406   | 0.1940 | 0.2080 |
| 3.6002        | 74.0  | 518  | 5.2307          | 0.5575   | 0.5935     | 2.4623 | 0.5575   | 0.5599   | 0.1959 | 0.2071 |
| 3.6002        | 75.0  | 525  | 5.2674          | 0.56     | 0.5877     | 2.4453 | 0.56     | 0.5587   | 0.1956 | 0.2033 |
| 3.6002        | 76.0  | 532  | 5.2263          | 0.5525   | 0.5907     | 2.5044 | 0.5525   | 0.5526   | 0.1862 | 0.2067 |
| 3.6002        | 77.0  | 539  | 5.2498          | 0.55     | 0.5938     | 2.4668 | 0.55     | 0.5467   | 0.2072 | 0.2059 |
| 3.6002        | 78.0  | 546  | 5.2671          | 0.545    | 0.5961     | 2.4394 | 0.545    | 0.5421   | 0.2056 | 0.2093 |
| 3.6002        | 79.0  | 553  | 5.2923          | 0.545    | 0.5950     | 2.4662 | 0.545    | 0.5455   | 0.1833 | 0.2058 |
| 3.6002        | 80.0  | 560  | 5.2854          | 0.555    | 0.5918     | 2.5010 | 0.555    | 0.5526   | 0.2040 | 0.2059 |
| 3.6002        | 81.0  | 567  | 5.3009          | 0.535    | 0.5955     | 2.4253 | 0.535    | 0.5319   | 0.1939 | 0.2101 |
| 3.6002        | 82.0  | 574  | 5.3016          | 0.535    | 0.5979     | 2.4528 | 0.535    | 0.5315   | 0.2020 | 0.2101 |
| 3.6002        | 83.0  | 581  | 5.3262          | 0.545    | 0.5990     | 2.4245 | 0.545    | 0.5422   | 0.1816 | 0.2081 |
| 3.6002        | 84.0  | 588  | 5.3206          | 0.535    | 0.5990     | 2.4519 | 0.535    | 0.5350   | 0.1959 | 0.2121 |
| 3.6002        | 85.0  | 595  | 5.3333          | 0.5375   | 0.5999     | 2.4909 | 0.5375   | 0.5352   | 0.1881 | 0.2109 |
| 3.6002        | 86.0  | 602  | 5.3407          | 0.535    | 0.6008     | 2.5019 | 0.535    | 0.5331   | 0.2087 | 0.2096 |
| 3.6002        | 87.0  | 609  | 5.3413          | 0.5425   | 0.6015     | 2.4753 | 0.5425   | 0.5402   | 0.2147 | 0.2101 |
| 3.6002        | 88.0  | 616  | 5.3716          | 0.5375   | 0.6041     | 2.4290 | 0.5375   | 0.5373   | 0.2234 | 0.2094 |
| 3.6002        | 89.0  | 623  | 5.3639          | 0.535    | 0.6010     | 2.4159 | 0.535    | 0.5319   | 0.2068 | 0.2108 |
| 3.6002        | 90.0  | 630  | 5.3742          | 0.5425   | 0.6030     | 2.4588 | 0.5425   | 0.5420   | 0.2021 | 0.2099 |
| 3.6002        | 91.0  | 637  | 5.3731          | 0.53     | 0.6046     | 2.4580 | 0.53     | 0.5284   | 0.2193 | 0.2122 |
| 3.6002        | 92.0  | 644  | 5.3919          | 0.54     | 0.6051     | 2.4317 | 0.54     | 0.5395   | 0.2057 | 0.2090 |
| 3.6002        | 93.0  | 651  | 5.3947          | 0.54     | 0.6049     | 2.4372 | 0.54     | 0.5385   | 0.2053 | 0.2092 |
| 3.6002        | 94.0  | 658  | 5.4070          | 0.535    | 0.6067     | 2.4600 | 0.535    | 0.5328   | 0.2297 | 0.2108 |
| 3.6002        | 95.0  | 665  | 5.4129          | 0.535    | 0.6071     | 2.4249 | 0.535    | 0.5345   | 0.2186 | 0.2104 |
| 3.6002        | 96.0  | 672  | 5.4137          | 0.535    | 0.6071     | 2.4580 | 0.535    | 0.5339   | 0.2192 | 0.2102 |
| 3.6002        | 97.0  | 679  | 5.4218          | 0.5325   | 0.6079     | 2.4584 | 0.5325   | 0.5316   | 0.2197 | 0.2112 |
| 3.6002        | 98.0  | 686  | 5.4236          | 0.5325   | 0.6083     | 2.4585 | 0.5325   | 0.5311   | 0.2154 | 0.2110 |
| 3.6002        | 99.0  | 693  | 5.4261          | 0.5325   | 0.6081     | 2.4569 | 0.5325   | 0.5316   | 0.2294 | 0.2114 |
| 3.6002        | 100.0 | 700  | 5.4262          | 0.535    | 0.6080     | 2.4569 | 0.535    | 0.5345   | 0.2120 | 0.2105 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo