
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-small_rvl_cdip_100_examples_per_class_kd_CE

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7493
- Accuracy: 0.6275
- Brier Loss: 0.5677
- Nll: 2.9769
- F1 Micro: 0.6275
- F1 Macro: 0.6250
- Ece: 0.2161
- Aurc: 0.1599

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll     | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:-------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 3.0120          | 0.085    | 0.9612     | 10.2074 | 0.085    | 0.0737   | 0.1585 | 0.8946 |
| No log        | 2.0   | 14   | 2.6380          | 0.12     | 0.9247     | 6.9313  | 0.12     | 0.1257   | 0.1753 | 0.8678 |
| No log        | 3.0   | 21   | 2.1951          | 0.36     | 0.7931     | 5.6390  | 0.36     | 0.3256   | 0.2066 | 0.4054 |
| No log        | 4.0   | 28   | 1.8405          | 0.445    | 0.6971     | 3.5387  | 0.445    | 0.4479   | 0.1889 | 0.3068 |
| No log        | 5.0   | 35   | 1.6213          | 0.525    | 0.6244     | 3.3423  | 0.525    | 0.5188   | 0.1821 | 0.2477 |
| No log        | 6.0   | 42   | 1.5983          | 0.5275   | 0.6177     | 3.1202  | 0.5275   | 0.5220   | 0.1781 | 0.2448 |
| No log        | 7.0   | 49   | 1.6214          | 0.54     | 0.6243     | 3.2514  | 0.54     | 0.5248   | 0.1758 | 0.2535 |
| No log        | 8.0   | 56   | 1.4964          | 0.5675   | 0.5862     | 2.6168  | 0.5675   | 0.5715   | 0.1585 | 0.2200 |
| No log        | 9.0   | 63   | 1.5696          | 0.575    | 0.5893     | 2.9901  | 0.575    | 0.5729   | 0.1851 | 0.2123 |
| No log        | 10.0  | 70   | 1.6620          | 0.54     | 0.6257     | 3.1275  | 0.54     | 0.5425   | 0.2353 | 0.2343 |
| No log        | 11.0  | 77   | 1.6901          | 0.585    | 0.5967     | 3.1708  | 0.585    | 0.5753   | 0.2006 | 0.1900 |
| No log        | 12.0  | 84   | 1.5686          | 0.61     | 0.5645     | 2.9975  | 0.61     | 0.6129   | 0.1904 | 0.1830 |
| No log        | 13.0  | 91   | 1.7390          | 0.5675   | 0.6159     | 3.0248  | 0.5675   | 0.5574   | 0.2200 | 0.2016 |
| No log        | 14.0  | 98   | 1.6423          | 0.59     | 0.5778     | 2.9212  | 0.59     | 0.5827   | 0.2015 | 0.1863 |
| No log        | 15.0  | 105  | 1.6262          | 0.61     | 0.5630     | 2.9492  | 0.61     | 0.6074   | 0.1950 | 0.1700 |
| No log        | 16.0  | 112  | 1.6987          | 0.5925   | 0.5791     | 3.0433  | 0.5925   | 0.5852   | 0.2123 | 0.1674 |
| No log        | 17.0  | 119  | 1.7256          | 0.5975   | 0.5782     | 3.0657  | 0.5975   | 0.5929   | 0.2214 | 0.1713 |
| No log        | 18.0  | 126  | 1.7127          | 0.6125   | 0.5697     | 2.9494  | 0.6125   | 0.6110   | 0.2044 | 0.1706 |
| No log        | 19.0  | 133  | 1.6961          | 0.62     | 0.5627     | 2.8745  | 0.62     | 0.6181   | 0.1972 | 0.1666 |
| No log        | 20.0  | 140  | 1.6784          | 0.6275   | 0.5565     | 2.9077  | 0.6275   | 0.6256   | 0.2005 | 0.1614 |
| No log        | 21.0  | 147  | 1.6699          | 0.62     | 0.5549     | 2.9148  | 0.62     | 0.6189   | 0.2089 | 0.1598 |
| No log        | 22.0  | 154  | 1.6705          | 0.62     | 0.5561     | 2.9207  | 0.62     | 0.6186   | 0.2036 | 0.1593 |
| No log        | 23.0  | 161  | 1.6749          | 0.62     | 0.5576     | 2.8938  | 0.62     | 0.6172   | 0.2017 | 0.1594 |
| No log        | 24.0  | 168  | 1.6811          | 0.62     | 0.5586     | 2.9303  | 0.62     | 0.6176   | 0.2064 | 0.1602 |
| No log        | 25.0  | 175  | 1.6870          | 0.625    | 0.5595     | 2.9457  | 0.625    | 0.6225   | 0.1996 | 0.1600 |
| No log        | 26.0  | 182  | 1.6905          | 0.625    | 0.5600     | 2.9438  | 0.625    | 0.6228   | 0.1957 | 0.1604 |
| No log        | 27.0  | 189  | 1.6920          | 0.625    | 0.5601     | 2.9207  | 0.625    | 0.6228   | 0.2030 | 0.1603 |
| No log        | 28.0  | 196  | 1.6928          | 0.6225   | 0.5596     | 2.9140  | 0.6225   | 0.6201   | 0.2104 | 0.1598 |
| No log        | 29.0  | 203  | 1.6934          | 0.6225   | 0.5596     | 2.9133  | 0.6225   | 0.6201   | 0.2171 | 0.1597 |
| No log        | 30.0  | 210  | 1.6952          | 0.6225   | 0.5600     | 2.9156  | 0.6225   | 0.6199   | 0.2175 | 0.1597 |
| No log        | 31.0  | 217  | 1.6962          | 0.6225   | 0.5604     | 2.9195  | 0.6225   | 0.6199   | 0.2151 | 0.1597 |
| No log        | 32.0  | 224  | 1.6982          | 0.625    | 0.5609     | 2.9466  | 0.625    | 0.6216   | 0.2052 | 0.1598 |
| No log        | 33.0  | 231  | 1.6996          | 0.625    | 0.5610     | 2.9468  | 0.625    | 0.6220   | 0.2073 | 0.1598 |
| No log        | 34.0  | 238  | 1.7008          | 0.625    | 0.5611     | 2.9223  | 0.625    | 0.6220   | 0.2099 | 0.1595 |
| No log        | 35.0  | 245  | 1.7028          | 0.625    | 0.5615     | 2.9159  | 0.625    | 0.6218   | 0.2062 | 0.1597 |
| No log        | 36.0  | 252  | 1.7053          | 0.6275   | 0.5621     | 2.9154  | 0.6275   | 0.6246   | 0.2166 | 0.1598 |
| No log        | 37.0  | 259  | 1.7078          | 0.625    | 0.5628     | 2.9132  | 0.625    | 0.6216   | 0.2113 | 0.1600 |
| No log        | 38.0  | 266  | 1.7098          | 0.6275   | 0.5631     | 2.9119  | 0.6275   | 0.6243   | 0.2209 | 0.1601 |
| No log        | 39.0  | 273  | 1.7112          | 0.625    | 0.5632     | 2.9136  | 0.625    | 0.6221   | 0.2164 | 0.1604 |
| No log        | 40.0  | 280  | 1.7122          | 0.625    | 0.5633     | 2.9183  | 0.625    | 0.6221   | 0.2206 | 0.1603 |
| No log        | 41.0  | 287  | 1.7134          | 0.6275   | 0.5635     | 2.9473  | 0.6275   | 0.6247   | 0.2192 | 0.1602 |
| No log        | 42.0  | 294  | 1.7142          | 0.625    | 0.5636     | 2.9477  | 0.625    | 0.6220   | 0.2172 | 0.1600 |
| No log        | 43.0  | 301  | 1.7152          | 0.6275   | 0.5634     | 2.9471  | 0.6275   | 0.6245   | 0.2090 | 0.1598 |
| No log        | 44.0  | 308  | 1.7160          | 0.6275   | 0.5634     | 2.9175  | 0.6275   | 0.6245   | 0.2074 | 0.1597 |
| No log        | 45.0  | 315  | 1.7172          | 0.6275   | 0.5637     | 2.9171  | 0.6275   | 0.6245   | 0.2138 | 0.1597 |
| No log        | 46.0  | 322  | 1.7188          | 0.63     | 0.5640     | 2.9184  | 0.63     | 0.6272   | 0.2138 | 0.1597 |
| No log        | 47.0  | 329  | 1.7204          | 0.63     | 0.5642     | 2.9171  | 0.63     | 0.6277   | 0.2146 | 0.1599 |
| No log        | 48.0  | 336  | 1.7220          | 0.63     | 0.5643     | 2.9178  | 0.63     | 0.6277   | 0.2088 | 0.1599 |
| No log        | 49.0  | 343  | 1.7233          | 0.6325   | 0.5643     | 2.9162  | 0.6325   | 0.6296   | 0.2114 | 0.1597 |
| No log        | 50.0  | 350  | 1.7244          | 0.6325   | 0.5644     | 2.9149  | 0.6325   | 0.6296   | 0.2117 | 0.1598 |
| No log        | 51.0  | 357  | 1.7253          | 0.6325   | 0.5645     | 2.9168  | 0.6325   | 0.6296   | 0.2078 | 0.1597 |
| No log        | 52.0  | 364  | 1.7260          | 0.63     | 0.5647     | 2.9198  | 0.63     | 0.6271   | 0.2002 | 0.1598 |
| No log        | 53.0  | 371  | 1.7268          | 0.63     | 0.5649     | 2.9230  | 0.63     | 0.6270   | 0.2068 | 0.1596 |
| No log        | 54.0  | 378  | 1.7271          | 0.6275   | 0.5649     | 2.9547  | 0.6275   | 0.6241   | 0.2019 | 0.1599 |
| No log        | 55.0  | 385  | 1.7281          | 0.6275   | 0.5652     | 2.9814  | 0.6275   | 0.6241   | 0.2084 | 0.1599 |
| No log        | 56.0  | 392  | 1.7293          | 0.6275   | 0.5652     | 2.9522  | 0.6275   | 0.6241   | 0.2086 | 0.1599 |
| No log        | 57.0  | 399  | 1.7306          | 0.6275   | 0.5653     | 2.9227  | 0.6275   | 0.6244   | 0.2160 | 0.1600 |
| No log        | 58.0  | 406  | 1.7315          | 0.6275   | 0.5654     | 2.9203  | 0.6275   | 0.6244   | 0.2140 | 0.1598 |
| No log        | 59.0  | 413  | 1.7322          | 0.6275   | 0.5655     | 2.9190  | 0.6275   | 0.6244   | 0.2229 | 0.1600 |
| No log        | 60.0  | 420  | 1.7333          | 0.6275   | 0.5657     | 2.9184  | 0.6275   | 0.6250   | 0.2150 | 0.1600 |
| No log        | 61.0  | 427  | 1.7343          | 0.63     | 0.5658     | 2.9166  | 0.63     | 0.6273   | 0.2304 | 0.1599 |
| No log        | 62.0  | 434  | 1.7351          | 0.63     | 0.5660     | 2.9230  | 0.63     | 0.6275   | 0.2154 | 0.1598 |
| No log        | 63.0  | 441  | 1.7354          | 0.63     | 0.5660     | 2.9476  | 0.63     | 0.6275   | 0.2056 | 0.1597 |
| No log        | 64.0  | 448  | 1.7359          | 0.63     | 0.5661     | 2.9483  | 0.63     | 0.6275   | 0.2050 | 0.1598 |
| No log        | 65.0  | 455  | 1.7366          | 0.6275   | 0.5661     | 2.9515  | 0.6275   | 0.6250   | 0.2053 | 0.1600 |
| No log        | 66.0  | 462  | 1.7371          | 0.6275   | 0.5661     | 2.9588  | 0.6275   | 0.6250   | 0.2110 | 0.1600 |
| No log        | 67.0  | 469  | 1.7378          | 0.6275   | 0.5663     | 2.9780  | 0.6275   | 0.6250   | 0.2108 | 0.1599 |
| No log        | 68.0  | 476  | 1.7384          | 0.6275   | 0.5663     | 2.9530  | 0.6275   | 0.6250   | 0.2150 | 0.1599 |
| No log        | 69.0  | 483  | 1.7392          | 0.63     | 0.5663     | 2.9631  | 0.63     | 0.6275   | 0.2114 | 0.1596 |
| No log        | 70.0  | 490  | 1.7398          | 0.63     | 0.5663     | 2.9778  | 0.63     | 0.6275   | 0.2129 | 0.1596 |
| No log        | 71.0  | 497  | 1.7402          | 0.63     | 0.5664     | 2.9544  | 0.63     | 0.6275   | 0.2227 | 0.1596 |
| 0.1799        | 72.0  | 504  | 1.7408          | 0.63     | 0.5665     | 2.9521  | 0.63     | 0.6275   | 0.2157 | 0.1596 |
| 0.1799        | 73.0  | 511  | 1.7412          | 0.63     | 0.5666     | 2.9508  | 0.63     | 0.6275   | 0.2262 | 0.1596 |
| 0.1799        | 74.0  | 518  | 1.7417          | 0.63     | 0.5666     | 2.9509  | 0.63     | 0.6272   | 0.2248 | 0.1596 |
| 0.1799        | 75.0  | 525  | 1.7420          | 0.63     | 0.5666     | 2.9555  | 0.63     | 0.6272   | 0.2219 | 0.1596 |
| 0.1799        | 76.0  | 532  | 1.7425          | 0.63     | 0.5667     | 2.9541  | 0.63     | 0.6268   | 0.2233 | 0.1596 |
| 0.1799        | 77.0  | 539  | 1.7430          | 0.63     | 0.5668     | 2.9773  | 0.63     | 0.6276   | 0.2133 | 0.1596 |
| 0.1799        | 78.0  | 546  | 1.7435          | 0.63     | 0.5668     | 2.9772  | 0.63     | 0.6276   | 0.2134 | 0.1597 |
| 0.1799        | 79.0  | 553  | 1.7439          | 0.63     | 0.5669     | 2.9514  | 0.63     | 0.6276   | 0.2142 | 0.1596 |
| 0.1799        | 80.0  | 560  | 1.7444          | 0.6325   | 0.5669     | 2.9499  | 0.6325   | 0.6303   | 0.2118 | 0.1594 |
| 0.1799        | 81.0  | 567  | 1.7451          | 0.6325   | 0.5669     | 2.9506  | 0.6325   | 0.6303   | 0.2078 | 0.1594 |
| 0.1799        | 82.0  | 574  | 1.7455          | 0.6325   | 0.5670     | 2.9617  | 0.6325   | 0.6303   | 0.2079 | 0.1594 |
| 0.1799        | 83.0  | 581  | 1.7459          | 0.6325   | 0.5671     | 2.9766  | 0.6325   | 0.6303   | 0.2130 | 0.1594 |
| 0.1799        | 84.0  | 588  | 1.7463          | 0.63     | 0.5672     | 2.9770  | 0.63     | 0.6278   | 0.2085 | 0.1597 |
| 0.1799        | 85.0  | 595  | 1.7466          | 0.6275   | 0.5672     | 2.9768  | 0.6275   | 0.6250   | 0.2111 | 0.1598 |
| 0.1799        | 86.0  | 602  | 1.7469          | 0.63     | 0.5673     | 2.9769  | 0.63     | 0.6278   | 0.2086 | 0.1597 |
| 0.1799        | 87.0  | 609  | 1.7472          | 0.6275   | 0.5673     | 2.9770  | 0.6275   | 0.6250   | 0.2140 | 0.1598 |
| 0.1799        | 88.0  | 616  | 1.7474          | 0.6275   | 0.5674     | 2.9771  | 0.6275   | 0.6250   | 0.2111 | 0.1598 |
| 0.1799        | 89.0  | 623  | 1.7477          | 0.6275   | 0.5674     | 2.9772  | 0.6275   | 0.6250   | 0.2112 | 0.1598 |
| 0.1799        | 90.0  | 630  | 1.7480          | 0.6275   | 0.5675     | 2.9770  | 0.6275   | 0.6250   | 0.2112 | 0.1598 |
| 0.1799        | 91.0  | 637  | 1.7483          | 0.6275   | 0.5675     | 2.9770  | 0.6275   | 0.6250   | 0.2112 | 0.1599 |
| 0.1799        | 92.0  | 644  | 1.7485          | 0.6275   | 0.5676     | 2.9769  | 0.6275   | 0.6250   | 0.2112 | 0.1598 |
| 0.1799        | 93.0  | 651  | 1.7486          | 0.6275   | 0.5676     | 2.9770  | 0.6275   | 0.6250   | 0.2112 | 0.1598 |
| 0.1799        | 94.0  | 658  | 1.7488          | 0.6275   | 0.5676     | 2.9770  | 0.6275   | 0.6250   | 0.2131 | 0.1598 |
| 0.1799        | 95.0  | 665  | 1.7489          | 0.6275   | 0.5676     | 2.9768  | 0.6275   | 0.6250   | 0.2143 | 0.1598 |
| 0.1799        | 96.0  | 672  | 1.7491          | 0.6275   | 0.5676     | 2.9768  | 0.6275   | 0.6250   | 0.2161 | 0.1599 |
| 0.1799        | 97.0  | 679  | 1.7492          | 0.6275   | 0.5676     | 2.9768  | 0.6275   | 0.6250   | 0.2161 | 0.1599 |
| 0.1799        | 98.0  | 686  | 1.7493          | 0.6275   | 0.5677     | 2.9768  | 0.6275   | 0.6250   | 0.2161 | 0.1599 |
| 0.1799        | 99.0  | 693  | 1.7493          | 0.6275   | 0.5677     | 2.9769  | 0.6275   | 0.6250   | 0.2161 | 0.1599 |
| 0.1799        | 100.0 | 700  | 1.7493          | 0.6275   | 0.5677     | 2.9769  | 0.6275   | 0.6250   | 0.2161 | 0.1599 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo