
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5_rand

This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.9130
- Accuracy: 0.6115
- Brier Loss: 0.5924
- Nll: 2.8591
- F1 Micro: 0.6115
- F1 Macro: 0.6137
- Ece: 0.2201
- Aurc: 0.1762

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| 4.0737        | 1.0   | 1000  | 3.8362          | 0.1948   | 0.8785     | 4.2153 | 0.1948   | 0.1339   | 0.0877 | 0.6770 |
| 3.2534        | 2.0   | 2000  | 3.0129          | 0.3772   | 0.7858     | 3.2453 | 0.3773   | 0.3171   | 0.1532 | 0.4263 |
| 2.802         | 3.0   | 3000  | 2.6716          | 0.4472   | 0.7236     | 3.0740 | 0.4472   | 0.4150   | 0.1479 | 0.3537 |
| 2.5803        | 4.0   | 4000  | 2.4882          | 0.4775   | 0.6581     | 2.9147 | 0.4775   | 0.4658   | 0.0774 | 0.2981 |
| 2.4445        | 5.0   | 5000  | 2.3888          | 0.4753   | 0.6741     | 2.8504 | 0.4753   | 0.4562   | 0.1529 | 0.2912 |
| 2.2315        | 6.0   | 6000  | 2.2439          | 0.5132   | 0.6404     | 2.7536 | 0.5132   | 0.5007   | 0.1179 | 0.2659 |
| 2.0849        | 7.0   | 7000  | 2.2140          | 0.5125   | 0.6441     | 2.8160 | 0.5125   | 0.5119   | 0.1522 | 0.2634 |
| 1.922         | 8.0   | 8000  | 2.0941          | 0.5298   | 0.6202     | 2.7843 | 0.5298   | 0.5267   | 0.1511 | 0.2384 |
| 1.8235        | 9.0   | 9000  | 2.0477          | 0.5453   | 0.5974     | 2.6273 | 0.5453   | 0.5505   | 0.1131 | 0.2252 |
| 1.6864        | 10.0  | 10000 | 2.0429          | 0.5575   | 0.6181     | 2.7309 | 0.5575   | 0.5391   | 0.1828 | 0.2172 |
| 1.5498        | 11.0  | 11000 | 2.0547          | 0.5597   | 0.6272     | 2.6772 | 0.5597   | 0.5444   | 0.1931 | 0.2237 |
| 1.4502        | 12.0  | 12000 | 2.0086          | 0.5707   | 0.6175     | 2.7419 | 0.5707   | 0.5592   | 0.1935 | 0.2153 |
| 1.3331        | 13.0  | 13000 | 2.0399          | 0.566    | 0.6150     | 2.8462 | 0.566    | 0.5660   | 0.1786 | 0.2175 |
| 1.19          | 14.0  | 14000 | 2.0207          | 0.5805   | 0.6209     | 2.7599 | 0.5805   | 0.5793   | 0.2133 | 0.2081 |
| 1.0615        | 15.0  | 15000 | 2.0089          | 0.5793   | 0.6224     | 2.9014 | 0.5793   | 0.5787   | 0.2157 | 0.2077 |
| 0.9517        | 16.0  | 16000 | 2.0644          | 0.5765   | 0.6300     | 2.9717 | 0.5765   | 0.5796   | 0.2264 | 0.2047 |
| 0.8343        | 17.0  | 17000 | 2.0990          | 0.5745   | 0.6473     | 3.0444 | 0.5745   | 0.5712   | 0.2502 | 0.2050 |
| 0.7191        | 18.0  | 18000 | 2.1704          | 0.5763   | 0.6527     | 3.0809 | 0.5763   | 0.5721   | 0.2539 | 0.2125 |
| 0.638         | 19.0  | 19000 | 2.1930          | 0.5807   | 0.6572     | 3.1618 | 0.5807   | 0.5792   | 0.2620 | 0.2120 |
| 0.5528        | 20.0  | 20000 | 2.1731          | 0.5885   | 0.6542     | 3.1412 | 0.5885   | 0.5898   | 0.2627 | 0.2067 |
| 0.4957        | 21.0  | 21000 | 2.2492          | 0.5763   | 0.6708     | 3.1701 | 0.5763   | 0.5760   | 0.2700 | 0.2232 |
| 0.4096        | 22.0  | 22000 | 2.3164          | 0.5707   | 0.6837     | 3.3874 | 0.5707   | 0.5706   | 0.2761 | 0.2260 |
| 0.3915        | 23.0  | 23000 | 2.3277          | 0.58     | 0.6813     | 3.4230 | 0.58     | 0.5814   | 0.2857 | 0.2165 |
| 0.3412        | 24.0  | 24000 | 2.2947          | 0.5813   | 0.6779     | 3.3373 | 0.5813   | 0.5854   | 0.2870 | 0.2076 |
| 0.3171        | 25.0  | 25000 | 2.2743          | 0.586    | 0.6720     | 3.3310 | 0.586    | 0.5848   | 0.2767 | 0.2104 |
| 0.2734        | 26.0  | 26000 | 2.2762          | 0.593    | 0.6696     | 3.3439 | 0.593    | 0.5977   | 0.2819 | 0.2044 |
| 0.2535        | 27.0  | 27000 | 2.2205          | 0.5845   | 0.6605     | 3.2712 | 0.5845   | 0.5864   | 0.2684 | 0.2060 |
| 0.229         | 28.0  | 28000 | 2.2961          | 0.5845   | 0.6821     | 3.2738 | 0.5845   | 0.5902   | 0.2896 | 0.2106 |
| 0.2202        | 29.0  | 29000 | 2.2698          | 0.5845   | 0.6752     | 3.2127 | 0.5845   | 0.5845   | 0.2835 | 0.2128 |
| 0.1922        | 30.0  | 30000 | 2.2511          | 0.5787   | 0.6731     | 3.3305 | 0.5787   | 0.5819   | 0.2770 | 0.2065 |
| 0.1857        | 31.0  | 31000 | 2.1847          | 0.5863   | 0.6598     | 3.2211 | 0.5863   | 0.5889   | 0.2692 | 0.2036 |
| 0.1678        | 32.0  | 32000 | 2.1752          | 0.5913   | 0.6551     | 3.0691 | 0.5913   | 0.5926   | 0.2680 | 0.2010 |
| 0.1587        | 33.0  | 33000 | 2.1107          | 0.5972   | 0.6392     | 3.0412 | 0.5972   | 0.6002   | 0.2495 | 0.2002 |
| 0.1432        | 34.0  | 34000 | 2.2079          | 0.5893   | 0.6593     | 3.1988 | 0.5893   | 0.5917   | 0.2640 | 0.2092 |
| 0.1291        | 35.0  | 35000 | 2.0788          | 0.592    | 0.6388     | 2.9891 | 0.592    | 0.5942   | 0.2587 | 0.1887 |
| 0.1245        | 36.0  | 36000 | 2.0521          | 0.601    | 0.6297     | 2.9432 | 0.601    | 0.6018   | 0.2464 | 0.1923 |
| 0.1169        | 37.0  | 37000 | 2.0669          | 0.5935   | 0.6294     | 3.0364 | 0.5935   | 0.5932   | 0.2425 | 0.1878 |
| 0.1068        | 38.0  | 38000 | 2.0197          | 0.606    | 0.6183     | 2.9448 | 0.606    | 0.6054   | 0.2375 | 0.1897 |
| 0.1031        | 39.0  | 39000 | 2.0022          | 0.6032   | 0.6163     | 2.8697 | 0.6032   | 0.6051   | 0.2429 | 0.1880 |
| 0.0961        | 40.0  | 40000 | 1.9965          | 0.6058   | 0.6147     | 2.9143 | 0.6058   | 0.6091   | 0.2393 | 0.1859 |
| 0.093         | 41.0  | 41000 | 1.9735          | 0.6082   | 0.6080     | 2.9346 | 0.6082   | 0.6100   | 0.2331 | 0.1834 |
| 0.0859        | 42.0  | 42000 | 1.9383          | 0.6092   | 0.6011     | 2.8692 | 0.6092   | 0.6105   | 0.2281 | 0.1792 |
| 0.0817        | 43.0  | 43000 | 1.9413          | 0.6115   | 0.5999     | 2.8908 | 0.6115   | 0.6124   | 0.2264 | 0.1802 |
| 0.0779        | 44.0  | 44000 | 1.9271          | 0.6112   | 0.6001     | 2.8695 | 0.6112   | 0.6127   | 0.2274 | 0.1791 |
| 0.0745        | 45.0  | 45000 | 1.9259          | 0.6135   | 0.5983     | 2.8467 | 0.6135   | 0.6154   | 0.2206 | 0.1786 |
| 0.0723        | 46.0  | 46000 | 1.9214          | 0.612    | 0.5964     | 2.8289 | 0.612    | 0.6136   | 0.2233 | 0.1778 |
| 0.0673        | 47.0  | 47000 | 1.9117          | 0.6128   | 0.5923     | 2.8226 | 0.6128   | 0.6148   | 0.2197 | 0.1768 |
| 0.0649        | 48.0  | 48000 | 1.9111          | 0.6152   | 0.5911     | 2.8575 | 0.6152   | 0.6164   | 0.2166 | 0.1757 |
| 0.0651        | 49.0  | 49000 | 1.9106          | 0.6148   | 0.5914     | 2.8675 | 0.6148   | 0.6169   | 0.2164 | 0.1761 |
| 0.0629        | 50.0  | 50000 | 1.9130          | 0.6115   | 0.5924     | 2.8591 | 0.6115   | 0.6137   | 0.2201 | 0.1762 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo