
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_MSE_rand

This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.8136
- Accuracy: 0.6262
- Brier Loss: 0.5539
- Nll: 2.6914
- F1 Micro: 0.6262
- F1 Macro: 0.6298
- Ece: 0.1916
- Aurc: 0.1624

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| 4.1455        | 1.0   | 1000  | 3.9403          | 0.1898   | 0.8859     | 5.5812 | 0.1898   | 0.1327   | 0.0656 | 0.6790 |
| 3.4397        | 2.0   | 2000  | 3.3498          | 0.346    | 0.8036     | 4.5993 | 0.346    | 0.2909   | 0.1060 | 0.4698 |
| 2.983         | 3.0   | 3000  | 3.1592          | 0.384    | 0.8070     | 3.7537 | 0.384    | 0.3451   | 0.1795 | 0.4280 |
| 2.7406        | 4.0   | 4000  | 2.7614          | 0.4447   | 0.6956     | 3.6359 | 0.4447   | 0.4361   | 0.0840 | 0.3307 |
| 2.5937        | 5.0   | 5000  | 2.7073          | 0.4462   | 0.7000     | 3.5795 | 0.4462   | 0.4321   | 0.1292 | 0.3212 |
| 2.3878        | 6.0   | 6000  | 2.3935          | 0.5012   | 0.6651     | 3.1888 | 0.5012   | 0.4842   | 0.1429 | 0.2829 |
| 2.2284        | 7.0   | 7000  | 2.3189          | 0.5022   | 0.6378     | 3.0628 | 0.5022   | 0.5109   | 0.1027 | 0.2630 |
| 2.0759        | 8.0   | 8000  | 2.3408          | 0.4993   | 0.6550     | 3.3921 | 0.4993   | 0.4923   | 0.1398 | 0.2640 |
| 1.9764        | 9.0   | 9000  | 2.0531          | 0.5563   | 0.5946     | 2.9188 | 0.5563   | 0.5619   | 0.1202 | 0.2170 |
| 1.8232        | 10.0  | 10000 | 2.1083          | 0.5505   | 0.6295     | 3.0945 | 0.5505   | 0.5445   | 0.1794 | 0.2322 |
| 1.7049        | 11.0  | 11000 | 2.0447          | 0.5653   | 0.6142     | 2.9718 | 0.5653   | 0.5605   | 0.1684 | 0.2207 |
| 1.6182        | 12.0  | 12000 | 2.0684          | 0.5637   | 0.6462     | 3.0100 | 0.5637   | 0.5595   | 0.2095 | 0.2272 |
| 1.4886        | 13.0  | 13000 | 1.9374          | 0.5735   | 0.6132     | 2.9415 | 0.5735   | 0.5806   | 0.1874 | 0.2042 |
| 1.3538        | 14.0  | 14000 | 2.0147          | 0.5895   | 0.6174     | 3.0835 | 0.5895   | 0.5851   | 0.1966 | 0.2109 |
| 1.2304        | 15.0  | 15000 | 1.9766          | 0.5867   | 0.6203     | 3.1471 | 0.5867   | 0.5846   | 0.2229 | 0.2091 |
| 1.1124        | 16.0  | 16000 | 1.8998          | 0.6008   | 0.6044     | 2.9169 | 0.6008   | 0.5943   | 0.2144 | 0.1911 |
| 1.0197        | 17.0  | 17000 | 1.9309          | 0.5955   | 0.6123     | 3.1166 | 0.5955   | 0.5979   | 0.2299 | 0.1876 |
| 0.8763        | 18.0  | 18000 | 1.9741          | 0.5952   | 0.6316     | 3.2227 | 0.5952   | 0.5971   | 0.2439 | 0.1957 |
| 0.8042        | 19.0  | 19000 | 1.9944          | 0.592    | 0.6318     | 3.1537 | 0.592    | 0.5898   | 0.2439 | 0.2024 |
| 0.7059        | 20.0  | 20000 | 1.9259          | 0.6082   | 0.6124     | 3.0665 | 0.6082   | 0.6093   | 0.2344 | 0.1889 |
| 0.632         | 21.0  | 21000 | 1.9444          | 0.6095   | 0.6148     | 3.0133 | 0.6095   | 0.6111   | 0.2281 | 0.1917 |
| 0.5641        | 22.0  | 22000 | 1.9830          | 0.5968   | 0.6282     | 3.0999 | 0.5968   | 0.5984   | 0.2442 | 0.1913 |
| 0.5138        | 23.0  | 23000 | 2.0190          | 0.5962   | 0.6331     | 3.1937 | 0.5962   | 0.5966   | 0.2501 | 0.2033 |
| 0.457         | 24.0  | 24000 | 1.9340          | 0.6075   | 0.6151     | 2.9559 | 0.6075   | 0.6096   | 0.2333 | 0.1888 |
| 0.3999        | 25.0  | 25000 | 1.9742          | 0.6048   | 0.6285     | 3.0455 | 0.6048   | 0.6080   | 0.2461 | 0.1939 |
| 0.3629        | 26.0  | 26000 | 1.9308          | 0.6142   | 0.6027     | 3.1686 | 0.6142   | 0.6169   | 0.2244 | 0.1850 |
| 0.3132        | 27.0  | 27000 | 1.9468          | 0.6175   | 0.6076     | 3.0271 | 0.6175   | 0.6189   | 0.2374 | 0.1863 |
| 0.2818        | 28.0  | 28000 | 1.9392          | 0.6095   | 0.6152     | 3.0499 | 0.6095   | 0.6079   | 0.2422 | 0.1894 |
| 0.2584        | 29.0  | 29000 | 1.8976          | 0.6202   | 0.6040     | 2.9355 | 0.6202   | 0.6204   | 0.2340 | 0.1834 |
| 0.228         | 30.0  | 30000 | 1.9111          | 0.617    | 0.6020     | 3.0272 | 0.617    | 0.6192   | 0.2336 | 0.1780 |
| 0.2041        | 31.0  | 31000 | 1.8513          | 0.6272   | 0.5835     | 2.8808 | 0.6272   | 0.6293   | 0.2222 | 0.1733 |
| 0.1834        | 32.0  | 32000 | 1.8501          | 0.6262   | 0.5782     | 2.8280 | 0.6262   | 0.6275   | 0.2142 | 0.1702 |
| 0.1613        | 33.0  | 33000 | 1.8250          | 0.6292   | 0.5712     | 2.8863 | 0.6292   | 0.6338   | 0.2021 | 0.1691 |
| 0.1437        | 34.0  | 34000 | 1.8457          | 0.6228   | 0.5773     | 2.9046 | 0.6228   | 0.6232   | 0.2114 | 0.1717 |
| 0.1275        | 35.0  | 35000 | 1.8088          | 0.6315   | 0.5646     | 2.8124 | 0.6315   | 0.6328   | 0.2039 | 0.1638 |
| 0.1127        | 36.0  | 36000 | 1.8204          | 0.6335   | 0.5647     | 2.7943 | 0.6335   | 0.6373   | 0.1993 | 0.1661 |
| 0.1026        | 37.0  | 37000 | 1.8070          | 0.631    | 0.5641     | 2.7537 | 0.631    | 0.6326   | 0.2015 | 0.1634 |
| 0.0894        | 38.0  | 38000 | 1.8068          | 0.63     | 0.5606     | 2.7461 | 0.63     | 0.6317   | 0.1998 | 0.1634 |
| 0.0785        | 39.0  | 39000 | 1.7894          | 0.6312   | 0.5550     | 2.7333 | 0.6312   | 0.6351   | 0.1963 | 0.1599 |
| 0.0696        | 40.0  | 40000 | 1.7996          | 0.6288   | 0.5607     | 2.7489 | 0.6288   | 0.6334   | 0.1986 | 0.1645 |
| 0.0626        | 41.0  | 41000 | 1.7963          | 0.6328   | 0.5532     | 2.7232 | 0.6328   | 0.6349   | 0.1933 | 0.1632 |
| 0.055         | 42.0  | 42000 | 1.7959          | 0.6268   | 0.5556     | 2.6877 | 0.6268   | 0.6298   | 0.1957 | 0.1617 |
| 0.0475        | 43.0  | 43000 | 1.8018          | 0.632    | 0.5522     | 2.7232 | 0.632    | 0.6354   | 0.1934 | 0.1598 |
| 0.0419        | 44.0  | 44000 | 1.7930          | 0.6325   | 0.5507     | 2.6842 | 0.6325   | 0.6361   | 0.1906 | 0.1612 |
| 0.0367        | 45.0  | 45000 | 1.8064          | 0.6265   | 0.5577     | 2.6772 | 0.6265   | 0.6299   | 0.1994 | 0.1632 |
| 0.0328        | 46.0  | 46000 | 1.8044          | 0.6228   | 0.5524     | 2.6611 | 0.6228   | 0.6263   | 0.1971 | 0.1620 |
| 0.0289        | 47.0  | 47000 | 1.8101          | 0.6248   | 0.5544     | 2.6841 | 0.6248   | 0.6284   | 0.1943 | 0.1624 |
| 0.0265        | 48.0  | 48000 | 1.8088          | 0.6242   | 0.5531     | 2.6870 | 0.6242   | 0.6283   | 0.1943 | 0.1622 |
| 0.0238        | 49.0  | 49000 | 1.8107          | 0.6255   | 0.5533     | 2.7007 | 0.6255   | 0.6292   | 0.1923 | 0.1621 |
| 0.022         | 50.0  | 50000 | 1.8136          | 0.6262   | 0.5539     | 2.6914 | 0.6262   | 0.6298   | 0.1916 | 0.1624 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo