
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5_rand

This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 6.1637
- Accuracy: 0.6275
- Brier Loss: 0.6026
- Nll: 2.9068
- F1 Micro: 0.6275
- F1 Macro: 0.6313
- Ece: 0.2499
- Aurc: 0.1609

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| 6.3322        | 1.0   | 1000  | 6.0794          | 0.1835   | 0.8928     | 6.5679 | 0.1835   | 0.1322   | 0.0627 | 0.6846 |
| 5.8198        | 2.0   | 2000  | 5.5963          | 0.3668   | 0.7821     | 3.5543 | 0.3668   | 0.3217   | 0.0967 | 0.4448 |
| 5.53          | 3.0   | 3000  | 5.4184          | 0.4225   | 0.7382     | 3.4217 | 0.4225   | 0.3848   | 0.1087 | 0.3778 |
| 5.3449        | 4.0   | 4000  | 5.1895          | 0.4655   | 0.6813     | 3.0794 | 0.4655   | 0.4562   | 0.1076 | 0.3029 |
| 5.2467        | 5.0   | 5000  | 5.1813          | 0.4592   | 0.6845     | 2.9944 | 0.4592   | 0.4430   | 0.1009 | 0.3125 |
| 5.1382        | 6.0   | 6000  | 5.0102          | 0.4998   | 0.6423     | 2.7804 | 0.4998   | 0.4926   | 0.1013 | 0.2660 |
| 5.0255        | 7.0   | 7000  | 4.9611          | 0.501    | 0.6350     | 2.7692 | 0.501    | 0.5085   | 0.0795 | 0.2690 |
| 4.9089        | 8.0   | 8000  | 4.9327          | 0.508    | 0.6204     | 2.6580 | 0.508    | 0.5068   | 0.0622 | 0.2565 |
| 4.8337        | 9.0   | 9000  | 4.8324          | 0.5467   | 0.5866     | 2.5636 | 0.5467   | 0.5419   | 0.0642 | 0.2274 |
| 4.747         | 10.0  | 10000 | 5.0170          | 0.5302   | 0.6080     | 2.7672 | 0.5302   | 0.5193   | 0.0622 | 0.2452 |
| 4.622         | 11.0  | 11000 | 4.8259          | 0.5593   | 0.5709     | 2.6791 | 0.5593   | 0.5520   | 0.0619 | 0.2090 |
| 4.5449        | 12.0  | 12000 | 4.7696          | 0.5675   | 0.5583     | 2.5273 | 0.5675   | 0.5678   | 0.0541 | 0.2016 |
| 4.447         | 13.0  | 13000 | 4.8718          | 0.5575   | 0.5775     | 2.7597 | 0.5575   | 0.5557   | 0.0575 | 0.2142 |
| 4.341         | 14.0  | 14000 | 4.7644          | 0.5897   | 0.5368     | 2.5797 | 0.5897   | 0.5930   | 0.0560 | 0.1835 |
| 4.2476        | 15.0  | 15000 | 4.8339          | 0.5905   | 0.5485     | 2.6684 | 0.5905   | 0.5903   | 0.0719 | 0.1872 |
| 4.1592        | 16.0  | 16000 | 4.7828          | 0.5877   | 0.5456     | 2.7300 | 0.5877   | 0.5877   | 0.0784 | 0.1832 |
| 4.0513        | 17.0  | 17000 | 4.8771          | 0.5885   | 0.5533     | 2.9097 | 0.5885   | 0.5930   | 0.0965 | 0.1867 |
| 3.9646        | 18.0  | 18000 | 4.8980          | 0.596    | 0.5499     | 2.8383 | 0.596    | 0.5948   | 0.1025 | 0.1797 |
| 3.8768        | 19.0  | 19000 | 4.9787          | 0.605    | 0.5551     | 2.8903 | 0.605    | 0.6050   | 0.1302 | 0.1765 |
| 3.7739        | 20.0  | 20000 | 5.1202          | 0.5945   | 0.5727     | 3.0393 | 0.5945   | 0.5935   | 0.1493 | 0.1821 |
| 3.7023        | 21.0  | 21000 | 5.1879          | 0.5998   | 0.5785     | 2.9570 | 0.5998   | 0.5991   | 0.1690 | 0.1807 |
| 3.6301        | 22.0  | 22000 | 5.2707          | 0.5933   | 0.5908     | 3.1177 | 0.5933   | 0.5971   | 0.1863 | 0.1829 |
| 3.5857        | 23.0  | 23000 | 5.2522          | 0.5887   | 0.5994     | 3.2051 | 0.5887   | 0.5949   | 0.1928 | 0.1857 |
| 3.5256        | 24.0  | 24000 | 5.3443          | 0.6102   | 0.5857     | 2.9687 | 0.6102   | 0.6084   | 0.1953 | 0.1760 |
| 3.4954        | 25.0  | 25000 | 5.3010          | 0.6045   | 0.5874     | 3.0184 | 0.6045   | 0.6053   | 0.1851 | 0.1807 |
| 3.46          | 26.0  | 26000 | 5.4451          | 0.5992   | 0.5994     | 3.0539 | 0.5992   | 0.6033   | 0.2053 | 0.1819 |
| 3.4086        | 27.0  | 27000 | 5.4299          | 0.608    | 0.5913     | 3.1127 | 0.608    | 0.6082   | 0.2027 | 0.1751 |
| 3.3769        | 28.0  | 28000 | 5.6979          | 0.601    | 0.6236     | 3.1077 | 0.601    | 0.6024   | 0.2396 | 0.1777 |
| 3.3238        | 29.0  | 29000 | 5.6090          | 0.611    | 0.6013     | 3.0875 | 0.611    | 0.6114   | 0.2238 | 0.1729 |
| 3.3011        | 30.0  | 30000 | 5.6356          | 0.6105   | 0.5991     | 2.9450 | 0.6105   | 0.6123   | 0.2243 | 0.1719 |
| 3.2708        | 31.0  | 31000 | 5.7634          | 0.604    | 0.6181     | 2.9119 | 0.604    | 0.6075   | 0.2402 | 0.1771 |
| 3.2556        | 32.0  | 32000 | 5.7042          | 0.617    | 0.6002     | 2.9324 | 0.617    | 0.6199   | 0.2263 | 0.1740 |
| 3.2213        | 33.0  | 33000 | 5.7388          | 0.603    | 0.6121     | 2.9240 | 0.603    | 0.6108   | 0.2345 | 0.1782 |
| 3.2138        | 34.0  | 34000 | 5.8008          | 0.6218   | 0.6001     | 2.9209 | 0.6218   | 0.6206   | 0.2284 | 0.1701 |
| 3.1994        | 35.0  | 35000 | 5.7350          | 0.6142   | 0.5967     | 2.9021 | 0.6142   | 0.6147   | 0.2294 | 0.1688 |
| 3.1776        | 36.0  | 36000 | 5.7487          | 0.609    | 0.6032     | 2.8651 | 0.609    | 0.6121   | 0.2329 | 0.1689 |
| 3.1606        | 37.0  | 37000 | 5.8022          | 0.6165   | 0.6075     | 2.8604 | 0.6165   | 0.6189   | 0.2398 | 0.1677 |
| 3.1405        | 38.0  | 38000 | 5.8133          | 0.6235   | 0.5949     | 2.8775 | 0.6235   | 0.6272   | 0.2319 | 0.1640 |
| 3.132         | 39.0  | 39000 | 5.8934          | 0.6232   | 0.5974     | 2.9324 | 0.6232   | 0.6274   | 0.2389 | 0.1639 |
| 3.1303        | 40.0  | 40000 | 5.8902          | 0.6288   | 0.5947     | 2.9049 | 0.6288   | 0.6322   | 0.2344 | 0.1634 |
| 3.1187        | 41.0  | 41000 | 5.9076          | 0.6215   | 0.5987     | 2.8584 | 0.6215   | 0.6261   | 0.2394 | 0.1630 |
| 3.0969        | 42.0  | 42000 | 5.9469          | 0.6265   | 0.5984     | 2.8509 | 0.6265   | 0.6309   | 0.2375 | 0.1631 |
| 3.0964        | 43.0  | 43000 | 5.9442          | 0.6252   | 0.5951     | 2.9309 | 0.6252   | 0.6291   | 0.2397 | 0.1607 |
| 3.0953        | 44.0  | 44000 | 6.0126          | 0.6238   | 0.5998     | 2.8956 | 0.6238   | 0.6274   | 0.2419 | 0.1630 |
| 3.0904        | 45.0  | 45000 | 6.0602          | 0.6295   | 0.5991     | 2.8669 | 0.6295   | 0.6334   | 0.2417 | 0.1609 |
| 3.0794        | 46.0  | 46000 | 6.0782          | 0.6282   | 0.6027     | 2.8830 | 0.6282   | 0.6321   | 0.2442 | 0.1616 |
| 3.0788        | 47.0  | 47000 | 6.1062          | 0.6275   | 0.6003     | 2.8472 | 0.6275   | 0.6316   | 0.2471 | 0.1610 |
| 3.0802        | 48.0  | 48000 | 6.1079          | 0.6285   | 0.5998     | 2.8916 | 0.6285   | 0.6322   | 0.2465 | 0.1600 |
| 3.0644        | 49.0  | 49000 | 6.1569          | 0.6275   | 0.6025     | 2.8941 | 0.6275   | 0.6314   | 0.2497 | 0.1610 |
| 3.0751        | 50.0  | 50000 | 6.1637          | 0.6275   | 0.6026     | 2.9068 | 0.6275   | 0.6313   | 0.2499 | 0.1609 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo