
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base-patch16-224-in21k-small_rvl_cdip-NK1000_hint

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 75.2362
- Accuracy: 0.829
- Brier Loss: 0.3192
- Nll: 1.9987
- F1 Micro: 0.8290
- F1 Macro: 0.8288
- Ece: 0.1536
- Aurc: 0.0562

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| 78.2197       | 1.0   | 1000  | 77.9227         | 0.6322   | 0.4884     | 2.2807 | 0.6322   | 0.6226   | 0.0652 | 0.1538 |
| 77.1947       | 2.0   | 2000  | 77.1839         | 0.727    | 0.3752     | 2.1097 | 0.7270   | 0.7290   | 0.0480 | 0.0917 |
| 76.83         | 3.0   | 3000  | 76.9771         | 0.7385   | 0.3662     | 2.0823 | 0.7385   | 0.7421   | 0.0612 | 0.0827 |
| 76.6666       | 4.0   | 4000  | 76.7764         | 0.7725   | 0.3383     | 1.9757 | 0.7725   | 0.7763   | 0.0808 | 0.0697 |
| 76.2784       | 5.0   | 5000  | 76.6028         | 0.773    | 0.3271     | 2.0074 | 0.7730   | 0.7737   | 0.0708 | 0.0668 |
| 75.9985       | 6.0   | 6000  | 76.4455         | 0.7875   | 0.3138     | 1.9622 | 0.7875   | 0.7874   | 0.0868 | 0.0593 |
| 75.7339       | 7.0   | 7000  | 76.2669         | 0.7973   | 0.3127     | 1.9661 | 0.7973   | 0.7978   | 0.1123 | 0.0546 |
| 75.504        | 8.0   | 8000  | 76.3972         | 0.7927   | 0.3373     | 2.0115 | 0.7927   | 0.7954   | 0.1318 | 0.0594 |
| 75.3558       | 9.0   | 9000  | 76.4614         | 0.7785   | 0.3628     | 2.1225 | 0.7785   | 0.7793   | 0.1501 | 0.0648 |
| 75.1485       | 10.0  | 10000 | 76.2131         | 0.795    | 0.3351     | 2.0123 | 0.795    | 0.7964   | 0.1467 | 0.0565 |
| 75.1098       | 11.0  | 11000 | 76.3161         | 0.798    | 0.3452     | 2.0430 | 0.798    | 0.8002   | 0.1569 | 0.0587 |
| 74.822        | 12.0  | 12000 | 76.2536         | 0.788    | 0.3637     | 2.0509 | 0.788    | 0.7877   | 0.1659 | 0.0607 |
| 74.8787       | 13.0  | 13000 | 76.2025         | 0.7965   | 0.3493     | 2.0401 | 0.7965   | 0.7981   | 0.1538 | 0.0582 |
| 74.7046       | 14.0  | 14000 | 76.1598         | 0.8075   | 0.3335     | 2.0388 | 0.8075   | 0.8058   | 0.1551 | 0.0525 |
| 74.6157       | 15.0  | 15000 | 76.0894         | 0.8003   | 0.3496     | 1.9931 | 0.8003   | 0.8006   | 0.1615 | 0.0566 |
| 74.6451       | 16.0  | 16000 | 76.0082         | 0.8065   | 0.3380     | 2.0005 | 0.8065   | 0.8060   | 0.1593 | 0.0530 |
| 74.3042       | 17.0  | 17000 | 76.0281         | 0.8075   | 0.3398     | 2.0028 | 0.8075   | 0.8097   | 0.1592 | 0.0544 |
| 74.3261       | 18.0  | 18000 | 75.9836         | 0.8063   | 0.3414     | 2.0447 | 0.8062   | 0.8066   | 0.1614 | 0.0566 |
| 74.1196       | 19.0  | 19000 | 75.8935         | 0.8103   | 0.3347     | 2.0211 | 0.8103   | 0.8121   | 0.1592 | 0.0595 |
| 74.2291       | 20.0  | 20000 | 75.9679         | 0.815    | 0.3329     | 2.0335 | 0.815    | 0.8141   | 0.1586 | 0.0572 |
| 74.268        | 21.0  | 21000 | 76.0052         | 0.8073   | 0.3442     | 2.0847 | 0.8073   | 0.8067   | 0.1633 | 0.0589 |
| 74.0436       | 22.0  | 22000 | 75.9529         | 0.8093   | 0.3454     | 2.1010 | 0.8093   | 0.8081   | 0.1625 | 0.0547 |
| 73.9289       | 23.0  | 23000 | 75.8841         | 0.8103   | 0.3420     | 2.0569 | 0.8103   | 0.8104   | 0.1625 | 0.0580 |
| 73.9519       | 24.0  | 24000 | 75.7295         | 0.8167   | 0.3320     | 2.0459 | 0.8167   | 0.8152   | 0.1575 | 0.0533 |
| 73.9333       | 25.0  | 25000 | 75.6503         | 0.8165   | 0.3296     | 1.9681 | 0.8165   | 0.8174   | 0.1586 | 0.0523 |
| 73.8239       | 26.0  | 26000 | 75.6156         | 0.8203   | 0.3245     | 2.0540 | 0.8203   | 0.8192   | 0.1546 | 0.0506 |
| 73.7011       | 27.0  | 27000 | 75.7075         | 0.8183   | 0.3312     | 2.0996 | 0.8183   | 0.8193   | 0.1594 | 0.0562 |
| 73.4822       | 28.0  | 28000 | 75.5065         | 0.8247   | 0.3184     | 2.0404 | 0.8247   | 0.8254   | 0.1535 | 0.0548 |
| 73.5787       | 29.0  | 29000 | 75.6063         | 0.8193   | 0.3295     | 2.0527 | 0.8193   | 0.8192   | 0.1591 | 0.0560 |
| 73.519        | 30.0  | 30000 | 75.5828         | 0.8163   | 0.3351     | 2.0151 | 0.8163   | 0.8173   | 0.1621 | 0.0582 |
| 73.6516       | 31.0  | 31000 | 75.4986         | 0.827    | 0.3147     | 2.0640 | 0.827    | 0.8272   | 0.1513 | 0.0539 |
| 73.5156       | 32.0  | 32000 | 75.5884         | 0.8147   | 0.3355     | 2.0634 | 0.8148   | 0.8137   | 0.1631 | 0.0556 |
| 73.4564       | 33.0  | 33000 | 75.3992         | 0.8233   | 0.3219     | 2.0498 | 0.8233   | 0.8227   | 0.1536 | 0.0526 |
| 73.3286       | 34.0  | 34000 | 75.4277         | 0.8197   | 0.3256     | 2.0222 | 0.8197   | 0.8213   | 0.1594 | 0.0540 |
| 73.3056       | 35.0  | 35000 | 75.3989         | 0.8285   | 0.3136     | 1.9681 | 0.8285   | 0.8303   | 0.1510 | 0.0566 |
| 73.3272       | 36.0  | 36000 | 75.4398         | 0.8233   | 0.3247     | 2.0504 | 0.8233   | 0.8247   | 0.1583 | 0.0553 |
| 73.2738       | 37.0  | 37000 | 75.3631         | 0.8207   | 0.3242     | 1.9921 | 0.8207   | 0.8211   | 0.1595 | 0.0546 |
| 73.2657       | 38.0  | 38000 | 75.3613         | 0.8245   | 0.3231     | 2.0715 | 0.8245   | 0.8232   | 0.1569 | 0.0548 |
| 73.2045       | 39.0  | 39000 | 75.3697         | 0.8223   | 0.3253     | 2.0207 | 0.8223   | 0.8213   | 0.1571 | 0.0557 |
| 73.1701       | 40.0  | 40000 | 75.3138         | 0.8277   | 0.3174     | 2.0071 | 0.8277   | 0.8282   | 0.1525 | 0.0557 |
| 73.1491       | 41.0  | 41000 | 75.3160         | 0.827    | 0.3183     | 2.0131 | 0.827    | 0.8271   | 0.1549 | 0.0573 |
| 73.1466       | 42.0  | 42000 | 75.3052         | 0.8297   | 0.3166     | 1.9978 | 0.8297   | 0.8296   | 0.1535 | 0.0558 |
| 73.0658       | 43.0  | 43000 | 75.3064         | 0.8293   | 0.3166     | 2.0293 | 0.8293   | 0.8292   | 0.1548 | 0.0570 |
| 73.1394       | 44.0  | 44000 | 75.2527         | 0.8285   | 0.3179     | 2.0172 | 0.8285   | 0.8284   | 0.1540 | 0.0554 |
| 73.2385       | 45.0  | 45000 | 75.2782         | 0.828    | 0.3181     | 2.0026 | 0.828    | 0.8280   | 0.1556 | 0.0570 |
| 73.2207       | 46.0  | 46000 | 75.2624         | 0.827    | 0.3194     | 1.9884 | 0.827    | 0.8268   | 0.1552 | 0.0559 |
| 73.2837       | 47.0  | 47000 | 75.2604         | 0.8285   | 0.3195     | 1.9982 | 0.8285   | 0.8283   | 0.1542 | 0.0566 |
| 73.0848       | 48.0  | 48000 | 75.2454         | 0.829    | 0.3188     | 1.9958 | 0.8290   | 0.8288   | 0.1535 | 0.0568 |
| 73.111        | 49.0  | 49000 | 75.2438         | 0.8283   | 0.3196     | 2.0019 | 0.8283   | 0.8282   | 0.1542 | 0.0567 |
| 73.0278       | 50.0  | 50000 | 75.2362         | 0.829    | 0.3192     | 1.9987 | 0.8290   | 0.8288   | 0.1536 | 0.0562 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo