
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# cdip-tiny_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.9475
- Accuracy: 0.8255
- Brier Loss: 0.2849
- Nll: 1.5880
- F1 Micro: 0.8255
- F1 Macro: 0.8276
- Ece: 0.1152
- Aurc: 0.0416

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 125  | 5.0886          | 0.506    | 0.6520     | 2.3481 | 0.506    | 0.4936   | 0.1512 | 0.2571 |
| No log        | 2.0   | 250  | 4.5869          | 0.639    | 0.5026     | 2.1689 | 0.639    | 0.6339   | 0.1308 | 0.1459 |
| No log        | 3.0   | 375  | 4.3135          | 0.7065   | 0.4267     | 2.0672 | 0.7065   | 0.7112   | 0.1248 | 0.1057 |
| 5.1406        | 4.0   | 500  | 4.1859          | 0.717    | 0.3906     | 2.0573 | 0.7170   | 0.7213   | 0.0822 | 0.0921 |
| 5.1406        | 5.0   | 625  | 3.9770          | 0.7615   | 0.3347     | 1.9365 | 0.7615   | 0.7597   | 0.0587 | 0.0699 |
| 5.1406        | 6.0   | 750  | 3.9247          | 0.7698   | 0.3189     | 1.8905 | 0.7698   | 0.7710   | 0.0411 | 0.0654 |
| 5.1406        | 7.0   | 875  | 3.8799          | 0.7728   | 0.3165     | 1.9278 | 0.7728   | 0.7800   | 0.0362 | 0.0644 |
| 3.778         | 8.0   | 1000 | 3.8415          | 0.786    | 0.3036     | 1.8576 | 0.786    | 0.7927   | 0.0417 | 0.0602 |
| 3.778         | 9.0   | 1125 | 3.7909          | 0.7977   | 0.2904     | 1.8383 | 0.7977   | 0.7996   | 0.0444 | 0.0547 |
| 3.778         | 10.0  | 1250 | 3.8216          | 0.794    | 0.2950     | 1.8486 | 0.7940   | 0.7953   | 0.0526 | 0.0534 |
| 3.778         | 11.0  | 1375 | 3.8084          | 0.797    | 0.2929     | 1.8672 | 0.797    | 0.8004   | 0.0653 | 0.0513 |
| 3.3945        | 12.0  | 1500 | 3.7547          | 0.8113   | 0.2801     | 1.8312 | 0.8113   | 0.8124   | 0.0548 | 0.0482 |
| 3.3945        | 13.0  | 1625 | 3.7730          | 0.8137   | 0.2825     | 1.8087 | 0.8137   | 0.8151   | 0.0762 | 0.0482 |
| 3.3945        | 14.0  | 1750 | 3.8090          | 0.807    | 0.2863     | 1.7529 | 0.807    | 0.8075   | 0.0713 | 0.0504 |
| 3.3945        | 15.0  | 1875 | 3.7612          | 0.8067   | 0.2886     | 1.7934 | 0.8067   | 0.8113   | 0.0741 | 0.0498 |
| 3.2666        | 16.0  | 2000 | 3.7760          | 0.809    | 0.2863     | 1.8104 | 0.809    | 0.8116   | 0.0779 | 0.0490 |
| 3.2666        | 17.0  | 2125 | 3.7504          | 0.8155   | 0.2765     | 1.7438 | 0.8155   | 0.8160   | 0.0798 | 0.0458 |
| 3.2666        | 18.0  | 2250 | 3.7798          | 0.8085   | 0.2858     | 1.7447 | 0.8085   | 0.8097   | 0.0844 | 0.0462 |
| 3.2666        | 19.0  | 2375 | 3.7784          | 0.8073   | 0.2876     | 1.7731 | 0.8073   | 0.8112   | 0.0832 | 0.0481 |
| 3.1995        | 20.0  | 2500 | 3.7772          | 0.8123   | 0.2862     | 1.7110 | 0.8123   | 0.8137   | 0.0887 | 0.0461 |
| 3.1995        | 21.0  | 2625 | 3.7531          | 0.8155   | 0.2780     | 1.6920 | 0.8155   | 0.8166   | 0.0860 | 0.0435 |
| 3.1995        | 22.0  | 2750 | 3.7922          | 0.8123   | 0.2850     | 1.7335 | 0.8123   | 0.8162   | 0.0957 | 0.0454 |
| 3.1995        | 23.0  | 2875 | 3.7857          | 0.8185   | 0.2760     | 1.7026 | 0.8185   | 0.8201   | 0.0870 | 0.0455 |
| 3.154         | 24.0  | 3000 | 3.7452          | 0.821    | 0.2724     | 1.6936 | 0.821    | 0.8234   | 0.0902 | 0.0425 |
| 3.154         | 25.0  | 3125 | 3.7485          | 0.8233   | 0.2734     | 1.6908 | 0.8233   | 0.8252   | 0.0889 | 0.0418 |
| 3.154         | 26.0  | 3250 | 3.7627          | 0.8197   | 0.2754     | 1.6656 | 0.8197   | 0.8212   | 0.0951 | 0.0424 |
| 3.154         | 27.0  | 3375 | 3.7635          | 0.821    | 0.2743     | 1.6732 | 0.821    | 0.8221   | 0.0955 | 0.0419 |
| 3.1227        | 28.0  | 3500 | 3.7829          | 0.821    | 0.2765     | 1.6749 | 0.821    | 0.8223   | 0.0980 | 0.0426 |
| 3.1227        | 29.0  | 3625 | 3.7738          | 0.8207   | 0.2752     | 1.6585 | 0.8207   | 0.8223   | 0.0936 | 0.0417 |
| 3.1227        | 30.0  | 3750 | 3.7622          | 0.822    | 0.2763     | 1.6672 | 0.822    | 0.8243   | 0.0942 | 0.0421 |
| 3.1227        | 31.0  | 3875 | 3.7884          | 0.824    | 0.2749     | 1.6566 | 0.824    | 0.8249   | 0.0980 | 0.0416 |
| 3.0998        | 32.0  | 4000 | 3.7948          | 0.8205   | 0.2780     | 1.6516 | 0.8205   | 0.8225   | 0.1004 | 0.0420 |
| 3.0998        | 33.0  | 4125 | 3.7831          | 0.8175   | 0.2787     | 1.6503 | 0.8175   | 0.8207   | 0.1022 | 0.0416 |
| 3.0998        | 34.0  | 4250 | 3.8119          | 0.8223   | 0.2785     | 1.6290 | 0.8223   | 0.8246   | 0.1004 | 0.0421 |
| 3.0998        | 35.0  | 4375 | 3.8186          | 0.8235   | 0.2798     | 1.6490 | 0.8235   | 0.8263   | 0.1019 | 0.0422 |
| 3.0845        | 36.0  | 4500 | 3.8304          | 0.8205   | 0.2821     | 1.6117 | 0.8205   | 0.8228   | 0.1062 | 0.0421 |
| 3.0845        | 37.0  | 4625 | 3.8128          | 0.8267   | 0.2758     | 1.6362 | 0.8267   | 0.8292   | 0.1007 | 0.0409 |
| 3.0845        | 38.0  | 4750 | 3.8488          | 0.8217   | 0.2812     | 1.6245 | 0.8217   | 0.8236   | 0.1080 | 0.0417 |
| 3.0845        | 39.0  | 4875 | 3.8459          | 0.826    | 0.2781     | 1.6239 | 0.826    | 0.8281   | 0.1050 | 0.0417 |
| 3.0726        | 40.0  | 5000 | 3.8527          | 0.8257   | 0.2790     | 1.6083 | 0.8257   | 0.8280   | 0.1078 | 0.0412 |
| 3.0726        | 41.0  | 5125 | 3.8496          | 0.829    | 0.2777     | 1.6026 | 0.8290   | 0.8304   | 0.1018 | 0.0412 |
| 3.0726        | 42.0  | 5250 | 3.8656          | 0.826    | 0.2803     | 1.6125 | 0.826    | 0.8283   | 0.1074 | 0.0412 |
| 3.0726        | 43.0  | 5375 | 3.8860          | 0.8253   | 0.2815     | 1.6029 | 0.8253   | 0.8273   | 0.1102 | 0.0415 |
| 3.0635        | 44.0  | 5500 | 3.8868          | 0.8225   | 0.2810     | 1.5939 | 0.8225   | 0.8248   | 0.1132 | 0.0414 |
| 3.0635        | 45.0  | 5625 | 3.9087          | 0.8247   | 0.2825     | 1.5956 | 0.8247   | 0.8268   | 0.1122 | 0.0414 |
| 3.0635        | 46.0  | 5750 | 3.9273          | 0.8243   | 0.2842     | 1.5863 | 0.8243   | 0.8263   | 0.1150 | 0.0415 |
| 3.0635        | 47.0  | 5875 | 3.9352          | 0.8247   | 0.2841     | 1.5859 | 0.8247   | 0.8268   | 0.1148 | 0.0416 |
| 3.0576        | 48.0  | 6000 | 3.9397          | 0.8253   | 0.2843     | 1.5907 | 0.8253   | 0.8274   | 0.1146 | 0.0416 |
| 3.0576        | 49.0  | 6125 | 3.9444          | 0.8255   | 0.2847     | 1.5886 | 0.8255   | 0.8276   | 0.1147 | 0.0416 |
| 3.0576        | 50.0  | 6250 | 3.9475          | 0.8255   | 0.2849     | 1.5880 | 0.8255   | 0.8276   | 0.1152 | 0.0416 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo