
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# cdip-tiny_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5622
- Accuracy: 0.8255
- Brier Loss: 0.2585
- Nll: 1.9229
- F1 Micro: 0.8255
- F1 Macro: 0.8273
- Ece: 0.0661
- Aurc: 0.0421

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 125  | 2.1936          | 0.55     | 0.5790     | 2.6092 | 0.55     | 0.5380   | 0.0723 | 0.2136 |
| No log        | 2.0   | 250  | 1.5136          | 0.666    | 0.4551     | 2.2832 | 0.666    | 0.6632   | 0.0686 | 0.1282 |
| No log        | 3.0   | 375  | 1.2758          | 0.6993   | 0.4127     | 2.2483 | 0.6993   | 0.6928   | 0.0836 | 0.1026 |
| 1.9726        | 4.0   | 500  | 1.1382          | 0.726    | 0.3794     | 2.0849 | 0.726    | 0.7288   | 0.0725 | 0.0878 |
| 1.9726        | 5.0   | 625  | 1.0625          | 0.7542   | 0.3523     | 2.1178 | 0.7542   | 0.7426   | 0.0755 | 0.0758 |
| 1.9726        | 6.0   | 750  | 0.8657          | 0.7775   | 0.3184     | 1.9516 | 0.7775   | 0.7783   | 0.0736 | 0.0618 |
| 1.9726        | 7.0   | 875  | 0.8285          | 0.7887   | 0.3089     | 2.0439 | 0.7887   | 0.7867   | 0.0724 | 0.0586 |
| 0.5092        | 8.0   | 1000 | 0.7988          | 0.7925   | 0.3056     | 2.0106 | 0.7925   | 0.7953   | 0.0700 | 0.0572 |
| 0.5092        | 9.0   | 1125 | 0.7783          | 0.7925   | 0.3060     | 1.9710 | 0.7925   | 0.7914   | 0.0822 | 0.0573 |
| 0.5092        | 10.0  | 1250 | 0.7640          | 0.796    | 0.3007     | 1.9819 | 0.796    | 0.7996   | 0.0801 | 0.0536 |
| 0.5092        | 11.0  | 1375 | 0.7705          | 0.7913   | 0.3048     | 1.9588 | 0.7913   | 0.7952   | 0.0857 | 0.0559 |
| 0.2071        | 12.0  | 1500 | 0.7328          | 0.8015   | 0.2937     | 1.9484 | 0.8015   | 0.8017   | 0.0760 | 0.0537 |
| 0.2071        | 13.0  | 1625 | 0.6946          | 0.811    | 0.2881     | 1.9173 | 0.811    | 0.8125   | 0.0824 | 0.0507 |
| 0.2071        | 14.0  | 1750 | 0.6902          | 0.8053   | 0.2880     | 1.9154 | 0.8053   | 0.8068   | 0.0738 | 0.0502 |
| 0.2071        | 15.0  | 1875 | 0.6756          | 0.8083   | 0.2840     | 1.9317 | 0.8083   | 0.8078   | 0.0761 | 0.0487 |
| 0.1424        | 16.0  | 2000 | 0.6684          | 0.8067   | 0.2852     | 1.9192 | 0.8067   | 0.8073   | 0.0765 | 0.0507 |
| 0.1424        | 17.0  | 2125 | 0.6548          | 0.8095   | 0.2816     | 1.9398 | 0.8095   | 0.8110   | 0.0758 | 0.0472 |
| 0.1424        | 18.0  | 2250 | 0.6477          | 0.8117   | 0.2762     | 1.9054 | 0.8117   | 0.8140   | 0.0759 | 0.0464 |
| 0.1424        | 19.0  | 2375 | 0.6423          | 0.8145   | 0.2794     | 1.9081 | 0.8145   | 0.8148   | 0.0774 | 0.0478 |
| 0.1102        | 20.0  | 2500 | 0.6312          | 0.8103   | 0.2771     | 1.9581 | 0.8103   | 0.8125   | 0.0746 | 0.0454 |
| 0.1102        | 21.0  | 2625 | 0.6299          | 0.8133   | 0.2720     | 1.9275 | 0.8133   | 0.8132   | 0.0758 | 0.0466 |
| 0.1102        | 22.0  | 2750 | 0.6148          | 0.8197   | 0.2691     | 1.9463 | 0.8197   | 0.8223   | 0.0681 | 0.0447 |
| 0.1102        | 23.0  | 2875 | 0.6132          | 0.8187   | 0.2700     | 1.9301 | 0.8187   | 0.8200   | 0.0691 | 0.0451 |
| 0.0931        | 24.0  | 3000 | 0.5995          | 0.8245   | 0.2649     | 1.9173 | 0.8245   | 0.8251   | 0.0640 | 0.0444 |
| 0.0931        | 25.0  | 3125 | 0.6020          | 0.8177   | 0.2697     | 1.9205 | 0.8178   | 0.8201   | 0.0723 | 0.0440 |
| 0.0931        | 26.0  | 3250 | 0.5914          | 0.8247   | 0.2617     | 1.9385 | 0.8247   | 0.8264   | 0.0667 | 0.0428 |
| 0.0931        | 27.0  | 3375 | 0.5833          | 0.822    | 0.2621     | 1.9390 | 0.822    | 0.8228   | 0.0658 | 0.0429 |
| 0.0789        | 28.0  | 3500 | 0.5884          | 0.8247   | 0.2626     | 1.9400 | 0.8247   | 0.8259   | 0.0619 | 0.0435 |
| 0.0789        | 29.0  | 3625 | 0.5771          | 0.8285   | 0.2568     | 1.9252 | 0.8285   | 0.8313   | 0.0612 | 0.0413 |
| 0.0789        | 30.0  | 3750 | 0.5815          | 0.823    | 0.2628     | 1.9413 | 0.823    | 0.8236   | 0.0676 | 0.0433 |
| 0.0789        | 31.0  | 3875 | 0.5789          | 0.8205   | 0.2617     | 1.9209 | 0.8205   | 0.8219   | 0.0667 | 0.0431 |
| 0.0686        | 32.0  | 4000 | 0.5775          | 0.8247   | 0.2616     | 1.9045 | 0.8247   | 0.8265   | 0.0674 | 0.0428 |
| 0.0686        | 33.0  | 4125 | 0.5744          | 0.827    | 0.2603     | 1.9088 | 0.827    | 0.8275   | 0.0656 | 0.0420 |
| 0.0686        | 34.0  | 4250 | 0.5685          | 0.824    | 0.2607     | 1.9372 | 0.824    | 0.8264   | 0.0647 | 0.0421 |
| 0.0686        | 35.0  | 4375 | 0.5649          | 0.8255   | 0.2584     | 1.9375 | 0.8255   | 0.8274   | 0.0694 | 0.0419 |
| 0.0596        | 36.0  | 4500 | 0.5629          | 0.8263   | 0.2574     | 1.9304 | 0.8263   | 0.8283   | 0.0651 | 0.0415 |
| 0.0596        | 37.0  | 4625 | 0.5622          | 0.8237   | 0.2579     | 1.9228 | 0.8237   | 0.8254   | 0.0644 | 0.0419 |
| 0.0596        | 38.0  | 4750 | 0.5623          | 0.8257   | 0.2579     | 1.9310 | 0.8257   | 0.8277   | 0.0650 | 0.0418 |
| 0.0596        | 39.0  | 4875 | 0.5625          | 0.827    | 0.2579     | 1.9311 | 0.827    | 0.8286   | 0.0668 | 0.0418 |
| 0.0538        | 40.0  | 5000 | 0.5633          | 0.8247   | 0.2590     | 1.9264 | 0.8247   | 0.8264   | 0.0671 | 0.0424 |
| 0.0538        | 41.0  | 5125 | 0.5607          | 0.8257   | 0.2575     | 1.9239 | 0.8257   | 0.8275   | 0.0621 | 0.0417 |
| 0.0538        | 42.0  | 5250 | 0.5605          | 0.8263   | 0.2569     | 1.9305 | 0.8263   | 0.8279   | 0.0620 | 0.0418 |
| 0.0538        | 43.0  | 5375 | 0.5613          | 0.8255   | 0.2581     | 1.9295 | 0.8255   | 0.8272   | 0.0672 | 0.0418 |
| 0.0512        | 44.0  | 5500 | 0.5616          | 0.8255   | 0.2581     | 1.9235 | 0.8255   | 0.8273   | 0.0636 | 0.0419 |
| 0.0512        | 45.0  | 5625 | 0.5624          | 0.8253   | 0.2585     | 1.9206 | 0.8253   | 0.8270   | 0.0646 | 0.0422 |
| 0.0512        | 46.0  | 5750 | 0.5622          | 0.8257   | 0.2582     | 1.9283 | 0.8257   | 0.8273   | 0.0647 | 0.0420 |
| 0.0512        | 47.0  | 5875 | 0.5616          | 0.825    | 0.2581     | 1.9283 | 0.825    | 0.8267   | 0.0640 | 0.0420 |
| 0.05          | 48.0  | 6000 | 0.5620          | 0.8257   | 0.2584     | 1.9262 | 0.8257   | 0.8275   | 0.0644 | 0.0421 |
| 0.05          | 49.0  | 6125 | 0.5622          | 0.8253   | 0.2585     | 1.9257 | 0.8253   | 0.8270   | 0.0630 | 0.0421 |
| 0.05          | 50.0  | 6250 | 0.5622          | 0.8255   | 0.2585     | 1.9229 | 0.8255   | 0.8273   | 0.0661 | 0.0421 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo