
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# rvlcdip-tiny_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6215
- Accuracy: 0.7963
- Brier Loss: 0.3076
- Nll: 1.6291
- F1 Micro: 0.7963
- F1 Macro: 0.7978
- Ece: 0.0919
- Aurc: 0.0682

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 125  | 1.3808          | 0.541    | 0.5996     | 3.3159 | 0.541    | 0.5235   | 0.1039 | 0.2209 |
| No log        | 2.0   | 250  | 1.0577          | 0.6525   | 0.4662     | 2.6310 | 0.6525   | 0.6396   | 0.0871 | 0.1302 |
| No log        | 3.0   | 375  | 0.9165          | 0.7075   | 0.4104     | 2.2685 | 0.7075   | 0.7041   | 0.0788 | 0.1048 |
| 1.3004        | 4.0   | 500  | 0.8505          | 0.7298   | 0.3804     | 2.1171 | 0.7298   | 0.7380   | 0.0622 | 0.0934 |
| 1.3004        | 5.0   | 625  | 0.8063          | 0.745    | 0.3603     | 2.1178 | 0.745    | 0.7359   | 0.0588 | 0.0814 |
| 1.3004        | 6.0   | 750  | 0.7441          | 0.7662   | 0.3348     | 1.9219 | 0.7663   | 0.7636   | 0.0545 | 0.0741 |
| 1.3004        | 7.0   | 875  | 0.6987          | 0.7732   | 0.3193     | 1.8601 | 0.7732   | 0.7741   | 0.0509 | 0.0697 |
| 0.4682        | 8.0   | 1000 | 0.7033          | 0.773    | 0.3240     | 1.8889 | 0.7730   | 0.7733   | 0.0516 | 0.0776 |
| 0.4682        | 9.0   | 1125 | 0.6973          | 0.7865   | 0.3151     | 1.9589 | 0.7865   | 0.7838   | 0.0441 | 0.0760 |
| 0.4682        | 10.0  | 1250 | 0.7068          | 0.7748   | 0.3252     | 2.0362 | 0.7748   | 0.7749   | 0.0515 | 0.0791 |
| 0.4682        | 11.0  | 1375 | 0.6988          | 0.7768   | 0.3285     | 1.9227 | 0.7768   | 0.7801   | 0.0555 | 0.0840 |
| 0.1899        | 12.0  | 1500 | 0.7048          | 0.7762   | 0.3303     | 1.9777 | 0.7762   | 0.7719   | 0.0627 | 0.0809 |
| 0.1899        | 13.0  | 1625 | 0.6842          | 0.7785   | 0.3240     | 1.9360 | 0.7785   | 0.7784   | 0.0614 | 0.0808 |
| 0.1899        | 14.0  | 1750 | 0.6993          | 0.7742   | 0.3319     | 1.9508 | 0.7742   | 0.7727   | 0.0731 | 0.0759 |
| 0.1899        | 15.0  | 1875 | 0.6936          | 0.7742   | 0.3333     | 1.9042 | 0.7742   | 0.7760   | 0.0717 | 0.0853 |
| 0.1304        | 16.0  | 2000 | 0.6818          | 0.7837   | 0.3233     | 1.9541 | 0.7837   | 0.7855   | 0.0713 | 0.0853 |
| 0.1304        | 17.0  | 2125 | 0.6757          | 0.78     | 0.3255     | 1.8818 | 0.78     | 0.7829   | 0.0755 | 0.0834 |
| 0.1304        | 18.0  | 2250 | 0.7018          | 0.781    | 0.3348     | 2.0078 | 0.7810   | 0.7829   | 0.0786 | 0.0876 |
| 0.1304        | 19.0  | 2375 | 0.6872          | 0.7775   | 0.3340     | 1.8345 | 0.7775   | 0.7786   | 0.0864 | 0.0787 |
| 0.11          | 20.0  | 2500 | 0.7054          | 0.7758   | 0.3379     | 1.9542 | 0.7758   | 0.7747   | 0.0731 | 0.0847 |
| 0.11          | 21.0  | 2625 | 0.7006          | 0.782    | 0.3371     | 1.8610 | 0.782    | 0.7813   | 0.0821 | 0.0891 |
| 0.11          | 22.0  | 2750 | 0.7046          | 0.775    | 0.3428     | 1.8464 | 0.775    | 0.7772   | 0.0833 | 0.0814 |
| 0.11          | 23.0  | 2875 | 0.6620          | 0.789    | 0.3201     | 1.8174 | 0.7890   | 0.7908   | 0.0761 | 0.0799 |
| 0.0979        | 24.0  | 3000 | 0.6886          | 0.783    | 0.3324     | 1.8706 | 0.7830   | 0.7848   | 0.0807 | 0.0773 |
| 0.0979        | 25.0  | 3125 | 0.6600          | 0.7847   | 0.3236     | 1.8218 | 0.7847   | 0.7863   | 0.0833 | 0.0749 |
| 0.0979        | 26.0  | 3250 | 0.6777          | 0.7798   | 0.3349     | 1.7189 | 0.7798   | 0.7812   | 0.0951 | 0.0752 |
| 0.0979        | 27.0  | 3375 | 0.6554          | 0.7857   | 0.3212     | 1.7356 | 0.7857   | 0.7888   | 0.0871 | 0.0709 |
| 0.087         | 28.0  | 3500 | 0.6460          | 0.7955   | 0.3140     | 1.7680 | 0.7955   | 0.7970   | 0.0761 | 0.0696 |
| 0.087         | 29.0  | 3625 | 0.6371          | 0.7935   | 0.3136     | 1.6350 | 0.7935   | 0.7946   | 0.0830 | 0.0706 |
| 0.087         | 30.0  | 3750 | 0.6334          | 0.7915   | 0.3127     | 1.7187 | 0.7915   | 0.7933   | 0.0857 | 0.0712 |
| 0.087         | 31.0  | 3875 | 0.6293          | 0.7977   | 0.3075     | 1.7781 | 0.7977   | 0.7999   | 0.0799 | 0.0661 |
| 0.0793        | 32.0  | 4000 | 0.6273          | 0.7973   | 0.3076     | 1.6439 | 0.7973   | 0.7976   | 0.0782 | 0.0695 |
| 0.0793        | 33.0  | 4125 | 0.6320          | 0.7933   | 0.3123     | 1.6486 | 0.7932   | 0.7954   | 0.0899 | 0.0679 |
| 0.0793        | 34.0  | 4250 | 0.6345          | 0.79     | 0.3154     | 1.6402 | 0.79     | 0.7903   | 0.0922 | 0.0675 |
| 0.0793        | 35.0  | 4375 | 0.6209          | 0.793    | 0.3098     | 1.6026 | 0.793    | 0.7943   | 0.0863 | 0.0630 |
| 0.0733        | 36.0  | 4500 | 0.6187          | 0.7947   | 0.3076     | 1.6282 | 0.7947   | 0.7967   | 0.0880 | 0.0666 |
| 0.0733        | 37.0  | 4625 | 0.6146          | 0.7957   | 0.3051     | 1.6186 | 0.7957   | 0.7971   | 0.0885 | 0.0623 |
| 0.0733        | 38.0  | 4750 | 0.6169          | 0.7983   | 0.3062     | 1.6182 | 0.7983   | 0.7996   | 0.0835 | 0.0650 |
| 0.0733        | 39.0  | 4875 | 0.6180          | 0.7953   | 0.3074     | 1.6241 | 0.7953   | 0.7975   | 0.0889 | 0.0655 |
| 0.0693        | 40.0  | 5000 | 0.6204          | 0.7977   | 0.3069     | 1.6048 | 0.7977   | 0.7987   | 0.0824 | 0.0659 |
| 0.0693        | 41.0  | 5125 | 0.6140          | 0.7967   | 0.3055     | 1.6065 | 0.7967   | 0.7986   | 0.0911 | 0.0662 |
| 0.0693        | 42.0  | 5250 | 0.6162          | 0.7957   | 0.3062     | 1.6182 | 0.7957   | 0.7971   | 0.0883 | 0.0655 |
| 0.0693        | 43.0  | 5375 | 0.6169          | 0.796    | 0.3058     | 1.6212 | 0.796    | 0.7976   | 0.0879 | 0.0662 |
| 0.0673        | 44.0  | 5500 | 0.6173          | 0.7973   | 0.3063     | 1.6161 | 0.7973   | 0.7990   | 0.0877 | 0.0666 |
| 0.0673        | 45.0  | 5625 | 0.6193          | 0.797    | 0.3070     | 1.6151 | 0.797    | 0.7986   | 0.0881 | 0.0678 |
| 0.0673        | 46.0  | 5750 | 0.6209          | 0.7963   | 0.3076     | 1.6211 | 0.7963   | 0.7979   | 0.0894 | 0.0678 |
| 0.0673        | 47.0  | 5875 | 0.6211          | 0.7977   | 0.3075     | 1.6284 | 0.7977   | 0.7993   | 0.0905 | 0.0691 |
| 0.0662        | 48.0  | 6000 | 0.6206          | 0.7967   | 0.3072     | 1.6289 | 0.7967   | 0.7983   | 0.0892 | 0.0673 |
| 0.0662        | 49.0  | 6125 | 0.6213          | 0.7965   | 0.3075     | 1.6262 | 0.7965   | 0.7980   | 0.0886 | 0.0684 |
| 0.0662        | 50.0  | 6250 | 0.6215          | 0.7963   | 0.3076     | 1.6291 | 0.7963   | 0.7978   | 0.0919 | 0.0682 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo