
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# rvlcdip-tiny_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 5.4561
- Accuracy: 0.802
- Brier Loss: 0.3399
- Nll: 1.6335
- F1 Micro: 0.802
- F1 Macro: 0.8037
- Ece: 0.1478
- Aurc: 0.0576

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 125  | 5.6533          | 0.5288   | 0.6696     | 3.8280 | 0.5288   | 0.4993   | 0.2170 | 0.2294 |
| No log        | 2.0   | 250  | 5.3016          | 0.6285   | 0.5364     | 2.7651 | 0.6285   | 0.6089   | 0.1861 | 0.1465 |
| No log        | 3.0   | 375  | 5.1153          | 0.696    | 0.4775     | 2.4052 | 0.696    | 0.6956   | 0.2003 | 0.1078 |
| 5.7003        | 4.0   | 500  | 4.9491          | 0.7358   | 0.3968     | 2.1532 | 0.7358   | 0.7375   | 0.1406 | 0.0813 |
| 5.7003        | 5.0   | 625  | 4.8556          | 0.754    | 0.3676     | 1.8243 | 0.754    | 0.7472   | 0.1001 | 0.0756 |
| 5.7003        | 6.0   | 750  | 4.8060          | 0.7625   | 0.3475     | 1.8558 | 0.7625   | 0.7636   | 0.0808 | 0.0696 |
| 5.7003        | 7.0   | 875  | 4.8301          | 0.7648   | 0.3320     | 1.7367 | 0.7648   | 0.7663   | 0.0434 | 0.0677 |
| 4.6459        | 8.0   | 1000 | 4.7883          | 0.7692   | 0.3305     | 1.8366 | 0.7692   | 0.7728   | 0.0532 | 0.0666 |
| 4.6459        | 9.0   | 1125 | 4.8347          | 0.7762   | 0.3282     | 1.7122 | 0.7762   | 0.7789   | 0.0610 | 0.0675 |
| 4.6459        | 10.0  | 1250 | 4.8679          | 0.7682   | 0.3338     | 1.8225 | 0.7682   | 0.7713   | 0.0634 | 0.0672 |
| 4.6459        | 11.0  | 1375 | 4.9875          | 0.7655   | 0.3521     | 1.9651 | 0.7655   | 0.7647   | 0.0914 | 0.0692 |
| 4.2436        | 12.0  | 1500 | 4.9708          | 0.77     | 0.3410     | 2.0195 | 0.7700   | 0.7694   | 0.0838 | 0.0684 |
| 4.2436        | 13.0  | 1625 | 4.9246          | 0.7752   | 0.3349     | 1.8150 | 0.7752   | 0.7758   | 0.0801 | 0.0666 |
| 4.2436        | 14.0  | 1750 | 4.9235          | 0.776    | 0.3327     | 1.8364 | 0.776    | 0.7782   | 0.0896 | 0.0628 |
| 4.2436        | 15.0  | 1875 | 4.9149          | 0.7817   | 0.3348     | 1.9243 | 0.7817   | 0.7857   | 0.0917 | 0.0650 |
| 4.0997        | 16.0  | 2000 | 4.8998          | 0.7837   | 0.3255     | 1.8326 | 0.7837   | 0.7874   | 0.0901 | 0.0637 |
| 4.0997        | 17.0  | 2125 | 4.9658          | 0.7792   | 0.3358     | 1.8156 | 0.7792   | 0.7815   | 0.1025 | 0.0640 |
| 4.0997        | 18.0  | 2250 | 4.9819          | 0.7905   | 0.3256     | 1.8605 | 0.7905   | 0.7919   | 0.1016 | 0.0613 |
| 4.0997        | 19.0  | 2375 | 5.0040          | 0.778    | 0.3417     | 1.9392 | 0.778    | 0.7800   | 0.1095 | 0.0638 |
| 4.0325        | 20.0  | 2500 | 5.0084          | 0.7817   | 0.3387     | 1.9882 | 0.7817   | 0.7833   | 0.1043 | 0.0642 |
| 4.0325        | 21.0  | 2625 | 5.0680          | 0.7805   | 0.3473     | 1.8641 | 0.7805   | 0.7803   | 0.1200 | 0.0631 |
| 4.0325        | 22.0  | 2750 | 5.0324          | 0.7808   | 0.3395     | 1.8541 | 0.7808   | 0.7835   | 0.1124 | 0.0620 |
| 4.0325        | 23.0  | 2875 | 5.0734          | 0.7845   | 0.3446     | 1.9087 | 0.7845   | 0.7884   | 0.1170 | 0.0625 |
| 3.99          | 24.0  | 3000 | 5.2144          | 0.782    | 0.3564     | 1.9540 | 0.782    | 0.7845   | 0.1293 | 0.0640 |
| 3.99          | 25.0  | 3125 | 5.0299          | 0.7873   | 0.3387     | 1.8106 | 0.7873   | 0.7887   | 0.1167 | 0.0614 |
| 3.99          | 26.0  | 3250 | 5.0673          | 0.792    | 0.3318     | 1.7538 | 0.792    | 0.7930   | 0.1134 | 0.0599 |
| 3.99          | 27.0  | 3375 | 5.0854          | 0.791    | 0.3379     | 1.8144 | 0.791    | 0.7932   | 0.1253 | 0.0586 |
| 3.9606        | 28.0  | 3500 | 5.0962          | 0.787    | 0.3403     | 1.7780 | 0.787    | 0.7884   | 0.1224 | 0.0592 |
| 3.9606        | 29.0  | 3625 | 5.0812          | 0.7877   | 0.3379     | 1.7721 | 0.7877   | 0.7900   | 0.1247 | 0.0592 |
| 3.9606        | 30.0  | 3750 | 5.1318          | 0.7905   | 0.3359     | 1.8105 | 0.7905   | 0.7931   | 0.1290 | 0.0597 |
| 3.9606        | 31.0  | 3875 | 5.0330          | 0.7953   | 0.3276     | 1.7361 | 0.7953   | 0.7978   | 0.1144 | 0.0584 |
| 3.9355        | 32.0  | 4000 | 5.0843          | 0.7975   | 0.3276     | 1.7556 | 0.7975   | 0.7990   | 0.1236 | 0.0560 |
| 3.9355        | 33.0  | 4125 | 5.1843          | 0.7995   | 0.3315     | 1.7084 | 0.7995   | 0.8004   | 0.1297 | 0.0575 |
| 3.9355        | 34.0  | 4250 | 5.1703          | 0.7987   | 0.3333     | 1.6918 | 0.7987   | 0.8000   | 0.1257 | 0.0580 |
| 3.9355        | 35.0  | 4375 | 5.1933          | 0.7937   | 0.3372     | 1.7084 | 0.7937   | 0.7941   | 0.1307 | 0.0561 |
| 3.9148        | 36.0  | 4500 | 5.1404          | 0.7987   | 0.3275     | 1.6423 | 0.7987   | 0.8011   | 0.1308 | 0.0547 |
| 3.9148        | 37.0  | 4625 | 5.1734          | 0.8017   | 0.3272     | 1.6836 | 0.8017   | 0.8034   | 0.1272 | 0.0572 |
| 3.9148        | 38.0  | 4750 | 5.2479          | 0.802    | 0.3322     | 1.7081 | 0.802    | 0.8032   | 0.1353 | 0.0550 |
| 3.9148        | 39.0  | 4875 | 5.1921          | 0.8      | 0.3320     | 1.6554 | 0.8000   | 0.8012   | 0.1334 | 0.0538 |
| 3.9001        | 40.0  | 5000 | 5.2477          | 0.801    | 0.3353     | 1.6333 | 0.801    | 0.8022   | 0.1390 | 0.0539 |
| 3.9001        | 41.0  | 5125 | 5.2140          | 0.801    | 0.3299     | 1.6370 | 0.801    | 0.8017   | 0.1340 | 0.0544 |
| 3.9001        | 42.0  | 5250 | 5.2660          | 0.807    | 0.3303     | 1.6090 | 0.807    | 0.8079   | 0.1339 | 0.0545 |
| 3.9001        | 43.0  | 5375 | 5.2884          | 0.8007   | 0.3319     | 1.6816 | 0.8007   | 0.8022   | 0.1394 | 0.0547 |
| 3.8892        | 44.0  | 5500 | 5.3358          | 0.804    | 0.3352     | 1.6399 | 0.804    | 0.8049   | 0.1387 | 0.0560 |
| 3.8892        | 45.0  | 5625 | 5.3545          | 0.8043   | 0.3349     | 1.6445 | 0.8043   | 0.8060   | 0.1408 | 0.0555 |
| 3.8892        | 46.0  | 5750 | 5.4026          | 0.8033   | 0.3373     | 1.6493 | 0.8033   | 0.8049   | 0.1439 | 0.0567 |
| 3.8892        | 47.0  | 5875 | 5.4195          | 0.8015   | 0.3386     | 1.6393 | 0.8015   | 0.8031   | 0.1468 | 0.0570 |
| 3.8834        | 48.0  | 6000 | 5.4409          | 0.803    | 0.3396     | 1.6392 | 0.803    | 0.8046   | 0.1458 | 0.0574 |
| 3.8834        | 49.0  | 6125 | 5.4501          | 0.8023   | 0.3395     | 1.6367 | 0.8023   | 0.8039   | 0.1468 | 0.0574 |
| 3.8834        | 50.0  | 6250 | 5.4561          | 0.802    | 0.3399     | 1.6335 | 0.802    | 0.8037   | 0.1478 | 0.0576 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo