
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# cdip-tiny_rvl_cdip-NK1000_kd_test

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4588
- Accuracy: 0.8227
- Brier Loss: 0.2552
- Nll: 1.9117
- F1 Micro: 0.8227
- F1 Macro: 0.8239
- Ece: 0.0507
- Aurc: 0.0420

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 125  | 1.3371          | 0.5423   | 0.5838     | 2.6217 | 0.5423   | 0.5341   | 0.0583 | 0.2219 |
| No log        | 2.0   | 250  | 0.9983          | 0.646    | 0.4725     | 2.3143 | 0.646    | 0.6407   | 0.0490 | 0.1432 |
| No log        | 3.0   | 375  | 0.8094          | 0.7085   | 0.3977     | 2.2571 | 0.7085   | 0.7034   | 0.0500 | 0.1017 |
| 1.2477        | 4.0   | 500  | 0.7633          | 0.7215   | 0.3806     | 2.2013 | 0.7215   | 0.7275   | 0.0447 | 0.0926 |
| 1.2477        | 5.0   | 625  | 0.7295          | 0.7505   | 0.3565     | 2.1741 | 0.7505   | 0.7417   | 0.0631 | 0.0775 |
| 1.2477        | 6.0   | 750  | 0.6706          | 0.7692   | 0.3321     | 2.1869 | 0.7692   | 0.7669   | 0.0614 | 0.0690 |
| 1.2477        | 7.0   | 875  | 0.6933          | 0.767    | 0.3344     | 2.1685 | 0.767    | 0.7650   | 0.0811 | 0.0676 |
| 0.3434        | 8.0   | 1000 | 0.6640          | 0.7778   | 0.3251     | 2.1666 | 0.7778   | 0.7795   | 0.0681 | 0.0650 |
| 0.3434        | 9.0   | 1125 | 0.6874          | 0.774    | 0.3328     | 2.1401 | 0.774    | 0.7739   | 0.0863 | 0.0660 |
| 0.3434        | 10.0  | 1250 | 0.6639          | 0.7795   | 0.3194     | 2.1335 | 0.7795   | 0.7800   | 0.0800 | 0.0618 |
| 0.3434        | 11.0  | 1375 | 0.6827          | 0.7728   | 0.3332     | 2.1140 | 0.7728   | 0.7771   | 0.0891 | 0.0650 |
| 0.1507        | 12.0  | 1500 | 0.6197          | 0.786    | 0.3106     | 2.1052 | 0.786    | 0.7873   | 0.0716 | 0.0586 |
| 0.1507        | 13.0  | 1625 | 0.6264          | 0.7823   | 0.3133     | 2.1077 | 0.7823   | 0.7834   | 0.0784 | 0.0595 |
| 0.1507        | 14.0  | 1750 | 0.5822          | 0.796    | 0.2964     | 2.0567 | 0.796    | 0.7983   | 0.0676 | 0.0549 |
| 0.1507        | 15.0  | 1875 | 0.5900          | 0.7923   | 0.3016     | 2.0704 | 0.7923   | 0.7936   | 0.0724 | 0.0541 |
| 0.107         | 16.0  | 2000 | 0.6044          | 0.7855   | 0.3099     | 2.0625 | 0.7855   | 0.7901   | 0.0730 | 0.0617 |
| 0.107         | 17.0  | 2125 | 0.5692          | 0.7973   | 0.2930     | 2.0627 | 0.7973   | 0.7990   | 0.0676 | 0.0528 |
| 0.107         | 18.0  | 2250 | 0.5836          | 0.7907   | 0.2984     | 2.0575 | 0.7907   | 0.7922   | 0.0749 | 0.0554 |
| 0.107         | 19.0  | 2375 | 0.5469          | 0.806    | 0.2835     | 2.0754 | 0.806    | 0.8060   | 0.0576 | 0.0498 |
| 0.0879        | 20.0  | 2500 | 0.5427          | 0.804    | 0.2892     | 2.0655 | 0.804    | 0.8089   | 0.0593 | 0.0528 |
| 0.0879        | 21.0  | 2625 | 0.5305          | 0.806    | 0.2777     | 2.0213 | 0.806    | 0.8070   | 0.0604 | 0.0495 |
| 0.0879        | 22.0  | 2750 | 0.5146          | 0.8113   | 0.2741     | 2.0127 | 0.8113   | 0.8121   | 0.0534 | 0.0480 |
| 0.0879        | 23.0  | 2875 | 0.5196          | 0.8107   | 0.2750     | 2.0261 | 0.8108   | 0.8117   | 0.0541 | 0.0489 |
| 0.0755        | 24.0  | 3000 | 0.5169          | 0.8123   | 0.2743     | 1.9561 | 0.8123   | 0.8127   | 0.0595 | 0.0478 |
| 0.0755        | 25.0  | 3125 | 0.5129          | 0.8073   | 0.2777     | 2.0020 | 0.8073   | 0.8089   | 0.0552 | 0.0491 |
| 0.0755        | 26.0  | 3250 | 0.4898          | 0.8177   | 0.2649     | 1.9710 | 0.8178   | 0.8177   | 0.0474 | 0.0451 |
| 0.0755        | 27.0  | 3375 | 0.4966          | 0.8155   | 0.2682     | 2.0075 | 0.8155   | 0.8163   | 0.0514 | 0.0458 |
| 0.0652        | 28.0  | 3500 | 0.4883          | 0.813    | 0.2690     | 1.9655 | 0.813    | 0.8141   | 0.0557 | 0.0465 |
| 0.0652        | 29.0  | 3625 | 0.4860          | 0.8185   | 0.2659     | 1.9593 | 0.8185   | 0.8194   | 0.0481 | 0.0456 |
| 0.0652        | 30.0  | 3750 | 0.4760          | 0.818    | 0.2600     | 1.9517 | 0.818    | 0.8194   | 0.0505 | 0.0441 |
| 0.0652        | 31.0  | 3875 | 0.4755          | 0.8195   | 0.2611     | 1.9593 | 0.8195   | 0.8196   | 0.0507 | 0.0440 |
| 0.0568        | 32.0  | 4000 | 0.4763          | 0.8155   | 0.2628     | 1.9508 | 0.8155   | 0.8161   | 0.0484 | 0.0451 |
| 0.0568        | 33.0  | 4125 | 0.4675          | 0.8225   | 0.2574     | 1.9474 | 0.8225   | 0.8238   | 0.0477 | 0.0433 |
| 0.0568        | 34.0  | 4250 | 0.4664          | 0.8207   | 0.2579     | 1.9478 | 0.8207   | 0.8220   | 0.0498 | 0.0431 |
| 0.0568        | 35.0  | 4375 | 0.4635          | 0.8213   | 0.2567     | 1.9233 | 0.8213   | 0.8219   | 0.0481 | 0.0427 |
| 0.0514        | 36.0  | 4500 | 0.4584          | 0.8245   | 0.2551     | 1.9196 | 0.8245   | 0.8260   | 0.0461 | 0.0424 |
| 0.0514        | 37.0  | 4625 | 0.4627          | 0.825    | 0.2557     | 1.9274 | 0.825    | 0.8256   | 0.0454 | 0.0424 |
| 0.0514        | 38.0  | 4750 | 0.4603          | 0.8213   | 0.2552     | 1.9319 | 0.8213   | 0.8221   | 0.0478 | 0.0425 |
| 0.0514        | 39.0  | 4875 | 0.4610          | 0.8245   | 0.2560     | 1.9337 | 0.8245   | 0.8252   | 0.0476 | 0.0424 |
| 0.0483        | 40.0  | 5000 | 0.4603          | 0.825    | 0.2559     | 1.9319 | 0.825    | 0.8262   | 0.0460 | 0.0421 |
| 0.0483        | 41.0  | 5125 | 0.4589          | 0.8253   | 0.2545     | 1.9317 | 0.8253   | 0.8260   | 0.0459 | 0.0421 |
| 0.0483        | 42.0  | 5250 | 0.4586          | 0.8245   | 0.2552     | 1.9192 | 0.8245   | 0.8260   | 0.0524 | 0.0420 |
| 0.0483        | 43.0  | 5375 | 0.4581          | 0.825    | 0.2552     | 1.9179 | 0.825    | 0.8263   | 0.0477 | 0.0421 |
| 0.0465        | 44.0  | 5500 | 0.4573          | 0.8245   | 0.2543     | 1.9187 | 0.8245   | 0.8257   | 0.0457 | 0.0417 |
| 0.0465        | 45.0  | 5625 | 0.4589          | 0.8225   | 0.2554     | 1.9184 | 0.8225   | 0.8235   | 0.0549 | 0.0421 |
| 0.0465        | 46.0  | 5750 | 0.4582          | 0.823    | 0.2547     | 1.9128 | 0.823    | 0.8242   | 0.0512 | 0.0420 |
| 0.0465        | 47.0  | 5875 | 0.4587          | 0.823    | 0.2551     | 1.9135 | 0.823    | 0.8241   | 0.0484 | 0.0420 |
| 0.0458        | 48.0  | 6000 | 0.4585          | 0.8235   | 0.2550     | 1.9127 | 0.8235   | 0.8246   | 0.0479 | 0.0420 |
| 0.0458        | 49.0  | 6125 | 0.4589          | 0.8227   | 0.2553     | 1.9117 | 0.8227   | 0.8238   | 0.0490 | 0.0421 |
| 0.0458        | 50.0  | 6250 | 0.4588          | 0.8227   | 0.2552     | 1.9117 | 0.8227   | 0.8239   | 0.0507 | 0.0420 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo