
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# cdip-small_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4315
- Accuracy: 0.8522
- Brier Loss: 0.2145
- Nll: 1.3474
- F1 Micro: 0.8522
- F1 Macro: 0.8535
- Ece: 0.0573
- Aurc: 0.0300

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 96
- eval_batch_size: 96
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 167  | 1.6705          | 0.6378   | 0.4837     | 2.4248 | 0.6378   | 0.6323   | 0.0655 | 0.1457 |
| No log        | 2.0   | 334  | 1.1423          | 0.7322   | 0.3740     | 1.9847 | 0.7322   | 0.7285   | 0.0695 | 0.0846 |
| 1.7909        | 3.0   | 501  | 0.9082          | 0.7682   | 0.3248     | 1.7674 | 0.7682   | 0.7676   | 0.0620 | 0.0642 |
| 1.7909        | 4.0   | 668  | 0.8494          | 0.7865   | 0.3082     | 1.7306 | 0.7865   | 0.7904   | 0.0665 | 0.0560 |
| 1.7909        | 5.0   | 835  | 0.7837          | 0.798    | 0.2988     | 1.6072 | 0.798    | 0.7953   | 0.0729 | 0.0553 |
| 0.4994        | 6.0   | 1002 | 0.6867          | 0.804    | 0.2862     | 1.5014 | 0.804    | 0.8059   | 0.0794 | 0.0471 |
| 0.4994        | 7.0   | 1169 | 0.7037          | 0.8157   | 0.2797     | 1.5533 | 0.8157   | 0.8178   | 0.0807 | 0.0478 |
| 0.4994        | 8.0   | 1336 | 0.6709          | 0.8163   | 0.2756     | 1.5297 | 0.8163   | 0.8166   | 0.0728 | 0.0478 |
| 0.2478        | 9.0   | 1503 | 0.6132          | 0.825    | 0.2576     | 1.4349 | 0.825    | 0.8247   | 0.0728 | 0.0398 |
| 0.2478        | 10.0  | 1670 | 0.6389          | 0.8235   | 0.2671     | 1.4455 | 0.8235   | 0.8266   | 0.0746 | 0.0419 |
| 0.2478        | 11.0  | 1837 | 0.6043          | 0.8257   | 0.2585     | 1.4609 | 0.8257   | 0.8293   | 0.0752 | 0.0403 |
| 0.1683        | 12.0  | 2004 | 0.5639          | 0.8327   | 0.2457     | 1.4470 | 0.8327   | 0.8350   | 0.0676 | 0.0375 |
| 0.1683        | 13.0  | 2171 | 0.5665          | 0.8317   | 0.2508     | 1.4054 | 0.8317   | 0.8324   | 0.0731 | 0.0388 |
| 0.1683        | 14.0  | 2338 | 0.5505          | 0.8403   | 0.2427     | 1.4059 | 0.8403   | 0.8408   | 0.0649 | 0.0377 |
| 0.131         | 15.0  | 2505 | 0.5321          | 0.836    | 0.2428     | 1.4078 | 0.836    | 0.8372   | 0.0684 | 0.0365 |
| 0.131         | 16.0  | 2672 | 0.5161          | 0.8373   | 0.2383     | 1.3900 | 0.8373   | 0.8373   | 0.0711 | 0.0368 |
| 0.131         | 17.0  | 2839 | 0.5177          | 0.8403   | 0.2371     | 1.3828 | 0.8403   | 0.8413   | 0.0633 | 0.0354 |
| 0.1071        | 18.0  | 3006 | 0.5113          | 0.8407   | 0.2377     | 1.3832 | 0.8407   | 0.8432   | 0.0718 | 0.0343 |
| 0.1071        | 19.0  | 3173 | 0.4949          | 0.8415   | 0.2332     | 1.3767 | 0.8415   | 0.8428   | 0.0667 | 0.0338 |
| 0.1071        | 20.0  | 3340 | 0.4857          | 0.848    | 0.2271     | 1.3664 | 0.848    | 0.8492   | 0.0615 | 0.0338 |
| 0.0877        | 21.0  | 3507 | 0.4812          | 0.847    | 0.2283     | 1.3360 | 0.847    | 0.8478   | 0.0602 | 0.0346 |
| 0.0877        | 22.0  | 3674 | 0.4715          | 0.8495   | 0.2243     | 1.3761 | 0.8495   | 0.8506   | 0.0560 | 0.0320 |
| 0.0877        | 23.0  | 3841 | 0.4622          | 0.8508   | 0.2206     | 1.3584 | 0.8508   | 0.8515   | 0.0557 | 0.0323 |
| 0.0694        | 24.0  | 4008 | 0.4432          | 0.8515   | 0.2167     | 1.3653 | 0.8515   | 0.8531   | 0.0555 | 0.0309 |
| 0.0694        | 25.0  | 4175 | 0.4467          | 0.8498   | 0.2193     | 1.3499 | 0.8498   | 0.8512   | 0.0581 | 0.0309 |
| 0.0694        | 26.0  | 4342 | 0.4412          | 0.8545   | 0.2162     | 1.3535 | 0.8545   | 0.8560   | 0.0534 | 0.0306 |
| 0.0586        | 27.0  | 4509 | 0.4402          | 0.8498   | 0.2180     | 1.3390 | 0.8498   | 0.8510   | 0.0597 | 0.0309 |
| 0.0586        | 28.0  | 4676 | 0.4408          | 0.8522   | 0.2174     | 1.3568 | 0.8522   | 0.8536   | 0.0576 | 0.0306 |
| 0.0586        | 29.0  | 4843 | 0.4391          | 0.851    | 0.2168     | 1.3429 | 0.851    | 0.8523   | 0.0585 | 0.0305 |
| 0.0549        | 30.0  | 5010 | 0.4371          | 0.853    | 0.2160     | 1.3389 | 0.853    | 0.8543   | 0.0573 | 0.0303 |
| 0.0549        | 31.0  | 5177 | 0.4382          | 0.8498   | 0.2168     | 1.3486 | 0.8498   | 0.8513   | 0.0602 | 0.0304 |
| 0.0549        | 32.0  | 5344 | 0.4372          | 0.853    | 0.2166     | 1.3501 | 0.853    | 0.8540   | 0.0591 | 0.0306 |
| 0.0527        | 33.0  | 5511 | 0.4379          | 0.852    | 0.2156     | 1.3546 | 0.852    | 0.8531   | 0.0576 | 0.0304 |
| 0.0527        | 34.0  | 5678 | 0.4353          | 0.8532   | 0.2154     | 1.3381 | 0.8532   | 0.8543   | 0.0574 | 0.0302 |
| 0.0527        | 35.0  | 5845 | 0.4347          | 0.8525   | 0.2148     | 1.3550 | 0.8525   | 0.8535   | 0.0591 | 0.0304 |
| 0.0511        | 36.0  | 6012 | 0.4311          | 0.8542   | 0.2141     | 1.3233 | 0.8542   | 0.8552   | 0.0572 | 0.0299 |
| 0.0511        | 37.0  | 6179 | 0.4323          | 0.852    | 0.2150     | 1.3332 | 0.852    | 0.8532   | 0.0586 | 0.0302 |
| 0.0511        | 38.0  | 6346 | 0.4321          | 0.8515   | 0.2152     | 1.3382 | 0.8515   | 0.8527   | 0.0583 | 0.0299 |
| 0.0494        | 39.0  | 6513 | 0.4335          | 0.8495   | 0.2152     | 1.3385 | 0.8495   | 0.8511   | 0.0593 | 0.0303 |
| 0.0494        | 40.0  | 6680 | 0.4323          | 0.852    | 0.2146     | 1.3603 | 0.852    | 0.8533   | 0.0576 | 0.0299 |
| 0.0494        | 41.0  | 6847 | 0.4309          | 0.8512   | 0.2143     | 1.3448 | 0.8512   | 0.8525   | 0.0570 | 0.0299 |
| 0.0477        | 42.0  | 7014 | 0.4327          | 0.8525   | 0.2149     | 1.3439 | 0.8525   | 0.8539   | 0.0580 | 0.0299 |
| 0.0477        | 43.0  | 7181 | 0.4309          | 0.8532   | 0.2140     | 1.3406 | 0.8532   | 0.8544   | 0.0560 | 0.0299 |
| 0.0477        | 44.0  | 7348 | 0.4308          | 0.8528   | 0.2141     | 1.3404 | 0.8528   | 0.8540   | 0.0573 | 0.0299 |
| 0.0466        | 45.0  | 7515 | 0.4317          | 0.8525   | 0.2147     | 1.3402 | 0.8525   | 0.8538   | 0.0580 | 0.0299 |
| 0.0466        | 46.0  | 7682 | 0.4317          | 0.8535   | 0.2144     | 1.3475 | 0.8535   | 0.8547   | 0.0553 | 0.0298 |
| 0.0466        | 47.0  | 7849 | 0.4314          | 0.8525   | 0.2143     | 1.3479 | 0.8525   | 0.8537   | 0.0559 | 0.0299 |
| 0.0465        | 48.0  | 8016 | 0.4314          | 0.8525   | 0.2143     | 1.3479 | 0.8525   | 0.8538   | 0.0559 | 0.0299 |
| 0.0465        | 49.0  | 8183 | 0.4316          | 0.8528   | 0.2145     | 1.3471 | 0.8528   | 0.8540   | 0.0573 | 0.0299 |
| 0.0465        | 50.0  | 8350 | 0.4315          | 0.8522   | 0.2145     | 1.3474 | 0.8522   | 0.8535   | 0.0573 | 0.0300 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo