
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base_rvl-cdip-tiny_rvl_cdip-NK1000_kd

This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.8222
- Accuracy: 0.5543
- Brier Loss: 0.6966
- Nll: 3.2790
- F1 Micro: 0.5543
- F1 Macro: 0.5553
- Ece: 0.2764
- Aurc: 0.2323

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 125  | 2.4938          | 0.1348   | 0.9105     | 6.1860 | 0.1348   | 0.0673   | 0.0701 | 0.7303 |
| No log        | 2.0   | 250  | 2.3746          | 0.1895   | 0.8922     | 5.4000 | 0.1895   | 0.1384   | 0.0653 | 0.6997 |
| No log        | 3.0   | 375  | 2.2780          | 0.198    | 0.8746     | 4.3716 | 0.198    | 0.1340   | 0.0676 | 0.6597 |
| 2.396         | 4.0   | 500  | 2.0622          | 0.302    | 0.8212     | 4.2504 | 0.302    | 0.2268   | 0.0655 | 0.5280 |
| 2.396         | 5.0   | 625  | 1.8529          | 0.3703   | 0.7693     | 3.3328 | 0.3703   | 0.3184   | 0.0727 | 0.4470 |
| 2.396         | 6.0   | 750  | 1.6847          | 0.423    | 0.7103     | 3.0730 | 0.4230   | 0.3879   | 0.0698 | 0.3567 |
| 2.396         | 7.0   | 875  | 1.5975          | 0.45     | 0.6817     | 3.0713 | 0.45     | 0.4139   | 0.0632 | 0.3257 |
| 1.7095        | 8.0   | 1000 | 1.5156          | 0.4768   | 0.6588     | 2.8252 | 0.4768   | 0.4517   | 0.0635 | 0.3029 |
| 1.7095        | 9.0   | 1125 | 1.4425          | 0.5018   | 0.6308     | 2.7656 | 0.5018   | 0.4812   | 0.0650 | 0.2728 |
| 1.7095        | 10.0  | 1250 | 1.4089          | 0.5092   | 0.6218     | 2.6715 | 0.5092   | 0.4894   | 0.0527 | 0.2642 |
| 1.7095        | 11.0  | 1375 | 1.3930          | 0.523    | 0.6150     | 2.6821 | 0.523    | 0.5261   | 0.0635 | 0.2584 |
| 1.3064        | 12.0  | 1500 | 1.4166          | 0.5205   | 0.6262     | 2.7691 | 0.5205   | 0.4991   | 0.0813 | 0.2639 |
| 1.3064        | 13.0  | 1625 | 1.3343          | 0.5312   | 0.5961     | 2.6475 | 0.5312   | 0.5194   | 0.0586 | 0.2383 |
| 1.3064        | 14.0  | 1750 | 1.3277          | 0.5417   | 0.5917     | 2.6528 | 0.5417   | 0.5361   | 0.0669 | 0.2327 |
| 1.3064        | 15.0  | 1875 | 1.3407          | 0.5312   | 0.5958     | 2.6880 | 0.5312   | 0.5356   | 0.0637 | 0.2378 |
| 1.0419        | 16.0  | 2000 | 1.2873          | 0.5545   | 0.5801     | 2.6042 | 0.5545   | 0.5509   | 0.0870 | 0.2193 |
| 1.0419        | 17.0  | 2125 | 1.3539          | 0.5375   | 0.6022     | 2.6706 | 0.5375   | 0.5329   | 0.0970 | 0.2376 |
| 1.0419        | 18.0  | 2250 | 1.3073          | 0.5543   | 0.5857     | 2.6217 | 0.5543   | 0.5502   | 0.1006 | 0.2200 |
| 1.0419        | 19.0  | 2375 | 1.3225          | 0.558    | 0.5886     | 2.6258 | 0.558    | 0.5530   | 0.1047 | 0.2206 |
| 0.8001        | 20.0  | 2500 | 1.3573          | 0.554    | 0.5955     | 2.7139 | 0.554    | 0.5489   | 0.1221 | 0.2200 |
| 0.8001        | 21.0  | 2625 | 1.4029          | 0.546    | 0.6150     | 2.7649 | 0.546    | 0.5456   | 0.1547 | 0.2274 |
| 0.8001        | 22.0  | 2750 | 1.4006          | 0.5525   | 0.6092     | 2.8131 | 0.5525   | 0.5504   | 0.1474 | 0.2246 |
| 0.8001        | 23.0  | 2875 | 1.4523          | 0.5513   | 0.6223     | 2.8803 | 0.5513   | 0.5448   | 0.1818 | 0.2269 |
| 0.5716        | 24.0  | 3000 | 1.4744          | 0.5495   | 0.6261     | 2.9958 | 0.5495   | 0.5525   | 0.1799 | 0.2253 |
| 0.5716        | 25.0  | 3125 | 1.5278          | 0.5445   | 0.6418     | 3.0853 | 0.5445   | 0.5485   | 0.1915 | 0.2321 |
| 0.5716        | 26.0  | 3250 | 1.5782          | 0.5433   | 0.6566     | 3.0618 | 0.5433   | 0.5448   | 0.2171 | 0.2333 |
| 0.5716        | 27.0  | 3375 | 1.6368          | 0.5375   | 0.6704     | 3.2249 | 0.5375   | 0.5389   | 0.2277 | 0.2401 |
| 0.3744        | 28.0  | 3500 | 1.6339          | 0.5445   | 0.6694     | 3.1689 | 0.5445   | 0.5447   | 0.2376 | 0.2338 |
| 0.3744        | 29.0  | 3625 | 1.6589          | 0.548    | 0.6714     | 3.1654 | 0.548    | 0.5469   | 0.2376 | 0.2319 |
| 0.3744        | 30.0  | 3750 | 1.7679          | 0.5353   | 0.6989     | 3.3537 | 0.5353   | 0.5387   | 0.2524 | 0.2558 |
| 0.3744        | 31.0  | 3875 | 1.7441          | 0.5475   | 0.6846     | 3.3716 | 0.5475   | 0.5501   | 0.2455 | 0.2395 |
| 0.2439        | 32.0  | 4000 | 1.7856          | 0.5365   | 0.6977     | 3.4176 | 0.5365   | 0.5443   | 0.2510 | 0.2462 |
| 0.2439        | 33.0  | 4125 | 1.7886          | 0.545    | 0.6997     | 3.3804 | 0.545    | 0.5454   | 0.2646 | 0.2379 |
| 0.2439        | 34.0  | 4250 | 1.8658          | 0.5275   | 0.7187     | 3.6006 | 0.5275   | 0.5300   | 0.2840 | 0.2482 |
| 0.2439        | 35.0  | 4375 | 1.8668          | 0.5387   | 0.7145     | 3.3922 | 0.5387   | 0.5391   | 0.2797 | 0.2453 |
| 0.1695        | 36.0  | 4500 | 1.8920          | 0.5288   | 0.7263     | 3.4756 | 0.5288   | 0.5320   | 0.2878 | 0.2507 |
| 0.1695        | 37.0  | 4625 | 1.8767          | 0.542    | 0.7146     | 3.5924 | 0.542    | 0.5357   | 0.2792 | 0.2469 |
| 0.1695        | 38.0  | 4750 | 1.8617          | 0.5435   | 0.7094     | 3.5434 | 0.5435   | 0.5467   | 0.2729 | 0.2440 |
| 0.1695        | 39.0  | 4875 | 1.8746          | 0.5525   | 0.7073     | 3.4325 | 0.5525   | 0.5514   | 0.2789 | 0.2434 |
| 0.1278        | 40.0  | 5000 | 1.8877          | 0.5435   | 0.7171     | 3.4872 | 0.5435   | 0.5438   | 0.2852 | 0.2393 |
| 0.1278        | 41.0  | 5125 | 1.8919          | 0.54     | 0.7219     | 3.4577 | 0.54     | 0.5456   | 0.2869 | 0.2487 |
| 0.1278        | 42.0  | 5250 | 1.8631          | 0.548    | 0.7089     | 3.4287 | 0.548    | 0.5502   | 0.2758 | 0.2390 |
| 0.1278        | 43.0  | 5375 | 1.8433          | 0.5475   | 0.7058     | 3.2993 | 0.5475   | 0.5468   | 0.2863 | 0.2335 |
| 0.0993        | 44.0  | 5500 | 1.8458          | 0.5505   | 0.7048     | 3.3852 | 0.5505   | 0.5528   | 0.2776 | 0.2378 |
| 0.0993        | 45.0  | 5625 | 1.8408          | 0.5443   | 0.7100     | 3.3510 | 0.5443   | 0.5490   | 0.2769 | 0.2392 |
| 0.0993        | 46.0  | 5750 | 1.8492          | 0.5477   | 0.7064     | 3.2989 | 0.5477   | 0.5496   | 0.2807 | 0.2363 |
| 0.0993        | 47.0  | 5875 | 1.8100          | 0.5497   | 0.6969     | 3.2853 | 0.5497   | 0.5534   | 0.2761 | 0.2341 |
| 0.0803        | 48.0  | 6000 | 1.8260          | 0.5523   | 0.6984     | 3.2543 | 0.5523   | 0.5532   | 0.2783 | 0.2326 |
| 0.0803        | 49.0  | 6125 | 1.8225          | 0.5563   | 0.6970     | 3.3070 | 0.5563   | 0.5573   | 0.2739 | 0.2327 |
| 0.0803        | 50.0  | 6250 | 1.8222          | 0.5543   | 0.6966     | 3.2790 | 0.5543   | 0.5553   | 0.2764 | 0.2323 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo