
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-small_rvl_cdip_100_examples_per_class_kd_CEKD_t5.0_a0.5

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2623
- Accuracy: 0.65
- Brier Loss: 0.4803
- Nll: 3.2676
- F1 Micro: 0.65
- F1 Macro: 0.6575
- Ece: 0.1722
- Aurc: 0.1414

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll     | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:-------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 3.4916          | 0.1075   | 0.9342     | 15.2561 | 0.1075   | 0.1132   | 0.1627 | 0.8874 |
| No log        | 2.0   | 50   | 2.6905          | 0.395    | 0.7423     | 8.7655  | 0.395    | 0.3694   | 0.1922 | 0.3538 |
| No log        | 3.0   | 75   | 2.1229          | 0.505    | 0.6157     | 5.2850  | 0.505    | 0.4830   | 0.1716 | 0.2424 |
| No log        | 4.0   | 100  | 1.9322          | 0.55     | 0.5842     | 4.6402  | 0.55     | 0.5501   | 0.1744 | 0.2156 |
| No log        | 5.0   | 125  | 1.8231          | 0.5575   | 0.5788     | 4.2830  | 0.5575   | 0.5494   | 0.1777 | 0.2091 |
| No log        | 6.0   | 150  | 1.7318          | 0.5875   | 0.5523     | 4.4127  | 0.5875   | 0.5864   | 0.1686 | 0.1950 |
| No log        | 7.0   | 175  | 1.6652          | 0.615    | 0.5325     | 3.8720  | 0.615    | 0.6192   | 0.1654 | 0.1740 |
| No log        | 8.0   | 200  | 1.5910          | 0.61     | 0.5233     | 3.2435  | 0.61     | 0.6097   | 0.1556 | 0.1702 |
| No log        | 9.0   | 225  | 1.7751          | 0.59     | 0.5610     | 3.9627  | 0.59     | 0.5839   | 0.1932 | 0.1965 |
| No log        | 10.0  | 250  | 1.5950          | 0.5975   | 0.5521     | 3.9360  | 0.5975   | 0.5922   | 0.1868 | 0.1886 |
| No log        | 11.0  | 275  | 1.6105          | 0.6      | 0.5459     | 4.2017  | 0.6      | 0.5960   | 0.1788 | 0.1696 |
| No log        | 12.0  | 300  | 1.5566          | 0.5975   | 0.5283     | 3.6344  | 0.5975   | 0.5957   | 0.1843 | 0.1758 |
| No log        | 13.0  | 325  | 1.5395          | 0.6225   | 0.5344     | 3.3755  | 0.6225   | 0.6327   | 0.1725 | 0.1721 |
| No log        | 14.0  | 350  | 1.5117          | 0.64     | 0.5193     | 3.7990  | 0.64     | 0.6366   | 0.1849 | 0.1659 |
| No log        | 15.0  | 375  | 1.5274          | 0.6225   | 0.5381     | 3.5126  | 0.6225   | 0.6198   | 0.1837 | 0.1689 |
| No log        | 16.0  | 400  | 1.3822          | 0.645    | 0.4848     | 3.5167  | 0.645    | 0.6501   | 0.1426 | 0.1384 |
| No log        | 17.0  | 425  | 1.4390          | 0.6325   | 0.5345     | 3.8558  | 0.6325   | 0.6406   | 0.1859 | 0.1624 |
| No log        | 18.0  | 450  | 1.3763          | 0.6425   | 0.4905     | 3.0232  | 0.6425   | 0.6446   | 0.1687 | 0.1388 |
| No log        | 19.0  | 475  | 1.5017          | 0.5925   | 0.5558     | 3.9738  | 0.5925   | 0.5699   | 0.2064 | 0.1827 |
| 0.7312        | 20.0  | 500  | 1.4216          | 0.64     | 0.5092     | 3.5054  | 0.64     | 0.6394   | 0.1885 | 0.1583 |
| 0.7312        | 21.0  | 525  | 1.3999          | 0.6325   | 0.5166     | 3.6206  | 0.6325   | 0.6342   | 0.1865 | 0.1586 |
| 0.7312        | 22.0  | 550  | 1.3555          | 0.6575   | 0.5092     | 3.5815  | 0.6575   | 0.6570   | 0.1748 | 0.1565 |
| 0.7312        | 23.0  | 575  | 1.3915          | 0.6375   | 0.5065     | 3.2269  | 0.6375   | 0.6367   | 0.1712 | 0.1485 |
| 0.7312        | 24.0  | 600  | 1.4116          | 0.64     | 0.5130     | 3.7646  | 0.64     | 0.6412   | 0.1690 | 0.1624 |
| 0.7312        | 25.0  | 625  | 1.3663          | 0.64     | 0.5160     | 3.0397  | 0.64     | 0.6471   | 0.1736 | 0.1575 |
| 0.7312        | 26.0  | 650  | 1.3717          | 0.63     | 0.5097     | 3.7950  | 0.63     | 0.6379   | 0.1823 | 0.1570 |
| 0.7312        | 27.0  | 675  | 1.3229          | 0.6425   | 0.4933     | 3.5568  | 0.6425   | 0.6498   | 0.1564 | 0.1470 |
| 0.7312        | 28.0  | 700  | 1.3638          | 0.6275   | 0.5124     | 3.2988  | 0.6275   | 0.6266   | 0.1916 | 0.1600 |
| 0.7312        | 29.0  | 725  | 1.3353          | 0.6475   | 0.5013     | 3.4126  | 0.6475   | 0.6407   | 0.1747 | 0.1558 |
| 0.7312        | 30.0  | 750  | 1.3788          | 0.6325   | 0.5172     | 3.4229  | 0.6325   | 0.6329   | 0.1629 | 0.1650 |
| 0.7312        | 31.0  | 775  | 1.3021          | 0.6525   | 0.4840     | 3.2418  | 0.6525   | 0.6571   | 0.1788 | 0.1412 |
| 0.7312        | 32.0  | 800  | 1.3127          | 0.6525   | 0.5058     | 3.1876  | 0.6525   | 0.6579   | 0.1879 | 0.1525 |
| 0.7312        | 33.0  | 825  | 1.3181          | 0.64     | 0.5023     | 3.1837  | 0.64     | 0.6459   | 0.1751 | 0.1529 |
| 0.7312        | 34.0  | 850  | 1.3071          | 0.6425   | 0.4954     | 3.5271  | 0.6425   | 0.6480   | 0.1615 | 0.1496 |
| 0.7312        | 35.0  | 875  | 1.2808          | 0.655    | 0.4904     | 3.2539  | 0.655    | 0.6606   | 0.1725 | 0.1448 |
| 0.7312        | 36.0  | 900  | 1.2766          | 0.68     | 0.4771     | 3.3397  | 0.68     | 0.6823   | 0.1645 | 0.1408 |
| 0.7312        | 37.0  | 925  | 1.2751          | 0.665    | 0.4837     | 3.3390  | 0.665    | 0.6728   | 0.1723 | 0.1446 |
| 0.7312        | 38.0  | 950  | 1.2658          | 0.67     | 0.4791     | 3.2603  | 0.67     | 0.6760   | 0.1781 | 0.1407 |
| 0.7312        | 39.0  | 975  | 1.2678          | 0.66     | 0.4814     | 3.1865  | 0.66     | 0.6682   | 0.1585 | 0.1414 |
| 0.0683        | 40.0  | 1000 | 1.2737          | 0.66     | 0.4840     | 3.3466  | 0.66     | 0.6658   | 0.1699 | 0.1434 |
| 0.0683        | 41.0  | 1025 | 1.2581          | 0.66     | 0.4769     | 3.1757  | 0.66     | 0.6660   | 0.1752 | 0.1398 |
| 0.0683        | 42.0  | 1050 | 1.2734          | 0.655    | 0.4833     | 3.1843  | 0.655    | 0.6600   | 0.1721 | 0.1440 |
| 0.0683        | 43.0  | 1075 | 1.2628          | 0.66     | 0.4802     | 3.2578  | 0.66     | 0.6670   | 0.1789 | 0.1403 |
| 0.0683        | 44.0  | 1100 | 1.2717          | 0.66     | 0.4837     | 3.2573  | 0.66     | 0.6651   | 0.1584 | 0.1433 |
| 0.0683        | 45.0  | 1125 | 1.2637          | 0.6475   | 0.4791     | 3.3419  | 0.6475   | 0.6545   | 0.1736 | 0.1408 |
| 0.0683        | 46.0  | 1150 | 1.2625          | 0.6575   | 0.4797     | 3.3403  | 0.6575   | 0.6642   | 0.1597 | 0.1406 |
| 0.0683        | 47.0  | 1175 | 1.2642          | 0.6525   | 0.4791     | 3.3527  | 0.6525   | 0.6592   | 0.1731 | 0.1416 |
| 0.0683        | 48.0  | 1200 | 1.2652          | 0.655    | 0.4816     | 3.2664  | 0.655    | 0.6623   | 0.1717 | 0.1413 |
| 0.0683        | 49.0  | 1225 | 1.2646          | 0.65     | 0.4806     | 3.3371  | 0.65     | 0.6568   | 0.1758 | 0.1419 |
| 0.0683        | 50.0  | 1250 | 1.2677          | 0.65     | 0.4812     | 3.4189  | 0.65     | 0.6575   | 0.1582 | 0.1427 |
| 0.0683        | 51.0  | 1275 | 1.2657          | 0.65     | 0.4813     | 3.3393  | 0.65     | 0.6565   | 0.1748 | 0.1413 |
| 0.0683        | 52.0  | 1300 | 1.2648          | 0.655    | 0.4813     | 3.3447  | 0.655    | 0.6629   | 0.1627 | 0.1419 |
| 0.0683        | 53.0  | 1325 | 1.2650          | 0.65     | 0.4813     | 3.3350  | 0.65     | 0.6565   | 0.1780 | 0.1414 |
| 0.0683        | 54.0  | 1350 | 1.2593          | 0.655    | 0.4790     | 3.3427  | 0.655    | 0.6620   | 0.1543 | 0.1399 |
| 0.0683        | 55.0  | 1375 | 1.2648          | 0.6525   | 0.4810     | 3.3368  | 0.6525   | 0.6592   | 0.1723 | 0.1414 |
| 0.0683        | 56.0  | 1400 | 1.2608          | 0.6525   | 0.4802     | 3.2599  | 0.6525   | 0.6603   | 0.1738 | 0.1411 |
| 0.0683        | 57.0  | 1425 | 1.2639          | 0.6525   | 0.4799     | 3.3437  | 0.6525   | 0.6599   | 0.1767 | 0.1413 |
| 0.0683        | 58.0  | 1450 | 1.2631          | 0.65     | 0.4810     | 3.3401  | 0.65     | 0.6578   | 0.1667 | 0.1416 |
| 0.0683        | 59.0  | 1475 | 1.2636          | 0.6525   | 0.4803     | 3.3411  | 0.6525   | 0.6594   | 0.1690 | 0.1416 |
| 0.0391        | 60.0  | 1500 | 1.2618          | 0.6525   | 0.4796     | 3.2684  | 0.6525   | 0.6600   | 0.1813 | 0.1413 |
| 0.0391        | 61.0  | 1525 | 1.2636          | 0.6525   | 0.4807     | 3.2704  | 0.6525   | 0.6595   | 0.1673 | 0.1413 |
| 0.0391        | 62.0  | 1550 | 1.2615          | 0.65     | 0.4794     | 3.2662  | 0.65     | 0.6575   | 0.1741 | 0.1413 |
| 0.0391        | 63.0  | 1575 | 1.2630          | 0.65     | 0.4803     | 3.3417  | 0.65     | 0.6575   | 0.1752 | 0.1411 |
| 0.0391        | 64.0  | 1600 | 1.2618          | 0.65     | 0.4801     | 3.2663  | 0.65     | 0.6575   | 0.1770 | 0.1413 |
| 0.0391        | 65.0  | 1625 | 1.2622          | 0.65     | 0.4802     | 3.2698  | 0.65     | 0.6575   | 0.1686 | 0.1412 |
| 0.0391        | 66.0  | 1650 | 1.2622          | 0.65     | 0.4802     | 3.3400  | 0.65     | 0.6575   | 0.1922 | 0.1412 |
| 0.0391        | 67.0  | 1675 | 1.2625          | 0.65     | 0.4802     | 3.2694  | 0.65     | 0.6575   | 0.1801 | 0.1413 |
| 0.0391        | 68.0  | 1700 | 1.2626          | 0.65     | 0.4803     | 3.2683  | 0.65     | 0.6575   | 0.1656 | 0.1414 |
| 0.0391        | 69.0  | 1725 | 1.2631          | 0.65     | 0.4806     | 3.2696  | 0.65     | 0.6575   | 0.1722 | 0.1413 |
| 0.0391        | 70.0  | 1750 | 1.2622          | 0.65     | 0.4802     | 3.2688  | 0.65     | 0.6575   | 0.1812 | 0.1412 |
| 0.0391        | 71.0  | 1775 | 1.2626          | 0.65     | 0.4803     | 3.2676  | 0.65     | 0.6575   | 0.1845 | 0.1412 |
| 0.0391        | 72.0  | 1800 | 1.2621          | 0.65     | 0.4801     | 3.2683  | 0.65     | 0.6575   | 0.1805 | 0.1411 |
| 0.0391        | 73.0  | 1825 | 1.2626          | 0.65     | 0.4804     | 3.2683  | 0.65     | 0.6575   | 0.1665 | 0.1413 |
| 0.0391        | 74.0  | 1850 | 1.2624          | 0.65     | 0.4803     | 3.2686  | 0.65     | 0.6575   | 0.1773 | 0.1412 |
| 0.0391        | 75.0  | 1875 | 1.2624          | 0.65     | 0.4803     | 3.2682  | 0.65     | 0.6575   | 0.1807 | 0.1412 |
| 0.0391        | 76.0  | 1900 | 1.2627          | 0.65     | 0.4804     | 3.2680  | 0.65     | 0.6575   | 0.1732 | 0.1414 |
| 0.0391        | 77.0  | 1925 | 1.2625          | 0.65     | 0.4803     | 3.2673  | 0.65     | 0.6575   | 0.1715 | 0.1412 |
| 0.0391        | 78.0  | 1950 | 1.2623          | 0.65     | 0.4803     | 3.2681  | 0.65     | 0.6575   | 0.1840 | 0.1413 |
| 0.0391        | 79.0  | 1975 | 1.2624          | 0.65     | 0.4803     | 3.2678  | 0.65     | 0.6575   | 0.1773 | 0.1413 |
| 0.0385        | 80.0  | 2000 | 1.2625          | 0.65     | 0.4803     | 3.2686  | 0.65     | 0.6575   | 0.1802 | 0.1414 |
| 0.0385        | 81.0  | 2025 | 1.2625          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1773 | 0.1413 |
| 0.0385        | 82.0  | 2050 | 1.2625          | 0.65     | 0.4803     | 3.2684  | 0.65     | 0.6575   | 0.1802 | 0.1414 |
| 0.0385        | 83.0  | 2075 | 1.2624          | 0.65     | 0.4803     | 3.2679  | 0.65     | 0.6575   | 0.1823 | 0.1413 |
| 0.0385        | 84.0  | 2100 | 1.2623          | 0.65     | 0.4803     | 3.2681  | 0.65     | 0.6575   | 0.1772 | 0.1413 |
| 0.0385        | 85.0  | 2125 | 1.2624          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 86.0  | 2150 | 1.2625          | 0.65     | 0.4804     | 3.2680  | 0.65     | 0.6575   | 0.1751 | 0.1414 |
| 0.0385        | 87.0  | 2175 | 1.2623          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1772 | 0.1413 |
| 0.0385        | 88.0  | 2200 | 1.2624          | 0.65     | 0.4803     | 3.2676  | 0.65     | 0.6575   | 0.1723 | 0.1414 |
| 0.0385        | 89.0  | 2225 | 1.2623          | 0.65     | 0.4803     | 3.2679  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 90.0  | 2250 | 1.2622          | 0.65     | 0.4802     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1413 |
| 0.0385        | 91.0  | 2275 | 1.2623          | 0.65     | 0.4803     | 3.2678  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 92.0  | 2300 | 1.2624          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 93.0  | 2325 | 1.2623          | 0.65     | 0.4803     | 3.2679  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 94.0  | 2350 | 1.2623          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 95.0  | 2375 | 1.2623          | 0.65     | 0.4803     | 3.2676  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 96.0  | 2400 | 1.2623          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 97.0  | 2425 | 1.2623          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 98.0  | 2450 | 1.2623          | 0.65     | 0.4803     | 3.2677  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 99.0  | 2475 | 1.2623          | 0.65     | 0.4803     | 3.2676  | 0.65     | 0.6575   | 0.1722 | 0.1414 |
| 0.0385        | 100.0 | 2500 | 1.2623          | 0.65     | 0.4803     | 3.2676  | 0.65     | 0.6575   | 0.1722 | 0.1414 |


### Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu113
- Datasets 2.12.0
- Tokenizers 0.12.1

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo