
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-small_rvl_cdip_100_examples_per_class_kd_MSE

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4673
- Accuracy: 0.6425
- Brier Loss: 0.4763
- Nll: 3.0680
- F1 Micro: 0.6425
- F1 Macro: 0.6485
- Ece: 0.1946
- Aurc: 0.1381

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll     | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:-------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 4.4851          | 0.06     | 0.9565     | 13.8276 | 0.06     | 0.0556   | 0.1688 | 0.9385 |
| No log        | 2.0   | 50   | 3.5619          | 0.3775   | 0.7827     | 6.2649  | 0.3775   | 0.3611   | 0.2331 | 0.3882 |
| No log        | 3.0   | 75   | 2.8990          | 0.5025   | 0.6453     | 4.7376  | 0.5025   | 0.4858   | 0.1689 | 0.2658 |
| No log        | 4.0   | 100  | 2.5972          | 0.515    | 0.5980     | 4.4210  | 0.515    | 0.4895   | 0.1605 | 0.2249 |
| No log        | 5.0   | 125  | 2.4353          | 0.56     | 0.5762     | 3.4885  | 0.56     | 0.5566   | 0.1548 | 0.2100 |
| No log        | 6.0   | 150  | 2.4157          | 0.5475   | 0.5864     | 3.8261  | 0.5475   | 0.5323   | 0.1837 | 0.2167 |
| No log        | 7.0   | 175  | 2.1786          | 0.6075   | 0.5203     | 3.4565  | 0.6075   | 0.6103   | 0.1403 | 0.1670 |
| No log        | 8.0   | 200  | 2.1082          | 0.63     | 0.5040     | 3.3570  | 0.63     | 0.6246   | 0.1580 | 0.1530 |
| No log        | 9.0   | 225  | 2.0472          | 0.625    | 0.5042     | 3.8572  | 0.625    | 0.6184   | 0.1552 | 0.1530 |
| No log        | 10.0  | 250  | 2.0589          | 0.6025   | 0.5468     | 3.5723  | 0.6025   | 0.5982   | 0.1781 | 0.1785 |
| No log        | 11.0  | 275  | 1.8965          | 0.65     | 0.4755     | 3.4466  | 0.65     | 0.6497   | 0.1605 | 0.1475 |
| No log        | 12.0  | 300  | 1.9014          | 0.6325   | 0.5066     | 3.0881  | 0.6325   | 0.6359   | 0.1658 | 0.1591 |
| No log        | 13.0  | 325  | 1.7904          | 0.6175   | 0.5162     | 3.4673  | 0.6175   | 0.6141   | 0.1525 | 0.1598 |
| No log        | 14.0  | 350  | 1.8624          | 0.625    | 0.5173     | 3.6824  | 0.625    | 0.6179   | 0.1567 | 0.1624 |
| No log        | 15.0  | 375  | 1.7083          | 0.6625   | 0.4817     | 3.1296  | 0.6625   | 0.6686   | 0.1651 | 0.1405 |
| No log        | 16.0  | 400  | 1.8848          | 0.59     | 0.5478     | 4.3761  | 0.59     | 0.5913   | 0.2083 | 0.1696 |
| No log        | 17.0  | 425  | 1.7238          | 0.6125   | 0.5229     | 3.1232  | 0.6125   | 0.6052   | 0.1833 | 0.1553 |
| No log        | 18.0  | 450  | 1.7126          | 0.625    | 0.5152     | 2.9267  | 0.625    | 0.6284   | 0.1747 | 0.1565 |
| No log        | 19.0  | 475  | 1.6459          | 0.6275   | 0.5024     | 2.9078  | 0.6275   | 0.6219   | 0.1766 | 0.1527 |
| 1.0542        | 20.0  | 500  | 1.6029          | 0.6275   | 0.4855     | 3.0931  | 0.6275   | 0.6316   | 0.1720 | 0.1414 |
| 1.0542        | 21.0  | 525  | 1.6566          | 0.6525   | 0.4847     | 3.0998  | 0.6525   | 0.6479   | 0.1558 | 0.1438 |
| 1.0542        | 22.0  | 550  | 1.6169          | 0.645    | 0.4894     | 3.0081  | 0.645    | 0.6471   | 0.1687 | 0.1400 |
| 1.0542        | 23.0  | 575  | 1.5322          | 0.6525   | 0.4557     | 3.3587  | 0.6525   | 0.6520   | 0.1428 | 0.1247 |
| 1.0542        | 24.0  | 600  | 1.5991          | 0.6475   | 0.4787     | 2.9349  | 0.6475   | 0.6444   | 0.1580 | 0.1450 |
| 1.0542        | 25.0  | 625  | 1.5625          | 0.6375   | 0.4926     | 3.0245  | 0.6375   | 0.6378   | 0.1641 | 0.1433 |
| 1.0542        | 26.0  | 650  | 1.5366          | 0.64     | 0.4884     | 3.3388  | 0.64     | 0.6461   | 0.1595 | 0.1453 |
| 1.0542        | 27.0  | 675  | 1.5686          | 0.65     | 0.4765     | 3.5120  | 0.65     | 0.6504   | 0.1625 | 0.1359 |
| 1.0542        | 28.0  | 700  | 1.5562          | 0.6475   | 0.4817     | 3.0348  | 0.6475   | 0.6488   | 0.1459 | 0.1388 |
| 1.0542        | 29.0  | 725  | 1.5213          | 0.6475   | 0.4719     | 3.2628  | 0.6475   | 0.6475   | 0.1634 | 0.1326 |
| 1.0542        | 30.0  | 750  | 1.5492          | 0.6675   | 0.4730     | 3.1693  | 0.6675   | 0.6679   | 0.1469 | 0.1415 |
| 1.0542        | 31.0  | 775  | 1.5311          | 0.65     | 0.4896     | 3.0881  | 0.65     | 0.6504   | 0.1815 | 0.1380 |
| 1.0542        | 32.0  | 800  | 1.5556          | 0.6475   | 0.4821     | 3.1829  | 0.6475   | 0.6491   | 0.1640 | 0.1405 |
| 1.0542        | 33.0  | 825  | 1.5471          | 0.6375   | 0.4846     | 3.4190  | 0.6375   | 0.6407   | 0.1628 | 0.1415 |
| 1.0542        | 34.0  | 850  | 1.4809          | 0.6575   | 0.4714     | 2.9136  | 0.6575   | 0.6612   | 0.1729 | 0.1338 |
| 1.0542        | 35.0  | 875  | 1.5256          | 0.66     | 0.4773     | 3.2303  | 0.66     | 0.6650   | 0.1746 | 0.1368 |
| 1.0542        | 36.0  | 900  | 1.4929          | 0.6675   | 0.4671     | 3.2360  | 0.6675   | 0.6698   | 0.1698 | 0.1309 |
| 1.0542        | 37.0  | 925  | 1.4923          | 0.645    | 0.4880     | 3.0567  | 0.645    | 0.6564   | 0.1764 | 0.1395 |
| 1.0542        | 38.0  | 950  | 1.5038          | 0.665    | 0.4672     | 3.2116  | 0.665    | 0.6661   | 0.1588 | 0.1343 |
| 1.0542        | 39.0  | 975  | 1.4708          | 0.6625   | 0.4669     | 3.1420  | 0.6625   | 0.6675   | 0.1683 | 0.1301 |
| 0.0522        | 40.0  | 1000 | 1.5153          | 0.6475   | 0.4865     | 3.1796  | 0.6475   | 0.6447   | 0.1639 | 0.1400 |
| 0.0522        | 41.0  | 1025 | 1.4705          | 0.6575   | 0.4642     | 3.2196  | 0.6575   | 0.6626   | 0.1440 | 0.1308 |
| 0.0522        | 42.0  | 1050 | 1.4844          | 0.6575   | 0.4722     | 3.2445  | 0.6575   | 0.6595   | 0.1746 | 0.1328 |
| 0.0522        | 43.0  | 1075 | 1.4957          | 0.6425   | 0.4828     | 3.1456  | 0.6425   | 0.6468   | 0.1499 | 0.1417 |
| 0.0522        | 44.0  | 1100 | 1.5179          | 0.645    | 0.4910     | 3.3921  | 0.645    | 0.6470   | 0.1861 | 0.1433 |
| 0.0522        | 45.0  | 1125 | 1.4878          | 0.6425   | 0.4839     | 3.2139  | 0.6425   | 0.6478   | 0.1720 | 0.1403 |
| 0.0522        | 46.0  | 1150 | 1.4666          | 0.655    | 0.4741     | 2.9333  | 0.655    | 0.6601   | 0.1813 | 0.1347 |
| 0.0522        | 47.0  | 1175 | 1.4954          | 0.6575   | 0.4776     | 3.2102  | 0.6575   | 0.6604   | 0.1842 | 0.1390 |
| 0.0522        | 48.0  | 1200 | 1.4976          | 0.645    | 0.4856     | 3.1539  | 0.645    | 0.6493   | 0.1549 | 0.1407 |
| 0.0522        | 49.0  | 1225 | 1.4772          | 0.64     | 0.4780     | 2.9845  | 0.64     | 0.6445   | 0.1826 | 0.1388 |
| 0.0522        | 50.0  | 1250 | 1.4584          | 0.65     | 0.4703     | 3.0776  | 0.65     | 0.6533   | 0.1685 | 0.1352 |
| 0.0522        | 51.0  | 1275 | 1.4828          | 0.6325   | 0.4844     | 3.1425  | 0.6325   | 0.6377   | 0.1641 | 0.1409 |
| 0.0522        | 52.0  | 1300 | 1.4676          | 0.6525   | 0.4737     | 3.1483  | 0.6525   | 0.6565   | 0.1773 | 0.1358 |
| 0.0522        | 53.0  | 1325 | 1.4675          | 0.6475   | 0.4791     | 3.1411  | 0.6475   | 0.6515   | 0.1820 | 0.1388 |
| 0.0522        | 54.0  | 1350 | 1.4724          | 0.645    | 0.4764     | 3.0744  | 0.645    | 0.6499   | 0.1847 | 0.1382 |
| 0.0522        | 55.0  | 1375 | 1.4689          | 0.6425   | 0.4769     | 3.2256  | 0.6425   | 0.6476   | 0.1839 | 0.1376 |
| 0.0522        | 56.0  | 1400 | 1.4660          | 0.6425   | 0.4760     | 2.9907  | 0.6425   | 0.6479   | 0.1906 | 0.1378 |
| 0.0522        | 57.0  | 1425 | 1.4663          | 0.645    | 0.4757     | 3.0722  | 0.645    | 0.6514   | 0.1705 | 0.1367 |
| 0.0522        | 58.0  | 1450 | 1.4678          | 0.65     | 0.4770     | 3.0710  | 0.65     | 0.6546   | 0.1794 | 0.1371 |
| 0.0522        | 59.0  | 1475 | 1.4717          | 0.64     | 0.4786     | 3.0737  | 0.64     | 0.6455   | 0.1889 | 0.1392 |
| 0.0064        | 60.0  | 1500 | 1.4691          | 0.645    | 0.4768     | 3.0688  | 0.645    | 0.6499   | 0.1815 | 0.1378 |
| 0.0064        | 61.0  | 1525 | 1.4689          | 0.64     | 0.4767     | 3.0688  | 0.64     | 0.6452   | 0.1846 | 0.1382 |
| 0.0064        | 62.0  | 1550 | 1.4689          | 0.64     | 0.4770     | 3.0674  | 0.64     | 0.6455   | 0.1937 | 0.1383 |
| 0.0064        | 63.0  | 1575 | 1.4687          | 0.6425   | 0.4767     | 3.0700  | 0.6425   | 0.6485   | 0.1897 | 0.1381 |
| 0.0064        | 64.0  | 1600 | 1.4674          | 0.6425   | 0.4764     | 3.0675  | 0.6425   | 0.6472   | 0.1855 | 0.1375 |
| 0.0064        | 65.0  | 1625 | 1.4681          | 0.6425   | 0.4766     | 3.0694  | 0.6425   | 0.6485   | 0.1917 | 0.1381 |
| 0.0064        | 66.0  | 1650 | 1.4681          | 0.6425   | 0.4766     | 3.0687  | 0.6425   | 0.6472   | 0.1905 | 0.1378 |
| 0.0064        | 67.0  | 1675 | 1.4667          | 0.645    | 0.4757     | 3.0681  | 0.645    | 0.6505   | 0.1899 | 0.1375 |
| 0.0064        | 68.0  | 1700 | 1.4683          | 0.6425   | 0.4771     | 3.0686  | 0.6425   | 0.6474   | 0.1871 | 0.1379 |
| 0.0064        | 69.0  | 1725 | 1.4672          | 0.64     | 0.4760     | 3.0679  | 0.64     | 0.6455   | 0.1932 | 0.1380 |
| 0.0064        | 70.0  | 1750 | 1.4673          | 0.6425   | 0.4763     | 3.0683  | 0.6425   | 0.6474   | 0.1955 | 0.1376 |
| 0.0064        | 71.0  | 1775 | 1.4676          | 0.645    | 0.4763     | 3.0680  | 0.645    | 0.6505   | 0.1921 | 0.1376 |
| 0.0064        | 72.0  | 1800 | 1.4674          | 0.6425   | 0.4763     | 3.0683  | 0.6425   | 0.6474   | 0.1946 | 0.1376 |
| 0.0064        | 73.0  | 1825 | 1.4675          | 0.6425   | 0.4763     | 3.0682  | 0.6425   | 0.6474   | 0.1946 | 0.1377 |
| 0.0064        | 74.0  | 1850 | 1.4674          | 0.6425   | 0.4763     | 3.0682  | 0.6425   | 0.6485   | 0.1945 | 0.1380 |
| 0.0064        | 75.0  | 1875 | 1.4674          | 0.64     | 0.4763     | 3.0680  | 0.64     | 0.6455   | 0.1960 | 0.1380 |
| 0.0064        | 76.0  | 1900 | 1.4675          | 0.64     | 0.4764     | 3.0682  | 0.64     | 0.6455   | 0.1972 | 0.1381 |
| 0.0064        | 77.0  | 1925 | 1.4675          | 0.6425   | 0.4763     | 3.0681  | 0.6425   | 0.6485   | 0.1947 | 0.1380 |
| 0.0064        | 78.0  | 1950 | 1.4674          | 0.6425   | 0.4763     | 3.0681  | 0.6425   | 0.6485   | 0.1958 | 0.1381 |
| 0.0064        | 79.0  | 1975 | 1.4674          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6474   | 0.1935 | 0.1376 |
| 0.0           | 80.0  | 2000 | 1.4673          | 0.6425   | 0.4763     | 3.0681  | 0.6425   | 0.6485   | 0.1958 | 0.1380 |
| 0.0           | 81.0  | 2025 | 1.4674          | 0.6425   | 0.4763     | 3.0681  | 0.6425   | 0.6485   | 0.1946 | 0.1380 |
| 0.0           | 82.0  | 2050 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1935 | 0.1380 |
| 0.0           | 83.0  | 2075 | 1.4674          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 84.0  | 2100 | 1.4674          | 0.6425   | 0.4763     | 3.0681  | 0.6425   | 0.6485   | 0.1958 | 0.1381 |
| 0.0           | 85.0  | 2125 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 86.0  | 2150 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 87.0  | 2175 | 1.4673          | 0.6425   | 0.4763     | 3.0681  | 0.6425   | 0.6485   | 0.1958 | 0.1381 |
| 0.0           | 88.0  | 2200 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 89.0  | 2225 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 90.0  | 2250 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 91.0  | 2275 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 92.0  | 2300 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 93.0  | 2325 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 94.0  | 2350 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1909 | 0.1381 |
| 0.0           | 95.0  | 2375 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 96.0  | 2400 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 97.0  | 2425 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 98.0  | 2450 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 99.0  | 2475 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |
| 0.0           | 100.0 | 2500 | 1.4673          | 0.6425   | 0.4763     | 3.0680  | 0.6425   | 0.6485   | 0.1946 | 0.1381 |


### Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu113
- Datasets 2.12.0
- Tokenizers 0.12.1

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo