
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-small_rvl_cdip_100_examples_per_class_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4583
- Accuracy: 0.655
- Brier Loss: 0.4857
- Nll: 2.9372
- F1 Micro: 0.655
- F1 Macro: 0.6591
- Ece: 0.1679
- Aurc: 0.1394

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll     | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:-------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 4.2264          | 0.1375   | 0.9289     | 15.9084 | 0.1375   | 0.1395   | 0.1536 | 0.8596 |
| No log        | 2.0   | 50   | 3.2078          | 0.405    | 0.7396     | 8.9647  | 0.405    | 0.3723   | 0.2073 | 0.3570 |
| No log        | 3.0   | 75   | 2.4477          | 0.4975   | 0.6180     | 5.3439  | 0.4975   | 0.4756   | 0.1714 | 0.2421 |
| No log        | 4.0   | 100  | 2.2058          | 0.545    | 0.5825     | 4.3028  | 0.545    | 0.5448   | 0.1681 | 0.2147 |
| No log        | 5.0   | 125  | 2.1459          | 0.5325   | 0.6143     | 4.3798  | 0.5325   | 0.5164   | 0.2012 | 0.2274 |
| No log        | 6.0   | 150  | 2.0457          | 0.5825   | 0.5625     | 4.1921  | 0.5825   | 0.5823   | 0.1712 | 0.2008 |
| No log        | 7.0   | 175  | 1.9438          | 0.575    | 0.5557     | 4.2405  | 0.575    | 0.5654   | 0.1805 | 0.1894 |
| No log        | 8.0   | 200  | 1.9821          | 0.5675   | 0.5766     | 3.8326  | 0.5675   | 0.5665   | 0.1815 | 0.2050 |
| No log        | 9.0   | 225  | 2.1566          | 0.5425   | 0.6068     | 4.2488  | 0.5425   | 0.5367   | 0.2053 | 0.2167 |
| No log        | 10.0  | 250  | 1.9672          | 0.5925   | 0.5692     | 4.3417  | 0.5925   | 0.5968   | 0.2005 | 0.2114 |
| No log        | 11.0  | 275  | 2.0417          | 0.5725   | 0.6080     | 3.6972  | 0.5725   | 0.5608   | 0.2005 | 0.2168 |
| No log        | 12.0  | 300  | 1.9432          | 0.585    | 0.5704     | 3.6005  | 0.585    | 0.5840   | 0.1976 | 0.1939 |
| No log        | 13.0  | 325  | 1.9031          | 0.585    | 0.5816     | 4.0984  | 0.585    | 0.5835   | 0.1996 | 0.1911 |
| No log        | 14.0  | 350  | 1.8994          | 0.5925   | 0.5897     | 4.2703  | 0.5925   | 0.5926   | 0.2211 | 0.2041 |
| No log        | 15.0  | 375  | 1.8136          | 0.6325   | 0.5297     | 4.5861  | 0.6325   | 0.6299   | 0.1622 | 0.1578 |
| No log        | 16.0  | 400  | 1.6961          | 0.5925   | 0.5300     | 4.0317  | 0.5925   | 0.5839   | 0.1909 | 0.1630 |
| No log        | 17.0  | 425  | 1.7687          | 0.61     | 0.5357     | 3.6514  | 0.61     | 0.6110   | 0.1715 | 0.1703 |
| No log        | 18.0  | 450  | 1.8963          | 0.6      | 0.5785     | 4.7474  | 0.6      | 0.5842   | 0.2168 | 0.1893 |
| No log        | 19.0  | 475  | 1.7545          | 0.6175   | 0.5506     | 4.4192  | 0.6175   | 0.6086   | 0.2006 | 0.1759 |
| 0.8611        | 20.0  | 500  | 1.7832          | 0.61     | 0.5546     | 4.0543  | 0.61     | 0.6099   | 0.2133 | 0.1662 |
| 0.8611        | 21.0  | 525  | 1.7788          | 0.5875   | 0.5718     | 3.8585  | 0.5875   | 0.5855   | 0.2084 | 0.1848 |
| 0.8611        | 22.0  | 550  | 1.6323          | 0.62     | 0.5184     | 3.6953  | 0.62     | 0.6146   | 0.1921 | 0.1588 |
| 0.8611        | 23.0  | 575  | 1.6384          | 0.6325   | 0.5431     | 3.5349  | 0.6325   | 0.6269   | 0.2042 | 0.1678 |
| 0.8611        | 24.0  | 600  | 1.7895          | 0.62     | 0.5588     | 4.2768  | 0.62     | 0.6169   | 0.1993 | 0.1885 |
| 0.8611        | 25.0  | 625  | 1.5712          | 0.6175   | 0.5111     | 3.1891  | 0.6175   | 0.6199   | 0.1777 | 0.1552 |
| 0.8611        | 26.0  | 650  | 1.6139          | 0.62     | 0.5284     | 3.0912  | 0.62     | 0.6238   | 0.1793 | 0.1599 |
| 0.8611        | 27.0  | 675  | 1.6449          | 0.6375   | 0.5190     | 4.0147  | 0.6375   | 0.6313   | 0.1794 | 0.1606 |
| 0.8611        | 28.0  | 700  | 1.6379          | 0.6325   | 0.5355     | 3.5225  | 0.6325   | 0.6300   | 0.1859 | 0.1693 |
| 0.8611        | 29.0  | 725  | 1.5486          | 0.6375   | 0.5202     | 3.1611  | 0.6375   | 0.6407   | 0.1908 | 0.1608 |
| 0.8611        | 30.0  | 750  | 1.5410          | 0.63     | 0.5074     | 3.2562  | 0.63     | 0.6340   | 0.1772 | 0.1424 |
| 0.8611        | 31.0  | 775  | 1.5033          | 0.6575   | 0.4973     | 3.3321  | 0.6575   | 0.6619   | 0.1802 | 0.1451 |
| 0.8611        | 32.0  | 800  | 1.6065          | 0.6375   | 0.5260     | 3.4264  | 0.6375   | 0.6451   | 0.2028 | 0.1670 |
| 0.8611        | 33.0  | 825  | 1.5188          | 0.6525   | 0.5028     | 3.5128  | 0.6525   | 0.6536   | 0.1813 | 0.1491 |
| 0.8611        | 34.0  | 850  | 1.5034          | 0.635    | 0.5005     | 3.4093  | 0.635    | 0.6345   | 0.1602 | 0.1506 |
| 0.8611        | 35.0  | 875  | 1.5711          | 0.66     | 0.5163     | 3.6591  | 0.66     | 0.6587   | 0.1884 | 0.1574 |
| 0.8611        | 36.0  | 900  | 1.5224          | 0.6475   | 0.5057     | 3.1773  | 0.6475   | 0.6491   | 0.1802 | 0.1526 |
| 0.8611        | 37.0  | 925  | 1.4781          | 0.6475   | 0.4938     | 3.3389  | 0.6475   | 0.6508   | 0.1753 | 0.1420 |
| 0.8611        | 38.0  | 950  | 1.4991          | 0.65     | 0.5005     | 3.4077  | 0.65     | 0.6541   | 0.1843 | 0.1482 |
| 0.8611        | 39.0  | 975  | 1.4613          | 0.6625   | 0.4848     | 3.2461  | 0.6625   | 0.6675   | 0.1647 | 0.1386 |
| 0.0907        | 40.0  | 1000 | 1.4824          | 0.64     | 0.4951     | 3.1830  | 0.64     | 0.6444   | 0.1779 | 0.1431 |
| 0.0907        | 41.0  | 1025 | 1.5224          | 0.6625   | 0.5004     | 3.4231  | 0.6625   | 0.6659   | 0.1769 | 0.1506 |
| 0.0907        | 42.0  | 1050 | 1.4882          | 0.6375   | 0.5013     | 3.0893  | 0.6375   | 0.6451   | 0.1844 | 0.1465 |
| 0.0907        | 43.0  | 1075 | 1.4852          | 0.665    | 0.4901     | 3.4025  | 0.665    | 0.6685   | 0.1869 | 0.1442 |
| 0.0907        | 44.0  | 1100 | 1.4744          | 0.65     | 0.4934     | 3.4829  | 0.65     | 0.6528   | 0.1836 | 0.1426 |
| 0.0907        | 45.0  | 1125 | 1.4735          | 0.66     | 0.4892     | 3.1763  | 0.66     | 0.6642   | 0.1666 | 0.1427 |
| 0.0907        | 46.0  | 1150 | 1.4690          | 0.65     | 0.4898     | 3.0960  | 0.65     | 0.6537   | 0.1642 | 0.1427 |
| 0.0907        | 47.0  | 1175 | 1.4773          | 0.6475   | 0.4909     | 3.2535  | 0.6475   | 0.6506   | 0.1749 | 0.1446 |
| 0.0907        | 48.0  | 1200 | 1.4632          | 0.6575   | 0.4884     | 3.1685  | 0.6575   | 0.6625   | 0.1750 | 0.1398 |
| 0.0907        | 49.0  | 1225 | 1.4712          | 0.66     | 0.4896     | 3.0915  | 0.66     | 0.6634   | 0.1697 | 0.1432 |
| 0.0907        | 50.0  | 1250 | 1.4630          | 0.655    | 0.4883     | 3.0953  | 0.655    | 0.6591   | 0.1650 | 0.1406 |
| 0.0907        | 51.0  | 1275 | 1.4607          | 0.66     | 0.4860     | 3.0153  | 0.66     | 0.6653   | 0.1665 | 0.1411 |
| 0.0907        | 52.0  | 1300 | 1.4646          | 0.6475   | 0.4889     | 3.0242  | 0.6475   | 0.6510   | 0.1713 | 0.1426 |
| 0.0907        | 53.0  | 1325 | 1.4717          | 0.6575   | 0.4904     | 3.0926  | 0.6575   | 0.6605   | 0.1789 | 0.1428 |
| 0.0907        | 54.0  | 1350 | 1.4554          | 0.645    | 0.4868     | 3.0882  | 0.645    | 0.6489   | 0.1664 | 0.1408 |
| 0.0907        | 55.0  | 1375 | 1.4581          | 0.6575   | 0.4855     | 3.0904  | 0.6575   | 0.6614   | 0.1602 | 0.1404 |
| 0.0907        | 56.0  | 1400 | 1.4588          | 0.655    | 0.4866     | 3.0910  | 0.655    | 0.6598   | 0.1722 | 0.1405 |
| 0.0907        | 57.0  | 1425 | 1.4582          | 0.6575   | 0.4859     | 3.0143  | 0.6575   | 0.6619   | 0.1540 | 0.1397 |
| 0.0907        | 58.0  | 1450 | 1.4613          | 0.6575   | 0.4865     | 3.0143  | 0.6575   | 0.6620   | 0.1659 | 0.1402 |
| 0.0907        | 59.0  | 1475 | 1.4593          | 0.655    | 0.4867     | 3.0140  | 0.655    | 0.6599   | 0.1583 | 0.1402 |
| 0.0478        | 60.0  | 1500 | 1.4593          | 0.655    | 0.4864     | 3.0148  | 0.655    | 0.6593   | 0.1657 | 0.1404 |
| 0.0478        | 61.0  | 1525 | 1.4588          | 0.655    | 0.4861     | 3.0165  | 0.655    | 0.6590   | 0.1757 | 0.1401 |
| 0.0478        | 62.0  | 1550 | 1.4598          | 0.6575   | 0.4864     | 3.0140  | 0.6575   | 0.6616   | 0.1528 | 0.1403 |
| 0.0478        | 63.0  | 1575 | 1.4595          | 0.6575   | 0.4865     | 3.0143  | 0.6575   | 0.6623   | 0.1538 | 0.1400 |
| 0.0478        | 64.0  | 1600 | 1.4591          | 0.655    | 0.4864     | 2.9404  | 0.655    | 0.6591   | 0.1669 | 0.1399 |
| 0.0478        | 65.0  | 1625 | 1.4568          | 0.655    | 0.4854     | 2.9393  | 0.655    | 0.6596   | 0.1644 | 0.1393 |
| 0.0478        | 66.0  | 1650 | 1.4569          | 0.655    | 0.4855     | 3.0146  | 0.655    | 0.6599   | 0.1619 | 0.1401 |
| 0.0478        | 67.0  | 1675 | 1.4592          | 0.655    | 0.4865     | 2.9380  | 0.655    | 0.6596   | 0.1540 | 0.1399 |
| 0.0478        | 68.0  | 1700 | 1.4580          | 0.66     | 0.4858     | 2.9406  | 0.66     | 0.6641   | 0.1850 | 0.1396 |
| 0.0478        | 69.0  | 1725 | 1.4591          | 0.655    | 0.4865     | 2.9381  | 0.655    | 0.6593   | 0.1651 | 0.1399 |
| 0.0478        | 70.0  | 1750 | 1.4586          | 0.655    | 0.4859     | 2.9388  | 0.655    | 0.6596   | 0.1773 | 0.1397 |
| 0.0478        | 71.0  | 1775 | 1.4585          | 0.6525   | 0.4862     | 2.9366  | 0.6525   | 0.6566   | 0.1644 | 0.1400 |
| 0.0478        | 72.0  | 1800 | 1.4582          | 0.66     | 0.4858     | 2.9385  | 0.66     | 0.6644   | 0.1809 | 0.1396 |
| 0.0478        | 73.0  | 1825 | 1.4577          | 0.65     | 0.4857     | 2.9374  | 0.65     | 0.6543   | 0.1715 | 0.1403 |
| 0.0478        | 74.0  | 1850 | 1.4578          | 0.6525   | 0.4857     | 2.9381  | 0.6525   | 0.6565   | 0.1748 | 0.1401 |
| 0.0478        | 75.0  | 1875 | 1.4583          | 0.65     | 0.4860     | 2.9371  | 0.65     | 0.6544   | 0.1661 | 0.1402 |
| 0.0478        | 76.0  | 1900 | 1.4582          | 0.65     | 0.4859     | 2.9369  | 0.65     | 0.6544   | 0.1760 | 0.1402 |
| 0.0478        | 77.0  | 1925 | 1.4585          | 0.65     | 0.4859     | 2.9367  | 0.65     | 0.6546   | 0.1609 | 0.1403 |
| 0.0478        | 78.0  | 1950 | 1.4580          | 0.65     | 0.4858     | 2.9372  | 0.65     | 0.6546   | 0.1626 | 0.1401 |
| 0.0478        | 79.0  | 1975 | 1.4578          | 0.6525   | 0.4857     | 2.9369  | 0.6525   | 0.6564   | 0.1706 | 0.1400 |
| 0.0457        | 80.0  | 2000 | 1.4584          | 0.6525   | 0.4859     | 2.9370  | 0.6525   | 0.6564   | 0.1712 | 0.1402 |
| 0.0457        | 81.0  | 2025 | 1.4587          | 0.6525   | 0.4860     | 2.9370  | 0.6525   | 0.6568   | 0.1631 | 0.1402 |
| 0.0457        | 82.0  | 2050 | 1.4584          | 0.6525   | 0.4859     | 2.9369  | 0.6525   | 0.6568   | 0.1631 | 0.1401 |
| 0.0457        | 83.0  | 2075 | 1.4581          | 0.65     | 0.4858     | 2.9369  | 0.65     | 0.6543   | 0.1703 | 0.1401 |
| 0.0457        | 84.0  | 2100 | 1.4581          | 0.6525   | 0.4858     | 2.9370  | 0.6525   | 0.6564   | 0.1588 | 0.1401 |
| 0.0457        | 85.0  | 2125 | 1.4582          | 0.6525   | 0.4858     | 2.9370  | 0.6525   | 0.6568   | 0.1723 | 0.1400 |
| 0.0457        | 86.0  | 2150 | 1.4582          | 0.6525   | 0.4858     | 2.9371  | 0.6525   | 0.6564   | 0.1724 | 0.1400 |
| 0.0457        | 87.0  | 2175 | 1.4582          | 0.6525   | 0.4858     | 2.9369  | 0.6525   | 0.6567   | 0.1720 | 0.1400 |
| 0.0457        | 88.0  | 2200 | 1.4582          | 0.6525   | 0.4858     | 2.9372  | 0.6525   | 0.6567   | 0.1606 | 0.1401 |
| 0.0457        | 89.0  | 2225 | 1.4583          | 0.6525   | 0.4858     | 2.9372  | 0.6525   | 0.6567   | 0.1665 | 0.1401 |
| 0.0457        | 90.0  | 2250 | 1.4583          | 0.6525   | 0.4857     | 2.9370  | 0.6525   | 0.6564   | 0.1688 | 0.1400 |
| 0.0457        | 91.0  | 2275 | 1.4583          | 0.6525   | 0.4858     | 2.9371  | 0.6525   | 0.6567   | 0.1695 | 0.1400 |
| 0.0457        | 92.0  | 2300 | 1.4583          | 0.655    | 0.4858     | 2.9372  | 0.655    | 0.6591   | 0.1660 | 0.1394 |
| 0.0457        | 93.0  | 2325 | 1.4583          | 0.6525   | 0.4857     | 2.9371  | 0.6525   | 0.6565   | 0.1645 | 0.1400 |
| 0.0457        | 94.0  | 2350 | 1.4583          | 0.6525   | 0.4858     | 2.9371  | 0.6525   | 0.6567   | 0.1665 | 0.1399 |
| 0.0457        | 95.0  | 2375 | 1.4583          | 0.6525   | 0.4858     | 2.9372  | 0.6525   | 0.6567   | 0.1704 | 0.1399 |
| 0.0457        | 96.0  | 2400 | 1.4583          | 0.655    | 0.4858     | 2.9372  | 0.655    | 0.6588   | 0.1660 | 0.1395 |
| 0.0457        | 97.0  | 2425 | 1.4582          | 0.6525   | 0.4857     | 2.9372  | 0.6525   | 0.6567   | 0.1704 | 0.1399 |
| 0.0457        | 98.0  | 2450 | 1.4582          | 0.655    | 0.4857     | 2.9372  | 0.655    | 0.6591   | 0.1679 | 0.1394 |
| 0.0457        | 99.0  | 2475 | 1.4583          | 0.6525   | 0.4857     | 2.9372  | 0.6525   | 0.6567   | 0.1704 | 0.1399 |
| 0.0456        | 100.0 | 2500 | 1.4583          | 0.655    | 0.4857     | 2.9372  | 0.655    | 0.6591   | 0.1679 | 0.1394 |


### Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu113
- Datasets 2.12.0
- Tokenizers 0.12.1

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo