
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_rvl_cdip_100_examples_per_class_kd_MSE

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7723
- Accuracy: 0.6025
- Brier Loss: 0.5295
- Nll: 3.6748
- F1 Micro: 0.6025
- F1 Macro: 0.6055
- Ece: 0.1688
- Aurc: 0.1708

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll     | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:-------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 4.7870          | 0.065    | 0.9655     | 17.0930 | 0.065    | 0.0550   | 0.1747 | 0.9357 |
| No log        | 2.0   | 50   | 3.9498          | 0.205    | 0.8858     | 9.5780  | 0.205    | 0.1863   | 0.1692 | 0.6618 |
| No log        | 3.0   | 75   | 3.3698          | 0.3675   | 0.7672     | 6.4908  | 0.3675   | 0.3392   | 0.1676 | 0.4195 |
| No log        | 4.0   | 100  | 2.9935          | 0.4075   | 0.6958     | 5.5595  | 0.4075   | 0.3820   | 0.1828 | 0.3327 |
| No log        | 5.0   | 125  | 2.8351          | 0.455    | 0.6591     | 4.8619  | 0.455    | 0.4351   | 0.1561 | 0.2833 |
| No log        | 6.0   | 150  | 2.8196          | 0.4725   | 0.6595     | 4.7785  | 0.4725   | 0.4367   | 0.1808 | 0.2790 |
| No log        | 7.0   | 175  | 2.6352          | 0.5075   | 0.6234     | 4.9881  | 0.5075   | 0.4886   | 0.1563 | 0.2493 |
| No log        | 8.0   | 200  | 2.5325          | 0.525    | 0.6162     | 4.3297  | 0.525    | 0.5026   | 0.1724 | 0.2365 |
| No log        | 9.0   | 225  | 2.5459          | 0.53     | 0.6099     | 5.1608  | 0.53     | 0.5148   | 0.1944 | 0.2350 |
| No log        | 10.0  | 250  | 2.5573          | 0.5325   | 0.6161     | 5.4495  | 0.5325   | 0.5212   | 0.2052 | 0.2397 |
| No log        | 11.0  | 275  | 2.3199          | 0.5675   | 0.5828     | 4.1247  | 0.5675   | 0.5626   | 0.1849 | 0.2071 |
| No log        | 12.0  | 300  | 2.2917          | 0.565    | 0.5758     | 4.1738  | 0.565    | 0.5694   | 0.1992 | 0.2023 |
| No log        | 13.0  | 325  | 2.2744          | 0.555    | 0.5974     | 4.2323  | 0.555    | 0.5544   | 0.1982 | 0.2203 |
| No log        | 14.0  | 350  | 2.1638          | 0.5625   | 0.5807     | 4.2049  | 0.5625   | 0.5629   | 0.1868 | 0.2049 |
| No log        | 15.0  | 375  | 2.1934          | 0.5575   | 0.5903     | 4.3813  | 0.5575   | 0.5614   | 0.1868 | 0.2022 |
| No log        | 16.0  | 400  | 2.1092          | 0.5625   | 0.5702     | 3.6094  | 0.5625   | 0.5700   | 0.1846 | 0.2011 |
| No log        | 17.0  | 425  | 2.0379          | 0.5875   | 0.5642     | 4.4351  | 0.5875   | 0.5822   | 0.2036 | 0.1959 |
| No log        | 18.0  | 450  | 2.0303          | 0.5825   | 0.5558     | 3.6847  | 0.5825   | 0.5820   | 0.1684 | 0.1881 |
| No log        | 19.0  | 475  | 2.0506          | 0.57     | 0.5749     | 4.0014  | 0.57     | 0.5708   | 0.1725 | 0.2027 |
| 1.5026        | 20.0  | 500  | 1.9932          | 0.5875   | 0.5524     | 3.8003  | 0.5875   | 0.5914   | 0.1843 | 0.1831 |
| 1.5026        | 21.0  | 525  | 2.0131          | 0.565    | 0.5643     | 4.0681  | 0.565    | 0.5635   | 0.1776 | 0.1957 |
| 1.5026        | 22.0  | 550  | 2.0162          | 0.5725   | 0.5712     | 3.7068  | 0.5725   | 0.5766   | 0.1934 | 0.1955 |
| 1.5026        | 23.0  | 575  | 1.9093          | 0.605    | 0.5381     | 3.7930  | 0.605    | 0.6032   | 0.1539 | 0.1749 |
| 1.5026        | 24.0  | 600  | 1.9607          | 0.575    | 0.5561     | 4.5740  | 0.575    | 0.5789   | 0.1782 | 0.1902 |
| 1.5026        | 25.0  | 625  | 1.8971          | 0.5825   | 0.5408     | 3.7290  | 0.5825   | 0.5754   | 0.1836 | 0.1751 |
| 1.5026        | 26.0  | 650  | 1.9217          | 0.5775   | 0.5537     | 3.8085  | 0.5775   | 0.5844   | 0.1725 | 0.1843 |
| 1.5026        | 27.0  | 675  | 1.9493          | 0.585    | 0.5606     | 3.6743  | 0.585    | 0.5953   | 0.1755 | 0.1882 |
| 1.5026        | 28.0  | 700  | 1.8884          | 0.585    | 0.5437     | 3.7865  | 0.585    | 0.5828   | 0.1801 | 0.1822 |
| 1.5026        | 29.0  | 725  | 1.9242          | 0.585    | 0.5479     | 3.9607  | 0.585    | 0.5856   | 0.1619 | 0.1817 |
| 1.5026        | 30.0  | 750  | 1.8767          | 0.5975   | 0.5470     | 3.7995  | 0.5975   | 0.5966   | 0.1599 | 0.1790 |
| 1.5026        | 31.0  | 775  | 1.8723          | 0.5925   | 0.5337     | 3.8962  | 0.5925   | 0.5972   | 0.1678 | 0.1729 |
| 1.5026        | 32.0  | 800  | 1.9093          | 0.585    | 0.5545     | 3.8776  | 0.585    | 0.5830   | 0.1902 | 0.1841 |
| 1.5026        | 33.0  | 825  | 1.8667          | 0.595    | 0.5363     | 3.8926  | 0.595    | 0.5917   | 0.1772 | 0.1745 |
| 1.5026        | 34.0  | 850  | 1.8403          | 0.59     | 0.5521     | 3.8560  | 0.59     | 0.5953   | 0.1711 | 0.1800 |
| 1.5026        | 35.0  | 875  | 1.8464          | 0.5925   | 0.5380     | 4.0376  | 0.5925   | 0.5970   | 0.1719 | 0.1756 |
| 1.5026        | 36.0  | 900  | 1.8441          | 0.5975   | 0.5411     | 3.7193  | 0.5975   | 0.6008   | 0.1569 | 0.1753 |
| 1.5026        | 37.0  | 925  | 1.8599          | 0.5875   | 0.5402     | 3.9139  | 0.5875   | 0.5908   | 0.1779 | 0.1789 |
| 1.5026        | 38.0  | 950  | 1.8559          | 0.6      | 0.5458     | 3.8970  | 0.6      | 0.5991   | 0.1583 | 0.1804 |
| 1.5026        | 39.0  | 975  | 1.8285          | 0.61     | 0.5370     | 3.6292  | 0.61     | 0.6155   | 0.1623 | 0.1722 |
| 0.0745        | 40.0  | 1000 | 1.8309          | 0.5975   | 0.5432     | 3.6865  | 0.5975   | 0.6017   | 0.1663 | 0.1821 |
| 0.0745        | 41.0  | 1025 | 1.8237          | 0.59     | 0.5348     | 3.6213  | 0.59     | 0.5921   | 0.1695 | 0.1738 |
| 0.0745        | 42.0  | 1050 | 1.8421          | 0.605    | 0.5360     | 3.8592  | 0.605    | 0.6048   | 0.1601 | 0.1743 |
| 0.0745        | 43.0  | 1075 | 1.8158          | 0.5975   | 0.5300     | 3.4537  | 0.5975   | 0.5953   | 0.1696 | 0.1707 |
| 0.0745        | 44.0  | 1100 | 1.8238          | 0.5875   | 0.5358     | 3.7706  | 0.5875   | 0.5923   | 0.1797 | 0.1754 |
| 0.0745        | 45.0  | 1125 | 1.8214          | 0.595    | 0.5463     | 3.4742  | 0.595    | 0.5981   | 0.1800 | 0.1770 |
| 0.0745        | 46.0  | 1150 | 1.8162          | 0.5925   | 0.5317     | 3.9260  | 0.5925   | 0.5950   | 0.1646 | 0.1733 |
| 0.0745        | 47.0  | 1175 | 1.8050          | 0.5975   | 0.5392     | 3.8322  | 0.5975   | 0.5979   | 0.1794 | 0.1763 |
| 0.0745        | 48.0  | 1200 | 1.8214          | 0.5975   | 0.5347     | 3.7965  | 0.5975   | 0.6009   | 0.1555 | 0.1746 |
| 0.0745        | 49.0  | 1225 | 1.7813          | 0.6      | 0.5294     | 3.8398  | 0.6      | 0.6005   | 0.1674 | 0.1688 |
| 0.0745        | 50.0  | 1250 | 1.8179          | 0.6075   | 0.5336     | 3.4690  | 0.6075   | 0.6112   | 0.1743 | 0.1748 |
| 0.0745        | 51.0  | 1275 | 1.7953          | 0.595    | 0.5380     | 3.7781  | 0.595    | 0.5990   | 0.1380 | 0.1727 |
| 0.0745        | 52.0  | 1300 | 1.7897          | 0.6      | 0.5323     | 3.7412  | 0.6      | 0.6013   | 0.1603 | 0.1707 |
| 0.0745        | 53.0  | 1325 | 1.8072          | 0.59     | 0.5428     | 3.5993  | 0.59     | 0.5947   | 0.1571 | 0.1773 |
| 0.0745        | 54.0  | 1350 | 1.7834          | 0.605    | 0.5219     | 3.7600  | 0.605    | 0.6049   | 0.1563 | 0.1671 |
| 0.0745        | 55.0  | 1375 | 1.7920          | 0.595    | 0.5361     | 3.5986  | 0.595    | 0.5978   | 0.1512 | 0.1717 |
| 0.0745        | 56.0  | 1400 | 1.8074          | 0.5925   | 0.5387     | 3.5383  | 0.5925   | 0.5962   | 0.1669 | 0.1741 |
| 0.0745        | 57.0  | 1425 | 1.7893          | 0.605    | 0.5346     | 3.6929  | 0.605    | 0.6039   | 0.1641 | 0.1681 |
| 0.0745        | 58.0  | 1450 | 1.7787          | 0.6      | 0.5317     | 3.7652  | 0.6      | 0.6004   | 0.1850 | 0.1726 |
| 0.0745        | 59.0  | 1475 | 1.7888          | 0.595    | 0.5323     | 3.4558  | 0.595    | 0.5975   | 0.1797 | 0.1732 |
| 0.0231        | 60.0  | 1500 | 1.8064          | 0.58     | 0.5332     | 3.7773  | 0.58     | 0.5839   | 0.1819 | 0.1762 |
| 0.0231        | 61.0  | 1525 | 1.7795          | 0.6075   | 0.5298     | 3.7998  | 0.6075   | 0.6086   | 0.1678 | 0.1704 |
| 0.0231        | 62.0  | 1550 | 1.7826          | 0.595    | 0.5318     | 3.6741  | 0.595    | 0.5916   | 0.1550 | 0.1715 |
| 0.0231        | 63.0  | 1575 | 1.7704          | 0.5925   | 0.5325     | 3.5942  | 0.5925   | 0.5941   | 0.1619 | 0.1712 |
| 0.0231        | 64.0  | 1600 | 1.7901          | 0.6025   | 0.5289     | 3.4459  | 0.6025   | 0.6054   | 0.2022 | 0.1712 |
| 0.0231        | 65.0  | 1625 | 1.7944          | 0.59     | 0.5381     | 3.7591  | 0.59     | 0.5910   | 0.1599 | 0.1756 |
| 0.0231        | 66.0  | 1650 | 1.7721          | 0.605    | 0.5256     | 3.5227  | 0.605    | 0.6045   | 0.1525 | 0.1677 |
| 0.0231        | 67.0  | 1675 | 1.7779          | 0.5975   | 0.5306     | 3.6792  | 0.5975   | 0.5994   | 0.1667 | 0.1714 |
| 0.0231        | 68.0  | 1700 | 1.7724          | 0.6      | 0.5250     | 3.7552  | 0.6      | 0.6022   | 0.1818 | 0.1683 |
| 0.0231        | 69.0  | 1725 | 1.7765          | 0.6025   | 0.5283     | 3.4264  | 0.6025   | 0.6019   | 0.1671 | 0.1700 |
| 0.0231        | 70.0  | 1750 | 1.7784          | 0.6      | 0.5276     | 3.6887  | 0.6      | 0.6053   | 0.1715 | 0.1703 |
| 0.0231        | 71.0  | 1775 | 1.7659          | 0.6      | 0.5282     | 3.6051  | 0.6      | 0.6006   | 0.1722 | 0.1691 |
| 0.0231        | 72.0  | 1800 | 1.7882          | 0.5975   | 0.5329     | 3.5950  | 0.5975   | 0.6016   | 0.1981 | 0.1716 |
| 0.0231        | 73.0  | 1825 | 1.7678          | 0.6      | 0.5287     | 3.6691  | 0.6      | 0.6032   | 0.1733 | 0.1696 |
| 0.0231        | 74.0  | 1850 | 1.7716          | 0.6      | 0.5286     | 3.7576  | 0.6      | 0.6013   | 0.1734 | 0.1692 |
| 0.0231        | 75.0  | 1875 | 1.7704          | 0.6      | 0.5299     | 3.5917  | 0.6      | 0.6016   | 0.1645 | 0.1709 |
| 0.0231        | 76.0  | 1900 | 1.7729          | 0.6      | 0.5298     | 3.6758  | 0.6      | 0.6024   | 0.1766 | 0.1710 |
| 0.0231        | 77.0  | 1925 | 1.7749          | 0.6      | 0.5308     | 3.6022  | 0.6      | 0.6030   | 0.1604 | 0.1717 |
| 0.0231        | 78.0  | 1950 | 1.7720          | 0.6      | 0.5294     | 3.6759  | 0.6      | 0.6017   | 0.1786 | 0.1708 |
| 0.0231        | 79.0  | 1975 | 1.7734          | 0.6025   | 0.5288     | 3.6765  | 0.6025   | 0.6048   | 0.1673 | 0.1698 |
| 0.0059        | 80.0  | 2000 | 1.7709          | 0.6      | 0.5286     | 3.6755  | 0.6      | 0.6020   | 0.1749 | 0.1704 |
| 0.0059        | 81.0  | 2025 | 1.7730          | 0.6      | 0.5295     | 3.6760  | 0.6      | 0.6020   | 0.1677 | 0.1708 |
| 0.0059        | 82.0  | 2050 | 1.7723          | 0.6025   | 0.5295     | 3.6756  | 0.6025   | 0.6055   | 0.1626 | 0.1708 |
| 0.0059        | 83.0  | 2075 | 1.7721          | 0.6025   | 0.5295     | 3.6741  | 0.6025   | 0.6055   | 0.1709 | 0.1708 |
| 0.0059        | 84.0  | 2100 | 1.7725          | 0.6025   | 0.5297     | 3.6747  | 0.6025   | 0.6048   | 0.1627 | 0.1709 |
| 0.0059        | 85.0  | 2125 | 1.7724          | 0.6025   | 0.5295     | 3.6751  | 0.6025   | 0.6055   | 0.1639 | 0.1707 |
| 0.0059        | 86.0  | 2150 | 1.7724          | 0.6025   | 0.5296     | 3.6751  | 0.6025   | 0.6055   | 0.1630 | 0.1708 |
| 0.0059        | 87.0  | 2175 | 1.7724          | 0.6025   | 0.5295     | 3.6749  | 0.6025   | 0.6055   | 0.1638 | 0.1707 |
| 0.0059        | 88.0  | 2200 | 1.7722          | 0.6025   | 0.5295     | 3.6752  | 0.6025   | 0.6055   | 0.1645 | 0.1708 |
| 0.0059        | 89.0  | 2225 | 1.7723          | 0.6025   | 0.5295     | 3.6747  | 0.6025   | 0.6055   | 0.1639 | 0.1708 |
| 0.0059        | 90.0  | 2250 | 1.7723          | 0.6025   | 0.5294     | 3.6750  | 0.6025   | 0.6055   | 0.1643 | 0.1708 |
| 0.0059        | 91.0  | 2275 | 1.7723          | 0.6025   | 0.5294     | 3.6750  | 0.6025   | 0.6055   | 0.1643 | 0.1708 |
| 0.0059        | 92.0  | 2300 | 1.7723          | 0.6025   | 0.5295     | 3.6747  | 0.6025   | 0.6055   | 0.1639 | 0.1708 |
| 0.0059        | 93.0  | 2325 | 1.7723          | 0.6025   | 0.5295     | 3.6749  | 0.6025   | 0.6055   | 0.1637 | 0.1707 |
| 0.0059        | 94.0  | 2350 | 1.7722          | 0.6025   | 0.5295     | 3.6749  | 0.6025   | 0.6055   | 0.1688 | 0.1708 |
| 0.0059        | 95.0  | 2375 | 1.7723          | 0.6025   | 0.5295     | 3.6748  | 0.6025   | 0.6055   | 0.1643 | 0.1708 |
| 0.0059        | 96.0  | 2400 | 1.7723          | 0.6025   | 0.5294     | 3.6748  | 0.6025   | 0.6055   | 0.1643 | 0.1707 |
| 0.0059        | 97.0  | 2425 | 1.7723          | 0.6025   | 0.5295     | 3.6748  | 0.6025   | 0.6055   | 0.1688 | 0.1708 |
| 0.0059        | 98.0  | 2450 | 1.7723          | 0.6025   | 0.5295     | 3.6749  | 0.6025   | 0.6055   | 0.1643 | 0.1708 |
| 0.0059        | 99.0  | 2475 | 1.7723          | 0.6025   | 0.5295     | 3.6749  | 0.6025   | 0.6055   | 0.1688 | 0.1708 |
| 0.0           | 100.0 | 2500 | 1.7723          | 0.6025   | 0.5295     | 3.6748  | 0.6025   | 0.6055   | 0.1688 | 0.1708 |


### Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu113
- Datasets 2.12.0
- Tokenizers 0.12.1

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo