
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-small_rvl_cdip_100_examples_per_class_kd_CEKD_t1.5_a0.9

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2366
- Accuracy: 0.63
- Brier Loss: 0.5035
- Nll: 2.8588
- F1 Micro: 0.63
- F1 Macro: 0.6311
- Ece: 0.1649
- Aurc: 0.1472

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll     | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:-------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 2.8887          | 0.1225   | 0.9306     | 15.9457 | 0.1225   | 0.1226   | 0.1434 | 0.8620 |
| No log        | 2.0   | 50   | 2.2120          | 0.3775   | 0.7577     | 9.7500  | 0.3775   | 0.3483   | 0.1992 | 0.3776 |
| No log        | 3.0   | 75   | 1.7681          | 0.495    | 0.6387     | 5.6935  | 0.495    | 0.4838   | 0.1885 | 0.2491 |
| No log        | 4.0   | 100  | 1.6420          | 0.5225   | 0.6038     | 5.2427  | 0.5225   | 0.5242   | 0.1757 | 0.2301 |
| No log        | 5.0   | 125  | 1.5877          | 0.545    | 0.5986     | 4.6187  | 0.545    | 0.5282   | 0.1808 | 0.2248 |
| No log        | 6.0   | 150  | 1.6460          | 0.5125   | 0.6162     | 3.9942  | 0.5125   | 0.5060   | 0.1962 | 0.2295 |
| No log        | 7.0   | 175  | 1.8436          | 0.5125   | 0.6538     | 4.1740  | 0.5125   | 0.4932   | 0.2299 | 0.2451 |
| No log        | 8.0   | 200  | 1.8205          | 0.545    | 0.6453     | 5.0752  | 0.545    | 0.5234   | 0.2057 | 0.2432 |
| No log        | 9.0   | 225  | 1.7399          | 0.55     | 0.6260     | 4.5896  | 0.55     | 0.5460   | 0.2057 | 0.2258 |
| No log        | 10.0  | 250  | 1.8559          | 0.55     | 0.6521     | 5.0532  | 0.55     | 0.5368   | 0.2209 | 0.2560 |
| No log        | 11.0  | 275  | 1.8636          | 0.5625   | 0.6488     | 4.6642  | 0.5625   | 0.5544   | 0.2335 | 0.2187 |
| No log        | 12.0  | 300  | 1.7461          | 0.55     | 0.6356     | 4.1298  | 0.55     | 0.5638   | 0.2047 | 0.2313 |
| No log        | 13.0  | 325  | 1.7468          | 0.5625   | 0.6281     | 4.5451  | 0.5625   | 0.5570   | 0.2224 | 0.2214 |
| No log        | 14.0  | 350  | 1.9616          | 0.545    | 0.6884     | 3.7999  | 0.545    | 0.5484   | 0.2691 | 0.2624 |
| No log        | 15.0  | 375  | 2.0977          | 0.5175   | 0.7138     | 4.3792  | 0.5175   | 0.5055   | 0.2658 | 0.2917 |
| No log        | 16.0  | 400  | 2.0238          | 0.5275   | 0.6896     | 4.5299  | 0.5275   | 0.5177   | 0.2664 | 0.2603 |
| No log        | 17.0  | 425  | 1.8687          | 0.535    | 0.6534     | 3.7356  | 0.535    | 0.5388   | 0.2490 | 0.2448 |
| No log        | 18.0  | 450  | 1.8210          | 0.5575   | 0.6492     | 4.3823  | 0.5575   | 0.5537   | 0.2533 | 0.2268 |
| No log        | 19.0  | 475  | 1.7610          | 0.555    | 0.6325     | 3.9697  | 0.555    | 0.5503   | 0.2292 | 0.2161 |
| 0.5398        | 20.0  | 500  | 1.7125          | 0.5825   | 0.6125     | 3.4176  | 0.5825   | 0.5731   | 0.2140 | 0.1859 |
| 0.5398        | 21.0  | 525  | 1.6296          | 0.5775   | 0.6163     | 3.6014  | 0.5775   | 0.5871   | 0.2236 | 0.2051 |
| 0.5398        | 22.0  | 550  | 1.5965          | 0.57     | 0.5908     | 3.7668  | 0.57     | 0.5712   | 0.2058 | 0.1883 |
| 0.5398        | 23.0  | 575  | 1.4828          | 0.5875   | 0.5646     | 3.7028  | 0.5875   | 0.5854   | 0.1944 | 0.1714 |
| 0.5398        | 24.0  | 600  | 1.3983          | 0.6075   | 0.5481     | 3.3608  | 0.6075   | 0.6107   | 0.1966 | 0.1628 |
| 0.5398        | 25.0  | 625  | 1.5241          | 0.5925   | 0.5866     | 3.3669  | 0.5925   | 0.6019   | 0.2069 | 0.1886 |
| 0.5398        | 26.0  | 650  | 1.5540          | 0.58     | 0.5780     | 3.5184  | 0.58     | 0.5710   | 0.2131 | 0.1857 |
| 0.5398        | 27.0  | 675  | 1.4653          | 0.6      | 0.5768     | 2.9877  | 0.6      | 0.6043   | 0.2166 | 0.1781 |
| 0.5398        | 28.0  | 700  | 1.4883          | 0.5925   | 0.5646     | 3.7789  | 0.5925   | 0.5910   | 0.2096 | 0.1746 |
| 0.5398        | 29.0  | 725  | 1.5738          | 0.59     | 0.5914     | 4.0558  | 0.59     | 0.5879   | 0.2150 | 0.1957 |
| 0.5398        | 30.0  | 750  | 1.4017          | 0.6025   | 0.5583     | 3.4791  | 0.6025   | 0.6023   | 0.2150 | 0.1752 |
| 0.5398        | 31.0  | 775  | 1.3500          | 0.61     | 0.5365     | 3.2560  | 0.61     | 0.6157   | 0.1988 | 0.1579 |
| 0.5398        | 32.0  | 800  | 1.2977          | 0.6375   | 0.5140     | 3.0503  | 0.6375   | 0.6395   | 0.1847 | 0.1534 |
| 0.5398        | 33.0  | 825  | 1.3471          | 0.6175   | 0.5406     | 3.1888  | 0.6175   | 0.6104   | 0.2077 | 0.1689 |
| 0.5398        | 34.0  | 850  | 1.2992          | 0.615    | 0.5219     | 2.8944  | 0.615    | 0.6191   | 0.1826 | 0.1574 |
| 0.5398        | 35.0  | 875  | 1.2733          | 0.6225   | 0.5124     | 2.9352  | 0.6225   | 0.6238   | 0.1588 | 0.1505 |
| 0.5398        | 36.0  | 900  | 1.2821          | 0.6175   | 0.5231     | 3.0142  | 0.6175   | 0.6169   | 0.1672 | 0.1553 |
| 0.5398        | 37.0  | 925  | 1.2819          | 0.61     | 0.5200     | 2.6874  | 0.61     | 0.6116   | 0.1847 | 0.1540 |
| 0.5398        | 38.0  | 950  | 1.2664          | 0.615    | 0.5145     | 2.9287  | 0.615    | 0.6159   | 0.1961 | 0.1528 |
| 0.5398        | 39.0  | 975  | 1.2584          | 0.6225   | 0.5134     | 3.0058  | 0.6225   | 0.6230   | 0.1747 | 0.1508 |
| 0.0507        | 40.0  | 1000 | 1.2562          | 0.615    | 0.5114     | 2.9269  | 0.615    | 0.6169   | 0.1815 | 0.1504 |
| 0.0507        | 41.0  | 1025 | 1.2525          | 0.6225   | 0.5101     | 2.9199  | 0.6225   | 0.6239   | 0.1770 | 0.1496 |
| 0.0507        | 42.0  | 1050 | 1.2573          | 0.62     | 0.5133     | 2.9195  | 0.62     | 0.6221   | 0.1824 | 0.1511 |
| 0.0507        | 43.0  | 1075 | 1.2536          | 0.6125   | 0.5131     | 2.9026  | 0.6125   | 0.6121   | 0.1820 | 0.1511 |
| 0.0507        | 44.0  | 1100 | 1.2543          | 0.6225   | 0.5109     | 3.0693  | 0.6225   | 0.6235   | 0.1647 | 0.1500 |
| 0.0507        | 45.0  | 1125 | 1.2526          | 0.6125   | 0.5117     | 2.9018  | 0.6125   | 0.6141   | 0.1788 | 0.1500 |
| 0.0507        | 46.0  | 1150 | 1.2432          | 0.615    | 0.5068     | 2.9042  | 0.615    | 0.6167   | 0.1762 | 0.1484 |
| 0.0507        | 47.0  | 1175 | 1.2485          | 0.6275   | 0.5098     | 2.8927  | 0.6275   | 0.6251   | 0.1590 | 0.1496 |
| 0.0507        | 48.0  | 1200 | 1.2576          | 0.6125   | 0.5140     | 2.8956  | 0.6125   | 0.6137   | 0.1824 | 0.1524 |
| 0.0507        | 49.0  | 1225 | 1.2468          | 0.62     | 0.5094     | 2.8918  | 0.62     | 0.6204   | 0.1832 | 0.1496 |
| 0.0507        | 50.0  | 1250 | 1.2479          | 0.6175   | 0.5102     | 2.8921  | 0.6175   | 0.6178   | 0.1706 | 0.1491 |
| 0.0507        | 51.0  | 1275 | 1.2393          | 0.6225   | 0.5057     | 2.8813  | 0.6225   | 0.6229   | 0.1784 | 0.1486 |
| 0.0507        | 52.0  | 1300 | 1.2463          | 0.6175   | 0.5085     | 2.8959  | 0.6175   | 0.6184   | 0.1669 | 0.1495 |
| 0.0507        | 53.0  | 1325 | 1.2391          | 0.62     | 0.5061     | 2.8828  | 0.62     | 0.6215   | 0.1803 | 0.1471 |
| 0.0507        | 54.0  | 1350 | 1.2538          | 0.6175   | 0.5121     | 2.8795  | 0.6175   | 0.6167   | 0.1680 | 0.1512 |
| 0.0507        | 55.0  | 1375 | 1.2407          | 0.625    | 0.5064     | 2.8830  | 0.625    | 0.6259   | 0.1842 | 0.1482 |
| 0.0507        | 56.0  | 1400 | 1.2488          | 0.62     | 0.5099     | 2.8769  | 0.62     | 0.6198   | 0.1568 | 0.1499 |
| 0.0507        | 57.0  | 1425 | 1.2402          | 0.625    | 0.5052     | 2.8778  | 0.625    | 0.6260   | 0.1616 | 0.1481 |
| 0.0507        | 58.0  | 1450 | 1.2457          | 0.625    | 0.5077     | 2.8786  | 0.625    | 0.6260   | 0.1759 | 0.1474 |
| 0.0507        | 59.0  | 1475 | 1.2430          | 0.6275   | 0.5073     | 2.8744  | 0.6275   | 0.6266   | 0.1652 | 0.1486 |
| 0.0319        | 60.0  | 1500 | 1.2399          | 0.625    | 0.5056     | 2.8767  | 0.625    | 0.6256   | 0.1701 | 0.1474 |
| 0.0319        | 61.0  | 1525 | 1.2460          | 0.63     | 0.5087     | 2.8758  | 0.63     | 0.6329   | 0.1865 | 0.1491 |
| 0.0319        | 62.0  | 1550 | 1.2410          | 0.6225   | 0.5058     | 2.8719  | 0.6225   | 0.6229   | 0.1752 | 0.1477 |
| 0.0319        | 63.0  | 1575 | 1.2418          | 0.63     | 0.5060     | 2.8746  | 0.63     | 0.6319   | 0.1692 | 0.1484 |
| 0.0319        | 64.0  | 1600 | 1.2424          | 0.6275   | 0.5069     | 2.8672  | 0.6275   | 0.6279   | 0.1903 | 0.1475 |
| 0.0319        | 65.0  | 1625 | 1.2413          | 0.63     | 0.5061     | 2.8747  | 0.63     | 0.6304   | 0.1737 | 0.1471 |
| 0.0319        | 66.0  | 1650 | 1.2385          | 0.6325   | 0.5039     | 2.8726  | 0.6325   | 0.6358   | 0.1792 | 0.1473 |
| 0.0319        | 67.0  | 1675 | 1.2368          | 0.625    | 0.5047     | 2.8661  | 0.625    | 0.6261   | 0.1843 | 0.1467 |
| 0.0319        | 68.0  | 1700 | 1.2370          | 0.6275   | 0.5039     | 2.8691  | 0.6275   | 0.6294   | 0.1724 | 0.1471 |
| 0.0319        | 69.0  | 1725 | 1.2382          | 0.63     | 0.5050     | 2.8659  | 0.63     | 0.6317   | 0.1698 | 0.1472 |
| 0.0319        | 70.0  | 1750 | 1.2396          | 0.6275   | 0.5051     | 2.8670  | 0.6275   | 0.6290   | 0.1790 | 0.1474 |
| 0.0319        | 71.0  | 1775 | 1.2378          | 0.625    | 0.5045     | 2.8637  | 0.625    | 0.6268   | 0.1742 | 0.1476 |
| 0.0319        | 72.0  | 1800 | 1.2360          | 0.625    | 0.5037     | 2.8669  | 0.625    | 0.6269   | 0.1778 | 0.1468 |
| 0.0319        | 73.0  | 1825 | 1.2390          | 0.63     | 0.5049     | 2.8638  | 0.63     | 0.6310   | 0.1711 | 0.1474 |
| 0.0319        | 74.0  | 1850 | 1.2372          | 0.625    | 0.5045     | 2.8640  | 0.625    | 0.6269   | 0.1817 | 0.1475 |
| 0.0319        | 75.0  | 1875 | 1.2375          | 0.63     | 0.5044     | 2.8640  | 0.63     | 0.6313   | 0.1703 | 0.1472 |
| 0.0319        | 76.0  | 1900 | 1.2372          | 0.6275   | 0.5041     | 2.8621  | 0.6275   | 0.6290   | 0.1794 | 0.1473 |
| 0.0319        | 77.0  | 1925 | 1.2374          | 0.63     | 0.5041     | 2.8629  | 0.63     | 0.6313   | 0.1722 | 0.1472 |
| 0.0319        | 78.0  | 1950 | 1.2367          | 0.6275   | 0.5039     | 2.8620  | 0.6275   | 0.6294   | 0.1704 | 0.1474 |
| 0.0319        | 79.0  | 1975 | 1.2371          | 0.6275   | 0.5039     | 2.8619  | 0.6275   | 0.6294   | 0.1639 | 0.1474 |
| 0.0314        | 80.0  | 2000 | 1.2372          | 0.63     | 0.5041     | 2.8612  | 0.63     | 0.6310   | 0.1750 | 0.1474 |
| 0.0314        | 81.0  | 2025 | 1.2368          | 0.63     | 0.5038     | 2.8613  | 0.63     | 0.6309   | 0.1648 | 0.1473 |
| 0.0314        | 82.0  | 2050 | 1.2370          | 0.63     | 0.5038     | 2.8607  | 0.63     | 0.6305   | 0.1782 | 0.1473 |
| 0.0314        | 83.0  | 2075 | 1.2368          | 0.63     | 0.5038     | 2.8609  | 0.63     | 0.6307   | 0.1686 | 0.1472 |
| 0.0314        | 84.0  | 2100 | 1.2368          | 0.63     | 0.5037     | 2.8603  | 0.63     | 0.6305   | 0.1667 | 0.1472 |
| 0.0314        | 85.0  | 2125 | 1.2366          | 0.63     | 0.5036     | 2.8601  | 0.63     | 0.6309   | 0.1686 | 0.1473 |
| 0.0314        | 86.0  | 2150 | 1.2367          | 0.6325   | 0.5037     | 2.8600  | 0.6325   | 0.6335   | 0.1751 | 0.1471 |
| 0.0314        | 87.0  | 2175 | 1.2369          | 0.63     | 0.5037     | 2.8598  | 0.63     | 0.6307   | 0.1730 | 0.1473 |
| 0.0314        | 88.0  | 2200 | 1.2367          | 0.63     | 0.5036     | 2.8595  | 0.63     | 0.6307   | 0.1657 | 0.1472 |
| 0.0314        | 89.0  | 2225 | 1.2366          | 0.63     | 0.5036     | 2.8597  | 0.63     | 0.6307   | 0.1680 | 0.1472 |
| 0.0314        | 90.0  | 2250 | 1.2366          | 0.63     | 0.5036     | 2.8594  | 0.63     | 0.6307   | 0.1580 | 0.1472 |
| 0.0314        | 91.0  | 2275 | 1.2366          | 0.63     | 0.5035     | 2.8593  | 0.63     | 0.6307   | 0.1677 | 0.1472 |
| 0.0314        | 92.0  | 2300 | 1.2367          | 0.63     | 0.5035     | 2.8593  | 0.63     | 0.6307   | 0.1616 | 0.1472 |
| 0.0314        | 93.0  | 2325 | 1.2366          | 0.63     | 0.5035     | 2.8590  | 0.63     | 0.6307   | 0.1625 | 0.1472 |
| 0.0314        | 94.0  | 2350 | 1.2366          | 0.6325   | 0.5035     | 2.8590  | 0.6325   | 0.6333   | 0.1586 | 0.1470 |
| 0.0314        | 95.0  | 2375 | 1.2366          | 0.63     | 0.5035     | 2.8591  | 0.63     | 0.6307   | 0.1580 | 0.1472 |
| 0.0314        | 96.0  | 2400 | 1.2366          | 0.63     | 0.5035     | 2.8589  | 0.63     | 0.6307   | 0.1695 | 0.1471 |
| 0.0314        | 97.0  | 2425 | 1.2366          | 0.63     | 0.5035     | 2.8589  | 0.63     | 0.6311   | 0.1648 | 0.1472 |
| 0.0314        | 98.0  | 2450 | 1.2366          | 0.63     | 0.5035     | 2.8588  | 0.63     | 0.6311   | 0.1695 | 0.1471 |
| 0.0314        | 99.0  | 2475 | 1.2366          | 0.6325   | 0.5035     | 2.8589  | 0.6325   | 0.6337   | 0.1724 | 0.1470 |
| 0.0312        | 100.0 | 2500 | 1.2366          | 0.63     | 0.5035     | 2.8588  | 0.63     | 0.6311   | 0.1649 | 0.1472 |


### Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu113
- Datasets 2.12.0
- Tokenizers 0.12.1

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo