
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 6_e_200-tiny_tobacco3482_kd_CEKD_t5.0_a0.9

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5583
- Accuracy: 0.82
- Brier Loss: 0.2563
- Nll: 1.8898
- F1 Micro: 0.82
- F1 Macro: 0.8009
- Ece: 0.1578
- Aurc: 0.0530

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 1.9764          | 0.23     | 0.8621     | 4.6756 | 0.23     | 0.1902   | 0.2733 | 0.7604 |
| No log        | 2.0   | 50   | 1.2764          | 0.535    | 0.5973     | 2.7212 | 0.535    | 0.4337   | 0.2769 | 0.2592 |
| No log        | 3.0   | 75   | 0.9774          | 0.68     | 0.4478     | 2.1874 | 0.68     | 0.5915   | 0.2144 | 0.1334 |
| No log        | 4.0   | 100  | 0.8047          | 0.755    | 0.3617     | 1.4629 | 0.755    | 0.7257   | 0.1850 | 0.0888 |
| No log        | 5.0   | 125  | 0.7616          | 0.765    | 0.3363     | 1.4885 | 0.765    | 0.7391   | 0.2017 | 0.0843 |
| No log        | 6.0   | 150  | 1.0029          | 0.72     | 0.4200     | 1.6550 | 0.72     | 0.7047   | 0.2303 | 0.1169 |
| No log        | 7.0   | 175  | 0.6286          | 0.825    | 0.2766     | 1.2493 | 0.825    | 0.7930   | 0.1954 | 0.0646 |
| No log        | 8.0   | 200  | 0.6859          | 0.82     | 0.2857     | 1.4847 | 0.82     | 0.7971   | 0.1837 | 0.0699 |
| No log        | 9.0   | 225  | 0.6365          | 0.81     | 0.2765     | 1.1457 | 0.81     | 0.7913   | 0.1604 | 0.0669 |
| No log        | 10.0  | 250  | 0.6085          | 0.81     | 0.2614     | 1.5809 | 0.81     | 0.7928   | 0.1874 | 0.0536 |
| No log        | 11.0  | 275  | 0.5900          | 0.84     | 0.2620     | 1.1457 | 0.8400   | 0.8308   | 0.1695 | 0.0674 |
| No log        | 12.0  | 300  | 0.8544          | 0.75     | 0.3667     | 1.9577 | 0.75     | 0.7330   | 0.1988 | 0.1329 |
| No log        | 13.0  | 325  | 0.5265          | 0.845    | 0.2278     | 1.2521 | 0.845    | 0.8209   | 0.1518 | 0.0399 |
| No log        | 14.0  | 350  | 0.5702          | 0.815    | 0.2567     | 1.5233 | 0.815    | 0.8032   | 0.1551 | 0.0519 |
| No log        | 15.0  | 375  | 0.5933          | 0.845    | 0.2581     | 1.4776 | 0.845    | 0.8341   | 0.1659 | 0.0738 |
| No log        | 16.0  | 400  | 0.5697          | 0.84     | 0.2496     | 1.6732 | 0.8400   | 0.8235   | 0.1470 | 0.0557 |
| No log        | 17.0  | 425  | 0.5471          | 0.825    | 0.2428     | 1.7010 | 0.825    | 0.8093   | 0.1406 | 0.0461 |
| No log        | 18.0  | 450  | 0.5696          | 0.825    | 0.2546     | 1.4095 | 0.825    | 0.7977   | 0.1461 | 0.0612 |
| No log        | 19.0  | 475  | 0.6544          | 0.805    | 0.2959     | 1.8251 | 0.805    | 0.7970   | 0.1681 | 0.0605 |
| 0.4416        | 20.0  | 500  | 0.5113          | 0.83     | 0.2327     | 1.4103 | 0.83     | 0.8093   | 0.1380 | 0.0541 |
| 0.4416        | 21.0  | 525  | 0.5255          | 0.84     | 0.2375     | 1.6750 | 0.8400   | 0.8220   | 0.1320 | 0.0462 |
| 0.4416        | 22.0  | 550  | 0.5889          | 0.835    | 0.2681     | 1.7850 | 0.835    | 0.8242   | 0.1507 | 0.0683 |
| 0.4416        | 23.0  | 575  | 0.5456          | 0.835    | 0.2492     | 1.8481 | 0.835    | 0.8137   | 0.1716 | 0.0550 |
| 0.4416        | 24.0  | 600  | 0.5661          | 0.83     | 0.2611     | 1.8434 | 0.83     | 0.8156   | 0.1618 | 0.0591 |
| 0.4416        | 25.0  | 625  | 0.5444          | 0.83     | 0.2484     | 1.7579 | 0.83     | 0.8091   | 0.1478 | 0.0530 |
| 0.4416        | 26.0  | 650  | 0.5418          | 0.83     | 0.2503     | 1.7188 | 0.83     | 0.8125   | 0.1564 | 0.0484 |
| 0.4416        | 27.0  | 675  | 0.5532          | 0.835    | 0.2540     | 1.8931 | 0.835    | 0.8146   | 0.1694 | 0.0514 |
| 0.4416        | 28.0  | 700  | 0.5492          | 0.835    | 0.2518     | 1.8959 | 0.835    | 0.8155   | 0.1505 | 0.0495 |
| 0.4416        | 29.0  | 725  | 0.5478          | 0.825    | 0.2505     | 1.8907 | 0.825    | 0.8069   | 0.1548 | 0.0503 |
| 0.4416        | 30.0  | 750  | 0.5478          | 0.835    | 0.2510     | 1.8881 | 0.835    | 0.8178   | 0.1467 | 0.0521 |
| 0.4416        | 31.0  | 775  | 0.5472          | 0.825    | 0.2505     | 1.8888 | 0.825    | 0.8064   | 0.1527 | 0.0510 |
| 0.4416        | 32.0  | 800  | 0.5522          | 0.83     | 0.2527     | 1.8927 | 0.83     | 0.8126   | 0.1449 | 0.0520 |
| 0.4416        | 33.0  | 825  | 0.5513          | 0.825    | 0.2524     | 1.8989 | 0.825    | 0.8064   | 0.1625 | 0.0509 |
| 0.4416        | 34.0  | 850  | 0.5465          | 0.835    | 0.2504     | 1.8880 | 0.835    | 0.8148   | 0.1519 | 0.0520 |
| 0.4416        | 35.0  | 875  | 0.5489          | 0.825    | 0.2515     | 1.8866 | 0.825    | 0.8064   | 0.1538 | 0.0510 |
| 0.4416        | 36.0  | 900  | 0.5508          | 0.825    | 0.2521     | 1.8922 | 0.825    | 0.8053   | 0.1356 | 0.0526 |
| 0.4416        | 37.0  | 925  | 0.5495          | 0.825    | 0.2522     | 1.8881 | 0.825    | 0.8064   | 0.1517 | 0.0514 |
| 0.4416        | 38.0  | 950  | 0.5483          | 0.825    | 0.2514     | 1.8859 | 0.825    | 0.8064   | 0.1749 | 0.0511 |
| 0.4416        | 39.0  | 975  | 0.5508          | 0.825    | 0.2524     | 1.8868 | 0.825    | 0.8064   | 0.1459 | 0.0514 |
| 0.0519        | 40.0  | 1000 | 0.5519          | 0.825    | 0.2529     | 1.8862 | 0.825    | 0.8064   | 0.1532 | 0.0513 |
| 0.0519        | 41.0  | 1025 | 0.5522          | 0.825    | 0.2530     | 1.8882 | 0.825    | 0.8064   | 0.1665 | 0.0519 |
| 0.0519        | 42.0  | 1050 | 0.5507          | 0.825    | 0.2525     | 1.8870 | 0.825    | 0.8064   | 0.1613 | 0.0508 |
| 0.0519        | 43.0  | 1075 | 0.5528          | 0.825    | 0.2536     | 1.8884 | 0.825    | 0.8064   | 0.1634 | 0.0517 |
| 0.0519        | 44.0  | 1100 | 0.5520          | 0.825    | 0.2531     | 1.8879 | 0.825    | 0.8064   | 0.1519 | 0.0525 |
| 0.0519        | 45.0  | 1125 | 0.5524          | 0.825    | 0.2535     | 1.8876 | 0.825    | 0.8053   | 0.1582 | 0.0515 |
| 0.0519        | 46.0  | 1150 | 0.5525          | 0.825    | 0.2534     | 1.8867 | 0.825    | 0.8064   | 0.1592 | 0.0519 |
| 0.0519        | 47.0  | 1175 | 0.5532          | 0.825    | 0.2539     | 1.8875 | 0.825    | 0.8064   | 0.1621 | 0.0521 |
| 0.0519        | 48.0  | 1200 | 0.5540          | 0.825    | 0.2540     | 1.8865 | 0.825    | 0.8064   | 0.1502 | 0.0522 |
| 0.0519        | 49.0  | 1225 | 0.5523          | 0.825    | 0.2538     | 1.8268 | 0.825    | 0.8072   | 0.1625 | 0.0514 |
| 0.0519        | 50.0  | 1250 | 0.5535          | 0.825    | 0.2539     | 1.8871 | 0.825    | 0.8064   | 0.1684 | 0.0517 |
| 0.0519        | 51.0  | 1275 | 0.5526          | 0.825    | 0.2534     | 1.8850 | 0.825    | 0.8064   | 0.1621 | 0.0519 |
| 0.0519        | 52.0  | 1300 | 0.5543          | 0.825    | 0.2543     | 1.8865 | 0.825    | 0.8064   | 0.1429 | 0.0521 |
| 0.0519        | 53.0  | 1325 | 0.5526          | 0.825    | 0.2538     | 1.8866 | 0.825    | 0.8064   | 0.1613 | 0.0515 |
| 0.0519        | 54.0  | 1350 | 0.5530          | 0.82     | 0.2538     | 1.8877 | 0.82     | 0.8009   | 0.1620 | 0.0518 |
| 0.0519        | 55.0  | 1375 | 0.5550          | 0.825    | 0.2547     | 1.8872 | 0.825    | 0.8064   | 0.1567 | 0.0522 |
| 0.0519        | 56.0  | 1400 | 0.5565          | 0.825    | 0.2552     | 1.8859 | 0.825    | 0.8064   | 0.1400 | 0.0523 |
| 0.0519        | 57.0  | 1425 | 0.5552          | 0.825    | 0.2548     | 1.8874 | 0.825    | 0.8064   | 0.1543 | 0.0520 |
| 0.0519        | 58.0  | 1450 | 0.5537          | 0.825    | 0.2542     | 1.8860 | 0.825    | 0.8064   | 0.1531 | 0.0516 |
| 0.0519        | 59.0  | 1475 | 0.5559          | 0.825    | 0.2551     | 1.8879 | 0.825    | 0.8064   | 0.1564 | 0.0525 |
| 0.0508        | 60.0  | 1500 | 0.5548          | 0.825    | 0.2545     | 1.8866 | 0.825    | 0.8064   | 0.1526 | 0.0522 |
| 0.0508        | 61.0  | 1525 | 0.5557          | 0.825    | 0.2550     | 1.8884 | 0.825    | 0.8064   | 0.1443 | 0.0524 |
| 0.0508        | 62.0  | 1550 | 0.5548          | 0.82     | 0.2546     | 1.8874 | 0.82     | 0.8009   | 0.1709 | 0.0527 |
| 0.0508        | 63.0  | 1575 | 0.5556          | 0.825    | 0.2551     | 1.8899 | 0.825    | 0.8064   | 0.1606 | 0.0524 |
| 0.0508        | 64.0  | 1600 | 0.5562          | 0.825    | 0.2553     | 1.8872 | 0.825    | 0.8064   | 0.1467 | 0.0527 |
| 0.0508        | 65.0  | 1625 | 0.5569          | 0.825    | 0.2554     | 1.8879 | 0.825    | 0.8064   | 0.1537 | 0.0524 |
| 0.0508        | 66.0  | 1650 | 0.5567          | 0.825    | 0.2555     | 1.8873 | 0.825    | 0.8064   | 0.1601 | 0.0525 |
| 0.0508        | 67.0  | 1675 | 0.5556          | 0.825    | 0.2550     | 1.8878 | 0.825    | 0.8064   | 0.1601 | 0.0527 |
| 0.0508        | 68.0  | 1700 | 0.5570          | 0.825    | 0.2555     | 1.8879 | 0.825    | 0.8064   | 0.1679 | 0.0528 |
| 0.0508        | 69.0  | 1725 | 0.5560          | 0.825    | 0.2553     | 1.8886 | 0.825    | 0.8064   | 0.1525 | 0.0521 |
| 0.0508        | 70.0  | 1750 | 0.5562          | 0.825    | 0.2553     | 1.8878 | 0.825    | 0.8064   | 0.1531 | 0.0528 |
| 0.0508        | 71.0  | 1775 | 0.5572          | 0.82     | 0.2557     | 1.8883 | 0.82     | 0.8009   | 0.1718 | 0.0530 |
| 0.0508        | 72.0  | 1800 | 0.5567          | 0.82     | 0.2555     | 1.8888 | 0.82     | 0.8009   | 0.1630 | 0.0525 |
| 0.0508        | 73.0  | 1825 | 0.5571          | 0.825    | 0.2556     | 1.8882 | 0.825    | 0.8064   | 0.1598 | 0.0528 |
| 0.0508        | 74.0  | 1850 | 0.5580          | 0.825    | 0.2561     | 1.8901 | 0.825    | 0.8064   | 0.1543 | 0.0530 |
| 0.0508        | 75.0  | 1875 | 0.5579          | 0.82     | 0.2561     | 1.8892 | 0.82     | 0.8009   | 0.1721 | 0.0530 |
| 0.0508        | 76.0  | 1900 | 0.5574          | 0.82     | 0.2559     | 1.8892 | 0.82     | 0.8009   | 0.1636 | 0.0528 |
| 0.0508        | 77.0  | 1925 | 0.5569          | 0.82     | 0.2557     | 1.8393 | 0.82     | 0.8009   | 0.1634 | 0.0526 |
| 0.0508        | 78.0  | 1950 | 0.5572          | 0.82     | 0.2558     | 1.8887 | 0.82     | 0.8009   | 0.1637 | 0.0530 |
| 0.0508        | 79.0  | 1975 | 0.5578          | 0.82     | 0.2560     | 1.8888 | 0.82     | 0.8009   | 0.1579 | 0.0530 |
| 0.0507        | 80.0  | 2000 | 0.5577          | 0.82     | 0.2559     | 1.8889 | 0.82     | 0.8009   | 0.1578 | 0.0532 |
| 0.0507        | 81.0  | 2025 | 0.5578          | 0.82     | 0.2560     | 1.8889 | 0.82     | 0.8009   | 0.1578 | 0.0533 |
| 0.0507        | 82.0  | 2050 | 0.5579          | 0.825    | 0.2561     | 1.8891 | 0.825    | 0.8064   | 0.1602 | 0.0528 |
| 0.0507        | 83.0  | 2075 | 0.5581          | 0.825    | 0.2562     | 1.8894 | 0.825    | 0.8064   | 0.1544 | 0.0528 |
| 0.0507        | 84.0  | 2100 | 0.5579          | 0.82     | 0.2561     | 1.8894 | 0.82     | 0.8009   | 0.1581 | 0.0531 |
| 0.0507        | 85.0  | 2125 | 0.5580          | 0.82     | 0.2561     | 1.8896 | 0.82     | 0.8009   | 0.1578 | 0.0528 |
| 0.0507        | 86.0  | 2150 | 0.5581          | 0.82     | 0.2562     | 1.8891 | 0.82     | 0.8009   | 0.1580 | 0.0532 |
| 0.0507        | 87.0  | 2175 | 0.5582          | 0.82     | 0.2562     | 1.8467 | 0.82     | 0.8009   | 0.1581 | 0.0528 |
| 0.0507        | 88.0  | 2200 | 0.5583          | 0.82     | 0.2562     | 1.8891 | 0.82     | 0.8009   | 0.1580 | 0.0531 |
| 0.0507        | 89.0  | 2225 | 0.5584          | 0.815    | 0.2563     | 1.8894 | 0.815    | 0.7976   | 0.1608 | 0.0534 |
| 0.0507        | 90.0  | 2250 | 0.5578          | 0.82     | 0.2561     | 1.8894 | 0.82     | 0.8009   | 0.1578 | 0.0530 |
| 0.0507        | 91.0  | 2275 | 0.5584          | 0.815    | 0.2563     | 1.8896 | 0.815    | 0.7976   | 0.1607 | 0.0532 |
| 0.0507        | 92.0  | 2300 | 0.5583          | 0.82     | 0.2562     | 1.8893 | 0.82     | 0.8009   | 0.1581 | 0.0531 |
| 0.0507        | 93.0  | 2325 | 0.5582          | 0.82     | 0.2562     | 1.8898 | 0.82     | 0.8009   | 0.1579 | 0.0530 |
| 0.0507        | 94.0  | 2350 | 0.5582          | 0.82     | 0.2562     | 1.8392 | 0.82     | 0.8009   | 0.1578 | 0.0530 |
| 0.0507        | 95.0  | 2375 | 0.5584          | 0.82     | 0.2563     | 1.8897 | 0.82     | 0.8009   | 0.1582 | 0.0531 |
| 0.0507        | 96.0  | 2400 | 0.5582          | 0.82     | 0.2562     | 1.8898 | 0.82     | 0.8009   | 0.1578 | 0.0530 |
| 0.0507        | 97.0  | 2425 | 0.5583          | 0.82     | 0.2563     | 1.8896 | 0.82     | 0.8009   | 0.1580 | 0.0530 |
| 0.0507        | 98.0  | 2450 | 0.5582          | 0.82     | 0.2562     | 1.8898 | 0.82     | 0.8009   | 0.1578 | 0.0530 |
| 0.0507        | 99.0  | 2475 | 0.5583          | 0.82     | 0.2563     | 1.8898 | 0.82     | 0.8009   | 0.1578 | 0.0530 |
| 0.0507        | 100.0 | 2500 | 0.5583          | 0.82     | 0.2563     | 1.8898 | 0.82     | 0.8009   | 0.1578 | 0.0530 |


### Framework versions

- Transformers 4.30.2
- Pytorch 1.13.1
- Datasets 2.13.1
- Tokenizers 0.13.3

adve email form letter memo news note report resume scientific