
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 6_e_200-tiny_tobacco3482_kd_CEKD_t2.5_a0.9

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5528
- Accuracy: 0.84
- Brier Loss: 0.2493
- Nll: 1.6062
- F1 Micro: 0.8400
- F1 Macro: 0.8256
- Ece: 0.1626
- Aurc: 0.0556

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 25   | 1.9962          | 0.24     | 0.8634     | 4.6099 | 0.24     | 0.1987   | 0.2924 | 0.7628 |
| No log        | 2.0   | 50   | 1.2785          | 0.545    | 0.5960     | 2.6707 | 0.545    | 0.4456   | 0.2808 | 0.2569 |
| No log        | 3.0   | 75   | 0.9740          | 0.685    | 0.4455     | 2.0688 | 0.685    | 0.5937   | 0.2231 | 0.1314 |
| No log        | 4.0   | 100  | 0.8052          | 0.75     | 0.3628     | 1.5271 | 0.75     | 0.7144   | 0.2031 | 0.0915 |
| No log        | 5.0   | 125  | 0.7531          | 0.77     | 0.3324     | 1.5448 | 0.7700   | 0.7348   | 0.1868 | 0.0829 |
| No log        | 6.0   | 150  | 0.9730          | 0.735    | 0.4050     | 1.5875 | 0.735    | 0.7229   | 0.1899 | 0.1118 |
| No log        | 7.0   | 175  | 0.6437          | 0.83     | 0.2790     | 1.3915 | 0.83     | 0.7996   | 0.1967 | 0.0612 |
| No log        | 8.0   | 200  | 0.6670          | 0.78     | 0.2984     | 1.2128 | 0.78     | 0.7429   | 0.1701 | 0.0716 |
| No log        | 9.0   | 225  | 0.6012          | 0.845    | 0.2521     | 1.4007 | 0.845    | 0.8208   | 0.1573 | 0.0581 |
| No log        | 10.0  | 250  | 0.6754          | 0.795    | 0.3063     | 1.4194 | 0.795    | 0.7638   | 0.2036 | 0.0759 |
| No log        | 11.0  | 275  | 0.5950          | 0.83     | 0.2554     | 1.1558 | 0.83     | 0.8053   | 0.1877 | 0.0529 |
| No log        | 12.0  | 300  | 0.7061          | 0.79     | 0.3153     | 1.6781 | 0.79     | 0.7676   | 0.1879 | 0.0848 |
| No log        | 13.0  | 325  | 0.6349          | 0.805    | 0.2806     | 1.3102 | 0.805    | 0.7767   | 0.1523 | 0.0667 |
| No log        | 14.0  | 350  | 0.5973          | 0.82     | 0.2677     | 1.5498 | 0.82     | 0.8020   | 0.1734 | 0.0567 |
| No log        | 15.0  | 375  | 0.6186          | 0.825    | 0.2792     | 1.3816 | 0.825    | 0.8170   | 0.1558 | 0.0672 |
| No log        | 16.0  | 400  | 0.5694          | 0.815    | 0.2662     | 1.1759 | 0.815    | 0.7962   | 0.1675 | 0.0559 |
| No log        | 17.0  | 425  | 0.5993          | 0.825    | 0.2793     | 1.2547 | 0.825    | 0.8112   | 0.1822 | 0.0647 |
| No log        | 18.0  | 450  | 0.6333          | 0.815    | 0.2844     | 1.6540 | 0.815    | 0.8024   | 0.1562 | 0.0622 |
| No log        | 19.0  | 475  | 0.5208          | 0.845    | 0.2349     | 1.2879 | 0.845    | 0.8155   | 0.1553 | 0.0494 |
| 0.4544        | 20.0  | 500  | 0.5412          | 0.86     | 0.2438     | 1.6726 | 0.8600   | 0.8465   | 0.1531 | 0.0485 |
| 0.4544        | 21.0  | 525  | 0.6171          | 0.825    | 0.2775     | 1.9997 | 0.825    | 0.8183   | 0.1464 | 0.0549 |
| 0.4544        | 22.0  | 550  | 0.5479          | 0.84     | 0.2447     | 1.5015 | 0.8400   | 0.8263   | 0.1481 | 0.0680 |
| 0.4544        | 23.0  | 575  | 0.5508          | 0.835    | 0.2491     | 1.8095 | 0.835    | 0.8209   | 0.1616 | 0.0469 |
| 0.4544        | 24.0  | 600  | 0.5597          | 0.825    | 0.2577     | 1.6676 | 0.825    | 0.8077   | 0.1572 | 0.0486 |
| 0.4544        | 25.0  | 625  | 0.5505          | 0.835    | 0.2535     | 1.6085 | 0.835    | 0.8166   | 0.1664 | 0.0524 |
| 0.4544        | 26.0  | 650  | 0.5347          | 0.84     | 0.2442     | 1.4694 | 0.8400   | 0.8288   | 0.1825 | 0.0505 |
| 0.4544        | 27.0  | 675  | 0.5333          | 0.84     | 0.2418     | 1.5809 | 0.8400   | 0.8280   | 0.1634 | 0.0521 |
| 0.4544        | 28.0  | 700  | 0.5417          | 0.84     | 0.2471     | 1.5289 | 0.8400   | 0.8231   | 0.1500 | 0.0503 |
| 0.4544        | 29.0  | 725  | 0.5369          | 0.845    | 0.2434     | 1.5333 | 0.845    | 0.8318   | 0.1690 | 0.0523 |
| 0.4544        | 30.0  | 750  | 0.5396          | 0.84     | 0.2448     | 1.5269 | 0.8400   | 0.8260   | 0.1689 | 0.0534 |
| 0.4544        | 31.0  | 775  | 0.5411          | 0.845    | 0.2459     | 1.5325 | 0.845    | 0.8289   | 0.1524 | 0.0514 |
| 0.4544        | 32.0  | 800  | 0.5429          | 0.845    | 0.2456     | 1.5239 | 0.845    | 0.8318   | 0.1550 | 0.0527 |
| 0.4544        | 33.0  | 825  | 0.5445          | 0.84     | 0.2468     | 1.5275 | 0.8400   | 0.8231   | 0.1626 | 0.0535 |
| 0.4544        | 34.0  | 850  | 0.5432          | 0.845    | 0.2461     | 1.5210 | 0.845    | 0.8289   | 0.1557 | 0.0533 |
| 0.4544        | 35.0  | 875  | 0.5438          | 0.845    | 0.2459     | 1.5269 | 0.845    | 0.8318   | 0.1564 | 0.0533 |
| 0.4544        | 36.0  | 900  | 0.5451          | 0.845    | 0.2466     | 1.5262 | 0.845    | 0.8289   | 0.1610 | 0.0541 |
| 0.4544        | 37.0  | 925  | 0.5415          | 0.85     | 0.2448     | 1.5254 | 0.85     | 0.8348   | 0.1667 | 0.0528 |
| 0.4544        | 38.0  | 950  | 0.5447          | 0.845    | 0.2461     | 1.5367 | 0.845    | 0.8318   | 0.1519 | 0.0535 |
| 0.4544        | 39.0  | 975  | 0.5437          | 0.85     | 0.2454     | 1.5223 | 0.85     | 0.8348   | 0.1605 | 0.0536 |
| 0.0607        | 40.0  | 1000 | 0.5445          | 0.845    | 0.2460     | 1.5252 | 0.845    | 0.8318   | 0.1610 | 0.0539 |
| 0.0607        | 41.0  | 1025 | 0.5460          | 0.845    | 0.2465     | 1.5925 | 0.845    | 0.8318   | 0.1416 | 0.0541 |
| 0.0607        | 42.0  | 1050 | 0.5466          | 0.84     | 0.2467     | 1.5304 | 0.8400   | 0.8260   | 0.1555 | 0.0542 |
| 0.0607        | 43.0  | 1075 | 0.5458          | 0.84     | 0.2464     | 1.5272 | 0.8400   | 0.8231   | 0.1633 | 0.0539 |
| 0.0607        | 44.0  | 1100 | 0.5460          | 0.85     | 0.2464     | 1.5459 | 0.85     | 0.8377   | 0.1534 | 0.0550 |
| 0.0607        | 45.0  | 1125 | 0.5464          | 0.85     | 0.2465     | 1.5390 | 0.85     | 0.8377   | 0.1471 | 0.0544 |
| 0.0607        | 46.0  | 1150 | 0.5462          | 0.85     | 0.2465     | 1.5972 | 0.85     | 0.8377   | 0.1549 | 0.0540 |
| 0.0607        | 47.0  | 1175 | 0.5475          | 0.85     | 0.2472     | 1.5910 | 0.85     | 0.8377   | 0.1592 | 0.0546 |
| 0.0607        | 48.0  | 1200 | 0.5482          | 0.845    | 0.2475     | 1.5943 | 0.845    | 0.8294   | 0.1548 | 0.0545 |
| 0.0607        | 49.0  | 1225 | 0.5475          | 0.845    | 0.2471     | 1.5922 | 0.845    | 0.8294   | 0.1534 | 0.0545 |
| 0.0607        | 50.0  | 1250 | 0.5476          | 0.85     | 0.2470     | 1.5908 | 0.85     | 0.8377   | 0.1539 | 0.0545 |
| 0.0607        | 51.0  | 1275 | 0.5480          | 0.845    | 0.2471     | 1.5990 | 0.845    | 0.8322   | 0.1545 | 0.0547 |
| 0.0607        | 52.0  | 1300 | 0.5479          | 0.85     | 0.2469     | 1.5917 | 0.85     | 0.8348   | 0.1688 | 0.0547 |
| 0.0607        | 53.0  | 1325 | 0.5479          | 0.845    | 0.2472     | 1.6052 | 0.845    | 0.8322   | 0.1545 | 0.0543 |
| 0.0607        | 54.0  | 1350 | 0.5490          | 0.85     | 0.2477     | 1.5948 | 0.85     | 0.8348   | 0.1610 | 0.0545 |
| 0.0607        | 55.0  | 1375 | 0.5489          | 0.85     | 0.2474     | 1.5967 | 0.85     | 0.8377   | 0.1543 | 0.0560 |
| 0.0607        | 56.0  | 1400 | 0.5499          | 0.845    | 0.2480     | 1.5939 | 0.845    | 0.8294   | 0.1561 | 0.0549 |
| 0.0607        | 57.0  | 1425 | 0.5492          | 0.845    | 0.2476     | 1.6048 | 0.845    | 0.8322   | 0.1570 | 0.0549 |
| 0.0607        | 58.0  | 1450 | 0.5497          | 0.845    | 0.2478     | 1.6004 | 0.845    | 0.8322   | 0.1724 | 0.0548 |
| 0.0607        | 59.0  | 1475 | 0.5496          | 0.85     | 0.2477     | 1.5982 | 0.85     | 0.8377   | 0.1634 | 0.0546 |
| 0.0589        | 60.0  | 1500 | 0.5497          | 0.845    | 0.2478     | 1.5969 | 0.845    | 0.8322   | 0.1592 | 0.0545 |
| 0.0589        | 61.0  | 1525 | 0.5492          | 0.85     | 0.2476     | 1.6095 | 0.85     | 0.8377   | 0.1630 | 0.0547 |
| 0.0589        | 62.0  | 1550 | 0.5507          | 0.845    | 0.2483     | 1.6060 | 0.845    | 0.8322   | 0.1649 | 0.0554 |
| 0.0589        | 63.0  | 1575 | 0.5490          | 0.845    | 0.2474     | 1.6021 | 0.845    | 0.8322   | 0.1635 | 0.0546 |
| 0.0589        | 64.0  | 1600 | 0.5508          | 0.845    | 0.2483     | 1.5970 | 0.845    | 0.8294   | 0.1697 | 0.0552 |
| 0.0589        | 65.0  | 1625 | 0.5505          | 0.84     | 0.2483     | 1.6023 | 0.8400   | 0.8256   | 0.1658 | 0.0553 |
| 0.0589        | 66.0  | 1650 | 0.5503          | 0.845    | 0.2481     | 1.6032 | 0.845    | 0.8322   | 0.1637 | 0.0546 |
| 0.0589        | 67.0  | 1675 | 0.5514          | 0.84     | 0.2486     | 1.6000 | 0.8400   | 0.8227   | 0.1649 | 0.0559 |
| 0.0589        | 68.0  | 1700 | 0.5516          | 0.84     | 0.2487     | 1.5979 | 0.8400   | 0.8227   | 0.1649 | 0.0550 |
| 0.0589        | 69.0  | 1725 | 0.5510          | 0.84     | 0.2485     | 1.6005 | 0.8400   | 0.8256   | 0.1639 | 0.0548 |
| 0.0589        | 70.0  | 1750 | 0.5510          | 0.84     | 0.2484     | 1.5990 | 0.8400   | 0.8256   | 0.1653 | 0.0549 |
| 0.0589        | 71.0  | 1775 | 0.5517          | 0.84     | 0.2487     | 1.6080 | 0.8400   | 0.8256   | 0.1640 | 0.0558 |
| 0.0589        | 72.0  | 1800 | 0.5525          | 0.84     | 0.2491     | 1.6069 | 0.8400   | 0.8227   | 0.1669 | 0.0558 |
| 0.0589        | 73.0  | 1825 | 0.5519          | 0.84     | 0.2488     | 1.6147 | 0.8400   | 0.8256   | 0.1638 | 0.0554 |
| 0.0589        | 74.0  | 1850 | 0.5519          | 0.84     | 0.2487     | 1.6027 | 0.8400   | 0.8256   | 0.1657 | 0.0558 |
| 0.0589        | 75.0  | 1875 | 0.5522          | 0.84     | 0.2490     | 1.6082 | 0.8400   | 0.8256   | 0.1717 | 0.0556 |
| 0.0589        | 76.0  | 1900 | 0.5523          | 0.84     | 0.2489     | 1.6022 | 0.8400   | 0.8256   | 0.1645 | 0.0553 |
| 0.0589        | 77.0  | 1925 | 0.5514          | 0.84     | 0.2486     | 1.6027 | 0.8400   | 0.8256   | 0.1635 | 0.0551 |
| 0.0589        | 78.0  | 1950 | 0.5518          | 0.84     | 0.2488     | 1.6007 | 0.8400   | 0.8256   | 0.1641 | 0.0556 |
| 0.0589        | 79.0  | 1975 | 0.5522          | 0.84     | 0.2490     | 1.6057 | 0.8400   | 0.8256   | 0.1637 | 0.0556 |
| 0.0588        | 80.0  | 2000 | 0.5520          | 0.84     | 0.2489     | 1.6110 | 0.8400   | 0.8256   | 0.1658 | 0.0552 |
| 0.0588        | 81.0  | 2025 | 0.5521          | 0.84     | 0.2489     | 1.6047 | 0.8400   | 0.8256   | 0.1659 | 0.0555 |
| 0.0588        | 82.0  | 2050 | 0.5521          | 0.84     | 0.2490     | 1.6015 | 0.8400   | 0.8256   | 0.1635 | 0.0551 |
| 0.0588        | 83.0  | 2075 | 0.5521          | 0.84     | 0.2489     | 1.6115 | 0.8400   | 0.8256   | 0.1637 | 0.0553 |
| 0.0588        | 84.0  | 2100 | 0.5523          | 0.84     | 0.2490     | 1.6033 | 0.8400   | 0.8256   | 0.1738 | 0.0553 |
| 0.0588        | 85.0  | 2125 | 0.5525          | 0.84     | 0.2491     | 1.6072 | 0.8400   | 0.8256   | 0.1658 | 0.0555 |
| 0.0588        | 86.0  | 2150 | 0.5521          | 0.84     | 0.2489     | 1.6057 | 0.8400   | 0.8256   | 0.1574 | 0.0553 |
| 0.0588        | 87.0  | 2175 | 0.5527          | 0.84     | 0.2492     | 1.6605 | 0.8400   | 0.8256   | 0.1610 | 0.0555 |
| 0.0588        | 88.0  | 2200 | 0.5526          | 0.84     | 0.2491     | 1.6056 | 0.8400   | 0.8256   | 0.1544 | 0.0556 |
| 0.0588        | 89.0  | 2225 | 0.5527          | 0.84     | 0.2492     | 1.6126 | 0.8400   | 0.8256   | 0.1547 | 0.0556 |
| 0.0588        | 90.0  | 2250 | 0.5525          | 0.84     | 0.2491     | 1.6059 | 0.8400   | 0.8256   | 0.1525 | 0.0556 |
| 0.0588        | 91.0  | 2275 | 0.5528          | 0.84     | 0.2492     | 1.6060 | 0.8400   | 0.8256   | 0.1604 | 0.0556 |
| 0.0588        | 92.0  | 2300 | 0.5526          | 0.84     | 0.2491     | 1.6080 | 0.8400   | 0.8256   | 0.1525 | 0.0555 |
| 0.0588        | 93.0  | 2325 | 0.5527          | 0.84     | 0.2492     | 1.6034 | 0.8400   | 0.8256   | 0.1547 | 0.0556 |
| 0.0588        | 94.0  | 2350 | 0.5526          | 0.84     | 0.2492     | 1.6040 | 0.8400   | 0.8256   | 0.1673 | 0.0555 |
| 0.0588        | 95.0  | 2375 | 0.5529          | 0.84     | 0.2493     | 1.6053 | 0.8400   | 0.8256   | 0.1545 | 0.0556 |
| 0.0588        | 96.0  | 2400 | 0.5526          | 0.84     | 0.2492     | 1.6050 | 0.8400   | 0.8256   | 0.1626 | 0.0555 |
| 0.0588        | 97.0  | 2425 | 0.5528          | 0.84     | 0.2492     | 1.6040 | 0.8400   | 0.8256   | 0.1686 | 0.0557 |
| 0.0588        | 98.0  | 2450 | 0.5528          | 0.84     | 0.2492     | 1.6068 | 0.8400   | 0.8256   | 0.1626 | 0.0555 |
| 0.0588        | 99.0  | 2475 | 0.5528          | 0.84     | 0.2492     | 1.6065 | 0.8400   | 0.8256   | 0.1626 | 0.0556 |
| 0.0588        | 100.0 | 2500 | 0.5528          | 0.84     | 0.2493     | 1.6062 | 0.8400   | 0.8256   | 0.1626 | 0.0556 |


### Framework versions

- Transformers 4.30.2
- Pytorch 1.13.1
- Datasets 2.13.1
- Tokenizers 0.13.3

adve email form letter memo news note report resume scientific