
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-tiny_tobacco3482_kd_MSE

This model is a fine-tuned version of [WinKawaks/vit-tiny-patch16-224](https://huggingface.co/WinKawaks/vit-tiny-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2482
- Accuracy: 0.82
- Brier Loss: 0.3226
- Nll: 0.8343
- F1 Micro: 0.82
- F1 Macro: 0.8090
- Ece: 0.2625
- Aurc: 0.0606

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 7    | 2.6314          | 0.165    | 1.0293     | 8.4704 | 0.165    | 0.0552   | 0.3856 | 0.8464 |
| No log        | 2.0   | 14   | 1.4800          | 0.12     | 0.9052     | 7.3409 | 0.12     | 0.0997   | 0.2469 | 0.8374 |
| No log        | 3.0   | 21   | 1.0627          | 0.34     | 0.8400     | 5.5369 | 0.34     | 0.2006   | 0.3075 | 0.5758 |
| No log        | 4.0   | 28   | 0.8429          | 0.42     | 0.7462     | 3.2442 | 0.4200   | 0.3362   | 0.3072 | 0.3692 |
| No log        | 5.0   | 35   | 0.7060          | 0.535    | 0.6558     | 2.7042 | 0.535    | 0.4295   | 0.2993 | 0.2677 |
| No log        | 6.0   | 42   | 0.5950          | 0.635    | 0.6056     | 2.0779 | 0.635    | 0.5021   | 0.3375 | 0.1948 |
| No log        | 7.0   | 49   | 0.4865          | 0.67     | 0.5486     | 1.4919 | 0.67     | 0.5384   | 0.3633 | 0.1737 |
| No log        | 8.0   | 56   | 0.4572          | 0.69     | 0.4897     | 1.4359 | 0.69     | 0.6106   | 0.2889 | 0.1355 |
| No log        | 9.0   | 63   | 0.3932          | 0.72     | 0.4496     | 1.0748 | 0.72     | 0.6261   | 0.2914 | 0.1092 |
| No log        | 10.0  | 70   | 0.3584          | 0.76     | 0.4091     | 1.1341 | 0.76     | 0.6747   | 0.2946 | 0.0937 |
| No log        | 11.0  | 77   | 0.3516          | 0.785    | 0.3906     | 1.0586 | 0.785    | 0.7422   | 0.3026 | 0.0762 |
| No log        | 12.0  | 84   | 0.3905          | 0.74     | 0.4155     | 1.1502 | 0.74     | 0.6677   | 0.2827 | 0.1211 |
| No log        | 13.0  | 91   | 0.3346          | 0.775    | 0.3640     | 1.0888 | 0.775    | 0.7397   | 0.2743 | 0.0771 |
| No log        | 14.0  | 98   | 0.3700          | 0.81     | 0.3728     | 0.9575 | 0.81     | 0.7941   | 0.3125 | 0.0709 |
| No log        | 15.0  | 105  | 0.3346          | 0.8      | 0.3631     | 0.9636 | 0.8000   | 0.7731   | 0.3113 | 0.0662 |
| No log        | 16.0  | 112  | 0.3084          | 0.785    | 0.3606     | 1.0243 | 0.785    | 0.7610   | 0.2883 | 0.0911 |
| No log        | 17.0  | 119  | 0.3266          | 0.785    | 0.3375     | 1.0301 | 0.785    | 0.7603   | 0.2659 | 0.0687 |
| No log        | 18.0  | 126  | 0.2987          | 0.805    | 0.3371     | 0.7961 | 0.805    | 0.7895   | 0.2764 | 0.0639 |
| No log        | 19.0  | 133  | 0.3468          | 0.815    | 0.3428     | 0.9841 | 0.815    | 0.7884   | 0.2824 | 0.0641 |
| No log        | 20.0  | 140  | 0.3111          | 0.81     | 0.3409     | 0.8873 | 0.81     | 0.8010   | 0.2978 | 0.0579 |
| No log        | 21.0  | 147  | 0.3042          | 0.8      | 0.3392     | 0.8057 | 0.8000   | 0.7802   | 0.2611 | 0.0556 |
| No log        | 22.0  | 154  | 0.2936          | 0.84     | 0.3359     | 0.8659 | 0.8400   | 0.8210   | 0.2920 | 0.0636 |
| No log        | 23.0  | 161  | 0.3284          | 0.815    | 0.3457     | 0.9494 | 0.815    | 0.8007   | 0.2793 | 0.0615 |
| No log        | 24.0  | 168  | 0.3184          | 0.805    | 0.3474     | 0.9087 | 0.805    | 0.7851   | 0.2793 | 0.0605 |
| No log        | 25.0  | 175  | 0.3081          | 0.805    | 0.3357     | 1.0425 | 0.805    | 0.7757   | 0.2781 | 0.0646 |
| No log        | 26.0  | 182  | 0.2743          | 0.815    | 0.3425     | 0.7898 | 0.815    | 0.8028   | 0.2814 | 0.0689 |
| No log        | 27.0  | 189  | 0.2873          | 0.81     | 0.3254     | 0.8795 | 0.81     | 0.7904   | 0.2755 | 0.0637 |
| No log        | 28.0  | 196  | 0.2650          | 0.82     | 0.3236     | 0.7462 | 0.82     | 0.7986   | 0.2714 | 0.0616 |
| No log        | 29.0  | 203  | 0.2756          | 0.815    | 0.3310     | 0.8227 | 0.815    | 0.7973   | 0.2766 | 0.0664 |
| No log        | 30.0  | 210  | 0.2711          | 0.83     | 0.3337     | 0.8340 | 0.83     | 0.8186   | 0.2992 | 0.0688 |
| No log        | 31.0  | 217  | 0.2779          | 0.825    | 0.3234     | 0.8173 | 0.825    | 0.8100   | 0.2579 | 0.0589 |
| No log        | 32.0  | 224  | 0.2679          | 0.82     | 0.3216     | 0.7441 | 0.82     | 0.8024   | 0.2940 | 0.0605 |
| No log        | 33.0  | 231  | 0.2633          | 0.805    | 0.3277     | 0.8046 | 0.805    | 0.7871   | 0.2710 | 0.0644 |
| No log        | 34.0  | 238  | 0.2705          | 0.805    | 0.3333     | 0.8661 | 0.805    | 0.7890   | 0.2626 | 0.0632 |
| No log        | 35.0  | 245  | 0.2624          | 0.815    | 0.3295     | 0.8568 | 0.815    | 0.7900   | 0.2865 | 0.0651 |
| No log        | 36.0  | 252  | 0.2654          | 0.805    | 0.3262     | 0.8075 | 0.805    | 0.7793   | 0.2726 | 0.0662 |
| No log        | 37.0  | 259  | 0.2697          | 0.805    | 0.3293     | 0.8143 | 0.805    | 0.7857   | 0.2587 | 0.0623 |
| No log        | 38.0  | 266  | 0.2548          | 0.805    | 0.3267     | 0.8028 | 0.805    | 0.7847   | 0.2606 | 0.0660 |
| No log        | 39.0  | 273  | 0.2740          | 0.83     | 0.3218     | 0.8270 | 0.83     | 0.8172   | 0.2697 | 0.0566 |
| No log        | 40.0  | 280  | 0.2572          | 0.81     | 0.3302     | 0.8573 | 0.81     | 0.7892   | 0.2722 | 0.0663 |
| No log        | 41.0  | 287  | 0.2528          | 0.81     | 0.3300     | 0.8454 | 0.81     | 0.7980   | 0.2555 | 0.0673 |
| No log        | 42.0  | 294  | 0.2590          | 0.815    | 0.3271     | 0.8393 | 0.815    | 0.8002   | 0.2554 | 0.0604 |
| No log        | 43.0  | 301  | 0.2654          | 0.825    | 0.3273     | 0.8100 | 0.825    | 0.8155   | 0.2687 | 0.0595 |
| No log        | 44.0  | 308  | 0.2506          | 0.805    | 0.3290     | 0.8551 | 0.805    | 0.7858   | 0.2425 | 0.0716 |
| No log        | 45.0  | 315  | 0.2615          | 0.82     | 0.3255     | 0.8624 | 0.82     | 0.8007   | 0.2773 | 0.0586 |
| No log        | 46.0  | 322  | 0.2487          | 0.815    | 0.3240     | 0.8416 | 0.815    | 0.7987   | 0.2757 | 0.0611 |
| No log        | 47.0  | 329  | 0.2674          | 0.845    | 0.3207     | 0.8407 | 0.845    | 0.8227   | 0.2963 | 0.0553 |
| No log        | 48.0  | 336  | 0.2522          | 0.805    | 0.3312     | 0.9376 | 0.805    | 0.7888   | 0.2714 | 0.0662 |
| No log        | 49.0  | 343  | 0.2547          | 0.81     | 0.3280     | 0.7847 | 0.81     | 0.7870   | 0.2696 | 0.0688 |
| No log        | 50.0  | 350  | 0.2523          | 0.81     | 0.3213     | 0.7968 | 0.81     | 0.7937   | 0.2599 | 0.0654 |
| No log        | 51.0  | 357  | 0.2526          | 0.815    | 0.3291     | 0.8022 | 0.815    | 0.7994   | 0.2888 | 0.0669 |
| No log        | 52.0  | 364  | 0.2568          | 0.835    | 0.3180     | 0.8317 | 0.835    | 0.8140   | 0.2750 | 0.0578 |
| No log        | 53.0  | 371  | 0.2496          | 0.82     | 0.3267     | 0.8442 | 0.82     | 0.8068   | 0.2825 | 0.0602 |
| No log        | 54.0  | 378  | 0.2602          | 0.82     | 0.3229     | 0.7963 | 0.82     | 0.8061   | 0.2714 | 0.0585 |
| No log        | 55.0  | 385  | 0.2477          | 0.81     | 0.3237     | 0.8278 | 0.81     | 0.7937   | 0.2511 | 0.0631 |
| No log        | 56.0  | 392  | 0.2508          | 0.83     | 0.3210     | 0.8302 | 0.83     | 0.8102   | 0.2706 | 0.0588 |
| No log        | 57.0  | 399  | 0.2454          | 0.815    | 0.3240     | 0.8377 | 0.815    | 0.7948   | 0.2607 | 0.0643 |
| No log        | 58.0  | 406  | 0.2488          | 0.815    | 0.3229     | 0.8308 | 0.815    | 0.7942   | 0.2569 | 0.0604 |
| No log        | 59.0  | 413  | 0.2510          | 0.82     | 0.3223     | 0.8314 | 0.82     | 0.8093   | 0.2779 | 0.0603 |
| No log        | 60.0  | 420  | 0.2499          | 0.82     | 0.3235     | 0.8401 | 0.82     | 0.8031   | 0.2578 | 0.0618 |
| No log        | 61.0  | 427  | 0.2478          | 0.81     | 0.3227     | 0.8315 | 0.81     | 0.7933   | 0.2645 | 0.0615 |
| No log        | 62.0  | 434  | 0.2460          | 0.82     | 0.3231     | 0.8364 | 0.82     | 0.8028   | 0.2795 | 0.0643 |
| No log        | 63.0  | 441  | 0.2489          | 0.825    | 0.3224     | 0.8337 | 0.825    | 0.8156   | 0.2759 | 0.0604 |
| No log        | 64.0  | 448  | 0.2482          | 0.825    | 0.3230     | 0.8320 | 0.825    | 0.8138   | 0.2753 | 0.0600 |
| No log        | 65.0  | 455  | 0.2462          | 0.815    | 0.3231     | 0.8354 | 0.815    | 0.8008   | 0.2551 | 0.0625 |
| No log        | 66.0  | 462  | 0.2470          | 0.815    | 0.3219     | 0.8338 | 0.815    | 0.8018   | 0.2729 | 0.0611 |
| No log        | 67.0  | 469  | 0.2457          | 0.81     | 0.3231     | 0.8336 | 0.81     | 0.7930   | 0.2587 | 0.0638 |
| No log        | 68.0  | 476  | 0.2472          | 0.815    | 0.3225     | 0.8334 | 0.815    | 0.8008   | 0.2706 | 0.0619 |
| No log        | 69.0  | 483  | 0.2473          | 0.825    | 0.3223     | 0.8357 | 0.825    | 0.8165   | 0.2668 | 0.0611 |
| No log        | 70.0  | 490  | 0.2481          | 0.81     | 0.3223     | 0.8343 | 0.81     | 0.7930   | 0.2685 | 0.0624 |
| No log        | 71.0  | 497  | 0.2472          | 0.825    | 0.3227     | 0.8338 | 0.825    | 0.8117   | 0.2839 | 0.0601 |
| 0.232         | 72.0  | 504  | 0.2472          | 0.815    | 0.3220     | 0.8345 | 0.815    | 0.8018   | 0.2617 | 0.0615 |
| 0.232         | 73.0  | 511  | 0.2486          | 0.82     | 0.3218     | 0.8321 | 0.82     | 0.8086   | 0.2768 | 0.0610 |
| 0.232         | 74.0  | 518  | 0.2468          | 0.815    | 0.3219     | 0.8338 | 0.815    | 0.8008   | 0.2717 | 0.0621 |
| 0.232         | 75.0  | 525  | 0.2470          | 0.82     | 0.3223     | 0.8325 | 0.82     | 0.8090   | 0.2625 | 0.0606 |
| 0.232         | 76.0  | 532  | 0.2474          | 0.825    | 0.3223     | 0.8322 | 0.825    | 0.8165   | 0.2723 | 0.0602 |
| 0.232         | 77.0  | 539  | 0.2476          | 0.805    | 0.3227     | 0.8345 | 0.805    | 0.7859   | 0.2589 | 0.0629 |
| 0.232         | 78.0  | 546  | 0.2479          | 0.82     | 0.3228     | 0.8336 | 0.82     | 0.8090   | 0.2674 | 0.0608 |
| 0.232         | 79.0  | 553  | 0.2478          | 0.82     | 0.3225     | 0.8349 | 0.82     | 0.8090   | 0.2624 | 0.0604 |
| 0.232         | 80.0  | 560  | 0.2477          | 0.81     | 0.3227     | 0.8337 | 0.81     | 0.7938   | 0.2577 | 0.0621 |
| 0.232         | 81.0  | 567  | 0.2478          | 0.82     | 0.3226     | 0.8336 | 0.82     | 0.8090   | 0.2670 | 0.0607 |
| 0.232         | 82.0  | 574  | 0.2480          | 0.825    | 0.3224     | 0.8340 | 0.825    | 0.8165   | 0.2673 | 0.0605 |
| 0.232         | 83.0  | 581  | 0.2479          | 0.82     | 0.3227     | 0.8347 | 0.82     | 0.8090   | 0.2564 | 0.0607 |
| 0.232         | 84.0  | 588  | 0.2480          | 0.82     | 0.3226     | 0.8342 | 0.82     | 0.8090   | 0.2625 | 0.0606 |
| 0.232         | 85.0  | 595  | 0.2480          | 0.82     | 0.3225     | 0.8339 | 0.82     | 0.8090   | 0.2625 | 0.0606 |
| 0.232         | 86.0  | 602  | 0.2479          | 0.825    | 0.3226     | 0.8339 | 0.825    | 0.8165   | 0.2677 | 0.0606 |
| 0.232         | 87.0  | 609  | 0.2479          | 0.82     | 0.3225     | 0.8339 | 0.82     | 0.8090   | 0.2624 | 0.0605 |
| 0.232         | 88.0  | 616  | 0.2481          | 0.825    | 0.3225     | 0.8343 | 0.825    | 0.8165   | 0.2675 | 0.0604 |
| 0.232         | 89.0  | 623  | 0.2481          | 0.825    | 0.3225     | 0.8341 | 0.825    | 0.8165   | 0.2722 | 0.0605 |
| 0.232         | 90.0  | 630  | 0.2481          | 0.82     | 0.3225     | 0.8341 | 0.82     | 0.8090   | 0.2625 | 0.0606 |
| 0.232         | 91.0  | 637  | 0.2481          | 0.82     | 0.3226     | 0.8345 | 0.82     | 0.8090   | 0.2629 | 0.0608 |
| 0.232         | 92.0  | 644  | 0.2481          | 0.825    | 0.3226     | 0.8342 | 0.825    | 0.8165   | 0.2675 | 0.0605 |
| 0.232         | 93.0  | 651  | 0.2481          | 0.825    | 0.3225     | 0.8340 | 0.825    | 0.8165   | 0.2675 | 0.0605 |
| 0.232         | 94.0  | 658  | 0.2481          | 0.82     | 0.3225     | 0.8343 | 0.82     | 0.8090   | 0.2625 | 0.0606 |
| 0.232         | 95.0  | 665  | 0.2482          | 0.82     | 0.3226     | 0.8345 | 0.82     | 0.8090   | 0.2627 | 0.0606 |
| 0.232         | 96.0  | 672  | 0.2482          | 0.82     | 0.3225     | 0.8343 | 0.82     | 0.8090   | 0.2627 | 0.0607 |
| 0.232         | 97.0  | 679  | 0.2482          | 0.82     | 0.3226     | 0.8344 | 0.82     | 0.8090   | 0.2627 | 0.0607 |
| 0.232         | 98.0  | 686  | 0.2482          | 0.82     | 0.3226     | 0.8344 | 0.82     | 0.8090   | 0.2626 | 0.0606 |
| 0.232         | 99.0  | 693  | 0.2482          | 0.82     | 0.3226     | 0.8343 | 0.82     | 0.8090   | 0.2625 | 0.0606 |
| 0.232         | 100.0 | 700  | 0.2482          | 0.82     | 0.3226     | 0.8343 | 0.82     | 0.8090   | 0.2625 | 0.0606 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

adve email form letter memo news note report resume scientific