
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-fashion-mnist

This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1755
- Accuracy: 0.9504

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 4
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Accuracy |
|:-------------:|:------:|:-----:|:---------------:|:--------:|
| 0.6439        | 0.0267 | 100   | 0.6483          | 0.7925   |
| 0.3972        | 0.0533 | 200   | 0.4405          | 0.8598   |
| 0.4898        | 0.08   | 300   | 0.4771          | 0.8344   |
| 0.4585        | 0.1067 | 400   | 0.4260          | 0.8533   |
| 0.4513        | 0.1333 | 500   | 0.4276          | 0.8582   |
| 0.3669        | 0.16   | 600   | 0.3700          | 0.8728   |
| 0.3053        | 0.1867 | 700   | 0.3351          | 0.8878   |
| 0.3537        | 0.2133 | 800   | 0.3868          | 0.8632   |
| 0.3253        | 0.24   | 900   | 0.2819          | 0.9023   |
| 0.6373        | 0.2667 | 1000  | 0.4660          | 0.8436   |
| 0.3327        | 0.2933 | 1100  | 0.2756          | 0.9068   |
| 0.2778        | 0.32   | 1200  | 0.3304          | 0.8892   |
| 0.2734        | 0.3467 | 1300  | 0.3733          | 0.8688   |
| 0.3481        | 0.3733 | 1400  | 0.3195          | 0.892    |
| 0.194         | 0.4    | 1500  | 0.2794          | 0.9059   |
| 0.3727        | 0.4267 | 1600  | 0.3116          | 0.8932   |
| 0.379         | 0.4533 | 1700  | 0.2742          | 0.9016   |
| 0.2764        | 0.48   | 1800  | 0.3533          | 0.8782   |
| 0.2362        | 0.5067 | 1900  | 0.2735          | 0.9062   |
| 0.333         | 0.5333 | 2000  | 0.2844          | 0.9065   |
| 0.2024        | 0.56   | 2100  | 0.3169          | 0.8871   |
| 0.2167        | 0.5867 | 2200  | 0.2575          | 0.9097   |
| 0.2368        | 0.6133 | 2300  | 0.2612          | 0.9103   |
| 0.3344        | 0.64   | 2400  | 0.2549          | 0.91     |
| 0.168         | 0.6667 | 2500  | 0.2792          | 0.9076   |
| 0.2709        | 0.6933 | 2600  | 0.2769          | 0.9034   |
| 0.2131        | 0.72   | 2700  | 0.2900          | 0.895    |
| 0.2265        | 0.7467 | 2800  | 0.2394          | 0.9141   |
| 0.3461        | 0.7733 | 2900  | 0.3260          | 0.8868   |
| 0.3012        | 0.8    | 3000  | 0.4391          | 0.8687   |
| 0.2332        | 0.8267 | 3100  | 0.2320          | 0.9189   |
| 0.2458        | 0.8533 | 3200  | 0.2460          | 0.9148   |
| 0.3271        | 0.88   | 3300  | 0.2724          | 0.9031   |
| 0.1846        | 0.9067 | 3400  | 0.2359          | 0.9173   |
| 0.1764        | 0.9333 | 3500  | 0.2712          | 0.9035   |
| 0.1818        | 0.96   | 3600  | 0.2453          | 0.9152   |
| 0.1628        | 0.9867 | 3700  | 0.2307          | 0.9189   |
| 0.2072        | 1.0133 | 3800  | 0.2309          | 0.9207   |
| 0.182         | 1.04   | 3900  | 0.2980          | 0.9015   |
| 0.1572        | 1.0667 | 4000  | 0.2553          | 0.917    |
| 0.2           | 1.0933 | 4100  | 0.2203          | 0.9216   |
| 0.1475        | 1.12   | 4200  | 0.2635          | 0.91     |
| 0.2729        | 1.1467 | 4300  | 0.2382          | 0.9151   |
| 0.2978        | 1.1733 | 4400  | 0.2469          | 0.9157   |
| 0.2117        | 1.2    | 4500  | 0.2546          | 0.9104   |
| 0.2361        | 1.2267 | 4600  | 0.2434          | 0.9143   |
| 0.3054        | 1.2533 | 4700  | 0.2272          | 0.9193   |
| 0.1032        | 1.28   | 4800  | 0.2392          | 0.9172   |
| 0.1405        | 1.3067 | 4900  | 0.2269          | 0.9205   |
| 0.2779        | 1.3333 | 5000  | 0.2037          | 0.9293   |
| 0.2025        | 1.3600 | 5100  | 0.2238          | 0.9231   |
| 0.3432        | 1.3867 | 5200  | 0.2428          | 0.9139   |
| 0.1422        | 1.4133 | 5300  | 0.2443          | 0.9181   |
| 0.2444        | 1.44   | 5400  | 0.2395          | 0.919    |
| 0.1836        | 1.4667 | 5500  | 0.2089          | 0.9277   |
| 0.2308        | 1.4933 | 5600  | 0.2120          | 0.926    |
| 0.1877        | 1.52   | 5700  | 0.2000          | 0.9305   |
| 0.2019        | 1.5467 | 5800  | 0.2278          | 0.9229   |
| 0.2829        | 1.5733 | 5900  | 0.1935          | 0.9315   |
| 0.1262        | 1.6    | 6000  | 0.2274          | 0.92     |
| 0.1152        | 1.6267 | 6100  | 0.2849          | 0.9082   |
| 0.2012        | 1.6533 | 6200  | 0.2272          | 0.921    |
| 0.1806        | 1.6800 | 6300  | 0.1932          | 0.9324   |
| 0.1769        | 1.7067 | 6400  | 0.2020          | 0.9293   |
| 0.2793        | 1.7333 | 6500  | 0.2052          | 0.927    |
| 0.0894        | 1.76   | 6600  | 0.2147          | 0.9238   |
| 0.2441        | 1.7867 | 6700  | 0.2020          | 0.93     |
| 0.2366        | 1.8133 | 6800  | 0.2125          | 0.9264   |
| 0.1992        | 1.8400 | 6900  | 0.1930          | 0.9316   |
| 0.1936        | 1.8667 | 7000  | 0.2038          | 0.93     |
| 0.2093        | 1.8933 | 7100  | 0.2100          | 0.9321   |
| 0.2183        | 1.92   | 7200  | 0.2287          | 0.9267   |
| 0.1483        | 1.9467 | 7300  | 0.1954          | 0.934    |
| 0.1828        | 1.9733 | 7400  | 0.1922          | 0.9345   |
| 0.1424        | 2.0    | 7500  | 0.1732          | 0.9388   |
| 0.1396        | 2.0267 | 7600  | 0.1920          | 0.9312   |
| 0.1433        | 2.0533 | 7700  | 0.1966          | 0.9316   |
| 0.0639        | 2.08   | 7800  | 0.1811          | 0.9358   |
| 0.1334        | 2.1067 | 7900  | 0.1962          | 0.9338   |
| 0.2618        | 2.1333 | 8000  | 0.2176          | 0.9307   |
| 0.1167        | 2.16   | 8100  | 0.1869          | 0.9369   |
| 0.0498        | 2.1867 | 8200  | 0.2008          | 0.9357   |
| 0.0647        | 2.2133 | 8300  | 0.2179          | 0.9295   |
| 0.1444        | 2.24   | 8400  | 0.1934          | 0.9368   |
| 0.1431        | 2.2667 | 8500  | 0.2257          | 0.9256   |
| 0.1464        | 2.2933 | 8600  | 0.1796          | 0.9397   |
| 0.1152        | 2.32   | 8700  | 0.1746          | 0.9422   |
| 0.1679        | 2.3467 | 8800  | 0.1796          | 0.9416   |
| 0.1404        | 2.3733 | 8900  | 0.1949          | 0.9357   |
| 0.2441        | 2.4    | 9000  | 0.1742          | 0.9421   |
| 0.1206        | 2.4267 | 9100  | 0.1953          | 0.9366   |
| 0.2064        | 2.4533 | 9200  | 0.1908          | 0.9371   |
| 0.0851        | 2.48   | 9300  | 0.1915          | 0.9369   |
| 0.1101        | 2.5067 | 9400  | 0.1830          | 0.9411   |
| 0.1081        | 2.5333 | 9500  | 0.1938          | 0.9387   |
| 0.1559        | 2.56   | 9600  | 0.1692          | 0.9435   |
| 0.0974        | 2.5867 | 9700  | 0.1735          | 0.9426   |
| 0.1344        | 2.6133 | 9800  | 0.1834          | 0.9411   |
| 0.0983        | 2.64   | 9900  | 0.1915          | 0.9367   |
| 0.0941        | 2.6667 | 10000 | 0.1842          | 0.9399   |
| 0.127         | 2.6933 | 10100 | 0.2004          | 0.938    |
| 0.1112        | 2.7200 | 10200 | 0.1829          | 0.9395   |
| 0.1898        | 2.7467 | 10300 | 0.1872          | 0.9384   |
| 0.088         | 2.7733 | 10400 | 0.1831          | 0.9417   |
| 0.1301        | 2.8    | 10500 | 0.1819          | 0.9408   |
| 0.129         | 2.8267 | 10600 | 0.1831          | 0.9394   |
| 0.1225        | 2.8533 | 10700 | 0.1778          | 0.9406   |
| 0.1084        | 2.88   | 10800 | 0.1754          | 0.9399   |
| 0.1159        | 2.9067 | 10900 | 0.1696          | 0.9432   |
| 0.1037        | 2.9333 | 11000 | 0.1731          | 0.9431   |
| 0.1173        | 2.96   | 11100 | 0.1817          | 0.9406   |
| 0.0524        | 2.9867 | 11200 | 0.1703          | 0.9439   |
| 0.0635        | 3.0133 | 11300 | 0.1689          | 0.9436   |
| 0.0662        | 3.04   | 11400 | 0.1726          | 0.9454   |
| 0.068         | 3.0667 | 11500 | 0.1777          | 0.9449   |
| 0.0441        | 3.0933 | 11600 | 0.1942          | 0.9408   |
| 0.0397        | 3.12   | 11700 | 0.1794          | 0.9478   |
| 0.0804        | 3.1467 | 11800 | 0.1859          | 0.9467   |
| 0.0193        | 3.1733 | 11900 | 0.1991          | 0.9431   |
| 0.1243        | 3.2    | 12000 | 0.1867          | 0.946    |
| 0.062         | 3.2267 | 12100 | 0.1877          | 0.9465   |
| 0.032         | 3.2533 | 12200 | 0.2086          | 0.9432   |
| 0.0177        | 3.2800 | 12300 | 0.1971          | 0.9458   |
| 0.0582        | 3.3067 | 12400 | 0.1875          | 0.9467   |
| 0.0584        | 3.3333 | 12500 | 0.1805          | 0.9484   |
| 0.0814        | 3.36   | 12600 | 0.1829          | 0.9487   |
| 0.1127        | 3.3867 | 12700 | 0.1875          | 0.9466   |
| 0.0515        | 3.4133 | 12800 | 0.1906          | 0.9452   |
| 0.0568        | 3.44   | 12900 | 0.1794          | 0.9488   |
| 0.0642        | 3.4667 | 13000 | 0.1820          | 0.9479   |
| 0.1252        | 3.4933 | 13100 | 0.1844          | 0.9491   |
| 0.0512        | 3.52   | 13200 | 0.1787          | 0.9495   |
| 0.0241        | 3.5467 | 13300 | 0.1772          | 0.9486   |
| 0.0239        | 3.5733 | 13400 | 0.1723          | 0.952    |
| 0.0796        | 3.6    | 13500 | 0.1792          | 0.9494   |
| 0.0507        | 3.6267 | 13600 | 0.1744          | 0.9513   |
| 0.0443        | 3.6533 | 13700 | 0.1745          | 0.9505   |
| 0.1451        | 3.68   | 13800 | 0.1796          | 0.9483   |
| 0.0799        | 3.7067 | 13900 | 0.1800          | 0.9491   |
| 0.0416        | 3.7333 | 14000 | 0.1799          | 0.9481   |
| 0.0758        | 3.76   | 14100 | 0.1767          | 0.9496   |
| 0.0472        | 3.7867 | 14200 | 0.1776          | 0.9495   |
| 0.0325        | 3.8133 | 14300 | 0.1745          | 0.9506   |
| 0.0388        | 3.84   | 14400 | 0.1748          | 0.951    |
| 0.0579        | 3.8667 | 14500 | 0.1763          | 0.9504   |
| 0.0784        | 3.8933 | 14600 | 0.1759          | 0.9508   |
| 0.0811        | 3.92   | 14700 | 0.1750          | 0.951    |
| 0.0204        | 3.9467 | 14800 | 0.1749          | 0.9508   |
| 0.0767        | 3.9733 | 14900 | 0.1757          | 0.9502   |
| 0.0661        | 4.0    | 15000 | 0.1755          | 0.9504   |


### Framework versions

- Transformers 4.47.1
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0

t - shirt / top trouser pullover dress coat sandal shirt sneaker bag ankle boot