


# vit-base-oxford-iiit-pets

This model is a fine-tuned version of [google/vit-base-patch16-224](https://huggingface.co/google/vit-base-patch16-224) on the pcuenq/oxford-pets dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2139
- Accuracy: 0.9350

## Model description

The Vision Transformer (ViT) is a transformer encoder model (BERT-like) pretrained on a large collection of images in a supervised fashion, namely ImageNet-21k, at a resolution of 224x224 pixels. 
Next, the model was fine-tuned on ImageNet (also referred to as ILSVRC2012), a dataset comprising 1 million images and 1,000 classes, also at resolution 224x224.

## Intended uses & limitations

### Intended Uses
This model is intended for image classification tasks, particularly those aligned with the ImageNet dataset's domain. 
It can also serve as a feature extractor for transfer learning on smaller, domain-specific datasets. 

### Limitations
This model may not generalize well to datasets that differ significantly from ImageNet. 
It is computationally intensive and may be unsuitable for use cases requiring low-latency predictions.

## Training and evaluation data
### Training Data
Pretraining Data: ImageNet-21k (14M images, 21k classes).
Fine-tuning Data: ImageNet ILSVRC2012 (1M images, 1k classes).

### Evaluation Data
Dataset: ImageNet ILSVRC2012 validation set.
Size: 50,000 images across 1,000 classes.
Metrics: Loss (0.2031), Accuracy (94.59%).


## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.3953        | 1.0   | 370  | 0.2863          | 0.9310   |
| 0.1918        | 2.0   | 740  | 0.2139          | 0.9391   |
| 0.165         | 3.0   | 1110 | 0.2008          | 0.9418   |
| 0.1476        | 4.0   | 1480 | 0.1912          | 0.9432   |
| 0.1359        | 5.0   | 1850 | 0.1872          | 0.9445   |


### Framework versions

- Transformers 4.47.1
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0
siamese birman shiba inu staffordshire bull terrier basset hound bombay japanese chin chihuahua german shorthaired pomeranian beagle english cocker spaniel american pit bull terrier ragdoll persian egyptian mau miniature pinscher sphynx maine coon keeshond yorkshire terrier havanese leonberger wheaten terrier american bulldog english setter boxer newfoundland bengal samoyed british shorthair great pyrenees abyssinian pug saint bernard russian blue scottish terrier