
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base-patch16-224-dmae-va-U5-100-3i

This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5087
- Accuracy: 0.8667

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 0.9   | 7    | 0.5069          | 0.8333   |
| 0.3296        | 1.94  | 15   | 0.5087          | 0.8667   |
| 0.2919        | 2.97  | 23   | 0.5190          | 0.8667   |
| 0.2572        | 4.0   | 31   | 0.6483          | 0.7667   |
| 0.2572        | 4.9   | 38   | 0.5785          | 0.8167   |
| 0.2229        | 5.94  | 46   | 0.5932          | 0.8333   |
| 0.1799        | 6.97  | 54   | 0.5272          | 0.85     |
| 0.1563        | 8.0   | 62   | 0.6124          | 0.85     |
| 0.1563        | 8.9   | 69   | 0.6798          | 0.8167   |
| 0.125         | 9.94  | 77   | 0.7356          | 0.7833   |
| 0.1343        | 10.97 | 85   | 0.5086          | 0.85     |
| 0.0906        | 12.0  | 93   | 0.7601          | 0.7667   |
| 0.103         | 12.9  | 100  | 0.8084          | 0.8      |
| 0.103         | 13.94 | 108  | 0.5612          | 0.85     |
| 0.1002        | 14.97 | 116  | 0.6454          | 0.8333   |
| 0.1107        | 16.0  | 124  | 0.7783          | 0.8      |
| 0.1036        | 16.9  | 131  | 0.7857          | 0.7833   |
| 0.1036        | 17.94 | 139  | 0.6504          | 0.8167   |
| 0.1248        | 18.97 | 147  | 0.6510          | 0.8167   |
| 0.1074        | 20.0  | 155  | 0.7813          | 0.7833   |
| 0.1038        | 20.9  | 162  | 0.6553          | 0.8      |
| 0.1052        | 21.94 | 170  | 0.6449          | 0.8333   |
| 0.1052        | 22.97 | 178  | 0.7444          | 0.8      |
| 0.0782        | 24.0  | 186  | 1.0751          | 0.6833   |
| 0.0952        | 24.9  | 193  | 0.6453          | 0.8333   |
| 0.0803        | 25.94 | 201  | 0.7794          | 0.8      |
| 0.0803        | 26.97 | 209  | 0.6160          | 0.8333   |
| 0.0947        | 28.0  | 217  | 0.6362          | 0.85     |
| 0.0702        | 28.9  | 224  | 0.7610          | 0.8167   |
| 0.0737        | 29.94 | 232  | 0.7924          | 0.8167   |
| 0.0644        | 30.97 | 240  | 0.9755          | 0.8      |
| 0.0644        | 32.0  | 248  | 0.8580          | 0.8333   |
| 0.0695        | 32.9  | 255  | 1.1410          | 0.7167   |
| 0.09          | 33.94 | 263  | 0.8442          | 0.8      |
| 0.0619        | 34.97 | 271  | 1.1689          | 0.7167   |
| 0.0619        | 36.0  | 279  | 0.7599          | 0.8333   |
| 0.0607        | 36.9  | 286  | 0.8498          | 0.8167   |
| 0.0509        | 37.94 | 294  | 0.8331          | 0.85     |
| 0.0666        | 38.97 | 302  | 0.8166          | 0.8167   |
| 0.0615        | 40.0  | 310  | 0.9394          | 0.7667   |
| 0.0615        | 40.9  | 317  | 0.8837          | 0.8      |
| 0.0503        | 41.94 | 325  | 0.8208          | 0.8333   |
| 0.0431        | 42.97 | 333  | 1.1271          | 0.75     |
| 0.0548        | 44.0  | 341  | 0.9044          | 0.7833   |
| 0.0548        | 44.9  | 348  | 0.9017          | 0.8      |
| 0.0414        | 45.94 | 356  | 1.1390          | 0.75     |
| 0.0609        | 46.97 | 364  | 0.8937          | 0.8      |
| 0.0556        | 48.0  | 372  | 0.8459          | 0.8      |
| 0.0556        | 48.9  | 379  | 1.0285          | 0.7667   |
| 0.0417        | 49.94 | 387  | 0.7379          | 0.85     |
| 0.0409        | 50.97 | 395  | 0.7817          | 0.8333   |
| 0.0206        | 52.0  | 403  | 0.7860          | 0.8167   |
| 0.0414        | 52.9  | 410  | 0.8414          | 0.8167   |
| 0.0414        | 53.94 | 418  | 0.8657          | 0.8      |
| 0.0329        | 54.97 | 426  | 0.8824          | 0.8      |
| 0.0394        | 56.0  | 434  | 0.7990          | 0.8333   |
| 0.0373        | 56.9  | 441  | 0.8101          | 0.8167   |
| 0.0373        | 57.94 | 449  | 0.8535          | 0.8      |
| 0.0418        | 58.97 | 457  | 0.9149          | 0.8167   |
| 0.0365        | 60.0  | 465  | 0.9278          | 0.8      |
| 0.0367        | 60.9  | 472  | 0.9064          | 0.8      |
| 0.0355        | 61.94 | 480  | 0.9610          | 0.7833   |
| 0.0355        | 62.97 | 488  | 0.9174          | 0.8167   |
| 0.0492        | 64.0  | 496  | 0.9877          | 0.7667   |
| 0.0326        | 64.9  | 503  | 1.0192          | 0.7833   |
| 0.0233        | 65.94 | 511  | 0.9588          | 0.8      |
| 0.0233        | 66.97 | 519  | 0.9829          | 0.7833   |
| 0.0251        | 68.0  | 527  | 1.0540          | 0.7667   |
| 0.0283        | 68.9  | 534  | 1.0556          | 0.7667   |
| 0.0307        | 69.94 | 542  | 1.0036          | 0.7833   |
| 0.0319        | 70.97 | 550  | 0.9294          | 0.8      |
| 0.0319        | 72.0  | 558  | 1.0077          | 0.8      |
| 0.0246        | 72.9  | 565  | 1.0298          | 0.7833   |
| 0.0205        | 73.94 | 573  | 1.0041          | 0.7833   |
| 0.0345        | 74.97 | 581  | 0.9182          | 0.7833   |
| 0.0345        | 76.0  | 589  | 0.9054          | 0.8333   |
| 0.0181        | 76.9  | 596  | 0.9338          | 0.8333   |
| 0.0287        | 77.94 | 604  | 0.9678          | 0.7833   |
| 0.0268        | 78.97 | 612  | 0.9841          | 0.7833   |
| 0.0293        | 80.0  | 620  | 1.0380          | 0.7667   |
| 0.0293        | 80.9  | 627  | 1.0837          | 0.7833   |
| 0.0222        | 81.94 | 635  | 1.0132          | 0.7667   |
| 0.033         | 82.97 | 643  | 0.9785          | 0.8      |
| 0.0227        | 84.0  | 651  | 0.9848          | 0.8      |
| 0.0227        | 84.9  | 658  | 0.9780          | 0.8      |
| 0.0295        | 85.94 | 666  | 0.9613          | 0.8167   |
| 0.0291        | 86.97 | 674  | 0.9753          | 0.8167   |
| 0.031         | 88.0  | 682  | 0.9831          | 0.8      |
| 0.031         | 88.9  | 689  | 0.9820          | 0.8      |
| 0.0233        | 89.94 | 697  | 0.9793          | 0.8      |
| 0.0195        | 90.32 | 700  | 0.9788          | 0.8      |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0

avanzada leve moderada no dmae