
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# swin-transformer2

This model is a fine-tuned version of [microsoft/swin-large-patch4-window12-384](https://huggingface.co/microsoft/swin-large-patch4-window12-384) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2129
- Accuracy: 0.6386
- F1: 0.6328

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch   | Step | Validation Loss | Accuracy | F1     |
|:-------------:|:-------:|:----:|:---------------:|:--------:|:------:|
| 1.6336        | 0.9840  | 46   | 1.6510          | 0.2530   | 0.1876 |
| 1.2894        | 1.9893  | 93   | 1.2218          | 0.4458   | 0.3780 |
| 1.0959        | 2.9947  | 140  | 1.1383          | 0.5060   | 0.3518 |
| 1.0467        | 4.0     | 187  | 0.9372          | 0.5542   | 0.4352 |
| 0.9879        | 4.9840  | 233  | 1.0139          | 0.5301   | 0.4718 |
| 0.9086        | 5.9893  | 280  | 0.8822          | 0.6627   | 0.6359 |
| 0.9776        | 6.9947  | 327  | 1.0269          | 0.5542   | 0.5139 |
| 0.9715        | 8.0     | 374  | 0.7964          | 0.5663   | 0.5588 |
| 0.9049        | 8.9840  | 420  | 0.7839          | 0.5904   | 0.5346 |
| 0.8697        | 9.9893  | 467  | 1.0379          | 0.5663   | 0.4921 |
| 0.882         | 10.9947 | 514  | 0.9132          | 0.5663   | 0.5379 |
| 0.832         | 12.0    | 561  | 0.8513          | 0.5783   | 0.5008 |
| 0.7475        | 12.9840 | 607  | 0.7612          | 0.6627   | 0.6427 |
| 0.9056        | 13.9893 | 654  | 0.8431          | 0.6145   | 0.5725 |
| 0.9978        | 14.9947 | 701  | 0.7221          | 0.7108   | 0.6983 |
| 0.6956        | 16.0    | 748  | 0.7545          | 0.6145   | 0.5888 |
| 0.7185        | 16.9840 | 794  | 0.6561          | 0.6627   | 0.6499 |
| 0.8139        | 17.9893 | 841  | 0.7512          | 0.6506   | 0.6386 |
| 0.6837        | 18.9947 | 888  | 0.6491          | 0.6988   | 0.6849 |
| 0.5191        | 20.0    | 935  | 0.7290          | 0.6386   | 0.6336 |
| 0.6538        | 20.9840 | 981  | 0.8000          | 0.6988   | 0.6621 |
| 0.7912        | 21.9893 | 1028 | 1.0183          | 0.6145   | 0.5824 |
| 0.6093        | 22.9947 | 1075 | 0.9124          | 0.6506   | 0.6396 |
| 0.5312        | 24.0    | 1122 | 0.9098          | 0.6024   | 0.5581 |
| 0.6654        | 24.9840 | 1168 | 1.0432          | 0.5422   | 0.5028 |
| 0.5798        | 25.9893 | 1215 | 0.7369          | 0.6627   | 0.6553 |
| 0.506         | 26.9947 | 1262 | 0.9057          | 0.6265   | 0.6236 |
| 0.4638        | 28.0    | 1309 | 0.7950          | 0.6867   | 0.6644 |
| 0.371         | 28.9840 | 1355 | 1.0368          | 0.6627   | 0.6473 |
| 0.4721        | 29.9893 | 1402 | 0.8129          | 0.6747   | 0.6673 |
| 0.54          | 30.9947 | 1449 | 1.0379          | 0.6627   | 0.6491 |
| 0.3978        | 32.0    | 1496 | 1.3857          | 0.5904   | 0.5481 |
| 0.3503        | 32.9840 | 1542 | 1.0920          | 0.6024   | 0.5847 |
| 0.4407        | 33.9893 | 1589 | 1.1912          | 0.5904   | 0.5505 |
| 0.3786        | 34.9947 | 1636 | 1.5071          | 0.6024   | 0.5915 |
| 0.3482        | 36.0    | 1683 | 1.1161          | 0.6386   | 0.6240 |
| 0.2695        | 36.9840 | 1729 | 1.2040          | 0.5904   | 0.5704 |
| 0.2296        | 37.9893 | 1776 | 1.5781          | 0.5181   | 0.4691 |
| 0.2922        | 38.9947 | 1823 | 1.3713          | 0.6024   | 0.5879 |
| 0.1511        | 40.0    | 1870 | 1.1638          | 0.6506   | 0.6553 |
| 0.2814        | 40.9840 | 1916 | 1.3384          | 0.6988   | 0.6939 |
| 0.2196        | 41.9893 | 1963 | 1.2872          | 0.6506   | 0.6330 |
| 0.2477        | 42.9947 | 2010 | 1.5322          | 0.6627   | 0.6375 |
| 0.3296        | 44.0    | 2057 | 1.3479          | 0.6506   | 0.6353 |
| 0.2015        | 44.9840 | 2103 | 1.2521          | 0.6145   | 0.6044 |
| 0.3476        | 45.9893 | 2150 | 1.2464          | 0.6747   | 0.6641 |
| 0.189         | 46.9947 | 2197 | 1.4480          | 0.6506   | 0.6235 |
| 0.1852        | 48.0    | 2244 | 1.3611          | 0.6747   | 0.6594 |
| 0.2798        | 48.9840 | 2290 | 1.4427          | 0.6988   | 0.6957 |
| 0.1523        | 49.9893 | 2337 | 1.3352          | 0.6506   | 0.6450 |
| 0.1224        | 50.9947 | 2384 | 1.8088          | 0.6386   | 0.6201 |
| 0.0926        | 52.0    | 2431 | 1.4695          | 0.6506   | 0.6296 |
| 0.2071        | 52.9840 | 2477 | 1.4673          | 0.6867   | 0.6806 |
| 0.1063        | 53.9893 | 2524 | 1.4862          | 0.7108   | 0.6975 |
| 0.1831        | 54.9947 | 2571 | 1.4666          | 0.6506   | 0.6161 |
| 0.158         | 56.0    | 2618 | 1.8832          | 0.6988   | 0.6673 |
| 0.26          | 56.9840 | 2664 | 1.5855          | 0.6386   | 0.5986 |
| 0.1697        | 57.9893 | 2711 | 1.2184          | 0.7470   | 0.7434 |
| 0.2024        | 58.9947 | 2758 | 1.3524          | 0.6867   | 0.6682 |
| 0.2495        | 60.0    | 2805 | 1.7523          | 0.6627   | 0.6427 |
| 0.1247        | 60.9840 | 2851 | 1.7007          | 0.6506   | 0.6372 |
| 0.1436        | 61.9893 | 2898 | 1.9171          | 0.6386   | 0.6120 |
| 0.1438        | 62.9947 | 2945 | 1.8998          | 0.6265   | 0.5897 |
| 0.1137        | 64.0    | 2992 | 2.4028          | 0.5904   | 0.5498 |
| 0.1619        | 64.9840 | 3038 | 1.7087          | 0.7470   | 0.7473 |
| 0.1105        | 65.9893 | 3085 | 1.6545          | 0.6988   | 0.6975 |
| 0.1597        | 66.9947 | 3132 | 1.8024          | 0.6747   | 0.6758 |
| 0.0338        | 68.0    | 3179 | 1.8962          | 0.6747   | 0.6706 |
| 0.1184        | 68.9840 | 3225 | 2.1642          | 0.7108   | 0.7102 |
| 0.0878        | 69.9893 | 3272 | 2.0974          | 0.6506   | 0.6610 |
| 0.0963        | 70.9947 | 3319 | 1.8719          | 0.7108   | 0.7162 |
| 0.0827        | 72.0    | 3366 | 1.7538          | 0.6988   | 0.7000 |
| 0.0933        | 72.9840 | 3412 | 1.9357          | 0.6988   | 0.6988 |
| 0.0593        | 73.9893 | 3459 | 1.9924          | 0.6506   | 0.6420 |
| 0.0423        | 74.9947 | 3506 | 2.2029          | 0.6627   | 0.6702 |
| 0.0311        | 76.0    | 3553 | 1.9236          | 0.7108   | 0.7155 |
| 0.1881        | 76.9840 | 3599 | 1.9606          | 0.6747   | 0.6787 |
| 0.0566        | 77.9893 | 3646 | 2.1122          | 0.6265   | 0.6206 |
| 0.0266        | 78.9947 | 3693 | 2.1469          | 0.6506   | 0.6536 |
| 0.1015        | 80.0    | 3740 | 2.0335          | 0.6506   | 0.6587 |
| 0.1083        | 80.9840 | 3786 | 2.2123          | 0.6506   | 0.6509 |
| 0.0161        | 81.9893 | 3833 | 2.3094          | 0.6988   | 0.7064 |
| 0.0194        | 82.9947 | 3880 | 2.3315          | 0.6145   | 0.6101 |
| 0.113         | 84.0    | 3927 | 2.5276          | 0.6867   | 0.6908 |
| 0.0653        | 84.9840 | 3973 | 2.0321          | 0.6265   | 0.6263 |
| 0.0684        | 85.9893 | 4020 | 2.0302          | 0.6627   | 0.6706 |
| 0.1724        | 86.9947 | 4067 | 2.5865          | 0.5904   | 0.5860 |
| 0.028         | 88.0    | 4114 | 2.3814          | 0.5904   | 0.5804 |
| 0.0528        | 88.9840 | 4160 | 2.2804          | 0.6386   | 0.6410 |
| 0.0341        | 89.9893 | 4207 | 2.0635          | 0.5783   | 0.5736 |
| 0.0074        | 90.9947 | 4254 | 2.3491          | 0.6024   | 0.5993 |
| 0.0165        | 92.0    | 4301 | 2.2152          | 0.6145   | 0.6036 |
| 0.0157        | 92.9840 | 4347 | 2.3380          | 0.6145   | 0.6036 |
| 0.0544        | 93.9893 | 4394 | 2.3319          | 0.6265   | 0.6221 |
| 0.0577        | 94.9947 | 4441 | 2.2671          | 0.6265   | 0.6221 |
| 0.1516        | 96.0    | 4488 | 2.2034          | 0.6265   | 0.6204 |
| 0.0318        | 96.9840 | 4534 | 2.1932          | 0.6265   | 0.6204 |
| 0.043         | 97.9893 | 4581 | 2.2178          | 0.6265   | 0.6204 |
| 0.0099        | 98.3957 | 4600 | 2.2129          | 0.6386   | 0.6328 |


### Framework versions

- Transformers 4.45.2
- Pytorch 2.4.0+cu121
- Datasets 3.1.0
- Tokenizers 0.20.1

lglo mglo hglo uglo mgso lgso hgso mws mwl