swinv2-tiny-patch4-window8-256-DMAE-U4

This model is a fine-tuned version of microsoft/swinv2-tiny-patch4-window8-256 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.5677
• Accuracy: 0.4565

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.86	3	7.9349	0.1087
No log	2.0	7	7.8133	0.1087
7.8893	2.86	10	7.4996	0.1087
7.8893	4.0	14	6.8656	0.1087
7.8893	4.86	17	6.3003	0.1087
6.7417	6.0	21	5.5397	0.1087
6.7417	6.86	24	5.0339	0.1087
6.7417	8.0	28	4.3869	0.1087
4.9507	8.86	31	3.9219	0.1087
4.9507	10.0	35	3.3482	0.1087
4.9507	10.86	38	2.9557	0.1087
3.5754	12.0	42	2.4984	0.1087
3.5754	12.86	45	2.2140	0.1087
3.5754	14.0	49	1.9120	0.1087
2.3517	14.86	52	1.7396	0.1087
2.3517	16.0	56	1.5677	0.4565
2.3517	16.86	59	1.4687	0.4565
1.65	18.0	63	1.3626	0.4565
1.65	18.86	66	1.3022	0.4565
1.3511	20.0	70	1.2465	0.4565
1.3511	20.86	73	1.2214	0.4565
1.3511	22.0	77	1.2086	0.4565
1.2045	22.86	80	1.2084	0.4565
1.2045	24.0	84	1.2099	0.4565
1.2045	24.86	87	1.2104	0.4565
1.1908	26.0	91	1.2096	0.4565
1.1908	26.86	94	1.2095	0.4565
1.1908	28.0	98	1.2087	0.4565
1.201	28.86	101	1.2076	0.4565
1.201	30.0	105	1.2078	0.4565
1.201	30.86	108	1.2081	0.4565
1.1807	32.0	112	1.2082	0.4565
1.1807	32.86	115	1.2082	0.4565
1.1807	34.0	119	1.2082	0.4565
1.2091	34.29	120	1.2082	0.4565

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0