mobilebert-uncased-squad-v2-16-11-2-16-11-2

This model is a fine-tuned version of badokorach/mobilebert-uncased-squad-v2-16-11-2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 25.2478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	489	11.9059
0.3138	2.0	978	14.2379
0.2761	3.0	1467	14.5768
0.1998	4.0	1956	15.1178
0.2174	5.0	2445	16.0960
0.1874	6.0	2934	16.4190
0.155	7.0	3423	16.1489
0.1099	8.0	3912	16.4489
0.1429	9.0	4401	18.7170
0.1202	10.0	4890	17.5416
0.0998	11.0	5379	19.1337
0.0669	12.0	5868	20.7663
0.0517	13.0	6357	22.7418
0.0735	14.0	6846	22.7180
0.057	15.0	7335	21.5752
0.0382	16.0	7824	24.7041
0.0287	17.0	8313	23.8935
0.0582	18.0	8802	23.1639
0.0539	19.0	9291	23.2207
0.0349	20.0	9780	23.2792
0.0279	21.0	10269	24.0050
0.0536	22.0	10758	25.2554
0.0245	23.0	11247	25.7729
0.015	24.0	11736	25.1429
0.0166	25.0	12225	25.2478

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0