plus_model

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2227
• Eader: {'precision': 0.6527777777777778, 'recall': 0.6308724832214765, 'f1': 0.6416382252559727, 'number': 149}
• Nswer: {'precision': 0.7374551971326165, 'recall': 0.7481818181818182, 'f1': 0.7427797833935018, 'number': 1100}
• Uestion: {'precision': 0.7554833468724614, 'recall': 0.7604251839738349, 'f1': 0.7579462102689487, 'number': 1223}
• Overall Precision: 0.7415
• Overall Recall: 0.7472
• Overall F1: 0.7443
• Overall Accuracy: 0.8544

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500

Training results

Training Loss	Epoch	Step	Validation Loss	Eader	Nswer	Uestion	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.1843	0.78	200	0.2061	{'precision': 0.6296296296296297, 'recall': 0.3422818791946309, 'f1': 0.4434782608695652, 'number': 149}	{'precision': 0.6472906403940887, 'recall': 0.5972727272727273, 'f1': 0.6212765957446809, 'number': 1100}	{'precision': 0.7054794520547946, 'recall': 0.6737530662305805, 'f1': 0.6892513592639062, 'number': 1223}	0.6767	0.6197	0.6470	0.7748
0.1356	1.56	400	0.1477	{'precision': 0.5701754385964912, 'recall': 0.436241610738255, 'f1': 0.49429657794676807, 'number': 149}	{'precision': 0.6569148936170213, 'recall': 0.6736363636363636, 'f1': 0.6651705565529622, 'number': 1100}	{'precision': 0.6554054054054054, 'recall': 0.713818479149632, 'f1': 0.6833659491193737, 'number': 1223}	0.6523	0.6792	0.6655	0.8189
0.102	2.33	600	0.1666	{'precision': 0.38974358974358975, 'recall': 0.5100671140939598, 'f1': 0.4418604651162791, 'number': 149}	{'precision': 0.6898444647758463, 'recall': 0.6854545454545454, 'f1': 0.687642498860009, 'number': 1100}	{'precision': 0.7024661893396977, 'recall': 0.7219950940310711, 'f1': 0.7120967741935483, 'number': 1223}	0.6731	0.6930	0.6829	0.8246
0.0836	3.11	800	0.1592	{'precision': 0.6307692307692307, 'recall': 0.5503355704697986, 'f1': 0.5878136200716846, 'number': 149}	{'precision': 0.6595012897678418, 'recall': 0.6972727272727273, 'f1': 0.6778612461334512, 'number': 1100}	{'precision': 0.7373653686826843, 'recall': 0.7277187244480785, 'f1': 0.7325102880658437, 'number': 1223}	0.6956	0.7035	0.6995	0.8436
0.0657	3.89	1000	0.1658	{'precision': 0.5869565217391305, 'recall': 0.5436241610738255, 'f1': 0.5644599303135888, 'number': 149}	{'precision': 0.7464788732394366, 'recall': 0.7227272727272728, 'f1': 0.7344110854503464, 'number': 1100}	{'precision': 0.7090620031796503, 'recall': 0.7293540474243663, 'f1': 0.7190648931882304, 'number': 1223}	0.7184	0.7152	0.7168	0.8457
0.0462	4.67	1200	0.1855	{'precision': 0.656, 'recall': 0.5503355704697986, 'f1': 0.5985401459854014, 'number': 149}	{'precision': 0.6961038961038961, 'recall': 0.730909090909091, 'f1': 0.7130820399113083, 'number': 1100}	{'precision': 0.7286392405063291, 'recall': 0.7530662305805397, 'f1': 0.7406513872135101, 'number': 1223}	0.7103	0.7310	0.7205	0.8427
0.0441	5.45	1400	0.1721	{'precision': 0.6538461538461539, 'recall': 0.5704697986577181, 'f1': 0.6093189964157705, 'number': 149}	{'precision': 0.7275179856115108, 'recall': 0.7354545454545455, 'f1': 0.7314647377938518, 'number': 1100}	{'precision': 0.7504065040650406, 'recall': 0.7547015535568274, 'f1': 0.7525479005299633, 'number': 1223}	0.7350	0.7350	0.7350	0.8555
0.0347	6.23	1600	0.2052	{'precision': 0.6312056737588653, 'recall': 0.5973154362416108, 'f1': 0.6137931034482759, 'number': 149}	{'precision': 0.715929203539823, 'recall': 0.7354545454545455, 'f1': 0.7255605381165919, 'number': 1100}	{'precision': 0.7475728155339806, 'recall': 0.7555192150449714, 'f1': 0.7515250101667346, 'number': 1223}	0.7268	0.7371	0.7319	0.8545
0.0294	7.0	1800	0.2374	{'precision': 0.6190476190476191, 'recall': 0.610738255033557, 'f1': 0.6148648648648649, 'number': 149}	{'precision': 0.7442075996292864, 'recall': 0.73, 'f1': 0.7370353373106929, 'number': 1100}	{'precision': 0.7593671940049959, 'recall': 0.7457072771872445, 'f1': 0.7524752475247525, 'number': 1223}	0.7441	0.7306	0.7373	0.8401
0.0239	7.78	2000	0.2227	{'precision': 0.647887323943662, 'recall': 0.6174496644295302, 'f1': 0.6323024054982819, 'number': 149}	{'precision': 0.7139061116031886, 'recall': 0.7327272727272728, 'f1': 0.7231942575145804, 'number': 1100}	{'precision': 0.7662229617304492, 'recall': 0.7530662305805397, 'f1': 0.7595876288659793, 'number': 1223}	0.7355	0.7358	0.7357	0.8472
0.0204	8.56	2200	0.2263	{'precision': 0.6375838926174496, 'recall': 0.6375838926174496, 'f1': 0.6375838926174496, 'number': 149}	{'precision': 0.7371737173717372, 'recall': 0.7445454545454545, 'f1': 0.7408412483039349, 'number': 1100}	{'precision': 0.7540453074433657, 'recall': 0.7620605069501226, 'f1': 0.7580317202114681, 'number': 1223}	0.7396	0.7468	0.7432	0.8539
0.0184	9.34	2400	0.2227	{'precision': 0.6527777777777778, 'recall': 0.6308724832214765, 'f1': 0.6416382252559727, 'number': 149}	{'precision': 0.7374551971326165, 'recall': 0.7481818181818182, 'f1': 0.7427797833935018, 'number': 1100}	{'precision': 0.7554833468724614, 'recall': 0.7604251839738349, 'f1': 0.7579462102689487, 'number': 1223}	0.7415	0.7472	0.7443	0.8544

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0.dev20230810
• Datasets 2.14.4
• Tokenizers 0.14.1