layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7276
• Answer: {'precision': 0.7205240174672489, 'recall': 0.8158220024721878, 'f1': 0.7652173913043478, 'number': 809}
• Header: {'precision': 0.2903225806451613, 'recall': 0.3025210084033613, 'f1': 0.2962962962962963, 'number': 119}
• Question: {'precision': 0.7903508771929825, 'recall': 0.8460093896713615, 'f1': 0.8172335600907029, 'number': 1065}
• Overall Precision: 0.7326
• Overall Recall: 0.8013
• Overall F1: 0.7654
• Overall Accuracy: 0.8111

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.735	1.0	10	1.5211	{'precision': 0.04572098475967175, 'recall': 0.048207663782447466, 'f1': 0.04693140794223826, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2578616352201258, 'recall': 0.26948356807511736, 'f1': 0.26354453627180896, 'number': 1065}	0.1658	0.1636	0.1647	0.4315
1.353	2.0	20	1.1828	{'precision': 0.18625954198473282, 'recall': 0.1508034610630408, 'f1': 0.16666666666666669, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.48657445077298617, 'recall': 0.5615023474178403, 'f1': 0.5213600697471664, 'number': 1065}	0.3808	0.3613	0.3708	0.5930
1.027	3.0	30	0.9100	{'precision': 0.53, 'recall': 0.5241038318912238, 'f1': 0.5270354257302672, 'number': 809}	{'precision': 0.16981132075471697, 'recall': 0.07563025210084033, 'f1': 0.10465116279069768, 'number': 119}	{'precision': 0.6330434782608696, 'recall': 0.6835680751173709, 'f1': 0.6573363431151242, 'number': 1065}	0.5796	0.5825	0.5811	0.7286
0.7694	4.0	40	0.7622	{'precision': 0.6295546558704453, 'recall': 0.7688504326328801, 'f1': 0.6922648859209795, 'number': 809}	{'precision': 0.22727272727272727, 'recall': 0.12605042016806722, 'f1': 0.16216216216216214, 'number': 119}	{'precision': 0.704, 'recall': 0.7436619718309859, 'f1': 0.7232876712328766, 'number': 1065}	0.6558	0.7170	0.6850	0.7708
0.6233	5.0	50	0.7113	{'precision': 0.6527196652719666, 'recall': 0.7713226205191595, 'f1': 0.7070821529745043, 'number': 809}	{'precision': 0.26136363636363635, 'recall': 0.19327731092436976, 'f1': 0.22222222222222224, 'number': 119}	{'precision': 0.6977309562398704, 'recall': 0.8084507042253521, 'f1': 0.7490213136146151, 'number': 1065}	0.6620	0.7566	0.7062	0.7895
0.531	6.0	60	0.6976	{'precision': 0.6386138613861386, 'recall': 0.7972805933250927, 'f1': 0.709180868609126, 'number': 809}	{'precision': 0.22972972972972974, 'recall': 0.14285714285714285, 'f1': 0.17616580310880825, 'number': 119}	{'precision': 0.7175572519083969, 'recall': 0.7943661971830986, 'f1': 0.7540106951871658, 'number': 1065}	0.6664	0.7566	0.7086	0.7866
0.4577	7.0	70	0.6823	{'precision': 0.675531914893617, 'recall': 0.7849196538936959, 'f1': 0.7261292166952545, 'number': 809}	{'precision': 0.21951219512195122, 'recall': 0.226890756302521, 'f1': 0.2231404958677686, 'number': 119}	{'precision': 0.7434819175777965, 'recall': 0.8300469483568075, 'f1': 0.7843833185448092, 'number': 1065}	0.6865	0.7757	0.7284	0.8001
0.3982	8.0	80	0.6871	{'precision': 0.6847710330138446, 'recall': 0.7948084054388134, 'f1': 0.7356979405034326, 'number': 809}	{'precision': 0.2621359223300971, 'recall': 0.226890756302521, 'f1': 0.24324324324324326, 'number': 119}	{'precision': 0.7569386038687973, 'recall': 0.8450704225352113, 'f1': 0.7985803016858917, 'number': 1065}	0.7037	0.7878	0.7434	0.8091
0.3614	9.0	90	0.6850	{'precision': 0.7039045553145337, 'recall': 0.8022249690976514, 'f1': 0.7498555748122473, 'number': 809}	{'precision': 0.2692307692307692, 'recall': 0.23529411764705882, 'f1': 0.25112107623318386, 'number': 119}	{'precision': 0.7635593220338983, 'recall': 0.8460093896713615, 'f1': 0.8026726057906458, 'number': 1065}	0.7153	0.7918	0.7516	0.8101
0.354	10.0	100	0.6937	{'precision': 0.7171270718232045, 'recall': 0.8022249690976514, 'f1': 0.7572928821470245, 'number': 809}	{'precision': 0.30275229357798167, 'recall': 0.2773109243697479, 'f1': 0.28947368421052627, 'number': 119}	{'precision': 0.7840616966580977, 'recall': 0.8591549295774648, 'f1': 0.8198924731182796, 'number': 1065}	0.7322	0.8013	0.7652	0.8140
0.2994	11.0	110	0.7161	{'precision': 0.7063236870310825, 'recall': 0.8145859085290482, 'f1': 0.7566016073478761, 'number': 809}	{'precision': 0.2631578947368421, 'recall': 0.29411764705882354, 'f1': 0.27777777777777773, 'number': 119}	{'precision': 0.7885816235504014, 'recall': 0.8300469483568075, 'f1': 0.808783165599268, 'number': 1065}	0.7215	0.7918	0.7550	0.8067
0.2908	12.0	120	0.7068	{'precision': 0.7208287895310797, 'recall': 0.8170580964153276, 'f1': 0.7659327925840093, 'number': 809}	{'precision': 0.3, 'recall': 0.2773109243697479, 'f1': 0.28820960698689957, 'number': 119}	{'precision': 0.7865266841644795, 'recall': 0.844131455399061, 'f1': 0.8143115942028986, 'number': 1065}	0.7341	0.7993	0.7653	0.8134
0.2689	13.0	130	0.7206	{'precision': 0.7186477644492911, 'recall': 0.8145859085290482, 'f1': 0.7636152954808806, 'number': 809}	{'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119}	{'precision': 0.7954345917471466, 'recall': 0.8507042253521127, 'f1': 0.822141560798548, 'number': 1065}	0.7331	0.8023	0.7662	0.8120
0.2527	14.0	140	0.7260	{'precision': 0.724972497249725, 'recall': 0.8145859085290482, 'f1': 0.7671711292200234, 'number': 809}	{'precision': 0.2892561983471074, 'recall': 0.29411764705882354, 'f1': 0.2916666666666667, 'number': 119}	{'precision': 0.7900696864111498, 'recall': 0.8516431924882629, 'f1': 0.8197017623136014, 'number': 1065}	0.7351	0.8033	0.7677	0.8104
0.2511	15.0	150	0.7276	{'precision': 0.7205240174672489, 'recall': 0.8158220024721878, 'f1': 0.7652173913043478, 'number': 809}	{'precision': 0.2903225806451613, 'recall': 0.3025210084033613, 'f1': 0.2962962962962963, 'number': 119}	{'precision': 0.7903508771929825, 'recall': 0.8460093896713615, 'f1': 0.8172335600907029, 'number': 1065}	0.7326	0.8013	0.7654	0.8111

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3