File size: 9,347 Bytes
77493fd 5c94a61 77493fd 5c94a61 77493fd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6843
- Answer: {'precision': 0.6938997821350763, 'recall': 0.7873918417799752, 'f1': 0.7376954255935148, 'number': 809}
- Header: {'precision': 0.27941176470588236, 'recall': 0.31932773109243695, 'f1': 0.2980392156862745, 'number': 119}
- Question: {'precision': 0.7749562171628721, 'recall': 0.8309859154929577, 'f1': 0.8019936565473492, 'number': 1065}
- Overall Precision: 0.7104
- Overall Recall: 0.7827
- Overall F1: 0.7448
- Overall Accuracy: 0.8076
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8035 | 1.0 | 10 | 1.6086 | {'precision': 0.007142857142857143, 'recall': 0.003708281829419036, 'f1': 0.004882017900732303, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.07957559681697612, 'recall': 0.028169014084507043, 'f1': 0.04160887656033287, 'number': 1065} | 0.0414 | 0.0166 | 0.0237 | 0.3175 |
| 1.4936 | 2.0 | 20 | 1.2735 | {'precision': 0.279126213592233, 'recall': 0.4264524103831891, 'f1': 0.3374083129584352, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4406651549508692, 'recall': 0.5474178403755868, 'f1': 0.48827470686767166, 'number': 1065} | 0.3626 | 0.4656 | 0.4077 | 0.6074 |
| 1.1259 | 3.0 | 30 | 0.9718 | {'precision': 0.47892074198988194, 'recall': 0.7021013597033374, 'f1': 0.569423558897243, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5904121110176619, 'recall': 0.6591549295774648, 'f1': 0.6228926353149955, 'number': 1065} | 0.5336 | 0.6372 | 0.5808 | 0.6760 |
| 0.8568 | 4.0 | 40 | 0.8421 | {'precision': 0.5595126522961574, 'recall': 0.7379480840543882, 'f1': 0.6364605543710021, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6669595782073814, 'recall': 0.7126760563380282, 'f1': 0.6890603722197005, 'number': 1065} | 0.6059 | 0.6804 | 0.6410 | 0.7310 |
| 0.7275 | 5.0 | 50 | 0.7430 | {'precision': 0.6401673640167364, 'recall': 0.7564894932014833, 'f1': 0.693484419263456, 'number': 809} | {'precision': 0.12643678160919541, 'recall': 0.09243697478991597, 'f1': 0.10679611650485439, 'number': 119} | {'precision': 0.6825, 'recall': 0.7690140845070422, 'f1': 0.7231788079470199, 'number': 1065} | 0.6429 | 0.7235 | 0.6808 | 0.7727 |
| 0.6109 | 6.0 | 60 | 0.6965 | {'precision': 0.6494736842105263, 'recall': 0.7626699629171817, 'f1': 0.7015349630471859, 'number': 809} | {'precision': 0.125, 'recall': 0.09243697478991597, 'f1': 0.10628019323671498, 'number': 119} | {'precision': 0.6780415430267063, 'recall': 0.8582159624413146, 'f1': 0.7575631993369251, 'number': 1065} | 0.6463 | 0.7737 | 0.7043 | 0.7862 |
| 0.5341 | 7.0 | 70 | 0.6816 | {'precision': 0.6745945945945946, 'recall': 0.7713226205191595, 'f1': 0.7197231833910035, 'number': 809} | {'precision': 0.22727272727272727, 'recall': 0.21008403361344538, 'f1': 0.21834061135371177, 'number': 119} | {'precision': 0.7435037720033529, 'recall': 0.8328638497652582, 'f1': 0.7856510186005314, 'number': 1065} | 0.6894 | 0.7707 | 0.7278 | 0.7920 |
| 0.4811 | 8.0 | 80 | 0.6577 | {'precision': 0.6800870511425462, 'recall': 0.7725587144622992, 'f1': 0.7233796296296297, 'number': 809} | {'precision': 0.2072072072072072, 'recall': 0.19327731092436976, 'f1': 0.2, 'number': 119} | {'precision': 0.7440132122213047, 'recall': 0.8460093896713615, 'f1': 0.7917398945518453, 'number': 1065} | 0.6912 | 0.7772 | 0.7317 | 0.7986 |
| 0.4241 | 9.0 | 90 | 0.6586 | {'precision': 0.6898454746136865, 'recall': 0.7725587144622992, 'f1': 0.7288629737609328, 'number': 809} | {'precision': 0.2535211267605634, 'recall': 0.3025210084033613, 'f1': 0.2758620689655173, 'number': 119} | {'precision': 0.751269035532995, 'recall': 0.8338028169014085, 'f1': 0.7903871829105474, 'number': 1065} | 0.6946 | 0.7772 | 0.7336 | 0.7991 |
| 0.3784 | 10.0 | 100 | 0.6511 | {'precision': 0.6879739978331527, 'recall': 0.7849196538936959, 'f1': 0.7332563510392609, 'number': 809} | {'precision': 0.2833333333333333, 'recall': 0.2857142857142857, 'f1': 0.2845188284518828, 'number': 119} | {'precision': 0.7590870667793744, 'recall': 0.8431924882629108, 'f1': 0.7989323843416369, 'number': 1065} | 0.7040 | 0.7863 | 0.7428 | 0.8046 |
| 0.3425 | 11.0 | 110 | 0.6611 | {'precision': 0.6975982532751092, 'recall': 0.7898640296662547, 'f1': 0.7408695652173912, 'number': 809} | {'precision': 0.26865671641791045, 'recall': 0.3025210084033613, 'f1': 0.2845849802371542, 'number': 119} | {'precision': 0.7737162750217581, 'recall': 0.8347417840375587, 'f1': 0.803071364046974, 'number': 1065} | 0.7112 | 0.7847 | 0.7462 | 0.8116 |
| 0.3225 | 12.0 | 120 | 0.6676 | {'precision': 0.6957470010905126, 'recall': 0.788627935723115, 'f1': 0.7392815758980301, 'number': 809} | {'precision': 0.27941176470588236, 'recall': 0.31932773109243695, 'f1': 0.2980392156862745, 'number': 119} | {'precision': 0.7806167400881058, 'recall': 0.831924882629108, 'f1': 0.8054545454545454, 'number': 1065} | 0.7139 | 0.7837 | 0.7472 | 0.8081 |
| 0.302 | 13.0 | 130 | 0.6698 | {'precision': 0.6956043956043956, 'recall': 0.7824474660074165, 'f1': 0.7364746945898778, 'number': 809} | {'precision': 0.2878787878787879, 'recall': 0.31932773109243695, 'f1': 0.302788844621514, 'number': 119} | {'precision': 0.7730434782608696, 'recall': 0.8347417840375587, 'f1': 0.8027088036117382, 'number': 1065} | 0.7117 | 0.7827 | 0.7455 | 0.8133 |
| 0.2915 | 14.0 | 140 | 0.6845 | {'precision': 0.6978260869565217, 'recall': 0.7935723114956736, 'f1': 0.742625795257374, 'number': 809} | {'precision': 0.2857142857142857, 'recall': 0.31932773109243695, 'f1': 0.30158730158730157, 'number': 119} | {'precision': 0.7771929824561403, 'recall': 0.831924882629108, 'f1': 0.8036281179138323, 'number': 1065} | 0.7141 | 0.7858 | 0.7482 | 0.8052 |
| 0.2872 | 15.0 | 150 | 0.6843 | {'precision': 0.6938997821350763, 'recall': 0.7873918417799752, 'f1': 0.7376954255935148, 'number': 809} | {'precision': 0.27941176470588236, 'recall': 0.31932773109243695, 'f1': 0.2980392156862745, 'number': 119} | {'precision': 0.7749562171628721, 'recall': 0.8309859154929577, 'f1': 0.8019936565473492, 'number': 1065} | 0.7104 | 0.7827 | 0.7448 | 0.8076 |
### Framework versions
- Transformers 4.22.2
- Pytorch 1.12.1+cu113
- Datasets 2.5.2
- Tokenizers 0.12.1
|