Hugging Face's logo

pabloma09
/
layoutlm-funsd

Token Classification
Transformers
TensorBoard
Safetensors
layoutlm
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
pabloma09 commited on
Commit
3270da6
·
verified ·
1 Parent(s): 37da2d9

End of training

Browse files
Files changed (5) hide show
  1. README.md +29 -29
  2. logs/events.out.tfevents.1710521086.2e14fe2784b5.604.0 +2 -2
  3. model.safetensors +1 -1
  4. preprocessor_config.json +0 -1
  5. tokenizer.json +16 -2
README.md CHANGED
@@ -17,14 +17,14 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 0.7065
21
- - Answer: {'precision': 0.7120708748615725, 'recall': 0.7948084054388134, 'f1': 0.7511682242990654, 'number': 809}
22
- - Header: {'precision': 0.31746031746031744, 'recall': 0.33613445378151263, 'f1': 0.32653061224489793, 'number': 119}
23
- - Question: {'precision': 0.7776809067131648, 'recall': 0.8375586854460094, 'f1': 0.8065099457504521, 'number': 1065}
24
- - Overall Precision: 0.7238
25
- - Overall Recall: 0.7903
26
- - Overall F1: 0.7556
27
- - Overall Accuracy: 0.7999
28
 
29
  ## Model description
30
 
@@ -54,28 +54,28 @@ The following hyperparameters were used during training:
54
 
55
  ### Training results
56
 
57
- | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
58
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
59
- | 1.7682 | 1.0 | 10 | 1.5613 | {'precision': 0.029469548133595286, 'recall': 0.037082818294190356, 'f1': 0.03284072249589491, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.14761904761904762, 'recall': 0.14553990610328638, 'f1': 0.14657210401891252, 'number': 1065} | 0.0895 | 0.0928 | 0.0911 | 0.3814 |
60
- | 1.4106 | 2.0 | 20 | 1.2321 | {'precision': 0.1530120481927711, 'recall': 0.15698393077873918, 'f1': 0.1549725442342892, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4595959595959596, 'recall': 0.5981220657276995, 'f1': 0.5197878416972664, 'number': 1065} | 0.3448 | 0.3833 | 0.3630 | 0.5884 |
61
- | 1.0722 | 3.0 | 30 | 0.9239 | {'precision': 0.45864661654135336, 'recall': 0.5278121137206427, 'f1': 0.4908045977011494, 'number': 809} | {'precision': 0.034482758620689655, 'recall': 0.008403361344537815, 'f1': 0.013513513513513513, 'number': 119} | {'precision': 0.6172006745362564, 'recall': 0.6873239436619718, 'f1': 0.6503776099511329, 'number': 1065} | 0.5405 | 0.5820 | 0.5605 | 0.7059 |
62
- | 0.813 | 4.0 | 40 | 0.7588 | {'precision': 0.6421404682274248, 'recall': 0.7119901112484549, 'f1': 0.675263774912075, 'number': 809} | {'precision': 0.17543859649122806, 'recall': 0.08403361344537816, 'f1': 0.11363636363636363, 'number': 119} | {'precision': 0.6691236691236692, 'recall': 0.7671361502347418, 'f1': 0.7147856517935257, 'number': 1065} | 0.6451 | 0.7040 | 0.6732 | 0.7672 |
63
- | 0.6435 | 5.0 | 50 | 0.6978 | {'precision': 0.6810631229235881, 'recall': 0.7601977750309024, 'f1': 0.7184579439252335, 'number': 809} | {'precision': 0.33766233766233766, 'recall': 0.2184873949579832, 'f1': 0.2653061224489796, 'number': 119} | {'precision': 0.6998394863563403, 'recall': 0.8187793427230047, 'f1': 0.7546516659454782, 'number': 1065} | 0.6797 | 0.7592 | 0.7172 | 0.7834 |
64
- | 0.5577 | 6.0 | 60 | 0.6803 | {'precision': 0.6797385620915033, 'recall': 0.7713226205191595, 'f1': 0.7226404169079328, 'number': 809} | {'precision': 0.31645569620253167, 'recall': 0.21008403361344538, 'f1': 0.25252525252525254, 'number': 119} | {'precision': 0.7280334728033473, 'recall': 0.8169014084507042, 'f1': 0.7699115044247787, 'number': 1065} | 0.6930 | 0.7622 | 0.7259 | 0.7920 |
65
- | 0.4844 | 7.0 | 70 | 0.6678 | {'precision': 0.6868798235942668, 'recall': 0.7700865265760197, 'f1': 0.726107226107226, 'number': 809} | {'precision': 0.3157894736842105, 'recall': 0.25210084033613445, 'f1': 0.2803738317757009, 'number': 119} | {'precision': 0.7511032656663724, 'recall': 0.7990610328638498, 'f1': 0.7743403093721565, 'number': 1065} | 0.7044 | 0.7546 | 0.7287 | 0.7968 |
66
- | 0.4396 | 8.0 | 80 | 0.6579 | {'precision': 0.7017543859649122, 'recall': 0.7911001236093943, 'f1': 0.7437536316095292, 'number': 809} | {'precision': 0.30927835051546393, 'recall': 0.25210084033613445, 'f1': 0.2777777777777778, 'number': 119} | {'precision': 0.7595486111111112, 'recall': 0.8215962441314554, 'f1': 0.7893549842129003, 'number': 1065} | 0.7149 | 0.7752 | 0.7439 | 0.8041 |
67
- | 0.3704 | 9.0 | 90 | 0.6669 | {'precision': 0.7118834080717489, 'recall': 0.7849196538936959, 'f1': 0.7466196355085244, 'number': 809} | {'precision': 0.3064516129032258, 'recall': 0.31932773109243695, 'f1': 0.31275720164609055, 'number': 119} | {'precision': 0.7565217391304347, 'recall': 0.8169014084507042, 'f1': 0.7855530474040633, 'number': 1065} | 0.7124 | 0.7742 | 0.7420 | 0.7986 |
68
- | 0.3414 | 10.0 | 100 | 0.6865 | {'precision': 0.7039911308203991, 'recall': 0.7849196538936959, 'f1': 0.7422559906487435, 'number': 809} | {'precision': 0.308411214953271, 'recall': 0.2773109243697479, 'f1': 0.29203539823008845, 'number': 119} | {'precision': 0.7589743589743589, 'recall': 0.8338028169014085, 'f1': 0.7946308724832215, 'number': 1065} | 0.7141 | 0.7807 | 0.7459 | 0.7986 |
69
- | 0.3189 | 11.0 | 110 | 0.6830 | {'precision': 0.7093922651933702, 'recall': 0.7935723114956736, 'f1': 0.7491248541423571, 'number': 809} | {'precision': 0.2972972972972973, 'recall': 0.2773109243697479, 'f1': 0.28695652173913044, 'number': 119} | {'precision': 0.7759078830823738, 'recall': 0.8225352112676056, 'f1': 0.7985414767547857, 'number': 1065} | 0.7231 | 0.7782 | 0.7496 | 0.8025 |
70
- | 0.2921 | 12.0 | 120 | 0.6976 | {'precision': 0.7194163860830527, 'recall': 0.792336217552534, 'f1': 0.7541176470588233, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.36134453781512604, 'f1': 0.34677419354838707, 'number': 119} | {'precision': 0.7757417102966842, 'recall': 0.8347417840375587, 'f1': 0.8041610131162371, 'number': 1065} | 0.7262 | 0.7893 | 0.7564 | 0.7987 |
71
- | 0.2775 | 13.0 | 130 | 0.6988 | {'precision': 0.7120708748615725, 'recall': 0.7948084054388134, 'f1': 0.7511682242990654, 'number': 809} | {'precision': 0.34234234234234234, 'recall': 0.31932773109243695, 'f1': 0.33043478260869563, 'number': 119} | {'precision': 0.7779735682819383, 'recall': 0.8291079812206573, 'f1': 0.8027272727272727, 'number': 1065} | 0.7278 | 0.7847 | 0.7552 | 0.8008 |
72
- | 0.2632 | 14.0 | 140 | 0.7080 | {'precision': 0.7189249720044792, 'recall': 0.7935723114956736, 'f1': 0.7544065804935369, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.3445378151260504, 'f1': 0.33884297520661155, 'number': 119} | {'precision': 0.7769028871391076, 'recall': 0.8338028169014085, 'f1': 0.8043478260869565, 'number': 1065} | 0.7277 | 0.7883 | 0.7567 | 0.8005 |
73
- | 0.2666 | 15.0 | 150 | 0.7065 | {'precision': 0.7120708748615725, 'recall': 0.7948084054388134, 'f1': 0.7511682242990654, 'number': 809} | {'precision': 0.31746031746031744, 'recall': 0.33613445378151263, 'f1': 0.32653061224489793, 'number': 119} | {'precision': 0.7776809067131648, 'recall': 0.8375586854460094, 'f1': 0.8065099457504521, 'number': 1065} | 0.7238 | 0.7903 | 0.7556 | 0.7999 |
74
 
75
 
76
  ### Framework versions
77
 
78
- - Transformers 4.36.2
79
- - Pytorch 2.1.2
80
- - Datasets 2.15.0
81
- - Tokenizers 0.15.0
 
17
 
18
  This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.0784
21
+ - Answer: {'precision': 0.39729990356798456, 'recall': 0.5092707045735476, 'f1': 0.44637053087757317, 'number': 809}
22
+ - Header: {'precision': 0.2601626016260163, 'recall': 0.2689075630252101, 'f1': 0.2644628099173554, 'number': 119}
23
+ - Question: {'precision': 0.5115384615384615, 'recall': 0.6244131455399061, 'f1': 0.5623678646934461, 'number': 1065}
24
+ - Overall Precision: 0.4508
25
+ - Overall Recall: 0.5564
26
+ - Overall F1: 0.4981
27
+ - Overall Accuracy: 0.6275
28
 
29
  ## Model description
30
 
 
54
 
55
  ### Training results
56
 
57
+ | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
58
+ |:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
59
+ | 1.7434 | 1.0 | 10 | 1.5471 | {'precision': 0.05161290322580645, 'recall': 0.03955500618046971, 'f1': 0.04478656403079076, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.27050359712230215, 'recall': 0.17652582159624414, 'f1': 0.21363636363636365, 'number': 1065} | 0.1673 | 0.1104 | 0.1330 | 0.3331 |
60
+ | 1.4356 | 2.0 | 20 | 1.3534 | {'precision': 0.21909424724602203, 'recall': 0.44252163164400493, 'f1': 0.29308227589029884, 'number': 809} | {'precision': 0.04411764705882353, 'recall': 0.025210084033613446, 'f1': 0.0320855614973262, 'number': 119} | {'precision': 0.2876712328767123, 'recall': 0.39436619718309857, 'f1': 0.33267326732673264, 'number': 1065} | 0.2470 | 0.3919 | 0.3030 | 0.4281 |
61
+ | 1.2677 | 3.0 | 30 | 1.2046 | {'precision': 0.24696645253390434, 'recall': 0.4276885043263288, 'f1': 0.31312217194570136, 'number': 809} | {'precision': 0.20652173913043478, 'recall': 0.15966386554621848, 'f1': 0.1800947867298578, 'number': 119} | {'precision': 0.3335304553518628, 'recall': 0.5295774647887324, 'f1': 0.40928882438316405, 'number': 1065} | 0.2918 | 0.4661 | 0.3589 | 0.4856 |
62
+ | 1.1386 | 4.0 | 40 | 1.1240 | {'precision': 0.28816326530612246, 'recall': 0.4363411619283066, 'f1': 0.34709931170108166, 'number': 809} | {'precision': 0.1875, 'recall': 0.17647058823529413, 'f1': 0.1818181818181818, 'number': 119} | {'precision': 0.3801391524351676, 'recall': 0.564319248826291, 'f1': 0.45427059712774, 'number': 1065} | 0.3341 | 0.4892 | 0.3971 | 0.5567 |
63
+ | 1.0425 | 5.0 | 50 | 1.0865 | {'precision': 0.31069609507640067, 'recall': 0.45241038318912236, 'f1': 0.3683945646703573, 'number': 809} | {'precision': 0.25609756097560976, 'recall': 0.17647058823529413, 'f1': 0.208955223880597, 'number': 119} | {'precision': 0.41022364217252394, 'recall': 0.6028169014084507, 'f1': 0.4882129277566539, 'number': 1065} | 0.3642 | 0.5163 | 0.4271 | 0.5740 |
64
+ | 1.0051 | 6.0 | 60 | 1.0745 | {'precision': 0.3435185185185185, 'recall': 0.45859085290482077, 'f1': 0.39280042350449973, 'number': 809} | {'precision': 0.22448979591836735, 'recall': 0.18487394957983194, 'f1': 0.20276497695852533, 'number': 119} | {'precision': 0.48720066061106526, 'recall': 0.5539906103286385, 'f1': 0.5184534270650263, 'number': 1065} | 0.4115 | 0.4932 | 0.4487 | 0.5916 |
65
+ | 0.9533 | 7.0 | 70 | 1.0560 | {'precision': 0.329126213592233, 'recall': 0.41903584672435107, 'f1': 0.36867862969004894, 'number': 809} | {'precision': 0.22950819672131148, 'recall': 0.23529411764705882, 'f1': 0.23236514522821577, 'number': 119} | {'precision': 0.41502463054187194, 'recall': 0.6328638497652582, 'f1': 0.5013015991074748, 'number': 1065} | 0.375 | 0.5223 | 0.4366 | 0.5919 |
66
+ | 0.8838 | 8.0 | 80 | 1.0296 | {'precision': 0.3531047265987025, 'recall': 0.47095179233621753, 'f1': 0.4036016949152542, 'number': 809} | {'precision': 0.211864406779661, 'recall': 0.21008403361344538, 'f1': 0.2109704641350211, 'number': 119} | {'precision': 0.45523941707147814, 'recall': 0.615962441314554, 'f1': 0.5235434956105347, 'number': 1065} | 0.4026 | 0.5329 | 0.4586 | 0.6141 |
67
+ | 0.8148 | 9.0 | 90 | 1.0582 | {'precision': 0.38949454905847375, 'recall': 0.4857849196538937, 'f1': 0.43234323432343236, 'number': 809} | {'precision': 0.2571428571428571, 'recall': 0.226890756302521, 'f1': 0.24107142857142855, 'number': 119} | {'precision': 0.5230125523012552, 'recall': 0.5868544600938967, 'f1': 0.5530973451327434, 'number': 1065} | 0.4526 | 0.5243 | 0.4858 | 0.6139 |
68
+ | 0.8139 | 10.0 | 100 | 1.0429 | {'precision': 0.37296260786193675, 'recall': 0.48084054388133496, 'f1': 0.42008639308855295, 'number': 809} | {'precision': 0.24786324786324787, 'recall': 0.24369747899159663, 'f1': 0.24576271186440676, 'number': 119} | {'precision': 0.46943078004216443, 'recall': 0.6272300469483568, 'f1': 0.5369774919614148, 'number': 1065} | 0.4204 | 0.5449 | 0.4747 | 0.6247 |
69
+ | 0.7228 | 11.0 | 110 | 1.0542 | {'precision': 0.38454106280193234, 'recall': 0.4919653893695921, 'f1': 0.4316702819956616, 'number': 809} | {'precision': 0.2702702702702703, 'recall': 0.25210084033613445, 'f1': 0.2608695652173913, 'number': 119} | {'precision': 0.5042536736272235, 'recall': 0.612206572769953, 'f1': 0.5530110262934691, 'number': 1065} | 0.4428 | 0.5419 | 0.4874 | 0.6257 |
70
+ | 0.7193 | 12.0 | 120 | 1.0835 | {'precision': 0.3971563981042654, 'recall': 0.5179233621755254, 'f1': 0.4495708154506438, 'number': 809} | {'precision': 0.26126126126126126, 'recall': 0.24369747899159663, 'f1': 0.25217391304347825, 'number': 119} | {'precision': 0.5153664302600472, 'recall': 0.6140845070422535, 'f1': 0.5604113110539846, 'number': 1065} | 0.4526 | 0.5529 | 0.4977 | 0.6268 |
71
+ | 0.687 | 13.0 | 130 | 1.0892 | {'precision': 0.4001823154056518, 'recall': 0.5426452410383189, 'f1': 0.4606505771248688, 'number': 809} | {'precision': 0.25742574257425743, 'recall': 0.2184873949579832, 'f1': 0.23636363636363636, 'number': 119} | {'precision': 0.5263157894736842, 'recall': 0.5915492957746479, 'f1': 0.5570291777188329, 'number': 1065} | 0.4572 | 0.5494 | 0.4991 | 0.6255 |
72
+ | 0.6515 | 14.0 | 140 | 1.0795 | {'precision': 0.398635477582846, 'recall': 0.5055624227441285, 'f1': 0.4457765667574932, 'number': 809} | {'precision': 0.25862068965517243, 'recall': 0.25210084033613445, 'f1': 0.25531914893617025, 'number': 119} | {'precision': 0.5205047318611987, 'recall': 0.6197183098591549, 'f1': 0.5657951135876553, 'number': 1065} | 0.4560 | 0.5514 | 0.4992 | 0.6262 |
73
+ | 0.6453 | 15.0 | 150 | 1.0784 | {'precision': 0.39729990356798456, 'recall': 0.5092707045735476, 'f1': 0.44637053087757317, 'number': 809} | {'precision': 0.2601626016260163, 'recall': 0.2689075630252101, 'f1': 0.2644628099173554, 'number': 119} | {'precision': 0.5115384615384615, 'recall': 0.6244131455399061, 'f1': 0.5623678646934461, 'number': 1065} | 0.4508 | 0.5564 | 0.4981 | 0.6275 |
74
 
75
 
76
  ### Framework versions
77
 
78
+ - Transformers 4.38.2
79
+ - Pytorch 2.2.1+cu121
80
+ - Datasets 2.18.0
81
+ - Tokenizers 0.15.2
logs/events.out.tfevents.1710521086.2e14fe2784b5.604.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6e5007c18203ec2c42916934f1a2fe612fe30ce43affd73afa49ce53a58a9b55
3
- size 13954
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a632109c0df3338e4db1a11b57590ebc7a4122c1ef06717e121c9f22ebb9fd7e
3
+ size 15738
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e2806276e5e109b99ed94a5d4ea572581bcb05075765c6664000d12967a553e9
3
  size 450558212
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ee163a2d0c3b03473702e54816ce63715412bc5d8b1b45949f88d973c05cf57
3
  size 450558212
preprocessor_config.json CHANGED
@@ -1,7 +1,6 @@
1
  {
2
  "apply_ocr": true,
3
  "do_resize": true,
4
- "feature_extractor_type": "LayoutLMv2FeatureExtractor",
5
  "image_processor_type": "LayoutLMv2ImageProcessor",
6
  "ocr_lang": null,
7
  "processor_class": "LayoutLMv2Processor",
 
1
  {
2
  "apply_ocr": true,
3
  "do_resize": true,
 
4
  "image_processor_type": "LayoutLMv2ImageProcessor",
5
  "ocr_lang": null,
6
  "processor_class": "LayoutLMv2Processor",
tokenizer.json CHANGED
@@ -1,7 +1,21 @@
1
  {
2
  "version": "1.0",
3
- "truncation": null,
4
- "padding": null,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  "added_tokens": [
6
  {
7
  "id": 0,
 
1
  {
2
  "version": "1.0",
3
+ "truncation": {
4
+ "direction": "Right",
5
+ "max_length": 512,
6
+ "strategy": "LongestFirst",
7
+ "stride": 0
8
+ },
9
+ "padding": {
10
+ "strategy": {
11
+ "Fixed": 512
12
+ },
13
+ "direction": "Right",
14
+ "pad_to_multiple_of": null,
15
+ "pad_id": 0,
16
+ "pad_type_id": 0,
17
+ "pad_token": "[PAD]"
18
+ },
19
  "added_tokens": [
20
  {
21
  "id": 0,