File size: 9,317 Bytes
8ab0026 3806568 8ab0026 3806568 8ab0026 3806568 8ab0026 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6689
- Answer: {'precision': 0.7029063509149623, 'recall': 0.8071693448702101, 'f1': 0.7514384349827388, 'number': 809}
- Header: {'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119}
- Question: {'precision': 0.7777777777777778, 'recall': 0.828169014084507, 'f1': 0.8021828103683492, 'number': 1065}
- Overall Precision: 0.7209
- Overall Recall: 0.7918
- Overall F1: 0.7547
- Overall Accuracy: 0.8158
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8306 | 1.0 | 10 | 1.6060 | {'precision': 0.026582278481012658, 'recall': 0.02595797280593325, 'f1': 0.026266416510318948, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.21528861154446177, 'recall': 0.1295774647887324, 'f1': 0.16178194607268465, 'number': 1065} | 0.1111 | 0.0798 | 0.0929 | 0.3733 |
| 1.4787 | 2.0 | 20 | 1.2612 | {'precision': 0.20019627085377822, 'recall': 0.2521631644004944, 'f1': 0.22319474835886213, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.419710544452102, 'recall': 0.571830985915493, 'f1': 0.4841017488076311, 'number': 1065} | 0.3291 | 0.4079 | 0.3643 | 0.5976 |
| 1.1115 | 3.0 | 30 | 0.9517 | {'precision': 0.466, 'recall': 0.5760197775030902, 'f1': 0.5152017689331123, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5697115384615384, 'recall': 0.6676056338028169, 'f1': 0.6147859922178989, 'number': 1065} | 0.5201 | 0.5906 | 0.5531 | 0.6834 |
| 0.8531 | 4.0 | 40 | 0.8275 | {'precision': 0.5730337078651685, 'recall': 0.7564894932014833, 'f1': 0.65210442194992, 'number': 809} | {'precision': 0.06521739130434782, 'recall': 0.025210084033613446, 'f1': 0.03636363636363636, 'number': 119} | {'precision': 0.6735751295336787, 'recall': 0.7323943661971831, 'f1': 0.7017543859649122, 'number': 1065} | 0.6140 | 0.6999 | 0.6542 | 0.7393 |
| 0.7059 | 5.0 | 50 | 0.7345 | {'precision': 0.6333687566418703, 'recall': 0.7367119901112484, 'f1': 0.6811428571428572, 'number': 809} | {'precision': 0.2, 'recall': 0.14285714285714285, 'f1': 0.16666666666666666, 'number': 119} | {'precision': 0.6966386554621848, 'recall': 0.7784037558685446, 'f1': 0.7352549889135255, 'number': 1065} | 0.6507 | 0.7235 | 0.6852 | 0.7712 |
| 0.5949 | 6.0 | 60 | 0.6931 | {'precision': 0.6376050420168067, 'recall': 0.7503090234857849, 'f1': 0.689381033503691, 'number': 809} | {'precision': 0.20430107526881722, 'recall': 0.15966386554621848, 'f1': 0.1792452830188679, 'number': 119} | {'precision': 0.6931637519872814, 'recall': 0.8187793427230047, 'f1': 0.7507533362031856, 'number': 1065} | 0.6505 | 0.7516 | 0.6974 | 0.7836 |
| 0.5143 | 7.0 | 70 | 0.6674 | {'precision': 0.6688172043010753, 'recall': 0.7688504326328801, 'f1': 0.7153536515238643, 'number': 809} | {'precision': 0.23478260869565218, 'recall': 0.226890756302521, 'f1': 0.23076923076923078, 'number': 119} | {'precision': 0.7146341463414634, 'recall': 0.8253521126760563, 'f1': 0.7660130718954249, 'number': 1065} | 0.6716 | 0.7667 | 0.7160 | 0.7933 |
| 0.4641 | 8.0 | 80 | 0.6507 | {'precision': 0.667016806722689, 'recall': 0.7849196538936959, 'f1': 0.7211811470755252, 'number': 809} | {'precision': 0.3142857142857143, 'recall': 0.2773109243697479, 'f1': 0.29464285714285715, 'number': 119} | {'precision': 0.7347280334728034, 'recall': 0.8244131455399061, 'f1': 0.7769911504424779, 'number': 1065} | 0.6865 | 0.7757 | 0.7284 | 0.8029 |
| 0.4063 | 9.0 | 90 | 0.6671 | {'precision': 0.6574074074074074, 'recall': 0.7898640296662547, 'f1': 0.7175743964065132, 'number': 809} | {'precision': 0.3114754098360656, 'recall': 0.31932773109243695, 'f1': 0.3153526970954357, 'number': 119} | {'precision': 0.747008547008547, 'recall': 0.8206572769953052, 'f1': 0.782102908277405, 'number': 1065} | 0.6851 | 0.7782 | 0.7287 | 0.8017 |
| 0.3643 | 10.0 | 100 | 0.6603 | {'precision': 0.6851063829787234, 'recall': 0.796044499381953, 'f1': 0.7364208118925099, 'number': 809} | {'precision': 0.3669724770642202, 'recall': 0.33613445378151263, 'f1': 0.3508771929824562, 'number': 119} | {'precision': 0.7674825174825175, 'recall': 0.8244131455399061, 'f1': 0.7949298325033951, 'number': 1065} | 0.7123 | 0.7837 | 0.7463 | 0.8069 |
| 0.3331 | 11.0 | 110 | 0.6691 | {'precision': 0.6928879310344828, 'recall': 0.7948084054388134, 'f1': 0.740356937248129, 'number': 809} | {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119} | {'precision': 0.7666666666666667, 'recall': 0.8206572769953052, 'f1': 0.7927437641723357, 'number': 1065} | 0.7088 | 0.7802 | 0.7428 | 0.8071 |
| 0.3193 | 12.0 | 120 | 0.6597 | {'precision': 0.6932059447983014, 'recall': 0.8071693448702101, 'f1': 0.7458595088520845, 'number': 809} | {'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119} | {'precision': 0.7721739130434783, 'recall': 0.8338028169014085, 'f1': 0.801805869074492, 'number': 1065} | 0.7152 | 0.7938 | 0.7524 | 0.8112 |
| 0.2972 | 13.0 | 130 | 0.6679 | {'precision': 0.7011866235167206, 'recall': 0.8034610630407911, 'f1': 0.7488479262672811, 'number': 809} | {'precision': 0.344, 'recall': 0.36134453781512604, 'f1': 0.3524590163934426, 'number': 119} | {'precision': 0.7716814159292036, 'recall': 0.8187793427230047, 'f1': 0.7945330296127562, 'number': 1065} | 0.7172 | 0.7852 | 0.7497 | 0.8145 |
| 0.2833 | 14.0 | 140 | 0.6684 | {'precision': 0.703023758099352, 'recall': 0.8046971569839307, 'f1': 0.7504322766570604, 'number': 809} | {'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119} | {'precision': 0.7769973661106233, 'recall': 0.8309859154929577, 'f1': 0.8030852994555354, 'number': 1065} | 0.7207 | 0.7923 | 0.7548 | 0.8163 |
| 0.2765 | 15.0 | 150 | 0.6689 | {'precision': 0.7029063509149623, 'recall': 0.8071693448702101, 'f1': 0.7514384349827388, 'number': 809} | {'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119} | {'precision': 0.7777777777777778, 'recall': 0.828169014084507, 'f1': 0.8021828103683492, 'number': 1065} | 0.7209 | 0.7918 | 0.7547 | 0.8158 |
### Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.3
- Tokenizers 0.13.3
|