File size: 9,282 Bytes
8ab0026 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6801
- Answer: {'precision': 0.6998916576381365, 'recall': 0.7985166872682324, 'f1': 0.745958429561201, 'number': 809}
- Header: {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119}
- Question: {'precision': 0.7674216027874564, 'recall': 0.8272300469483568, 'f1': 0.7962042476276546, 'number': 1065}
- Overall Precision: 0.7123
- Overall Recall: 0.7852
- Overall F1: 0.7470
- Overall Accuracy: 0.8102
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8269 | 1.0 | 10 | 1.5983 | {'precision': 0.0274949083503055, 'recall': 0.03337453646477132, 'f1': 0.03015075376884422, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.10761154855643044, 'recall': 0.07699530516431925, 'f1': 0.08976464148877941, 'number': 1065} | 0.0625 | 0.0547 | 0.0583 | 0.3800 |
| 1.4969 | 2.0 | 20 | 1.3148 | {'precision': 0.14439140811455847, 'recall': 0.14956736711990112, 'f1': 0.14693381906496664, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3501683501683502, 'recall': 0.39061032863849765, 'f1': 0.36928539724811366, 'number': 1065} | 0.2651 | 0.2694 | 0.2672 | 0.5643 |
| 1.1839 | 3.0 | 30 | 1.0447 | {'precision': 0.40492957746478875, 'recall': 0.4264524103831891, 'f1': 0.41541240216736913, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5461672473867596, 'recall': 0.5887323943661972, 'f1': 0.5666516041572526, 'number': 1065} | 0.486 | 0.4877 | 0.4869 | 0.6655 |
| 0.9261 | 4.0 | 40 | 0.8452 | {'precision': 0.5813715455475946, 'recall': 0.7021013597033374, 'f1': 0.6360582306830906, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6362068965517241, 'recall': 0.6929577464788732, 'f1': 0.6633707865168539, 'number': 1065} | 0.6035 | 0.6553 | 0.6283 | 0.7335 |
| 0.7296 | 5.0 | 50 | 0.7494 | {'precision': 0.6223849372384938, 'recall': 0.7354758961681088, 'f1': 0.6742209631728047, 'number': 809} | {'precision': 0.16666666666666666, 'recall': 0.06722689075630252, 'f1': 0.09580838323353293, 'number': 119} | {'precision': 0.6773356401384083, 'recall': 0.7352112676056338, 'f1': 0.7050877982890591, 'number': 1065} | 0.6417 | 0.6954 | 0.6675 | 0.7676 |
| 0.6079 | 6.0 | 60 | 0.6998 | {'precision': 0.629399585921325, 'recall': 0.7515451174289246, 'f1': 0.6850704225352113, 'number': 809} | {'precision': 0.12222222222222222, 'recall': 0.09243697478991597, 'f1': 0.10526315789473684, 'number': 119} | {'precision': 0.6630686198920586, 'recall': 0.8075117370892019, 'f1': 0.7281964436917865, 'number': 1065} | 0.6286 | 0.7421 | 0.6806 | 0.7881 |
| 0.5267 | 7.0 | 70 | 0.6732 | {'precision': 0.6366427840327533, 'recall': 0.7688504326328801, 'f1': 0.696528555431131, 'number': 809} | {'precision': 0.20652173913043478, 'recall': 0.15966386554621848, 'f1': 0.1800947867298578, 'number': 119} | {'precision': 0.7107296137339055, 'recall': 0.7774647887323943, 'f1': 0.7426008968609865, 'number': 1065} | 0.6576 | 0.7371 | 0.6951 | 0.7864 |
| 0.4762 | 8.0 | 80 | 0.6606 | {'precision': 0.6577319587628866, 'recall': 0.788627935723115, 'f1': 0.71725688589095, 'number': 809} | {'precision': 0.25663716814159293, 'recall': 0.24369747899159663, 'f1': 0.25, 'number': 119} | {'precision': 0.731665228645384, 'recall': 0.7962441314553991, 'f1': 0.762589928057554, 'number': 1065} | 0.6757 | 0.7602 | 0.7155 | 0.7936 |
| 0.4175 | 9.0 | 90 | 0.6566 | {'precision': 0.6815761448349308, 'recall': 0.7911001236093943, 'f1': 0.7322654462242563, 'number': 809} | {'precision': 0.25892857142857145, 'recall': 0.24369747899159663, 'f1': 0.2510822510822511, 'number': 119} | {'precision': 0.7513089005235603, 'recall': 0.8084507042253521, 'f1': 0.7788331071913162, 'number': 1065} | 0.6964 | 0.7677 | 0.7303 | 0.8021 |
| 0.374 | 10.0 | 100 | 0.6592 | {'precision': 0.6956521739130435, 'recall': 0.7911001236093943, 'f1': 0.7403123192596877, 'number': 809} | {'precision': 0.319672131147541, 'recall': 0.3277310924369748, 'f1': 0.32365145228215775, 'number': 119} | {'precision': 0.7637931034482759, 'recall': 0.831924882629108, 'f1': 0.7964044943820225, 'number': 1065} | 0.7107 | 0.7852 | 0.7461 | 0.8070 |
| 0.3406 | 11.0 | 110 | 0.6666 | {'precision': 0.7, 'recall': 0.796044499381953, 'f1': 0.7449392712550607, 'number': 809} | {'precision': 0.3305084745762712, 'recall': 0.3277310924369748, 'f1': 0.32911392405063294, 'number': 119} | {'precision': 0.7656387665198238, 'recall': 0.815962441314554, 'f1': 0.79, 'number': 1065} | 0.7142 | 0.7787 | 0.7451 | 0.8071 |
| 0.332 | 12.0 | 120 | 0.6704 | {'precision': 0.6941798941798942, 'recall': 0.8108776266996292, 'f1': 0.7480045610034207, 'number': 809} | {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119} | {'precision': 0.7660311958405546, 'recall': 0.8300469483568075, 'f1': 0.7967552951780081, 'number': 1065} | 0.7118 | 0.7918 | 0.7496 | 0.8078 |
| 0.3061 | 13.0 | 130 | 0.6787 | {'precision': 0.6908108108108109, 'recall': 0.7898640296662547, 'f1': 0.7370242214532873, 'number': 809} | {'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119} | {'precision': 0.7669305189094108, 'recall': 0.8187793427230047, 'f1': 0.7920072661217076, 'number': 1065} | 0.7093 | 0.7787 | 0.7424 | 0.8091 |
| 0.2879 | 14.0 | 140 | 0.6781 | {'precision': 0.6951871657754011, 'recall': 0.8034610630407911, 'f1': 0.7454128440366973, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.33613445378151263, 'f1': 0.33472803347280333, 'number': 119} | {'precision': 0.7746478873239436, 'recall': 0.8262910798122066, 'f1': 0.7996365288505224, 'number': 1065} | 0.7166 | 0.7878 | 0.7505 | 0.8099 |
| 0.2831 | 15.0 | 150 | 0.6801 | {'precision': 0.6998916576381365, 'recall': 0.7985166872682324, 'f1': 0.745958429561201, 'number': 809} | {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119} | {'precision': 0.7674216027874564, 'recall': 0.8272300469483568, 'f1': 0.7962042476276546, 'number': 1065} | 0.7123 | 0.7852 | 0.7470 | 0.8102 |
### Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.2
- Tokenizers 0.13.3
|