layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6996
• Answer: {'precision': 0.7310267857142857, 'recall': 0.8096415327564895, 'f1': 0.7683284457478006, 'number': 809}
• Header: {'precision': 0.34328358208955223, 'recall': 0.3865546218487395, 'f1': 0.36363636363636365, 'number': 119}
• Question: {'precision': 0.7695652173913043, 'recall': 0.8309859154929577, 'f1': 0.7990970654627539, 'number': 1065}
• Overall Precision: 0.7275
• Overall Recall: 0.7958
• Overall F1: 0.7601
• Overall Accuracy: 0.8046

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7625	1.0	10	1.5222	{'precision': 0.044633368756641874, 'recall': 0.0519159456118665, 'f1': 0.048, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.24774774774774774, 'recall': 0.25821596244131456, 'f1': 0.25287356321839083, 'number': 1065}	0.1546	0.1591	0.1568	0.4691
1.4019	2.0	20	1.2094	{'precision': 0.14961832061068703, 'recall': 0.1211372064276885, 'f1': 0.13387978142076504, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4884910485933504, 'recall': 0.5380281690140845, 'f1': 0.5120643431635388, 'number': 1065}	0.3671	0.3367	0.3512	0.5863
1.0618	3.0	30	0.9399	{'precision': 0.46959459459459457, 'recall': 0.515451174289246, 'f1': 0.4914555097230407, 'number': 809}	{'precision': 0.08571428571428572, 'recall': 0.025210084033613446, 'f1': 0.03896103896103897, 'number': 119}	{'precision': 0.5851735015772871, 'recall': 0.6967136150234742, 'f1': 0.6360908701243034, 'number': 1065}	0.5304	0.5830	0.5554	0.7161
0.8074	4.0	40	0.7904	{'precision': 0.5879345603271984, 'recall': 0.7107540173053152, 'f1': 0.6435366536094012, 'number': 809}	{'precision': 0.13846153846153847, 'recall': 0.07563025210084033, 'f1': 0.09782608695652173, 'number': 119}	{'precision': 0.6505823627287853, 'recall': 0.7342723004694836, 'f1': 0.6898985443317159, 'number': 1065}	0.6085	0.6854	0.6446	0.7583
0.6495	5.0	50	0.7379	{'precision': 0.6703662597114317, 'recall': 0.7466007416563659, 'f1': 0.7064327485380117, 'number': 809}	{'precision': 0.24509803921568626, 'recall': 0.21008403361344538, 'f1': 0.22624434389140272, 'number': 119}	{'precision': 0.6588145896656535, 'recall': 0.8140845070422535, 'f1': 0.7282654346913062, 'number': 1065}	0.6451	0.7506	0.6939	0.7819
0.5577	6.0	60	0.6976	{'precision': 0.6381909547738693, 'recall': 0.7849196538936959, 'f1': 0.7039911308203992, 'number': 809}	{'precision': 0.2, 'recall': 0.15966386554621848, 'f1': 0.17757009345794392, 'number': 119}	{'precision': 0.7116512992455993, 'recall': 0.7971830985915493, 'f1': 0.7519929140832595, 'number': 1065}	0.6583	0.7541	0.7030	0.7832
0.4844	7.0	70	0.6800	{'precision': 0.6827225130890052, 'recall': 0.8059332509270705, 'f1': 0.7392290249433106, 'number': 809}	{'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119}	{'precision': 0.7390542907180385, 'recall': 0.7924882629107981, 'f1': 0.7648391481649298, 'number': 1065}	0.6908	0.7667	0.7268	0.7927
0.4359	8.0	80	0.6745	{'precision': 0.6886291179596175, 'recall': 0.8009888751545118, 'f1': 0.7405714285714285, 'number': 809}	{'precision': 0.23846153846153847, 'recall': 0.2605042016806723, 'f1': 0.24899598393574301, 'number': 119}	{'precision': 0.7412765957446809, 'recall': 0.8178403755868544, 'f1': 0.7776785714285714, 'number': 1065}	0.6901	0.7777	0.7313	0.7954
0.3775	9.0	90	0.6712	{'precision': 0.7014115092290988, 'recall': 0.7985166872682324, 'f1': 0.746820809248555, 'number': 809}	{'precision': 0.32456140350877194, 'recall': 0.31092436974789917, 'f1': 0.31759656652360513, 'number': 119}	{'precision': 0.745531914893617, 'recall': 0.8225352112676056, 'f1': 0.7821428571428571, 'number': 1065}	0.7054	0.7822	0.7419	0.7999
0.374	10.0	100	0.6731	{'precision': 0.7161716171617162, 'recall': 0.8046971569839307, 'f1': 0.7578579743888243, 'number': 809}	{'precision': 0.3135593220338983, 'recall': 0.31092436974789917, 'f1': 0.31223628691983124, 'number': 119}	{'precision': 0.7558644656820156, 'recall': 0.8169014084507042, 'f1': 0.7851985559566786, 'number': 1065}	0.7153	0.7817	0.7471	0.8028
0.3142	11.0	110	0.6817	{'precision': 0.7205720572057206, 'recall': 0.8096415327564895, 'f1': 0.7625145518044238, 'number': 809}	{'precision': 0.302158273381295, 'recall': 0.35294117647058826, 'f1': 0.3255813953488373, 'number': 119}	{'precision': 0.7670405522001725, 'recall': 0.8347417840375587, 'f1': 0.7994604316546762, 'number': 1065}	0.7186	0.7958	0.7552	0.8042
0.2936	12.0	120	0.6858	{'precision': 0.7281767955801105, 'recall': 0.8145859085290482, 'f1': 0.7689614935822637, 'number': 809}	{'precision': 0.3442622950819672, 'recall': 0.35294117647058826, 'f1': 0.3485477178423237, 'number': 119}	{'precision': 0.7717770034843205, 'recall': 0.831924882629108, 'f1': 0.8007230004518754, 'number': 1065}	0.7297	0.7963	0.7615	0.8077
0.2783	13.0	130	0.6950	{'precision': 0.7270718232044199, 'recall': 0.8133498145859085, 'f1': 0.7677946324387398, 'number': 809}	{'precision': 0.33076923076923076, 'recall': 0.36134453781512604, 'f1': 0.34538152610441764, 'number': 119}	{'precision': 0.7693661971830986, 'recall': 0.8206572769953052, 'f1': 0.79418446160836, 'number': 1065}	0.7255	0.7903	0.7565	0.8045
0.258	14.0	140	0.6993	{'precision': 0.7293986636971047, 'recall': 0.8096415327564895, 'f1': 0.767428236672525, 'number': 809}	{'precision': 0.3409090909090909, 'recall': 0.37815126050420167, 'f1': 0.3585657370517928, 'number': 119}	{'precision': 0.7655709342560554, 'recall': 0.8309859154929577, 'f1': 0.7969383160738407, 'number': 1065}	0.7251	0.7953	0.7586	0.8039
0.2615	15.0	150	0.6996	{'precision': 0.7310267857142857, 'recall': 0.8096415327564895, 'f1': 0.7683284457478006, 'number': 809}	{'precision': 0.34328358208955223, 'recall': 0.3865546218487395, 'f1': 0.36363636363636365, 'number': 119}	{'precision': 0.7695652173913043, 'recall': 0.8309859154929577, 'f1': 0.7990970654627539, 'number': 1065}	0.7275	0.7958	0.7601	0.8046

Framework versions

• Transformers 4.50.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0