Classifying the core textual components of a scientific paper-title, author, body text, etc.-is a critical first step in automated scientific document understanding. Previous work has shown how using elementary layout information, i.e., each token's 2D position on the page, leads to more accurate classification. We introduce new methods for incorporating VIsual LAyout structures (VILA)  , e.g., the grouping of page texts into text lines or text blocks, into language models to further improve performance. We show that the I-VILA approach, which simply adds special tokens denoting boundaries between layout structures into model inputs, can lead to +1~4.5 F1 Score improvements in token classificationtasks . Moreover, we design a hierarchical model  H-VILA that encodes these layout structures and record a up-to 70\% efficiency boost without hurting prediction accuracy. The experiments are conducted on a newly curated evaluation suite, S2-VLUE, with a novel metric measuring VILA awareness and a new dataset covering 19 scientific disciplines with gold annotations . Pre-trained weights, benchmark datasets, and source code will be available at URL
Classifying the core textual components of a scientific paper-title, author, body text, etc.-is a critical first step in automated scientific document understanding. Previous work has shown how using elementary layout information, i.e., each token's 2D position on the page, leads to more accurate classification. We introduce new methods for incorporating VIsual LAyout (VILA) structures , e.g., the grouping of page texts into text lines or text blocks, into language models to further improve performance. We show that the I-VILA approach, which simply adds special tokens denoting the boundaries of layout structures into model inputs, can lead to 1.9\% Macro F1 improvements for token classification . Moreover, we design a hierarchical model , H-VILA , that encodes the text based on layout structures and record an up-to 47\% inference time reduction with less than 1.5\% Macro F1 loss for the text classification models. Experiments are conducted on a newly curated evaluation suite, S2-VLUE, with a novel metric measuring classification uniformity within visual groups and a new dataset of gold annotations covering papers from 19 scientific disciplines . Pre-trained weights, benchmark datasets, and source code will be available at URL
D
structures
A
structures
R
boundaries between
the boundaries of
R
+1~4.5
1.9\% Macro
R
Score improvements in token classificationtasks
improvements for token classification
A
,
R
that encodes these
, that encodes the text based on
R
a
an
R
70\% efficiency boost without hurting prediction accuracy. The experiments
47\% inference time reduction with less than 1.5\% Macro F1 loss for the text classification models. Experiments
R
VILA awareness
classification uniformity within visual groups
R
covering
of gold annotations covering papers from
D
with gold annotations