ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents
<|reference_start|>ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents: Multimodal key information extraction (KIE) models have been studied extensively on semi-structured documents. However, their investigation on unstructured documents is an emerging research topic. The paper presents an approach to adapt a multimodal transformer (i.e., ViBERTgrid previously explored on semi-structured documents) for unstructured financial documents, by incorporating a BiLSTM-CRF layer. The proposed ViBERTgrid BiLSTM-CRF model demonstrates a significant improvement in performance (up to 2 percentage points) on named entity recognition from unstructured documents in financial domain, while maintaining its KIE performance on semi-structured documents. As an additional contribution, we publicly released token-level annotations for the SROIE dataset in order to pave the way for its use in multimodal sequence labeling models.<|reference_end|>
@article{pala2024vibertgrid,
    title={ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from
  Unstructured Financial Documents},
    author={Furkan Pala, Mehmet Yasin Akp{i}nar, Onur Deniz, G"ulc{s}en
  Eryiu{g}it},
    journal={arXiv preprint arXiv:2409.15004},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.15004},
    primaryClass={cs.AI cs.CL cs.CV cs.IR}
}