title: DocGraphLM: Documental Graph Language Model for Information Extraction
summary: Advances in Visually Rich Document Understanding (VrDU) have enabled
  information extraction and question answering over documents with complex
  layouts. Two tropes of architectures have emerged -- transformer-based models
  inspired by LLMs, and Graph Neural Networks. In this paper, we introduce
  DocGraphLM, a novel framework that combines pre-trained language models with
  graph semantics. To achieve this, we propose 1) a joint encoder architecture to
  represent documents, and 2) a novel link prediction approach to reconstruct
  document graphs. DocGraphLM predicts both directions and distances between
  nodes using a convergent joint loss function that prioritizes neighborhood
  restoration and downweighs distant node detection. Our experiments on three
  SotA datasets show consistent improvement on IE and QA tasks with the adoption
  of graph features. Moreover, we report that adopting the graph features
  accelerates convergence in the learning process during training, despite being
  solely constructed through link prediction.
paper_authors._id: 659b6ac6eff07dcf1ff16a0c
paper_authors.name: Dongsheng Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-08T11:37:20.833Z
paper_authors._id: 64863e4e60b30c3615b92812
paper_authors.avatarUrl: /avatars/e4a271b53e567f9423bcd9e62cf77889.svg
paper_authors.fullname: Dongsheng Wang
paper_authors.type: user
paper_authors.user: dswang2011
paper_authors._id: 659b6ac6eff07dcf1ff16a0d
paper_authors.name: Zhiqiang Ma
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-01-08T03:23:50.788Z
paper_authors._id: 64b702e8fa7eabaae5f0cbfb
paper_authors.avatarUrl: /avatars/512690f60997ae81b79b999a634a1861.svg
paper_authors.fullname: Zhiqiang Ma
paper_authors.type: user
paper_authors.user: zqmaster
paper_authors._id: 659b6ac6eff07dcf1ff16a0e
paper_authors.name: Armineh Nourbakhsh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-08T08:47:23.876Z
paper_authors._id: 5e79436f30dc073f817a2b91
paper_authors.avatarUrl: /avatars/5ffac317818689106be1a00ee69cded0.svg
paper_authors.fullname: Armi Nouri
paper_authors.type: user
paper_authors.user: arminouri
paper_authors._id: 659b6ac6eff07dcf1ff16a0f
paper_authors.name: Kang Gu
paper_authors._id: 659b6ac6eff07dcf1ff16a10
paper_authors.name: Sameena Shah
paper_title: DocGraphLM: Documental Graph Language Model for Information Extraction
paper_summary: Advances in Visually Rich Document Understanding (VrDU) have enabled
  information extraction and question answering over documents with complex
  layouts. Two tropes of architectures have emerged -- transformer-based models
  inspired by LLMs, and Graph Neural Networks. In this paper, we introduce
  DocGraphLM, a novel framework that combines pre-trained language models with
  graph semantics. To achieve this, we propose 1) a joint encoder architecture to
  represent documents, and 2) a novel link prediction approach to reconstruct
  document graphs. DocGraphLM predicts both directions and distances between
  nodes using a convergent joint loss function that prioritizes neighborhood
  restoration and downweighs distant node detection. Our experiments on three
  SotA datasets show consistent improvement on IE and QA tasks with the adoption
  of graph features. Moreover, we report that adopting the graph features
  accelerates convergence in the learning process during training, despite being
  solely constructed through link prediction.
paper_ai_keywords: transformer-based models LLMs (Large Language Models) Graph Neural Networks DocGraphLM joint encoder architecture document graphs link prediction approach convergent joint loss function neighborhood restoration distant node detection SotA datasets information extraction (IE) question answering (QA)