  We propose a novel framework for modeling the interaction between graphical
structures and the natural language text associated with their nodes and edges.
Existing approaches typically fall into two categories. On group ignores the
relational structure by converting them into linear sequences and then utilize
the highly successful Seq2Seq models. The other side ignores the sequential
nature of the text by representing them as fixed-dimensional vectors and apply
graph neural networks. Both simplifications lead to information loss.
  Our proposed method utilizes both the graphical structure as well as the
sequential nature of the texts. The input to our model is a set of text
segments associated with the nodes and edges of the graph, which are then
processed with a transformer encoder-decoder model, equipped with a
self-attention mechanism that is aware of the graphical relations between the
nodes containing the segments. This also allows us to use BERT-like models that
are already trained on large amounts of text.
  While the proposed model has wide applications, we demonstrate its
capabilities on data-to-text generation tasks. Our approach compares favorably
against state-of-the-art methods in four tasks without tailoring the model
architecture. We also provide an early demonstration in a novel practical
application -- generating clinical notes from the medical entities mentioned
during clinical visits.

Computation and Language