  Knowledge-grounded dialogue systems are challenging to build due to the lack
of training data and heterogeneous knowledge sources. Existing systems perform
poorly on unseen topics due to limited topics covered in the training data. In
addition, heterogeneous knowledge sources make it challenging for systems to
generalize to other tasks because knowledge sources in different knowledge
representations require different knowledge encoders. To address these
challenges, we present PLUG, a language model that homogenizes different
knowledge sources to a unified knowledge representation for knowledge-grounded
dialogue generation tasks. PLUG is pre-trained on a dialogue generation task
conditioned on a unified essential knowledge representation. It can generalize
to different downstream knowledge-grounded dialogue generation tasks with a few
training examples. The empirical evaluation on two benchmarks shows that our
model generalizes well across different knowledge-grounded tasks. It can
achieve comparable performance with state-of-the-art methods under a
fully-supervised setting and significantly outperforms other methods in
zero-shot and few-shot settings.

Computation and Language