Self Attention Architectures for Answer Agnostic Neural Question Generation
Neural architectures based on self attention, such as Transformers, recently attracted interest from the research community, and obtained significant improvements over the state of the art in several tasks. We explore how Transformers can be adapted to the task of Neural Question Generation without constraining the model to focus on a specific answer passage. We study the effect of several strategies to deal with out of vocabulary words such as copy mechanisms, placeholders, and contextual word embeddings. We report improvements obtained over the state of the art on the SQuAD dataset according to automated metrics (BLEU, ROUGE) as well as qualitative human assessments of the system outputs.
ACL