GENERATIVE QUESTION ANSWERING: LEARNING TO ANSWER THE WHOLE QUESTION
Discriminative question answering models can overfit to superficial biases in datasets, because their loss function saturates when any clue makes the answer likely. We introduce generative models of the joint distribution of questions and answers, which are trained to explain the whole question, not just to answer it. Our question answering (QA) model is implemented by learning a prior over answers, and a conditional language model to generate the question given the answerallowing scalable and interpretable many-hop reasoning as the question is generated word-by-word. Our model achieves competitive performance with comparable discriminative models on the SQUAD and CLEVR benchmarks, indicating that it is a more general architecture for language understanding and reasoning than previous work. The model greatly improves generalisation both from biased training data and to adversarial testing data, achieving state-of-the-art results on ADVERSARIALSQUAD.
This paper proposes a generative approach to textual QA on SQUAD and visual QA on CLEVR dataset, where, a joint distribution over the question and answer space, given the context (image or Wikipedia paragraphs) is learned (p(q,a|c)). During inference the answer is selected by argmax p(q,a|c) that is equal to p(a|c,q) if the question is given. Authors propose an architecture shown in Fig. 3 of the paper, where generation of each question word is condition on the corresponding answer, context and all the previous words generated in the question so far. The results compared to discriminative models are worse on SQUAD and CLEVR. Nevertheless, authors show that given the nature of the model that captures more complex relationships, the proposed model performs better than other models on a subset of SQUAD that they have created based on answer type (number/date/people), and also on adversarial SQUAD. 
Comments / questions:
The paper is well written, except for a few parts mentioned below, all the equations / components are explained clearly. The motivation of the paper is clearly stated as using generative modelling in (V)QA to overcome biases in these systems, e.g., answering questions by just using word matching and ignoring the context (context=image or Wikipedia paragraph). I have the following questions / comments about the paper which addressing them by authors will help to better understand/evaluate the paper:
1.	In page 3 on the top of section 2.3, can authors provide a more clear explanation of the additional 32-dimensional embedding added to each word representation? Also in Table 2, please add an ablation how much gain are you getting from this?
2.	In the same page (page 3), section 2.4, paragraph 2, put the equation in a separate line and number it + clearly explain how you have calculated s^{endpoints} and s{length}.
3.	In page 4 section 2.5.2 paragraph 2, the way the bias term is calculated and the incentive behind it is not clear. Can authors elaborate on this?
4.	In page 6 section 3.2 the first paragraph authors claim that their model is performing multihop reasoning on CLEVR, while there is no explicit component in their model to perform multiple rounds of reasoning. Can authors clarify their statement? 
5.	In section 3.3 the third paragraph, where authors explain the question agnostic baselines, can they clarify what they mean by “the first answer of the correct type”? 
6.	In Table 5 and section 3.4 the second paragraph, authors are stating that “… The improvement may be due to the model’s attempt to explain all question words, some of which may be unlikely under the distractor”. It is very important that the authors do a complete ablation study similar to that of Table 2 to clarify how much gain is achieved using each component of generative model. 
7.	In page 8 under related works: 
a.	In paragraph 2 where authors state “Duan et al. (2017) and Tang et al. (2017) train answering and generation models with separate parameters, but add a regularisation term that encourages the models to be consistent. They focus on answer sentence selection, so performance cannot easily be compared with our work.”. I do not agree that the performance can not be compared, it is easily comparable by labeling a sentence containing the answer interval as the answer sentence. Can authors provide comparison of their work with that of Duan et al. (2017) and Tang et al. (2017)?
b.	In the same paragraph as 7.a, the authors have briefly mentioned “Echihabi & Marcu (2003) describe an earlier method for answering questions in terms of the distribution of questions given answers.” Can they provide a more clear explanation of this work and its relation to / difference with their work? 
////////////
I would like to thank authors for providing detailed answers to my questions. After reading their feedback, I am now willing to change my score to accept.