FLOWQA: GRASPING FLOW IN HISTORY FOR CONVERSATIONAL MACHINE COMPREHENSION
Conversational machine comprehension requires the understanding of the conversation history, such as previous question/answer pairs, the document context and the current question. To enable traditional, single-turn models to encode the history comprehensively, we introduce FLOW, a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure. Compared to approaches that concatenate previous questions/answers as input, FLOW integrates the latent semantics of the conversation history more deeply. Our model, FLOWQA, shows superior performance on two recently proposed conversational challenges (+7.2% F 1 on CoQA and +4.0% on QuAC). The effectiveness of FLOW also shows in other tasks. By reducing sequential instruction understanding to conversational machine comprehension, FLOWQA outperforms the best models on all three domains in SCONE, with +1.8% to +4.4% improvement in accuracy.
The paper proposes a method to model the flow of context in multi-turn machine comprehension (MC) tasks. The proposed model achieves amazing improvements in the two recent conversational MC tasks as well as an instruction understanding task. I am very impressed by the improvements and the ablation test that actually shows the effectiveness of the FLOW mechanism they proposed.
However, this paper has a lack of clarity (especially, Section 3) which makes it difficult to follow and easy to lose the major contribution points of the work. I summarized the weaknesses as follows:
# lack of motivation and its validation
The paper should have more motivational questions at the beginning of why such flow information is necessary for the task. Authors already mentioned about some of it in Figure 1 and here: “such as phrases and facts in the context, for answering the previous questions, and hence provide additional clues on what the current conversation is revolving around”. However, the improvement of absolute scores in the Experiment section didn’t provide anything related to the motivation they mentioned. Have you actually found the real examples in the testing set that are correctly predicted by the FLOW model but not by the baseline? Are they actually referring to the “phrases and facts in the context”, “additional clues on what the current conversation is revolving around”? Another simple test authors can try is to show the attention between the context in a flow and question and see whether appropriate clues are actually activated given the question. 
# unclear definition of “flow”
The term “flow” is actually little over-toned in my opinion. Initially, I thought that flow is a sequence of latent information in a dialog (i.e., question-answer) but it turns to be a sequence of the context of the passage. The term “flow” is more likely a sequence of latent and hierarchical movement of the information in my opinion. What is your exact definition of “flow” here? Do you believe that the proposed architecture (i.e., RNN sequence of context) appropriately take into account that? RNN sequence of the passage context actually means your attention over the passage given the question in turn, right? If yes, it shouldn’t be called a flow. 
# Lack of clarity in Section 3
Different points of contributions are mixed together in Section 3 by themselves or with other techniques proposed by others. For example, the authors mention the computational efficiency of their alternating structure in Figure 2 compared to sequential implementation. However, none of the experiment validates its efficiency. If the computational efficiency is not your major point, Figure 2 and 3 are actually unnecessary but rather they should be briefly mentioned in the implementation details in the later section. Also, are Figure 2 and 3 really necessary? 
Section 3.1 and 3.3.1 are indeed very difficult to parse: This is mainly because authors like to introduce a new concept of “flow” but actually, it’s nothing more than a thread of a context in dialog turns. This makes the whole points very hyped up and over-toned like proposing a new “concept”. Also, the section introduces so many new terms (“context integration”. “Flow”, “integration layers”, “conversational flow”, “integration-flow”) without clear definition and example. The name itself looks not intuitive to me, too. I highly recommend authors provide a high-level description of the “flow” mechanism at first and then describe why/how it works without any technical terms. If you can provide a single example where “flow” can help with, it would be nicer to follow it.
# Some questions on the experiment
The FLOW method seems to have much more computation than single-turn baselines (i.e., BiDAF). Any comparison on computational cost?
In Table 3, most of the improvements for QuAC come from the encoding N answer spans to the context embeddings (N-ans). Did you also compare with (Yatskar, 2018) with the same setting as N-ans? 
I would be curious to see for each context representation (c), which of the feature(e.g., c, em, g) affect the improvement the most? Any ablation on this?
The major and the most contribution of the model is probably the RNN of the context representations and concatenation of the context and question at turn in Equation (4). For example, have you tested whether simple entity matching or coreference links over the question thread can help the task in some sense? 
Lastly for the model design, which part of the proposed method could be general enough to other tasks? Is the proposed method task-specific so only applicable to conversational MC tasks or restricted sequential semantic parsing tasks?