CAN AI GENERATE LOVE ADVICE?: TOWARD NEURAL ANSWER GENERATION FOR NON-FACTOID QUESTIONS
Deep learning methods that extract answers for non-factoid questions from QA sites are seen as critical since they can assist users in reaching their next decisions through conversations with AI systems. The current methods, however, have the following two problems: (1) They can not understand the ambiguous use of words in the questions as word usage can strongly depend on the context (e.g. the word “relationship” has quite different meanings in the categories of Love advice and other categories). As a result, the accuracies of their answer selections are not good enough. (2) The current methods can only select from among the answers held by QA sites and can not generate new ones. Thus, they can not answer the questions that are somewhat different with those stored in QA sites. Our solution, Neural Answer Construction Model, tackles these problems as it: (1) Incorporates the biases of semantics behind questions (e.g. categories assigned to questions) into word embeddings while also computing them regardless of the semantics. As a result, it can extract answers that suit the contexts of words used in the question as well as following the common usage of words across semantics. This improves the accuracy of answer selection. (2) Uses biLSTM to compute the embeddings of questions as well as those of the sentences often used to form answers (e.g. sentences representing conclusions or those supplementing the conclusions). It then simultaneously learns the optimum combination of those sentences as well as the closeness between the question and those sentences. As a result, our model can construct an answer that corresponds to the situation that underlies the question; it fills the gap between answer selection and generation and is the first model to move beyond the current simple answer selection model for non-factoid QAs. Evaluations using datasets created for love advice stored in the Japanese QA site, Oshiete goo, indicate that our model achieves 20 % higher accuracy in answer creation than the strong baselines. Our model is practical and has already been applied to the love advice service in Oshiete goo.
This paper proposes a neural architecture for answering non-factoid questions. The author*s model improves over previous neural models for answer sentence selection. Experiments are conducted on a Japanese love advice corpus; the coolest part of the paper for me was that the model was actually rolled out to the public and its answers were rated twice as good as actual human contributors! It was hard for me to determine the novelty of the contribution. The authors mention that their model *fills the gap between answer selection and generation*; however, no generation is actually performed by the model! Instead, the model appears to be very similar to the QA-LSTM of Tan et al., 2015 except that there are additional terms in the objective to handle conclusion and supplementary sentences. The structure of the answer is fixed to a predefined template (e.g., conclusion --> supplementary), so the model is not really learning how to order the sentences. The other contribution is the *word embedding with semantics* portion described in sec 4.1, which is essentially just the paragraph vector model except with *titles* and *categories* instead of paragraphs. While the result of the paper is a model that has actually demonstrated real-life usefulness, the technical contributions do not strike me as novel enough for publication at ICLR. Other comments: - One major issue with the reliance of the model on the template is that you can*t evaluate on commonly-used non-factoid QA datasets such as InsuranceQA. If the template were not fixed beforehand (but possibly learned by the model), you could conceivably evaluate on different datasets. - The examples in Table 4 don*t show a clear edge in answer quality to your model; QA-LSTM seems to choose good answers as well. - Doesn*t the construction model have an advantage over the vanilla QA-LSTM in that it knows which sentences are conclusions and which are supplementary? Or does QA-LSTM also get this distinction?
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
CAN AI GENERATE LOVE ADVICE?: TOWARD NEURAL ANSWER GENERATION FOR NON-FACTOID QUESTIONS
Deep learning methods that extract answers for non-factoid questions from QA sites are seen as critical since they can assist users in reaching their next decisions through conversations with AI systems. The current methods, however, have the following two problems: (1) They can not understand the ambiguous use of words in the questions as word usage can strongly depend on the context (e.g. the word “relationship” has quite different meanings in the categories of Love advice and other categories). As a result, the accuracies of their answer selections are not good enough. (2) The current methods can only select from among the answers held by QA sites and can not generate new ones. Thus, they can not answer the questions that are somewhat different with those stored in QA sites. Our solution, Neural Answer Construction Model, tackles these problems as it: (1) Incorporates the biases of semantics behind questions (e.g. categories assigned to questions) into word embeddings while also computing them regardless of the semantics. As a result, it can extract answers that suit the contexts of words used in the question as well as following the common usage of words across semantics. This improves the accuracy of answer selection. (2) Uses biLSTM to compute the embeddings of questions as well as those of the sentences often used to form answers (e.g. sentences representing conclusions or those supplementing the conclusions). It then simultaneously learns the optimum combination of those sentences as well as the closeness between the question and those sentences. As a result, our model can construct an answer that corresponds to the situation that underlies the question; it fills the gap between answer selection and generation and is the first model to move beyond the current simple answer selection model for non-factoid QAs. Evaluations using datasets created for love advice stored in the Japanese QA site, Oshiete goo, indicate that our model achieves 20 % higher accuracy in answer creation than the strong baselines. Our model is practical and has already been applied to the love advice service in Oshiete goo.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes a neural architecture for answering non-factoid questions. The author*s model improves over previous neural models for answer sentence selection. Experiments are conducted on a Japanese love advice corpus; the coolest part of the paper for me was that the model was actually rolled out to the public and its answers were rated twice as good as actual human contributors! It was hard for me to determine the novelty of the contribution. The authors mention that their model *fills the gap between answer selection and generation*; however, no generation is actually performed by the model! Instead, the model appears to be very similar to the QA-LSTM of Tan et al., 2015 except that there are additional terms in the objective to handle conclusion and supplementary sentences. The structure of the answer is fixed to a predefined template (e.g., conclusion --> supplementary), so the model is not really learning how to order the sentences. The other contribution is the *word embedding with semantics* portion described in sec 4.1, which is essentially just the paragraph vector model except with *titles* and *categories* instead of paragraphs. While the result of the paper is a model that has actually demonstrated real-life usefulness, the technical contributions do not strike me as novel enough for publication at ICLR. Other comments: - One major issue with the reliance of the model on the template is that you can*t evaluate on commonly-used non-factoid QA datasets such as InsuranceQA. If the template were not fixed beforehand (but possibly learned by the model), you could conceivably evaluate on different datasets. - The examples in Table 4 don*t show a clear edge in answer quality to your model; QA-LSTM seems to choose good answers as well. - Doesn*t the construction model have an advantage over the vanilla QA-LSTM in that it knows which sentences are conclusions and which are supplementary? Or does QA-LSTM also get this distinction?
assistant
This paper proposes a neural architecture for answering non-factoid questions. The author*s model improves over previous neural models for answer sentence selection. Experiments are conducted on a Japanese love advice corpus; the coolest part of the paper for me was that the model was actually rolled out to the public and its answers were rated twice as good as actual human contributors! It was hard for me to determine the novelty of the contribution. The authors mention that their model *fills the gap between answer selection and generation*; however, no generation is actually performed by the model! Instead, the model appears to be very similar to the QA-LSTM of Tan et al., 2015 except that there are additional terms in the objective to handle conclusion and supplementary sentences. The structure of the answer is fixed to a predefined template (e.g., conclusion --> supplementary), so the model is not really learning how to order the sentences. The other contribution is the *word embedding with semantics* portion described in sec 4.1, which is essentially just the paragraph vector model except with *titles* and *categories* instead of paragraphs. While the result of the paper is a model that has actually demonstrated real-life usefulness, the technical contributions do not strike me as novel enough for publication at ICLR. Other comments: - One major issue with the reliance of the model on the template is that you can*t evaluate on commonly-used non-factoid QA datasets such as InsuranceQA. If the template were not fixed beforehand (but possibly learned by the model), you could conceivably evaluate on different datasets. - The examples in Table 4 don*t show a clear edge in answer quality to your model; QA-LSTM seems to choose good answers as well. - Doesn*t the construction model have an advantage over the vanilla QA-LSTM in that it knows which sentences are conclusions and which are supplementary? Or does QA-LSTM also get this distinction?