Summarizing Answers in Non Factoid Community Question Answering
We aim at summarizing answers in community question answering (CQA) While most previous work focuses on factoid question answering, we focus on the non factoid question answering. Unlike factoid CQA, non factoid question answering usually requires passages as answers. The shortness, sparsity and diversity of answers form interesting challenges for summarization. To tackle these challenges, we propose a sparse coding based summarization strategy that includes three core ingredients: short document expansion, sentence vectorization, and a sparse coding optimization framework. Specifically, we extend each answer in a question answering thread to a more comprehensive representation via entity linking and sentence ranking strategies. From answers extended in this manner, each sentence is represented as a feature vector trained from a short text convolutional neural network model. We then use these sentence representations to estimate the saliency of candidate sentences via a sparse coding framework that jointly considers candidate sentences and Wikipedia sentences as reconstruction items. Given the saliency vectors for all candidate sentences, we extract sentences to generate an answer summary based on a maximal marginal relevance algorithm. Experimental results on a benchmark data collection confirm the effectiveness of our proposed method in answer summarization of non factoid CQA, and moreover, its significant improvement compared to state of the art baselines in terms of ROUGE metrics.
WSDM