Definition: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Input: The development of methods to deal with the informative contents of the text units in the matching process is a major challenge in automatic summary evaluation systems that use fixed n-gram matching. The limitation causes inaccurate matching between units in a peer and reference summaries. The present study introduces a new Keyphrase based Summary Evaluator (KpEval) for evaluating automatic summaries. The KpEval relies on the keyphrases since they convey the most important concepts of a text. In the evaluation process, the keyphrases are used in their lemma form as the matching text unit. The system was applied to evaluate different summaries of Arabic multi-document data set presented at TAC2011. The results showed that the new evaluation technique correlates well with the known evaluation systems: Rouge-1, Rouge-2, Rouge-SU4, and AutoSummENG–MeMoG. KpEval has the strongest correlation with AutoSummENG–MeMoG, Pearson and spearman correlation coefficient measures are 0.8840, 0.9667 respectively. General Terms Automatic summary evaluation, Automatic summarization, Keyphrase extraction, Natural language processing, computational linguistics, Information retrieval.

Keyphrase based Evaluation of Automatic Text Summarization