The demand for abstractive dialog summary is growing in real-world applications. For example, customer service center or hospitals would like to summarize customer service interaction and doctor-patient interaction. However, few researchers explored abstractive summarization on dialogs due to the lack of suitable datasets. We propose an abstractive dialog summarization dataset based on MultiWOZ (Budzianowski et al., 2018). If we directly apply previous state-of-the-art document summarization methods on dialogs, there are two significant drawbacks: the informative entities such as restaurant names are difficult to preserve, and the contents from different dialog domains are sometimes mismatched. To address these two drawbacks, we propose Scaffold Pointer Network (SPNet) to utilize the existing annotation on speaker role, semantic slot and dialog domain. SPNet incorporates these semantic scaffolds for dialog summarization. Since ROUGE cannot capture the two drawbacks mentioned, we also propose a new evaluation metric that considers critical informative entities in the text. On MultiWOZ, our proposed SPNet outperforms state-of-the-art abstractive summarization methods on all the automatic and human evaluation metrics.
Authors proposed an enhanced Pointer-Generator model called SPNet. The key difference between SPNet and PG are the separate handling or using of speaker role, semantic slot and domain labels. Authors also proposed a new metrics called Critical Information Completeness (CIC) to address ROUGE's weakness in assessing if key information is missing in the output.
SPNet considers speak role by using separate encoders for each speaker in the dialog. The hidden state vectors of all speakers are concatenated for next layer. 
Semantic slot is modeled by delexicalizing the input, i.e. replacing values (18:00) with their semantic category (time). The actual value is later recovered from input text by copying over the corresponding raw tokens according to the attention layer. The domain labels are incorporated by combining categorization task loss into the final training loss.
Authors used the MultiWoz dataset to evaluate the model and compared it with state-of-the-art Pointer-Generator and Transformer models. ROUGE and proposed CIC metrics all show clear improvements in SPNet. The best performance was observed when all three improvements over SPNet are leveraged. Authors also provided example generated summary and discussed the difference between SPNet PG and baseline. An additional human evaluation was conducted which confirmed the quality gain.
The main concern of Reviewer is the inconsistency in the paper.
1) Authors claimed to "propose an abstractive dialog summarization dataset based on MultiWOZ (Budzianowski et al., 2018)" in the abstract and introduction, which sounds like part of their contribution is creating a new dataset, but in experiment section there's no discussion about how the dataset was created or used at all. The same claim reappeared as the first sentence in the conclusion section.
2) Authors emphasized two drawbacks in the beginning of the paper, but didn't discuss or show any evidence of those drawbacks from data later.
The above inconsistency suggests the paper may not be quite ready for publication.
Other issues found by Reviewer:
1) In equation (7), value() seems to be the word while on the right hand side it's a numerical value (max a_i^t). Did Authors mean argmax?
2) In Table 1, dialog domain seems to provide very marginal improvement, does it justify the complexity added?
3) In Section 4.3, why do we need to train a customized embedding? The process and parameter for the embedding training was not described.
4) In Section 4.3 "batch size to eight" better be consistent as "batch size to 8" (minor issue).