Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequenceto-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder trained so that the mean of the representations of the input reviews decodes to a reasonable summary-review. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews.
Overall and positives:
The paper investigates the problem of multidocument summarization
without paired documents to summary data, thus using an unsupervised
approach. The main model is constructed using a pair of locked
autoencoders and decoders. The model is trained to optimize the
combination of 1. Loss between reconstructions of the original reviews
(from the encoded reviews) and original the reviews, 2. And the
average similarity of the encoded version of the docs with the encoded
representation of the summary, generated from the mean representation
of the given documents.
By comparing with a few simple baseline models, the authors were able
to demonstrate the potential of the design against several naive
approaches (on real datasets, YELP and AMAZON reviews). 
The necessity of several model components is demonstrated
through ablation studies. The paper is relatively well structured and
complete. The topic of the paper fits well with ICLR. The paper
provides decent technical contributions with some novel ideas about
multi-doc summary learning models without a (supervised) paired
data set.
Comments / Issues
[ issue 6 is most important ]
1.  Problem presentation. The problem was not properly introduced and
elaborated. In fact, there is not a formal and mathematical
introduction of the problem, input, output, dataset and model
parameters. The notations used are not very clearly defined and are
quite handwavy, (e.g. what is V, dimensions of inputs x_i was not
mentioned until much later in the paper). The authors should make
these more precise. Similar problem with presentations of the models,
parameters, and hyperparameters.
3.  How does non-equal weighted linear combinations of l_rec and l_sim
change the results? Other variation of the overall loss function? How
do we see the loss function interaction in the training, validation
and test data? With the proposed model, these could be interesting to
observe.
4.  In equation two, the decoder seems to be very directly affecting
the quality of the output summary. Teacher forcing was used to train
the decoder in part (1) of the model, but without ground truth, I
would expect more discussions and experiments on how the Gumbel
softmax trick affect or help the performance of the output.
5.  Baseline models and metrics
(1) There should be more details on how the language model is trained,
some examples, and how the reviews are generated from the language
model as a base model (in supplement?).
(2). It is difficult to get a sense of how these metrics corresponds
to the actual perceived quality of the summary from the
presentation. (see next)
(3). It will be more relevant to evaluate the proposed design
vs. other neural models, and/or more tested and proved methods.
6. The rating classifier (CLF) is intriguing, but it's not clearly
explained and its effect on the evaluation of the performance is not
clear: One of the key metrics used in the evaluation relies on the
output rating of a classifier, CLF, that predicts reader ratings on
reviews (eg on YELP).  The classifier is said to have 72%
accuracy. First, the accuracy is not clearly defined, and the details
of the classifier and its training is not explained (what features are
its input, is the output ordinal regression).  Equation 4 is not
explained clearly: what does 'comparing' in 'by comparing the
predicted rating given the summary rating..' mean?  The classifier may
have good performance, but it's unclear how this accuracy should
affect the results of the model comparisons.
The CLF is used to evaluate the rating of output
reviews from various models. There is no justification these outputs
are in the same space or generally the same type of document with the
training sample (assuming real Yelp reviews).  That is probably
particularly true for concatenation of the reviews, and the CLF classifier
scores the concatenation very high (or  eq 4 somehow leads to highest value
for the concatenation of reviews )... It's not clear whether such a classifier is 
beneficial in this context.
7. Summary vs Reviews. It seems that the model is built on an implicit
assumption that the output summary of the multi-doc should be
sufficiently similar with the individual input docs.  This may be not
true in many cases, which affects whether the approach generalizes.
Doc inputs could be covering different aspects of the review subject
(heterogeneity among the input docs, including topics, sentiment etc),
or they could have very different writing styles or length compared to
a summary.  The evaluation metrics may not work well in such
scenarios.  Maybe some pre-classification or clustering of the inputs,
and then doing summarization for each, would  help?  In the conclusions section, the
authors do mention summarizing negative and positive reviews
separately.