Opinion summarization is the task of automatically generating summaries that encapsulate information from multiple user reviews. We present Semantic Autoencoder (SemAE) to perform extractive opinion summarization in an unsupervised manner. SemAE uses dictionary learning to implicitly capture semantic information from the review and learns a latent representation of each sentence over semantic units. A semantic unit is supposed to capture an abstract semantic concept. Our extractive summarization algorithm leverages the representations to identify representative opinions among hundreds of reviews. SemAE is also able to perform controllable summarization to generate aspect-specific summaries. We report strong performance on SPACE and AMAZON datasets, and perform experiments to investigate the functioning of our model. Our code is publicly available at https://github.com/brcsomnath/SemAE. Opinion summarization is the task of automatically generating digests for an entity (e.g. a product, a hotel, a service, etc.), from user opinions in online forums. Automatic opinion summaries enable faster comparison, search, and better consumer feedback understanding (Hu and Liu, 2004; Pang, 2008; Medhat et al., 2014). Although there has been significant progress towards summarization (Rush et al., 2015; Nallapati et al., 2016; Cheng and Lapata, 2016; See et al., 2017; Narayan et al., 2018; Liu et al., 2018), existing approaches rely on human-annotated reference summaries, which are scarce for opinion summarization. For opinion summarization, human annotators need to read hundreds of reviews per entity across different sources for writing a summary, which may not be feasible. This lack of labeled training data has prompted a series of works to leverage unsupervised or weakly-supervised techniques for opinion summarization (Mei et al., 2007; Titov and McDonald, 2008; Angelidis and Lapata, 2018a; Angelidis et al., 2021). Recent works in this direction have focused on performing opinion summarization in an abstractive setting (Coavoux et al., 2019; Isonuma et al., 2019; Brainskas et al., 2020; Amplayo et al., 2021b; Iso et al., 2021; Wang and Wan, 2021). Abstractive models are able to produce fluent summaries using novel phrases. However, they suffer from problems common in text generation like hallucination (Rohrbach et al., 2018), text degeneration (Holtzman et al., 2020), and topic drift (Sun et al., 2020). Also, these approaches have been evaluated on small scales (10 reviews per entity or fewer), which does not reveal their utility in the real world where there are hundreds of reviews per entity. To overcome these issues, another thread of works focuses on extractive opinion summarization, which creates summaries by selecting review sentences to reflect the popular opinions corresponding to an entity. A recently proposed extractive summarization approach is Quantized Transformer (QT) (Angelidis et al., 2021), which leverages vector quantization (van den Oord et al., 2017) for assigning texts to a latent representation that is supposed to capture a semantic sense. However, a text phrase can encapsulate multiple semantic senses, making this representation learning approach restrictive. Building on the framework introduced by QT, we introduce an unsupervised extractive model, Semantic Autoencoder (SemAE), which learns a representation of text over latent semantic units using dictionary learning (Dumitrescu and Irofti, 2018). Similar to QT, SemAE leverages Transformer (Vaswani et al., 2017) for sentence reconstruction to simultaneously learn latent semantic units and sentence representations. However, while QT assigns texts to a latent representation (codebook), SemAE models text as a combination of semantics and forms a distribution over latent units (dictionary). This allows sentence rep-1209 resentations to capture fine-grained and diverse semantics. Unlike QT that relies on identification of aspect-specific head representations, we achieve controllable summarization by utilizing information-theoretic measures (such as relevance, redundancy, etc) on sentence representations. Our sentence selection algorithm is more flexible and allows a broader spectrum of controllable summarization. We experimentally show strong performance on two opinion summarization datasets. Our main contributions are:  We present Semantic Autoencoder (SemAE), which learns representation of sentences over latent semantic units. We introduce novel inference algorithms for general and controllable summarization utilizing information-theoretic measures. We show that SemAE outperforms previous methods using automatic and human evaluations. We perform analysis to understand how the learnt representations align with human semantics. Unsupervised opinion summarization can be conducted either abstractively or extractively. Abstractive approaches aim to summarize the opinion text using novel phrases. Traditional statistical approaches create abstractive summaries using graphical paths (Ganesan et al., 2010) or hand-written templates (Di Fabbrizio et al., 2014). Recent neural approaches leverage the encoder-decoder architecture to aggregate information from multiple reviews and generate summaries accordingly (Chu and Liu, 2019; Brainskas et al., 2020; Iso et al., 2021; Wang and Wan, 2021). In contrast to abstractive approaches, extractive approaches rank and select a subset of salient sentences from reviews to form a concise summary (Kim et al., 2011). Saliency computation has been explored using traditional frequency-based approaches (Nenkova and Vanderwende, 2005), similarity with the centroid in the representation space (Radev et al., 2004), and lexical similarity with all sentences in a graph-based representation (Erkan and Radev, 2004). Weakly supervised approaches (Angelidis and Lapata, 2018a; Zhao and Chaturvedi, 2020) extract opinions based on their aspect specificity, and nature of sentiment polarity. discussed in Section 1. It is also similar to neural topic model-based approaches (Iyyer et al., 2016; He et al., 2017; Angelidis and Lapata, 2018a) that use a variant of dictionary learning (Elad and Aharon, 2006; Olshausen and Field, 1997) to represent text as a combination of specific semantics (e.g. aspect, relationships etc). In contrast to these models, where text from same topics are trained to have similar representations using max-margin loss, SemAE uses an autoencoder setup to capture diverse latent semantics. We follow the task setup in (Angelidis et al., 2021), where given a set of entities (e.g. hotels), a review set R e = { r 1 , r 2 , . . . } is provided for each entity e , where each review r i is a sequence of sentences { s 1 , s 2 , . . . } . The review set R e covers a range of aspects A = { a 1 , a 2 , . . . } relating to the domain (e.g. service , location for hotels). We denote S e to be the set of sentences from all reviews for an entity e . SemAE is evaluated to perform two types of extractive opinion summarization introduced by Angelidis et al. (2021): (a) general summarization , which involves selecting a subset of sentences O e  S e such that it best represents the reviews in R e , and (b) aspect summarization , where the generated summary O ( a ) e  S e focuses on a specific aspect a  A . The intuition behind Semantic Autoencoder is that instead of representing text as a single latent semantic unit, we represent text as a distribution over latent semantic units using dictionary learning . Learning semantic representations over a common dictionary makes them structurally aligned, enabling comparison of sentences using information-theoretic measures. Semantic Autoencoder consists of three stages (i) sentence encoding an input sentence s is converted into a multi-head representation ( H heads) using Transformer encoder { s h } Hh =1 ; (ii) reconstruction a latent representation of head vectors s h is formed over elements of the dictionary D  RK  d , to produce reconstructed representations z = { z h } H h =1 ; and (iii) sentence decoding a Transformer-based decoder takes as input the reconstructed representations z to produce the output sentence  s . SemAE is trained on the sentence reconstruction task. The overall workflow of SemAE 1210 Sentence Encoder Sentence Decoder D   K  d  1  2  3 s 1 s 2 s 3 z 1 z 2 z 3 Input ( s ) Reconstruction (  s )  Figure 1: An example workflow of SemAE. We follow the setup of QT (Angelidis et al., 2021) for sentence encoding. Each sentence s starts with a special token [SNT] , which is fed to a Transformer-based encoder. We only consider the final-layer representation of the [SNT] token s snt  R d . The sentence representation s snt is split into H contiguous vectors { s (cid:48) h } Hh =1 , where s (cid:48) h  R d/H . A multi-head representation is formed by passing s (cid:48) h through a layer-normalization layer: s h = LN( s (cid:48) h WT + b ) (1) where W  R d  d/H , b  R d are trainable parameters and s h  R d is the h th head representation. For each s h , we obtain a latent representation  h over the dictionary D , by reconstructing the encoded sentence representation s h as shown below z h =  h D,  h = softmax( s h DT ) (2) where the reconstructed vector z h  R d , and the latent representation  h  RK . We hypothesize that the dictionary D captures the representation of latent semantic units, and  h captures the degree to which the text encapsulates a certain semantic. The vectors formed z = { z h } Hh =1 are forwarded to the decoder for sentence reconstruction. The dictionary D and s h are updated simultaneously using backpropagation. For summarization (Section 5), different from QT, we consider  h (not z h ) as the sentence representation. We employ a Transformer-based decoder that takes as input the reconstructed representations z = { z h } Hh =1 . MultiHead( z , z , t ) attention module in the decoder takes z as key and value, and the target tokens t as the query. The reconstructed sentence is generated from the decoder as  s = Decoder( z , t ) . As our goal is sentence reconstruction, we set the target tokens to be same as the input sentence s . Prior work (Angelidis et al., 2021) has also used a similar Transformer-based decoder for sentence reconstruction but they attend directly over quantized head vector formed using codebook elements. A sentence can capture only a small number of semantic senses. We ensure this by enforcing sparsity constraints on the representations  h , so that z h is a combination of only a few semantic units. The encoder, reconstructor and decoder are trained together to minimize the loss function: L = LCE ( s,  s )+  1 (cid:88) h |  h | +  2 (cid:88) h H (  h ) (3) where LCE is the reconstruction cross-entropy loss of the decoder, and to ensure sparsity of  h we penalize the L1-norm ( |  h | ) and its entropy H (  h ) . We leverage the latent representations  h generated by SemAE to perform opinion summarization. 1 5.1 General Summarization For obtaining the general summary of an entity, we first compute a mean representation of all the review sentences in S e , which represents the aggregate distribution over semantic units. Thereafter, the general summary is obtained as the collection of sentences that resemble the mean distribution. Mathematically, every sentence s is associated with a representation over dictionary elements  s = [  1 , . . . ,  H ] , where  s  RH  K . We form the mean representation of review sentences for an entity S e over dictionary elements as:   = 1 | S e | (cid:88) s  S e  s (4) where  s is the representation for sentence s  S e . For general summarization, we compute the relevance score R (  ) for each sentence s based on its similarity with the mean representation   : R (  s ) = ( ,  s ) =  (cid:88) h KL(  h ,  sh ) (5) 1 We experimented with different variations of the sentence selection scheme using  h in Appendix A.4. where  sh is latent representation of sentence s for the h th head. ( x, y ) denotes the similarity between two representations x and y . It is implemented as negation of the sum of KL-divergence between head representations. We also experimented with other divergence metrics and observed similar summarization performance (Appendix A.3). We rank sentences according to descending order of R (  ) and select the top N (a constant hyper-parameter, N < | S e | ) sentences as the summary O e (shown in Figure 2). The extracted summary is a concatenation of the text from N selected input sentences (Input ( s ) in Figure 1). However, modeling relevance only using (  ,  ) results in selection of similar sentences. We overcome this by designing variations of our system that have additional information-theoretic constraints. (a) Redundancy : We introduce diversity in the generated summary by penalizing sentences that have a high similarity value with already selected sentences. This is achieved by adding the redundancy term in relevance score: R (  s ,  O e ) = ( ,  s )   max s (cid:48)   O e (  s (cid:48) ,  s ) (6) where  O e is the set of sentences selected so far for the summary. The selection routine proceeds in a greedy fashion by choosing s 0 = arg max s  S e ( ,  s ) when  O e =  . (b) Aspect-awareness : Another drawback with sentence selection using (  ,  ) is that the summary frequently switches context among different aspects (example shown in Table 7). To mitigate this issue, we identify the aspect of a review sentence using occurrences of aspect-denoting keywords provided in the dataset (Section 5.2). We then cluster the sentences into aspect-specific buckets { S ( a 1 ) e , S ( a 2 ) e , . . . } and rank sentences within each bucket. We ignore sentences that are not part of any bucket. We select sentences using two different strategies:  We iterate over sentence buckets { S ( a i ) e } and select the first m sentences ranked according to R (  s ) , from each bucket. We prevent selection of similar sentences from a bucket by introducing the redundancy term. We iterate over individual buckets and select first m sentences ranked according to their relevance R (  s ,  O ( a ) e ) (Equation 6). SemAE can perform aspect summarization without needing additional training. For this, we require a small set of keywords to identify sentences that talk about an aspect. For example, food aspect is captured using keywords: breakfast, buffet etc. For a given aspect a , let the keyword set be Q a = { w 1 , w 2 , . . . } . We use Q a to identify a set of sentences S ( a ) e for each entity e , belonging to aspect a from a held-out dev set S dev . Similar to general summarization, we proceed by computing the mean representation of sentences S ( a ) e belonging to the aspect a :   ( a ) = 1 | S ( a ) e | (cid:88) s  S ( a ) e  s (7) We then select sentences most similar to the mean representation as the summary. (a) Informativeness : Sentences selected for aspect summarization should talk about the aspect but not the general information. We model informativeness (Peyrard, 2019) by ensuring that a selected sentence representation  s resembles the aspect mean   ( a ) , but is divergent from the overall representation mean   , for a given entity e . For an aspect a , we iterate over sentences in S ( a ) e and compute the relevance score for a sentence s as follows: R a (  s ) = (  ( a ) ,  s )   ( ,  s ) (8) We rank sentences s  S e according to their aspect-specific relevance score R a (  ) , and select first N sentences as the summary for aspect O ( a ) e . 2 6 Experimental Setup In this section, we discuss the experimental setup, results and analysis. 2 We experimented with incorporating the informativeness term in general summarization also but did not find it useful (see Appendix A.3 for more details). The dataset statistics are reported in Table 1. Test sets of both datasets contain three human-written general summaries per entity. The SPACE corpus was created in a two-step process of sentence selection and then summarization of selected sentences by annotators (further details in Appendix A.2). SPACE dataset also provides human-written summaries for six different aspects of hotels: building , cleanliness , food , location , rooms , and service . We build on the implementation framework introduced by Angelidis et al. (2021) for our experiments. We used a 3-layer Transformer with 4 attention heads as the encoder and decoder. The input and hidden dimensions are 320. The encoder and decoder for SemAE was trained for 4 warmup epochs, before the dictionary learning based reconstruction component was introduced. We split the encoded vector into H = 8 head representations. We have K = 1024 dictionary elements, each with dimension d = 320 . The dictionary elements are initialized using k -means clustering of review sentence representations. All hyperparameters were tuned on the development set (see Appendix A.1 for more details). We report ROUGE F-scores that compares the overlap between generated text with gold summaries. For SPACE dataset, we measure how much general summaries cover different aspects by computing the mean ROUGE-L score with the gold aspect summaries (denoted by RLASP ). We also compute perplexity (PPL) score to evaluate the readability of summaries. Perplexity is computed using cross-entropy loss from a BERT-base model. We measure aspect coverage of a system, by computing the average number of distinct aspects NASP in the generated summaries. Lastly, to evaluate repetition in summaries, we compute the percentage of distinct n -grams ( n = 2 ). Following prior work (Angelidis et al., 2021), we compare SemAE with three types of systems: (a) Best Review systems: We report the performance of Centroid method, where reviews are encoded using BERT or SentiNeutron (Radford et al., 2017), and the review most similar to the mean representation is selected. (b) Abstractive systems: We report the performance of Opinosis (Ganesan et al., 2010) (a graph-based approach), MeanSum (Chu and Liu, 2019), CopyCat (Brainskas et al., 2020) and AceSum (Am-playo et al., 2021a) summarization models. (c) Extractive systems: We report the performance of LexRank (Erkan and Radev, 2004), where sentences were encoded using BERT, SentiNeutron or tf-idf vector. We also report the performance achieved by selecting review sentences randomly . General Summarization : We present the results of general summarization on SPACE dataset in Table 2. SemAE and its variants show strong improvements over previous state-of-the-art QT, and other baselines, across all ROUGE metrics. They also outperform abstractive systems (like CopyCat and Meansum) by a large margin, which shows that SemAE can effectively select relevant sentences from a large pool of reviews. All variants of SemAE outperform other models in RLASP metric, showcasing that general summaries from SemAE cover aspects better than baselines. We compiled some baseline results from Angelidis et al. (2021). We further evaluate the quality of the summaries, for all variations of SemAE along with our strongest baseline QT, using other automatic metrics in Table 3. The first row in Table 3 reports the performance of QT, which achieves the highest distinct n -gram score, but has poor perplexity score. This shows that QT generates summaries with diverse text but they are not coherent. SemAE achieves the best perplexity score (sec-ond row in Table 3) but produces less diverse text (lowest distinct n -gram score). The third row in Table 3 reports the performance of SemAE with redundancy term. Comparing rows 2 and 3 of Table 3, we observe that the summaries from SemAE 1213 SPACE [General] R1 R2 RL RLASPB e s t R e v i e w Centroid SENTI 27.36 5.81 15.15 8.77 Centroid BERT 31.33 5.78 16.54 9.35 Oracle SENTI 32.14 7.52 17.43 9.29 Oracle BERT 33.21 8.33 18.02 9.67 A b s t r ac t Opinosis (Ganesan et al.) 28.76 4.57 15.96 11.68 MeanSum (Chu and Liu) 34.95 7.49 19.92 14.52 Copycat (Brainskas et al.) 36.66 8.87 20.90 14.15 AceSum (Amplayo et al.) 40.37 11.51 23.23 E x t r ac t Random 26.24 3.58 14.72 11.53 LexRank TF-IDF 29.85 5.87 17.56 11.84 LexRank SENTI 30.56 4.75 17.19 12.11 LexRank BERT 31.41 5.05 18.12 13.29 AceSum EXT (Amplayo et al.) 35.50 7.82 20.09 QT (Angelidis et al.) 38.66 10.22 21.90 14.26 SemAE 42.48 13.48 26.40 15.23 w/ redun. (w/ redundancy) have more distinct n -grams (less repetition), while falling behind in perplexity and aspect coverage. Performance results for aspect-aware variants of SemAE are reported in last two rows of Table 3. We observe that iteratively covering aspects reduces repetition (increase in distinct-n score). As expected the mean aspect-coverage ( E [ NASP ] ) improves in aspect-aware SemAE variants. However, a slight drop in aspect-coverage is observed when the redundancy term is introduced (last row in Table 3). We also observe an increase in perplexity for aspect-aware variants, which can be caused due to multiple changes in aspect context. Overall, SemAE (w/ aspect + redundancy) is able to produce diverse text with a high aspect coverage and a decent perplexity score, appearing to be the best performing model. similar performance, with SemAE achieving the best performance among all extractive summarization system. SemAE falls short of only abstractive summarization systems that have the advantage of generating novel phrases not present in the input reviews. Also, while SemAE beats most baselines for AMAZON dataset, the performance gain isn't as much as SPACE dataset. We believe this is because the number of reviews per entity in AMAZON (8) is much lower compared to SPACE (100). As SemAE is dependent on the mean representation   , having more reviews helps in capturing the popular opinion distribution accurately. 4 For practical purposes, opinion summarization systems are useful when there are hundreds or more reviews per entity. A larger improvement on SPACE shows the efficacy of SemAE in the real world. Aspect Summarization : For aspect summarization, we compare against four unsupervised systems MeanSum, CopyCat, LexRank and QT on the SPACE dataset. For general summarizers: MeanSum, CopyCat and LexRank, sentence embeddings retrieved from BERT (Vaswani et al., 2017) were clustered using k -means and each cluster S ( a ) e was assigned an aspect a based on frequency of aspect-denoting keywords in the cluster's sentences. The models then produced summaries for each aspect a given the input set S ( a ) e . All models including 4 We observed a drop in performance when the number of reviews/entity in SPACE dataset was reduced (experimental details in Section 6.6). SPACE [Aspect] Building Cleanliness Food Location Rooms Service R1 R2 RL MeanSum (Chu and Liu) 13.25 19.24 13.01 18.41 17.81 20.40 23.24 3.72 17.02 CopyCat (Brainskas et al.) 17.10 15.90 14.53 20.31 17.30 20.05 24.95 4.82 17.53 LexRank BERT (Erkan and Radev) 14.73 25.10 17.56 23.28 18.24 26.01 27.72 7.54 20.82 QT (Angelidis et al.) 16.45 25.12 17.79 23.63 21.61 26.07 28.95 8.34 21.77 SemAE 20.04 23.72 23.57 25.33 25.29 26.90 31.24 10.43 24.14 w/o informativeness 18.38 24.08 19.03 23.32 23.89 25.05 27.85 8.61 22.29 SemAE, use the same aspect-denoting keywords. Evaluation results on SPACE are reported in Table 5. SemAE outperforms the state-of-the-art QT in all aspects except cleanliness , where the performance is comparable. We observe that adding the informativeness term ( ( ,  s ) in Equation 8) helps improve the specificity of the aspect thereby boosting performance. SemAE also shows significant gains in terms of average ROUGE-1/2 and ROUGE-L across different aspects. Human Evaluation : We performed human evaluations for the general and aspect summaries. We evaluated general summaries from QT, best performing variant SemAE (w/ aspect + redundancy) and gold summary. Summaries were judged by 3 human annotators on three criteria: informativeness , coherence and non-redundancy . The judges were presented summaries in a pairwise manner and asked to select which one was bet-ter/worse/similar. The scores (-100 to +100) were computed using Best-Worst Scaling (Louviere et al., 2015). The first half of Table 6 reports the evaluation results, where we observe that SemAE (w/ aspect + redundancy) outperforms our strongest baseline, QT, for all criteria (statistical significance information provided in the caption of Table 6). However, summaries generated from both systems Figure 3: Visualization of UMAP projections of dictionary elements. We also evaluated aspect summaries generated by SemAE and QT in a similar manner. Aspect summaries were judged based on two criteria: aspect informativeness (usefulness of opinions for a specific aspect, consistent with reference) and aspect specificity (how specific the summary is for an aspect without considering other factors). The bottom half of Table 6 reports the results for aspect summaries. We observe that both QT and SemAE produce aspect-specific summaries. However, SemAE shows a statistically significant improvement over QT in aspect informativeness. Latent Dictionary Interpretation. In this section, we investigate the semantic meanings learnt by individual dictionary elements, D k . We visualized the UMAP projection (McInnes et al., 2018) of dictionary element representations (shown in Figure 3). For different runs of SemAE, we found that the dictionary representations converged into clusters as shown in Figure 3 (elements are color-coded according to their cluster identities as assigned by k -means algorithm with k =12). We hypothesize that the clusters should capture certain semantic meaning. We explore this hypothesis by identifying sentences sharing similar representations with the mean representations 1215 SemAE SemAE (w/ redun.) SemAE (w/ aspect) SemAE (w/ aspect + redun.) The staff is great. {  1 , . . . ,  K } for each cluster. For each head h in the encoder (Section 4.1), we compute cosine similarity of sentences with cluster means. Table 8 shows some examples of sentences having highest similarity with a cluster mean  k for a head representation h . We observe in most cases sentences closest to a cluster share a similar semantic meaning. For hotel reviews, we observe that sentences often talk about a specific aspect like service, food and rooms, as shown for ( h, k ) configurations (3, 5), (2, 8) and (5, 8) in Table 8. The clusters sometimes capture certain coarse semantics like presence of a word or phrase (e.g. config. (0, 10) in Table 8). It can also capture high-level semantics like the experience of a customer (e.g. config. (6, 0)). It was interesting to observe that a single cluster can capture different semantics for distinct heads (cluster 8 in configurations (2, 8) and (5, 8)). Qualitative Examples. Table 7 shows summaries generated by SemAE and its variants for the SPACE dataset. While the summary generated by SemAE talks about location , staff & service multiple times (shown as highlighted text), summary from SemAE (w/ redundancy) doesn't have that repetition. Also, the summary generated by SemAE switches context frequently. For example, the aspect of the first three sentences changes from service  location  service. We observe that compared to SemAE, both aspect-aware SemAE variants generate summaries without abrupt context switches. The summary generated by SemAE (w/ aspect) covers aspects like service, hotel, food and rooms sequentially, but sentences referring to an aspect are quite similar. SemAE (w/ aspect + redundancy) overcomes this shortcoming, and introduces diversity among the aspect-specific sentences. Training Data Efficiency. We analyze the performance of SemAE, QT and CopyCat for general summarization (ROUGE-1) on SPACE for varying training data fractions in Table 9. We observe that both QT and SemAE perform well with low training data. However, SemAE outperforms QT in all 1216 low resource settings. SemAE (with 10% data) yields significant ROUGE-1 improvements over QT (with access to 100% data). Impact of number of reviews. We investigate whether SemAE's performance gain on SPACE is due to the larger number of reviews available (reviews per entity  AMAZON : 8, SPACE : 100). Specifically, we perform ablation experiments by reducing the number of reviews/entity in SPACE dataset. We remove user reviews with low relevance scores (relevance score of a review is the average R (  ) of its sentences). Table 10 reports the performance of SemAE with different number of reviews/entity in the test set. We observe a gradual decline in ROUGE-1 score when the reviews/entity is reduced, which shows that having more reviews per entity helps in better extractive summarization. Additional Controllable Summarization . We showcase that SemAE can perform different forms of controllable summarization. Specifically, we perform sentiment-based summarization using a small number (10) of seed sentences belonging to positive , negative and neutral sentiment class. Seed sentences were annotated using the rule-based system VADER (Hutto and Gilbert, 2014). An example of sentiment-based summarization is shown in Table 11. We observe SemAE is able to generate summaries aligning with the seed sentiments. We also perform multi-aspect summarization using SemAE, by controlling the aspect of the selected sentences. Table 12 showcases an example of multi-aspect summarization. An interesting observation is that SemAE is able to select sentences, which have mutliple aspects (shown in blue ) and not in-dependent sentences from different aspects. These experiments show that SemAE is able capture and leverage granular semantics for summarization. In Appendix A.5, we perform additional analysis to investigate the head-wise analysis, efficacy of sparsity constraints, dictionary evolution, and qualitatively compare SemAE with baselines (QT and CopyCat). We proposed a novel opinion summarization approach using Semantic Autoencoder, which encodes text as a representation over latent semantic units. We perform extractive summarization by selecting sentences using information-theoretic measures over representations obtained from SemAE. Our experiments reveal that dictionary element representations from SemAE form clusters, which capture distinct semantics. Our model provides fine-grained control to users to model surface-level text attributes (like redundancy, informativeness etc.) in the representation space. SemAE outperforms existing extractive opinion summarization methods on SPACE and AMAZON datasets. Finally, SemAE representations can be leveraged to explore different forms of control on the summary generation (e.g. multi-aspect sumamrization) using our inference framework. Future works can focus on better representation learning systems to handle use-cases with noisy or sparse textual data. This work was supported in part by NSF grants IIS2112635 and IIS2047232. We also thank the anonymous reviewers for their thoughtful and constructive comments. We do not foresee any ethical concerns from the technology presented in this work. We used publicly available datasets, and do not annotate any data manually. The datasets used have reviews in English language. Human evaluations for summarization were performed on Amazon Mechanical Turks (AMT) platform. Human judges were compensated at a wage rate of $15 per hour.
abstain method abstain abstain method abstain objective other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain result abstain result objective objective result method other other other other other other other other other method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain other abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain method method method abstain method method method method abstain method abstain result abstain abstain result abstain abstain abstain abstain result abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result other abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain objective result result method abstain objective abstain other other method method abstain abstain abstain