Dependency trees have been intensively used with graph neural networks for aspect-based sentiment classification. Though being effective, such methods rely on external dependency parsers, which can be unavailable for low-resource languages or perform worse in low-resource domains. In addition, dependency trees are also not optimized for aspect-based sentiment classification. In this paper, we propose an aspect-specific and language-agnostic discrete latent opinion tree model as an alternative structure to explicit dependency trees. To ease the learning of complicated structured latent variables, we build a connection between aspect-to-context attention scores and syntactic distances, inducing trees from the attention scores. Results on six English benchmarks, one Chinese dataset and one Korean dataset show that our model can achieve competitive performance and interpretability. Aspect-based sentiment classification (ABSA) is the task of recognizing the sentiment polarities of specific aspect categories or aspect terms in a given sentence (Jiang et al., 2011; Dong et al., 2014; Wang et al., 2016; Tang et al., 2016; Li et al., 2018; Du et al., 2019; Sun et al., 2019a; Seoh et al., 2021; Xiao et al., 2021). Different from document-level sentiment analysis, different aspect terms in the same document can bear different sentiment polarities. For example, given a restaurant review  decor is nice though service can be spotty", the corresponding sentiment labels of decor and service are positive and negative, respectively. How to locate the corresponding opinion contexts for each aspect term is a key challenge for ABSA. To this end, recent efforts leverage dependency trees (Zhang et al., 2019; Sun et al., 2019a; Wang et al., 2020). Syntactic dependencies have been shown to better capture the interaction between the aspect and the opinion contexts (Huang decor is be nice though service can spotty (b) Induced tree for decor. (c) Induced tree for service. et al., 2020; Tang et al., 2020). For example, in Figure1(a), using syntactic relations, we can find that the corresponding opinion words for decor and service are nice and spotty, respectively. Despite its effectiveness, dependency syntax has the following limitations. First, dependency parsers can be unavailable for low-resource languages or perform worse in low-resource domains (Duong et al., 2015; Rotman and Reichart, 2019; Vania et al., 2019; Kurniawan et al., 2021). Second, dependency trees are also not optimized for aspect-based sentiment classification. Previous studies transform dependency trees to aspect-specific forms by hand-crafted rules (Dong et al., 2014; Nguyen and Shirai, 2015; Wang et al., 2020) to improve the aspect sentiment classification performance. However, the tree structure is adjusted mainly by the node hierarchy, without optimizing dependency relations for ABSA. In this paper, we explore a simple method to induce a discrete opinion tree structure automatically for each aspect. Two examples are shown in Figure 1. In particular, given a target and a sentence, 2051 our algorithm induces a tree structure recursively according to a set of attention scores, calculated using a neural layer on top of BERT representation of the sentence (Devlin et al., 2019). Starting with the root node, the algorithm builds a tree by selecting one child node on each side of a current node and recursively continue the partition process to obtain a binarized and lexicalized tree structure. The resulting tree serves as the input structure and is fed into graph convolutional networks (Kipf and Welling, 2017) for learning the sentiment classifier. We study policy-based reinforcement learning (Williams, 1992) to train the tree inducer. One challenge is that the generated policy can be easily remembered by the BERT encoder, which leads to insufficient explorations (Shi et al., 2019). To alleviate this issue, we propose a set of regularizers to help BERT-based policy generations. Although our method is conceptually simple and straightforward for the inference stage, we show that it has a deep theoretic grounding. In particular, the attention based tree induction parsers trained using the policy network can be viewed as a simplified version to a standard latent tree structured VAE model (Kingma and Welling, 2014; Yin et al., 2018), where the KL divergence between the prior and the posterior tree probabilities is approximated by attention-based syntactic distance measures (Shen et al., 2018a). Experiments on six English benchmarks, a Chinese hotel review dataset and a Korean automotive review dataset show the effectiveness of our proposed models. The discrete structure also makes it easy to interpret the classification results. In addition, our algorithm is faster, smaller and more accurate than a full variational latent tree variable model. To our knowledge, we are the first to learn aspect-specific discrete opinion tree structures with BERT. We make our code publicly available at https://github.com/CCSoleil/dotGCN . Figure 2 shows the architecture of our proposed model. Given an input sentence x and a specific aspect term a , we induce an opinion tree t according to a recognition network Q  ( t | x, a ) , where  is the set of network parameters. We apply multi-layered graph convolutional networks (GCNs) over the BERT output vectors to model the structural relations in the opinion tree and extract aspect-specific features. Finally, we use an attention-based clas-Figure 2: The model architecture. where  is the set of parameters. To train the model, RL is used for Q  ( t | x, a ) (Section 2.3) and standard backpropagation is used for training P  ( y | x, a, t ) (Section 2.2). Opinion Tree Denote the input sentence as x = w 1 w 2 . . . w n and the aspect as a = w b w b +1 . . . w e . [ b, e ] is a continuous span of [1 , n ] . w i is the i th word. As shown in Figure 1, the opinion tree for a is a binarized tree. Each node contains a word span and at most two children. a is placed at the root node. Except for the root node, 1 each node contains only one word. An in-order traversal over t can recover the original sentence. Ideally, the nodes near the root node should contain the corresponding opinion words, such as nice for decor and spotty for service. Algorithm 1 shows the process of building an opinion tree t for a that conforms to the above conditions using a node score function v , where v i indicates the informative score of the i -th word contributing to the sentiment polarity y of a . v ji is the corresponding scores of words in the span [ i, j ] . We first make the aspect span [ b, e ] as the root node and then build its left and right children from the spans [1 , b  1] and [ e +1 , n ] , respectively. To build the left or right subtree, we first select the element with the largest score in the span as the root node of the subtrees and then recursively use the build_tree call for the corresponding span partitions. Calculating v Following Song et al. (2019), we feed the inputs [CLS] w 1 w 2 . . . w n [SEP] w b w b +1 . . . w e  to BERT 2 to obtain the aspect-specific sentence representation H , and then calculate a set 1 A case study in Appendix shows an example of a root node containing multiple words grilled alaskan king salmon. Input : The scores v n 1 , the aspect span [ b, e ] ; //build the root node ; root  new TreeNode; root.words = w b w b +1 . . . w e ; // w i is the i -th word. root.left = build_tree ( v b  1 1 , 1, b-1); root.right = build_tree( v ne +1 , e+1, n); build_tree( v ji , i , j ): if i > j : return None; node  new TreeNode; k  arg max k   [ i,j ] v k  ; node.words = w k ; node.left = build_tree( v k  1 i , i , k  1 ); node.right = build_tree( v jk +1 , k + 1 , j ); return node; Output : root ; Algorithm 1: Aspect-specific construction algorithm given a scoring function v . where u p , W p and W a,p are model parameters,  is the ReLU activation function, h a is the aspect representation by sum pooling from H b H b +1 . . . H e . in Q  ( t | x, a ) contains the model parameters of BERT, u p , W p and W a,p . Graph Representation Given t and H , we use GCNs to learn the representation vectors for each word. We convert t to an undirected graph G . Specifically, we take each word as a node in G and design the adjacency matrix A  R n  n of G by considering four types of edges. First, we include self loops for each word. Second, we fully connect each word within the aspect term. Third, for the child node w j of the root node, we link w j to each word in a . Last, we consider edges in t between single word nodes except the root node. Formally, A is given by A i,j =    1 if i = j, (self-loops) 1 if i  ( b, e ) and j  ( b, e ) , (aspect words) 1 if i  [ b, e ] and a is the parent node of w j 1 if w i is the parent or a child node of w j 0 otherwise. (2) A is ensured to be symmetric by Eq 2. We then use GCNs to capture the structured relations between word pairs, given the adjacency matrix A between nodes and the representation matrix of the ( l  1) -th layer H l  1  R n  d , the l -th layer representation H l given by a GCN is, H l = f ( AH l  1 W l + b l ) , (3) where f is an activation function (i.e., ReLU), W l  R d  d and b l  R d are the model parameters for the l -th layer. The input to the first GCN layer H 0 is H given by the sentence encoder. Target Aspect Representation We consider both the representation vector of the [CLS] token ( H 0 cls ) and the aspect vectors given by the last GCN layer ( H Nb , H Nb +1 . . . , H Ne ) as the aspect-specific representation vector to query the input sentence representation H 0 . The final aspect-specific feature representation c over the input sentence representation is given by an attention layer,  t = ( H 0 t ) T ( H 0 cls + e (cid:88) i = b H Ni ) ,  = softmax (  ) , c =  H 0 , (4) where  t is the attention scores of a to w t ,  is the normalized scores and c is the final feature. Output layers use c for computing the sentiment polarity scores. The final sentiment distribution is given by a softmax classifier, p = softmax ( W c c + b c ) , (5) where W c and b c are model parameters and p is the predicted distribution. Cross Entropy Loss The classifier is trained by maximizing the log-likelihood of the training samples. Formally, the objective is to minimize L sup =  | D | (cid:88) i =1 (cid:88) a  x i log p i,y a , (6) where | D | is the size of training data, y a is the sentiment label of a in the i -th example x i and p i,y a is the classification probability for a , which is given by Eq 5. The set of model parameters  in P  ( y | x, a, t ) includes GCN blocks and the classifier parameters in Eq 5. Tree Distance Regularized Loss Following Pouran Ben Veyseh et al. (2020), we introduce a syntax constraint to regularize the attention weights. Ideally, the words near to the root node should receive high attention weights. Given an opinion tree t , we compute the tree distance d i for each word i using the length of the shortest path to the root. Given the distances and the attention scores  , we use the KL divergence to encourage the aspect term to attend the contexts with shorter distances. sampling to explore more discrete structures. Since the tree sampling process is a discrete decision making procedure, it is non-differentiable. The gradient can be propagated from L sup in Eq 6 to t and  , but can not be further propagated from t to  . Therefore, we use the policy gradient given by REINFORCE (Williams, 1992) to optimize  of the policy network (Section 2.3). Suppose that the reward function for a latent tree t is R t , the goal of reinforcement learning is to minimize the negative expected reward function, For each t , we use the sentiment log-likelihood log P  ( y | x, t, a ) as R t . Using REINFORCE, the gradient of L rl with respect to  is,  L rl  =  EQ  ( t | x,a ) [ R t  log Q  ( t | x, a )  ] (9) log Q  ( t | x, a ) is the log-likelihood of the generated sample t , which can be decomposed to a sum of log-likelihood at each tree-building step. According to Algorithm 1, each call of build_tree ( v ji , i, j ) involves selecting an action k from the span [ i, j ] given the scores v nm . The action space contains j  i +1 actions. The log-likelihood of this action is given by, log  k = log exp( v k ) (cid:80) jl = i exp( v l ) , i  k  j. In particular, we use v p in Eq 1 as the score function v . Enumerating all possible trees to calculate the expectation term in Eq 9 is intractable, and we use a Monte Carlo method (Rubinstein and Kroese, 2016), approximating the training objective by taking M samples, EQ  ( t | x,a ) [ R t  log Q  ( t | x, a )  ]  1 MM (cid:88) i =1 R t i  log Q  ( t i | x, a )  . Attention Consistency Loss Instead of solely relying on the reinforced gradient to train the policy network, we also apply an attention consistency loss to directly supervise the policy network. Note that there are two attention scores in our model. The first is the attention score s p defined in Eq 1, which is trained by the reinforcement learning algorithm. The second is the attention score  defined in Eq 4 for extracting useful context features for the aspect-specific classifier. is trained via end-to-end back propagation. Intuitively, words that receive the largest attention scores should be effective opinion words of the target aspect. Therefore, it should be put closer to the root node by the policy network. To this end, we enforce a consistent regularization between the two attention scores so that polarity oriented attention  can be directly used to supervise the scoring policy s p . Formally, L att is given by, L att = KL ( where L sup is the supervised loss, L rl is the reinforcement learning loss, L att is a novel attention consistency loss and L td is a loss to guide the attention score distributions by tree constraints. rl ,  att and  td are hyper-parameters. Interestingly, L sup , L rl and L att can be unified in a theoretic framework using variational inference (Kingma and Welling, 2014). We show in this section, that our method can be viewed as a stronger extension to a latent tree VAE model. To model P  ( y | x, a ) , we introduce a latent discrete structured variable t . Formally, the training objective is to minimize the negative log-likelihood, LMLE =  log P ( y | x, a,  ) =  log (cid:88) t P  ( y, t | x, a ) , (14) Eq 14 calculates log-of-sum over all possible trees t , which is exponential. Eq 14 can be approximated by the evidence lower bound (ELBO) using variational parameters  (Kingma and Welling, 2014; Yin et al., 2018), LELBO =  E q  ( t | x,y,a ) [log P  ( y | x, a, t )] + KL (cid:16) q  ( t | x, y, a ) , p  ( t | x, a ) (cid:17) , (15) where p  ( t | x, a ) is the prior distribution for generating latent trees, q  ( t | x, y, a ) is the corresponding posterior distribution, log P  ( y | x, a, t ) is the log-likelihood function by assuming 2054 that the latent tree t is already known, and E q  ( t | x,y,a ) [log P  ( y | x, a, t )] is the expected log-likelihood function over q  ( t | x, y, a ) by considering all the potential trees. The KL term acts as a regularizer to force the matching of the prior and the posterior distributions. During training, q  ( t | x, y, a ) is used to induce the tree. For inference, p  ( t | x, a ) is used since y is still unknown. In practice, a scale hyper-parameter  can be used to control the behaviour of the KL term (Bow-man et al., 2016b), LELBO =  E q  ( t | x,y,a ) [log P  ( y | x, a, t )] +  KL (cid:16) q  ( t | x, y, a ) || p  ( t | x, a ) (cid:17) . The first term is an expectation term and the second term is a KL term . Eq 16 is a standard VAE model for the ABSA task, which, however, has not been discussed in the research literature. It can be trained using the tree entropy (Kim et al., 2019b) and neural mutual information estimation (Fang et al., 2019). However, both are slow because they both need to consider a large batch of tree samples. To model q  ( t | x, y, a ) , we instead calculate a score function s q for the posterior by a MLP layer similar to Eq 1, s q = softmax (cid:16) u q  ( W q H  + W a,q h  a ) (cid:17) , (17) where u q , W q and W a,q are parameters, H  and h  a are the posterior sentence and aspect representations respectively given y . To ensure that y can guide the encoder, we feed the input sequence together with y to BERT by using [CLS] w 1 w 2 . . . w n [SEP] w f w f +1 . . . w e y to obtain H  . Our method can be regarded as a novel simplification to the above model, which can be shown by correlating the expectation term and the KL term defined in Eq 16 with the attention scores in Eq 1 and Eq 4, respectively. In particular, we consider converting t into a special type of tree distance, namely the aspect-to-context attention scores. Then we delegate the probability distribution over structured tree samples to a set of attention scores. Intuitively, if the attention scores are similar, the generated trees should be highly similar. E q  ( t | x,y,a ) [log P  ( y | x, a, t )]  = E q  ( t | x,y,a ) [log P  ( y | x, a, t )  log q  ( t | x, y, a )  ] Assuming that the posterior q  ( t | x, y, a ) is approximate to Q  ( t | x, a ) given by the recognition network, Eq 18 is equivalent to L rl in Eq 11. Approximate KL Term The KL term resembles L att in Eq 12 for  =  att , namely KL (cid:16) q  ( t | x, y, a ) || p  ( t | x, a ) (cid:17)  KL (  , s p ) . First, we delegate the probability distribution over tree samples to a set of attention scores. In particular, we use s p and s q as the proxies for p  ( t | x, a ) and q  ( t | x, y, a ) , respectively. This is equivalent to say that the posterior scores s q and the prior score s p are fed to Algorithm 1 to derive the corresponding trees during training. Second, since both s q and the attention score  in Eq 4 are directly supervised by the output label y , we can safely assume that s q   . Then the KL term KL ( s q , s p ) in Eq 16 becomes KL (  , s p ) , which is the attention-based regularization loss defined in Eq 12. 4 Experiments We perform experiments on eight aspect-based sentiment analysis benchmarks, including six English datasets, one Chinese dataset, and one Korean datase. The data statistics is shown in Appendix A.3. We use Stanza (Qi et al., 2020) as the external parser to produce dependency parses for comparing with dependency tree based models, reporting accuracy (Acc.) and macro-f1 (F1) scores for each model. More details are presented in Appendix A.1. MAMS Jiang et al. (2019) provide a recent challenge dataset with 4,297 sentences and 11,186 aspects. We take it as the main dataset because it is a large-scale multi-aspect dataset with more aspects in each sentence compared to the other datasets. MAMS-small is a small version of MAMS. Chinese hotel reviews dataset Liu et al. (2020) provide manually annotated 6,339 targets and 2,071 items for multi-target sentiment analysis. by Pontiki et al. (2014), restaurant reviews of SemEval 2014 task 4 ( Rest14 ; Pontiki et al. 2014), SemEval 2015 task 12 ( Rest15 ; Pontiki et al. 2015) and SemEval 2016 task 5 ( Rest16 ; Pontiki et al. 2016). These datasets are pre-processed following Tang et al. (2016) and Zhang et al. (2019). We denote our model as dotGCN ( d iscrete o pinion t ree GCN), making comparisons with BERT-based models, including models without using trees and dependency tree based models. In addition, the variational inference baseline (Section 3.1) is denoted as viGCN . Baselines are (1) BERT-SPC is a simple baseline by fine-tuning the vector of [CLS] of BERT from Jiang et al. (2019); (2) AEN . Song et al. (2019) use an attentional encoder with BERT; (3) CapsNet . Jiang et al. (2019) combine capsule network with BERT; (4) Hard-Span . Hu et al. (2019) use RL to determine aspect-specific opinion spans; (5) depGCN . Zhang et al. (2019) applies aspect-specific GCNs over dependency trees; (6) RGAT . Wang et al. (2020) use relational graph attention networks over aspect-centered dependency trees to incorporate the dependency edge type information; (7) SAGAT . Huang et al. (2020) use graph attention network and BERT, exploring both syntax and semantic information in the sequence; (8) DGEDT . Tang et al. (2020) jointly consider BERT outputs and dependency tree based representations by a bidirectional GCN. (9) kumaGCN . Chen et al. (2020) combine the dependency trees and latent graphs induced by self-attention neural networks; 4.2 Development Results We perform development experiments using MAMS since this is the largest dataset and the examples are more challenging compared to the other datasets. We implement three baselines, in-Method MAMS Small Multilingual Acc F1 Acc F1 Ch-F1 Ko-F1 BERT-SPC 82.22 -79.44 -CapsNet 83.39 -80.91 -CapsNet-DR 82.97 -80.09 --BERT-SPC  83.01 82.76 80.91 80.39 80.92 61.17 depGCN + L  td 84.36 83.88 81.59 80.81 NA NA kumaGCN + L  td 84.37 83.83 81.59 81.10 NA NA dotGCN 84.95 84.44 82.34 81.73 81.53 62.78 Table 2: Results on two MAMS datasets and the multilingual review datasets. cluding BERT-SPC, depGCN and kumaGCN. For fair comparison, we also combine depGCN and kumaGCN with the syntax regularization loss in Eq 7 by calculating syntactic distances on the input dependency trees with respect to the aspect terms. Table 1 shows the results on MAMS validation-set. BERT-SPC achieves 84.08 accuracy and 83.52 F1. Surprisingly, the dependency tree based models cannot outperform BERT-SPC, which verifies the limitation of using cross-domain dependency parsers for this task. kumaGCN outperforms depGCN due to its ability to include an implicit latent graph. Adding the syntax regularization loss generally improves the model performance of syntax-based models. In particular, kumaGCN + L td is on par with BERT-SPC. viGCN outperforms kumaGCN + L td and depGCN + L td , which shows the potential of structured latent tree models. Our dotGCN model achieves 84.53 accuracy and 83.97 F1, outperforming all the baselines by a large margin, which empirically shows the induced discrete opinion tree is promising to this task. Compared to viGCN, our model gives better scores. In addition, our model converges nearly 1.8 times faster (0.66h/epoch v.s. 1.25h/epoch) than viGCN. dotGCN does not have to calculate the true posterior distribution over structured tree samples and thus largely reduce computation overhead. Ablation Study Table 1 shows ablation studies on MAMS validation set by removing three proposed loss items during training, namely L td , L rl and L att . We can observe that the model performance degrades after removing either one of them. Removing the syntax regularization loss L td slightly hurts the performance. Without using the attention consistency loss L att , the model falls behind BERT-SPC, which suggests the importance of our proposed attention consistency regularizations. Excluding the reinforcement learning loss leads to the 2056 Model Twitter Laptop Rest14 Rest15 Rest16 Average Acc F1 Acc F1 Acc F1 Acc F1 Acc F1 Acc F1 AEN 75.14 74.15 76.96 73.67 84.29 77.22 ---RGAT 76.15 74.88 78.21 74.07 86.60 81.35 ---BERT-SPC  73.41 72.38 80.56 77.20 84.55 75.74 83.03 63.92 90.75 74.00 82.46 72.65 depGCN  75.58 74.58 81.19 77.67 85.00 78.79 84.13 67.28 91.39 74.25 83.46 74.51 SAGAT 75.40 74.17 80.37 76.94 85.08 77.94 ---DGEDT 77.90 75.40 79.80 75.60 86.30 80.00 84.00 71.00 91.90 79.00 83.98 76.2 depGCN + L  td 75.49 76.73 79.31 75.84 86.43 80.72 84.69 70.89 92.37 79.40 83.66 76.72 dotGCN 78.11 77.00 81.03 78.10 86.16 80.49 85.24 72.74 93.18 82.32 84.74 78.13 Table 3: Results on five SemEval datasets. biggest performance drop (Acc: 84 . 53  83 . 48 ) among the three settings. This shows that the reinforcement learning component plays a central role in the full model. MAMS Table 2 shows the results of dotGCN and the baselines from Jiang et al. (2019) on the MAMS test set. We implement BERT-SPC, denoted as BERT-SPC  , which outperforms the BERT-SPC model of Jiang et al. (2019). Compared to baselines (BERT-SPC, CapsNet, CapsNet-DR and BERT-SPC  ) without using dependency trees, dotGCN gives significantly better results ( p < 0 . 01 ). For fair comparison with dependency tree based models, we also implement depGCN+ L  td and kumaGCN+ L  td . depGCN+ L  td achieves 84.36 accuracy and 83.88 F1 on the MAMS test set. kumaGCN+ L  td gives similar results with 84.37 accuracy and 83.83 F1. Our dotGCN outperforms all the baselines, giving 84.95 accuracy and 84.44 F1. In terms of the averaged accuracy of F1 scores on MAMS and MAMS-small, dotGCN is significantly better than depGCN and kumaGCN ( p < 0 . 05 ). The results demonstrate that the induced aspect-specific discrete opinion trees are promising to handle multi-aspect sentiment tasks. Multilingual The results 3 on the Chinese hotel review dataset are shown in Table 2. dotGCN outperforms the baseline BERT-SPC  by 0.72 accuracy points and 0.61 F1, respectively. The result shows that our model can be generalized across languages without relying on language-specific parsers. On the Korean dataset, we obtain 5.20 accuracy and 11.61 F1 improvements compared to the LCF-BERT (Zeng et al., 2019), which is the 3 Since the Hotel dataset is based on Chinese characters, there are no annotated words. best BERT-based model. These results show that our model can be well generalized to multiple languages and may potentially benefits low-resource languages for this task. SemEval Table 3 shows the results of our model on the SemEval datasets. First, tree based graph neural network models are generally better than BERT-SPC. On the five datasets, which are relatively small, our model still achieves competitive performances in terms of the averaged F1 and accuracy scores as shown in Table 3. In particular, our model in general outperforms depGCN and depGCN + L  td on four out of five datasets, which verifies that the reinforced discrete opinion trees can be promising structured representations compared to auto-parsed dependency trees. We also compare our models with span-based reinforcement learning models (Hard-Span; Hu et al. (2019)) on the dataset of laptops and restaurants preprocessed by Tay et al. (2018). As shown in Table 4, our model outperforms Hard-Span by 2.55 accuracy points on laptops 4 . On restaurants, our model achieves a comparable result to Hard-Span. It shows that the opinion tree is a better representation compared to an opinion span. Figure 3a and Figure 3b show the induced tree and dependency parse for the aspect term scallops, respectively. The opinion words unique and tasty 4 Since the code of span-based RL methods is not publicly available, we do not include a significant test here. (b) The corresponding dependency tree by Stanza. In the induced tree by dotGCN, the opinion word tasty and unique are 2 and 3 depths from the aspect scallops respectively, which shows that dotGCN can potentially handle complex interactions among aspects and opinion contexts. In addition, the tree induced by dotGCN is binarized, and the root node can contain multiple words as shown in Figure 4a. Figure 4a and Figure 4b show the induced trees for two aspect terms with different sentiment polarities. For creme brulee, the policy network assigns high weights to both delicious and savory. Interestingly, it assigns a higher weight to delicious than savory, though savory is closer to its aspect term than delicious. For appetizer, the word interesting receives higher attention scores than the other two sentiment words. These results show that dotGCN is able to distinguish different sentiment contexts for different aspect terms in the same sentence. Distances between Aspect Terms and Opinion Words Figure 5 shows the distances between aspect terms and opinion words. We use the annotated opinion words of Rest16 provided by Fan et al. (2019) to compare our induced trees and dependency trees. The distances calculated over the original sequences are also included. We can observe creme brulee 7 , 8 the 6 but 5 appetizer 1 the 0 interesting 3 was 2 , 4 delicious 13 savory 11 was 9 very 10 and 12 . (b) An induced tree for appetizer. that the distance distribution over the sequences is relatively flat compared to that over tree structures. For the two tree structures, nearly 90% of opinion words are within 3 depths from the aspect terms. The distance distribution of our induced trees is similar to that of the dependency trees, which empirically demonstrates that induced discrete trees are able to capture the interactions between aspect terms and opinions. By treating dependency trees as gold standard, our tree inducer obtains 35.4% unlabeled attachment scores (UAS), which shows the induced trees are significantly different from the dependency trees although both can connect opinion words with aspect terms. cation accuracy of the MAMS test set with respect to the aspect frequency. For aspect terms which appear in the training corpus, both methods give similar results. However, for unseen aspects, dotGCN gives better results than depGCN. This is potentially due to the severe parsing errors for the low-frequent aspects. dotGCN does not depend on external parsers and thus can circumvent this problem. It empirically suggests that the induced tree structures have strong robustness for capturing the aspect-opinion interactions compared to depGCN. Tree Induction for ABSA There has been much work on unsupervised discrete induction (Bow-man et al., 2016a; Shen et al., 2018b; Kim et al., 2019b,a; Jin et al., 2019; Cao et al., 2020; Yang et al., 2021; Dai et al., 2021), which aims to obtain general constituent trees without explicit syntax annotations and task-dependent supervised signals. We focus on learning task-specific tree-structures for ABSA, where the tree is fully binarized and lexicalized. Choi et al. (2018) propose Gumbel Tree-LSTM for learning task-specific tree for semantic compositions. Similarly, Maillard et al. (2019) propose an unsupervised chart parser for jointly learning sentence embeddings and syntax. However, they focus on sentence-level tasks and do not consider aspect information. Aspect-level Sentiment Classification Much recent work has explored neural attention mechanism to this task (Tang et al., 2016; Ma et al., 2017; Li et al., 2018; Liang et al., 2019). Among tree-based methods, Zhang et al. (2019) and Sun et al. (2019b) encode dependency tree using GCN for aspect-level sentiment analysis; Zhao et al. (2019) use GCN to model fully connected graphs between aspect terms; Wang et al. (2020) use relational graph attention networks to incorporate the dependency edge type information, and construct aspect-specific graph structures; Barnes et al. (2021) attempt to directly predict dependency-based sentiment graphs. Tang et al. (2020) use duel-transformer structure to enhance the dependency graph for this task. Our work is similar in that we also consider the structure dependencies, but different in that we rely on automatically induced tree structures instead of external parses. Chen et al. (2020) propose to induce aspect-specific latent graph by sampling from self-attention-based Hard Kumaraswamy distributions (Bastings et al.). However, to achieve competitive performance, their method still requires a combination of external dependency parse trees and the induced latent graphs. Sun et al. (2019a) and Xu et al. (2019) constructed aspect related auxiliary sentences as inputs to BERT (Devlin et al., 2019) for strong contextual encoders. Xu et al. (2019) proposed BERT-based post training for enhancing domain-specific contextual representations for aspect sentiment analysis. Our work shares a similar feature extraction approach, but differently we focus on inducing latent trees for ABSA. We proposed a method to induce aspect-specific discrete opinion trees for aspect-based sentiment analysis, obtaining trees by viewing aspect-to-context attention scores as syntactic distances. The attention scores are trained using both RL and a novel attention-based regularization. Our model empirically achieves competitive performance compared with dependency tree based models, while being independent of parsers. We also provide a theoretic view of our method using variational inference. Zhiyang Teng and Yue Zhang are the corresponding authors. Our thanks to anonymous reviewers for their insightful comments and suggestions. We appreciate Prof. Pengyuan Liu sharing the Chinese Hotel dataset, Prof. Jingjing Wang sharing the reinforcement learning code of Wang et al. (2019), Mr. Chuang Fan helping obtain the MAMS-Small dataset, Prof. Hwanjo Yu and Mr. Dongmin Hyun sharing the Korean automotive datasets, Prof. De-jiang Dou and Mr. Amir Veyseh responding to our questions when reproducing their results on MAMS, Mr. Zhen Wu for releasing their codes of Wu et al. (2020) upon our request. We thank Dr. Xuebin Wang for providing us with 2 V100 GPU cards for use. This publication is conducted with the financial support of Pioneer and Leading Goose R&D Program of Zhejiang under Grant Number 2022SDXHDX0003.
abstain abstain abstain objective method result other abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain objective abstain method abstain abstain method abstain objective result abstain objective abstain abstain objective other abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain other abstain abstain method method method abstain abstain abstain method other abstain method abstain method other abstain abstain abstain other abstain method abstain method method abstain abstain abstain method abstain method other abstain abstain abstain method other method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain other abstain abstain method abstain abstain abstain abstain abstain abstain method abstain method abstain method abstain method abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain result result abstain result result abstain result result abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other method other other other other other other method other other other other method objective abstain result method other other other other other