In the literature, existing studies always consider Aspect Sentiment Classification (ASC) as an independent sentence-level classification problem aspect by aspect, which largely ignore the document-level sentiment preference information, though obviously such information is crucial for alleviating the information deficiency problem in ASC. In this paper, we explore two kinds of sentiment preference information inside a document, i.e., contextual sentiment consistency w.r.t. the same aspect (namely intra-aspect sentiment consistency) and contextual sentiment tendency w.r.t. all the related aspects (namely inter-aspect sentiment tendency). On the basis, we propose a Cooperative Graph Attention Networks (Co-GAN) approach for cooperatively learning the aspect-related sentence representation. Specifically, two graph attention networks are leveraged to model above two kinds of document-level sentiment preference information respectively, followed by an interactive mechanism to integrate the two-fold preference. Detailed evaluation demonstrates the great advantage of the proposed approach to ASC over the state-of-the-art baselines. This justifies the importance of the document-level sentiment preference information to ASC and the effectiveness of our approach capturing such information. Aspect Sentiment Classification (ASC), a fine-grained sentiment classification task in the field of sentiment analysis (Pang and Lee, 2007; Li et al., 2010), aims to identify the sentiment polarity (e.g., positive , negative or neutral ) for each aspect discussed inside a sentence. For example, the sentence  The restaurant has quite low price but the food tastes not good  would be assigned with a positive polarity for the aspect price and with a negative  Corresponding Author: Jingjing Wang. Intra-Aspect Sentiment Consistency Document 1: S1: Excellent food, although the interior could use some help. Category = FOOD#QUALITY , polarity = positive Category = AMBIENCE#GENERAL , polarity = negative S2: The space kind of feels like an Alice in Wonderland setting, without it trying to be that. Category = AMBIENCE#GENERAL , polarity = negative S3: I paid just about $60 for a good meal, tough :) Category = FOOD#QUALITY , polarity = positive Category = FOOD#PRICES , polarity = positive Inter-Aspect Sentiment Tendency Document 2: S1 : If you've ever been along with the river in Weehawken you have an idea of the top of view the chart house has to offer. Category = LOCATION#GENERAL , polarity = positive S2: Add to that great service and great food at a reasonable price and you have yourself the beginning of a great evening. Category = SERVICE#GENERAL , polarity = positive Category = FOOD#QUALITY , polarity = positive Category = FOOD#PRICES , polarity = positive S3: The lava cake dessert was incredible and I recommend it. Category = FOOD#QUALITY , polarity = positive Figure 1: Two documents from SemEval 2016 (Pontiki et al. (2016)) datasets, where aspect category is defined as the entity E and attribute A pair (i.e., E#A ). polarity for the aspect food . Over the past decade, the ASC task has been drawing more and more interests (Tang et al., 2016b; Wang et al., 2018) due to its wide applications, such as e-commerce customer service (Jing et al., 2015), public opinion mining (Wang et al., 2019c) and Question Answering (Wang et al., 2019a). In the literature, given the ASC datasets (Pon-tiki et al. (2016)) where aspects (i.e., entity and attribute) are manually annotated comprehensively sentence by sentence, previous studies model the aspect sentiment independently sentence by sentence, which suffer from the problem of ignoring the document-level sentiment preference information. In this study, we argue that such document-level sentiment preference information is crucial to 3668 remedy the information deficiency issue in ASC. On one hand, we assume that the sentences in a document involving the same aspect tend to have the same sentiment polarity on this aspect. For instance, in Document 1 , both the sentence S1 and S2 involve aspect AMBIENCE#GENERAL . Although it is difficult to infer the negative sentiment for aspect AMBIENCE#GENERAL through the clause without it trying to be that in S2 , we can infer that the sentiment of aspect AMBI-ENCE#GENERAL is more likely to be negative according to S1 . This is because it is easier to infer negative for aspect AMBIENCE#GENERAL through the clause interior could use some help in S1 . Therefore, a well-behaved approach should capture the contextual sentiment consistency w.r.t. the same aspect (namely intra-aspect consistency for short) information. On the other hand, we assume that the sentences in a document tend to have the same sentiment polarity on all the related aspects. For the example of Document 2 where the sentence S2 involves multiple aspects, it is really hard to precisely predict the sentiment polarity for each aspect. However, when taken the context into consideration, the sentiment polarity for each aspect in S2 is largely possible to be positive , since all the neighboring sentences express the positive sentiment polarity for their aspects. Therefore, a well-behaved model should capture the contextual sentiment tendency w.r.t. all the related aspects (namely inter-aspect tendency for short) information. To well accommodate the above two kinds of document-level sentiment preference information, we propose a Cooperative Graph Attention Networks (CoGAN) approach to ASC. Specifically, two graph attention networks are constructed to model the two-fold sentiment preference with the attention weight to measure the preference-degree. Furthermore, considering that the two-fold preference can jointly influence the sentiment polarities for aspects, we propose an interactive mechanism to jointly model the two-fold preference for obtaining better aspect-related sentence representation. Detailed evaluation shows our proposed CoGAN approach significantly outperforms the state-of-the-art baselines, including the three top-performed systems from SemEval-2015 Task 12 and SemEval-2016 Task 5 (Pontiki et al., 2015, 2016). In this section, we first review the Aspect Sentiment Classification (ASC) task, and then introduce the related studies on graph-based neural networks. Aspect Sentiment Classification. The ASC task aims to predict the sentiment polarity for each aspect discussed inside a sentence. Existing studies mainly focus on utilizing various approaches (e.g., attention mechanism and memory network) to align each aspect and the sentence for learning aspect-related sentence representation. Wang et al. (2016) propose an attention-based LSTM in order to explore the potential correlation of aspects and sentiment polarities in ASC. Wang et al. (2018) propose a hierarchical attention network to incorporate both words and clauses information for ASC. He et al. (2018a) propose an attention-based approach to incorporate the aspect-related syntactic information for ASC. Tang et al. (2016b) and Chen et al. (2017) design deep memory networks to align the aspect and sentence for ASC. Lin et al. (2019) propose a semantic and context-aware memory network to integrate aspect-related semantic parsing information for performing ASC. Wang et al. (2019a) and Wang et al. (2019b) leverage reinforcement learning grounded approaches to select aspect-relevant words for ASC. Recently, a few studies have recognized the information deficiency problem in ASC and attempted to using external information to improve the performance of ASC. He et al. (2018b) and Chen and Qian (2019) incorporate the knowledge from document-level sentiment classification to improve the performance of ASC. Ma et al. (2018) propose an extension of LSTM to integrate the commonsense knowledge into the recurrent encoder for improving the performance of ASC. In addition, it is worthwhile to note that Haz-arika et al. (2018) also investigate the inter-aspect sentiment dependency for ASC, but is limited to capture this information inside a single sentence. In summary, all the above studies ignore the document-level sentiment preference information, which can be leveraged to effectively mitigate the information deficiency problem in ASC. Graph-based Neural Networks. In recent years, graph-based neural networks have received more and more attentions. As a pioneer, Kipf and Welling (2017) present a simplified graph neural network model, called graph convolutional networks (GCN), which has been exported to several tasks such as scene recognition (Yuan et al., 2019), 3669 semi-supervised node classification (Zhang et al., 2019b), text-to-SQL parsing (Bogin et al., 2019) and relation extraction (Sahu et al., 2019). On this basis, some other improved Graph-based Neural Networks are proposed. Morris et al. (2019) propose a generalization of Graph-based Neural Networks, so-called k-dimensional GNNs (k-GNNs), which can take higher-order graph structures at multiple scales into account. Cao et al. (2019) propose a novel Multi-channel Graph Neural Network model to learn alignment-oriented knowledge graph embeddings by robustly encoding two knowledge graphs via multiple channels. More recently, there exist several studies also adopting graph-based neural networks to ASC. For instance, Hou et al. (2019) and Zhang et al. (2019a) build GCN over the dependency tree of a sentence to exploit syntactical information and word dependencies for learning better aspect-related sentence representation for ASC. Different from all the above studies, this paper proposes a novel Cooperative Graph Attention Networks approach to capture the document-level sentiment preference information in ASC. To our best knowledge, this is the first attempt to incorporate this information for the ASC task. In this section, we formulate the Aspect Sentiment Classification (ASC) task as follows. In each document D with sentences 1 { s 1 , s 2 , ..., s I } , given a sentence s i , i  { 1 , 2 , ..., I } and its aspect a k , k  { 1 , 2 , ..., K } , the ASC task aims to predict the sentiment polarity (cid:2) for aspect a k by automatically learning the aspect-related sentence representation r i of sentence s i . Here, I is the number of sentence s i , and K is the number of aspect a k inside the document. In this paper, we propose a Cooperative Graph Attention Networks (CoGAN) approach with two types of Graph Attention Networks (GAN) to incorporate the two-fold preference information respectively. Figure 2 shows the overall architecture of the CoGAN approach which consists of five major blocks: 1) Encoding Block; 2) Intra-Aspect Consistency Modeling Block; 3) Inter-Aspect Tendency Modeling Block; 4) Interaction Block. 5) Softmax 1 Like Pontiki et al. (2015), all aspects of every sentence are unrolled in a document. Graph Attention Network (GAN) (Velickovic et al., 2017) is a new graph neural network architecture including attention mechanism, which enables specifying different attention weights to different vertices in a neighborhood. In principle, GAN can aggregate the features of neighboring nodes and also can propagate the information of a vertex to its nearest neighbors. From this regard, GAN is capable of sufficiently modeling local contextual information for learning the representation of each vertex. Formally, given a graph G ( V, E ) where V and E denote the vertices and edges respectively, GAN updates each new vertex vector  h i of vertex v i by considering neighboring vertices' vectors { h j } I j =1 with the following formulas:  h i = tanh( I (cid:2) j =1  ij W h j + b )  ij = exp( f ( w (cid:3) [ W h i ; W h j ])) (cid:3) It =1 exp( f ( w (cid:3) [ W h i ; W h t ])) (1) where  ij is the attention weight (i.e., the edge weight) between vertex v i and vertex v j . f (  ) is a LeakyReLU activation function. [; ] denotes vector concatenation. W  R d  d and w  R 2 d are the trainable parameters. As a text encoding mechanism, BERT (Devlin et al., 2019) can be fine-tuned to create state-of-the-art models for a range of NLP tasks, e.g., text classification and natural language inference. In our approach, we use BERT-base 2 (uncased) model to encode both the aspect and the sentence as follows. Aspect Encoding. Since an aspect a k consists of an entity e entity and an attribute e attribute (Pon-tiki et al., 2015), we process the entity-attribute pair ( e entity , e attribute ) into the input pair format of BERT as: [CLS] e entity [SEP] e attribute [SEP] Then, we feed the entity-attribute pair into BERT and regard the mark [CLS] representation as the aspect vector e k  R d of the aspect a k . 2 https://github.com/google-research/bert 3670 Sentence Input:  Excellent food, although the interior could use some help. Sentence Encoding. We borrow the approach proposed by Sun et al. (2019) to generate the aspect-related sentence representation, which has achieved promising performance for the ASC task. Following Sun et al. (2019), we first process the sentence s i and its corresponding aspect a k into the input pair format of BERT as: [CLS] s i [SEP] question ( a k ) [SEP] where question (  ) denotes the construction of auxiliary question sentence for aspect a k proposed by Sun et al. (2019). For example, the auxiliary sentence for aspect FOOD#PRICE is constructed as  what do you think of the food and price? . Then, we similarly feed the above pair into BERT (shared with aspect encoding) and obtain the aspect-related sentence vector v i  R d of the sentence s i . Further, we fine-tune BERT and update both the aspect vector e k and sentence vector v i according to Eq.(8). In our approach, we propose a consistency-aware GAN to model the intra-aspect consistency. Given a document D with sentences { s 1 , s 2 , ..., s I } , the consistency-aware GAN is denoted as a bipartite graph G ( S (cid:4) A, E sa ) . Here, S and A are two disjoint sets of vertices, denoting the sentence vertices and the aspect vertices respectively. E sa is the set of the edge between the sentence s i  S and its corresponding aspect a k  A in the document D . On the basis, the intra-aspect consistency is formulated as that sentence vertices { s i } I (cid:3) i =1 sharing the same neighboring aspect vertex a k and located in the same document tend to have the same sentiment for this aspect a k . Here, I (cid:4) denotes the number of sentences sharing the same aspect a k . Nevertheless, there still possibly exist some sentiment inconsistency cases. Considering all the scenarios above, we use the graph attention mechanism (Velickovic et al., 2017) to measure the preference-degree, where the attention weight (preference-degree) is computed as the edge weight between the sentence vertex s i and the aspect vertex a k in a document. Specifically, according to Eq.(1), the attention weight  ik between sentence s i and aspect a k is computed as follows:  ik = exp( f ( w (cid:3) [ W v v i ; W e e k ])) (cid:3) I (cid:3) t =1 exp( f ( w (cid:3) [ W v v t ; W e e k ])) (2) where W v , W e  R d  d and w  R 2 d are the trainable parameters. As a vertex in G ( S (cid:4) A, E sa ) , the sentence s i is then encoded as the aspect-related sentence representation  v ( intra ) i according to the following proposed formula by modifying Eq.(1). where (cid:3) I (cid:3) j =1  jk W v j is the vector representation of the aspect vertex a k , which is weighted added to the sentence vector v i for enhancing the aspect-related sentence representation. W  R d  d , b  R d are trainable parameters. In our approach, we leverage a tendency-aware GAN to model the inter-aspect tendency. Given a document D with sentences { s 1 , s 2 , ...s I } , the 3671 tendency-aware GAN is denoted as an undirected graph G ( S, E ss ) . Here, S is the set of sentence vertices. E ss is the set of the edge between sentence s i  S and sentence s j  S in the document D . On the basis, the inter-aspect tendency assump-tion is formulated as that the sentence vertex s i tend to have the same sentiment with the neighboring sentence vertices { s j } Ij =1 inside a same document. Similar to intra-aspect consistency modeling block, according to Eq.(1), the following formula is applied to compute the attention weight  ij between the sentence vertex s i and the sentence vertex s j from the same document:  ij = exp( f ( w (cid:3) [ W 1 v i ; W 2 v j ])) (cid:3) It =1 exp( f ( w (cid:3) [ W 1 v i ; W 2 v t ])) (4) where W 1 , W 2  R d  d and w  R 2 d are the trainable parameters. As a vertex in G ( S, E ss ) , according to Eq.(1), sentence s i is encoded as the new sentence representation  v ( inter ) i , i.e.,  v ( inter ) i = tanh( I (cid:2) j =1  ij W  v j + b  ) (5) where W   R d  d , b   R d are the parameters. Since the above two-fold preference can jointly affect the sentiment for aspect a k in s i , we make the two-fold preference pairwisely interact with each other for cooperatively boosting the performance. Especially, after obtaining the two sentence representations  v ( inter ) i and  v ( intra ) i of sentence s i from the above two-fold preference modeling blocks, we propose an interactive mechanism to make an interaction between the two vectors instead of simply concatenating them. This is because a simple vector concatenation does not account for any interactions between the latent features of the two-fold preference, which is insufficient for cooperatively modeling the two-fold preference. In detail, this interactive mechanism leverages two strategies to learn the sentence representation. Pyramid Layers. As proposed in He et al. (2016), the model using a small number of hidden units for higher layers can learn more abstractive features. Inspired by this, we add pyramid hidden layers (see Figure 2) on the concatenated vector for interacting the latent features of the two-fold preference, where the bottom layer is the widest and each successive layer has a smaller number of neurons. More specifically, the sentence vector  v li  R 2 d  ( 12 ) l  1 of the l -th layer is defined as:  v li = tanh( W l  v l  1 i + b l ) (6) where  v 1 i = [ v ( inter ) i ;  v ( intra ) i ] and adding one layer will make the dimension of the sentence vector half. W l  R 2 d  ( 12 ) l  1  2 d  ( 12 ) l  2 and b l are the trainable parameters. l  [1 , L ] denotes the layer index. Adaptive Layer-Fusion. To sufficiently fuse the sentence representations at different level of abstractions, an adaptive fusion mechanism is proposed to fuse the representations of all layers for computing the final sentence vector r i  R d of vertex s i as follows: r i = tanh( W r ( L (cid:5) l =1  i  v li ) + b r ) (7) where (cid:6) denotes the concatenation of multiple vectors. W r and b r  R d are the trainable parameters. L is the number of added layers and set to be 4 fine-tuned according to the development data. = [  1 , ...,  L ] is a normalized weights vector to weigh each layer, which is learned during training. After obtaining the final sentence vector r i of sentence s i , we feed it to a softmax classifier m = W r i + b , where m  RC is output vector; W and b are the trainable parameters. Then, the probability of labeling sentence with sentiment polarity (cid:2)  [1 , C ] is computed by p  ( (cid:2) | r i ) = exp( m (cid:2) ) (cid:2) C =1 exp( m  ) . Finally, the label with the highest probability stands for the predicted sentiment polarity for the aspect a k . We use cross-entropy loss function to train our model end-to-end given a set of training data ( s i , a k , y i ) from corpus C , where s i is the i -th sentence to be predicted, a k is its corresponding aspect and y i is the ground-truth sentiment polarity for aspect a k . The objective of learning  is to minimize the loss function as follows: J (  ) = E ( s i ,a k ,y i ) C [  log p  ( y i | r i )]+  2 ||  || 22 (8) where E denotes the expectation-maximization. denotes the sampling operation. denotes all the trainable parameters of our CoGAN approach. is a L 2 regularization. Data Settings. We conduct experiments on four datasets 3 , i.e., two datasets ( restaurant15 and laptop15 ) released by SemEval-2015 Task 12 (Pontiki et al., 2015) and the other two datasets ( restaurant16 and laptop16 ) released by SemEval-2016 Task 5 (Pontiki et al., 2016), to verify the effectiveness of our proposed approach. Wherein, each dataset averagely consists of about 442 documents and one document averagely contains 4.9 sentences. Moreover, each sentence is annotated with one or multiple aspects and a sentiment polarity (i.e., positive , negative or neutral ) for each aspect. Additionally, we set aside 10% from the training set as the development data to tune the hyper-parameters. Implementation Details. In our experiments, all hyper-parameters are tuned according to the development set. Specifically, BERT is optimized by the Adam optimizer (Kingma and Ba, 2015), where  1 = 0 . 9 and the initial learning rate is 10  4 . Other parameters of BERT are following (Devlin et al., 2019). For our CoGAN approach, we adopt another Adam optimizer with an initial learning rate of 10  3 and  1 = 0 . 95 for cross-entropy training. The regularization weight of parameters is 10  5 . The dropout rate is 0.25. All matrix and vector parameters of the layers are initialized by the Glorot uniform (Glorot and Bengio, 2010). Evaluation Metrics. The performance is evaluated using standard Accuracy (Acc.) and Macro-F1 (F1) (Wang et al., 2017). Moreover, t -test is used to evaluate the significance of the performance difference (Yang and Liu, 1999). Baselines. We give the following baseline approaches for comparison in order to comprehensively evaluate the performance of our approach. 1) TC-LSTM. This approach extends LSTM by considering the aspect information where a forward LSTM and a backward one towards the aspect are adopted (Tang et al., 2016). 2) ATAE-LSTM. This approach models the aspect-related context words via attention-based LSTM (Wang et al., 2016). 3) RAM. This approach captures importance of context words for a specific aspect with a deep memory network and the results of multiple attentions are non-linearly combined with a recurrent neural network (Chen et al., 2017). 4) IAN. This approach is an interactive learning approach, which 3 Detail statistics can be seen in Pontiki et al. (2015, 2016). models the contexts and aspects via LSTM and then interactively learns attentions in the contexts and aspects (Ma et al., 2017). 5) Clause-Level ATT. This approach employs hierarchical attention to incorporate the clause information for ASC (Wang et al., 2018). 6) LSTM+synATT+TarRep. This approach employs syntax-aware attention to learn aspect-related representation for ASC. This is a state-of-the-art approach proposed by He et al. (2018a). 7) BERT. This approach transforms ASC from a single sentence classification task to a sentence pair classification task. In our implementation, we regard the pair of sentence and its aspect as the input pair of BERT-base model (Devlin et al., 2018) for performing ASC. 8) CADMN. This approach employs attention model to attend on relevant aspects for enhancing the aspect representation. This is a state-of-the-art approach proposed by Song et al. (2019). 9) IMN. This approach is a multi-task learning approach, which employs a novel message passing mechanism to better exploit the correlation among the tasks related to ASC. This is a state-of-the-art approach proposed by He et al. (2019). 10) BERT-QA. This approach is an extension of the above BERT baseline proposed by Sun et al. (2019). In this study, we adopt BERT-pair-QA-M in our implementation. This is another state-of-the-art approach for ASC. 11) Sentiue. This is the best-performed system in SemEval-2015 Task 12 (Saias, 2015), which achieves the best accuracy scores in both the laptop15 and restaurant15 domains. 12) XRCE. This is the best-performed system in SemEval-2016 Task 5 (Pontiki et al., 2016), which achieves the best accuracy score in the restaurant16 domain. 13) IIT-TUDA. This is also the best-performed system in SemEval-2016 Task 5 (Pontiki et al., 2016), while achieving the best accuracy score in the laptop16 domain. 15) CoGAN w/o Intra-Aspect Consistency. Our approach only modeling Inter-Aspect Tendency. 16) CoGAN w/o Inter-Aspect Tendency. Our approach only modeling Intra-Aspect Consistency. 17) CoGAN w/o Interactive Mechanism. Our approach only concatenating the two vectors  v ( inter ) i and  v ( intra ) i instead of using interaction block to integrate them. 4.2 Experimental Results Table 1 shows the performance comparison of different approaches. From the table, we can see that, all state-of-the-art approaches, such as Clause-3673 Approaches Restaurant15 Laptop15 Restaurant16 Laptop16 Acc. Level ATT , CADMN and IMN , perform better than TC-LSTM . This result demonstrates the effectiveness of using a proper attention mechanism to learn the aspect-related sentence representation for performing the ASC task. The BERT-based approaches, i.e., BERT and BERT-QA , perform better than the above approaches on almost all datasets. This result encourages to utilize the pre-trained BERT model as the aspect and sentence encoder for the ASC task. Furthermore, our approach CoGAN w/o Intra-Aspect Consistency and CoGAN w/o Inter-Aspect Tendency outperform most of the above state-of-the-art approaches. This encourages to model the intra-aspect consistency or inter-aspect tendency information for the ASC task. In comparison, when incorporating both the two-fold sentiment preference information, our approach CoGAN outperforms all the above baseline approaches and even significantly outperforms ( p -value < 0.05) all three top-performed systems from SemEval-2015 Task 12 and SemEval-2016 Task 5, i.e., Sentiue , XRCE and IIT-TUDA on all four datasets. Impressively, compared to TC-LSTM , our approach achieves the average improvement of 11.6% ( Accuracy ), 14.3%( Macro-F1 ) on the two restaurant datasets and 9.1% ( Accuracy ), 12.6%( Macro-F1 ) on the two laptop datasets. Significance test shows that these improvements are all significant ( p -value < 0.01). These results highlight the importance of incorporating both the intra-aspect consistency and inter-aspect tendency information in a document for the ASC task. In addition, it is worthwhile to note that CoGAN outperforms CoGAN w/o Interactive Mechanism , which encourages to employ the proposed interactive mechanism to cooperatively integrate the two-fold sentiment preference information. We provide a qualitative analysis of our CoGAN approach on the test sets of the restaurant16 and laptop16 datasets respectively. Figure 3 shows two documents, along with their predicted sentiment for each aspect, and probabilities of the ground-truth label by different approaches. From this fig-ure, we can see that: 1) For the example of Document 1, it is difficult to infer the sentiment for aspect LAPTOP#MISCELLANEOUS (to classify) in S1 since the long sentence S1 involves syntactic complications. Despite this, S8 expresses explicit negative polarity for the same aspect LAP-TOP#MISCELLANEOUS . Considering this intra-aspect consistency information, our CoGAN approach can still give the correct negative for aspect LAPTOP#MISCELLANEOUS , while both BERT 3674 Document 1 (Intra-Aspect Consistency) Document 2 (Inter-Aspect Tendency) BERT ( positive )  P( negative )=0.21 IMN ( positive )  P( negative )=0.26 CoGAN(Our Approach) ( negative )  P( negative )=0.82 BERT ( negative )  P( positive )=0.25 IMN ( neutral )  P( positive )=0.28 CoGAN(Our Approach) ( positive )  P( positive )=0.87 S1: I would've given 5 stars, had it not been for the hours of updates I've had to do to this upon arrival. and IMN give wrong predictions. This justifies the effectiveness of the intra-aspect consistency information for ASC. 2) For the example of Document 2, it is rather difficult to infer the sentiment for aspect RESTAURANT#GENERAL (to classify) in S1 , since the sentence S1  hidden little jem.  is too short and can not provide sufficient information to predict the positive polarity for aspect RESTAURANT#GENERAL . Despite this, CoGAN considering the inter-aspect tendency information can still give the positive for aspect RESTAU-RANT#GENERAL . This is reasonable because take the whole context into consideration, this restaurant has good reputations due to its service and food. To better illustrate the effectiveness of modeling the intra-aspect consistency and inter-aspect tendency information, we systematically investigate both sentiment preference phenomena in all the four evaluation datasets respectively. Specifically, we sample 200 sentence 4 pairs inside each dataset and calculate the ratio that the two sentences in the pair have 4 Sentences are repeated in a document according to the unrolled aspects. For instance, a sentence with two aspects will be repeated twice, each sentence with only one aspect. the same sentiment for their corresponding aspects. Especially, we propose three sampling strategies as follows. 1) Randomly Sampling: randomly selecting sentence pairs inside each dataset. 2) Inter-Aspect Tendency Sampling: randomly selecting the sentence pairs under the premise that each two sentences should be located in the same document. 3) Intra-Aspect Consistency Sampling: randomly selecting the sentence pairs under the premise that each two sentences should be located in the same document and should have the same aspect. Figure 4 shows the statistical results of the three sampling strategies on all four datasets. From this figure, we can see that Inter-Aspect Tendency Sampling and Intra-Aspect Consistency Sampling impressively outperform Randomly Sampling by 27.9 % and 34% respectively. Moreover, the average ratio of the two highest sampling strategies is up to 84.1%. This is the reason for the effectiveness of our CoGAN approach to ASC, and encourages to leverage CoGAN for incorporating the two-fold sentiment preference information. We randomly analyzed 100 error cases and roughly categorized them into 5 classes briefly introduced as follows. (1) 29% of errors are due to the occurrence of negation words, e.g.,  Nothing really came across as outstanding. . CoGAN incorrectly predicts positive polarity, inspiring us to optimize CoGAN for capturing negation scope better. (2) 27% are due to incorrectly recognizing neutral instances. The shortage of neutral training examples makes it hard to recognize neutral instances, inspiring us to use data augmentation to enlarge the scale of neutral data. (3) 24% are due to the implicit sentiment expression, e.g.,  There is definitely more to 3675 say... . CoGAN incorrectly predicts positive polarity instead of negative. (4) 12% are due to too short sentences (e.g. with less than 5 words), inspiring us to incorporate external ConceptNet knowledgebase to enhance the semantic representation. (5) 8% are due to comparative opinions, e.g.,  I've had better frozen pizza . CoGAN incorrectly predicts positive, inspiring us to investigate whether incorporating syntactic information can remedy this issue. In this paper, we propose a novel Cooperative Graph Attention Networks (CoGAN) approach to Aspect Sentiment Classification (ASC). The main idea of the proposed approach is to incorporate two kinds of sentiment preference information (i.e., the intra-aspect consistency and inter-aspect tendency) in a document for remedying the information deficiency problem in ASC. Experimental results on four datasets from SemEval-2015 and 2016 demonstrate that our approach significantly outperforms a number of competitive baselines, including all the three best-performed systems in the shared tasks of both SemEval-2015 and 2016. In our future work, we would like to improve the performance of the ASC task by using unlabeled data since our graph-based neural network approach is easy to add unlabeled data. Moreover, we would like to apply our approach to other sentiment analysis tasks, e.g., aspect-oriented opinion summarization and multi-label emotion detection. We thank our anonymous reviewers for their helpful comments. This work was supported by three NSFC grants, i.e., No.61751206, No.61672366 and No.61525205. This work was also supported by a Project funded by China Postdoctoral Science Foundation No.2019M661930, a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD), and a joint research project of Alibaba Group and Soochow University.
abstain objective objective abstain abstain abstain abstain abstain objective objective objective objective objective objective abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain objective abstain objective abstain objective other other other other other other other other other other other other other other other other other other other other other other other objective objective method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain method abstain other other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain method objective abstain abstain abstain abstain result abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain result result abstain abstain abstain method abstain result abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain result abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain objective abstain abstain result method other other other