Document-level event factuality identification is an important subtask in event factuality and is crucial for discourse understanding in Natural Language Processing (NLP). Previous studies mainly suffer from the scarcity of suitable corpus and effective methods. To solve these two issues, we first construct a corpus annotated with both documentand sentence-level event factuality information on both English and Chinese texts. Then we present an LSTM neural network based on adversarial training with both intraand inter-sequence attentions to identify document-level event factuality. Experimental results show that our neural network model can outperform various baselines on the constructed corpus. Document-level event factuality identification is the task of deciding the commitment of relevant sources towards the factual nature of an event, and to determine whether an event is a fact, a possibility, or an impossible situation from the view of document. Identifying document-level factuality of events requires comprehensive understanding of documents. As illustrated in Figure 1 where events are in bold , the event  reach  (including its other forms) have various factuality values in different sentences. For example, in paragraph 2,  reach  is impossible/CTaccording to the negative word  denied , while in paragraph 3,  reach  is possible/PS+ due to the speculative word  may . The main contents of this document is  Mexico denied that they will reach an agreement with the U.S. on the new trade deal , and the document-level factuality of the event  reach  is CT-. Document-level event factuality identification is fundamental for document-level NLP applications, such as machine reading comprehension, which aims to have machines read a text passage According to Politico.com, it is said the United States will reach (CT+) an agreement with Mexico on the new trade deal that will replace North American Free Trade Agreement (NAFTA) before December, 2017. However, Mexican Economy Minister Ildefonso Guajardo denied that they plan to reach (CT-) any agreement with the U.S. on the trade deal talks. We are not going to sacrifice the quality of an agreement because of pressure of time. We will keep engaged. he said. Just two days ago, Guajardo said the two sides may reach (PS+) an agreement within hours. The government has not been informed that any agreement will be reached (CT-) yet, said another two Mexican officials.Duringthe past few weeks, the U.S. has been negotiating with Mexico on the new trade deal and has achieved much progress. Thus, some media speculate that they will possibly reach (PS+) an agreement. But now it seems that the negotiations will continue before they can get a good deal. and then answer questions about the text. According to the document in Figure 1, the answer of the following question should be  No , which is consistent with the document-level factuality of the event  reach  (CT-): Q : Does the U.S. reach an agreement with Mexico on the new trade deal before December 2017? A : No. Previous studies mostly reported on sentence-level event factuality identification tasks. On one hand, due to the scarcity of document-level event factuality corpus, these studies only considered the corpora annotated with sentence-level event factuality information, such as ACE 2005 1 , LU (Diab et al., 2009), FactBank (Saur and Pustejovsky, 2009), and UDS-IH2 (Rudinger et al., 2018). On the other hand, previous studies only con-1 https://catalog.ldc.upenn.edu/LDC2006T06 sidered information within sentences, using rules (Saur, 2008; Saur and Pustejovsky, 2012), machine learning models (de Marneffe et al., 2012; Werner et al., 2015; Baly et al., 2018), and combinations of them (Qian et al., 2015; Stanovsky et al., 2017) for modeling. Neural network models have also recently been used for the sentence-level event factuality identification (He et al., 2017; Rudinger et al., 2018; Qian et al., 2018). According to Figure 1, document-level event factuality can not be deduced from each sentence-level factuality separately, but depends on the comprehensive semantic information of sentences. However, no suitable model for document-level task has been proposed yet. To solve the issues above, this paper focuses on document-level event factuality identification. Our contributions can be summarized as follows. 1) We construct a document-level event factuality corpus, i.e. DLEF, on both English and Chinese texts. To our best knowledge, this is the first document-level event factuality corpus. The statistics on the corpora and the experimental results show that our corpus can sufficiently reflect linguistic characteristics of news texts, and provide adequate support on resource for research. 2) We propose an LSTM neural network with both intraand inter-sequence attentions to identify document-level event factuality, and consider dependency paths from speculative and negative cues to the event and sentences containing the event as features. Due to the diversity of various contents of the texts in DLEF corpus, we employ Adversarial Training to improve the robustness of our model. Experimental results show that our model is superior to various baselines. The corpus and code of this paper will be released at https://github.com/qz011/dlef . This section introduces our D ocumentL evel E vent F actuality (DLEF) corpus, including the source, detailed guidelines for both documentand sentence-level event factuality, and the main statistics of the corpus. News texts contain sufficient speculative and negative information that is significant for event factuality identification, and usually focus on one event with a specific topic. Moreover, FactBank (Saur + u CT CT+ CT-CTu PS PS+ PS-(NA) U (NA) (NA) Uu Table 1: Event factuality values. and Pustejovsky, 2009), the sentence-level event factuality corpus, is also based on news texts. Therefore, we choose news texts in both English and Chinese to construct our corpus. The English corpus consists of 1727 documents from January 2017 to January 2018, among which 1506 documents are from China Daily 2 , and 221 documents are from Sina Bilingual News 3 . The Chinese corpus consists of 4649 documents from Sina News 4 . These news documents cover various topics, e.g., politics, economy, culture, military, and society, which can reflect the heterogeneity of language in news texts. Saur (2008) employed modality and polarity to describe event factuality values. Modality conveys the certainty degree of events, such as certain (CT), probable (PR), and possible (PS), while polarity expresses whether the event happened, including positive (+) and negative (-). We use the factuality values in Table 1 according to Saur (2008). Both PR and PS are speculative values and share similar certainty degrees in our corpus, and are merged into PS . U/u means underspecified . PSu and U+/are not applicable (NA) and are not considered. Although CTu is applicable, neither document-level nor sentence-level event can be annotated as CTu in our corpus. We adopt the definition of events proposed by TimeML (Pustejovsky et al., 2003) and consider the events that can be critical for computing the factuality. To ensure that the task is meaningful, we focus on the events that have various types of sentence-level factuality values. If there is more than one suitable event in a document, we annotate them separately. First, the annotation of document-level event factuality is based on the definition, i.e., determining the factuality of an event from the view of 2 http://www.chinadaily.com.cn 3 http://roll.edu.sina.com.cn/english/syxw/index.shtml 4 http://news.sina.com.cn the document requires to understand the semantic of the document, including various sentence-level event factuality. Second, sentence-level event factuality is essential for document-level task, which makes sense when documentand sentence-level factuality of events have different values. Therefore, we annotate the sentence-level event factuality as follows: CTevents are negated by negative cues. For example, the events  enter  and  merger  are governed by negative cues  impossible  and  denied  in sentence S1 and S2, respectively. (S2) Sinopec responded to National Business Daily, and denied the rumors of a merger with PetroChina. PS+ events (e.g.  improve  and  fallen ) are governed by speculative cues (e.g.,  impossible  and  denied ), just as illustrated in sentence S4 and S5. (S4) We think that further investigation may help to improve the treatment of people with similar infections. PSevents are governed by both speculative and negative cues. Different from CT-, PSmeans incompletely negation. For example, the PS-event  noticed  is governed by the speculative cue  probably  and the negative cue  not  in sentence S6, and  fall  is modified by the cues  may  and  not  in sentence S7. (S6) The bus driver had probably not noticed the truck early enough. (S7) Oil prices may not fall sharply due to the strong global demand. Uu events can appear in questions (e.g.,  considering  in sentence S8) and in the intensional contexts with underspecified semantics (e.g.,  raises  in sentence S9): (S8) Is France considering to leave EU? (S9) The US dollar's declination can not be reversed even if the Federal Reserve raises rates three times. The task is trivial if most documents have only one type of sentence-level factuality value, and in this case, document-level factuality probably shares the same value. To understand the usefulness of document-level event factuality identification and DLEF corpus, we launched the statistics of documents with n different types of sentence-level event factuality values shown in Table 2. From the table we can find that for English corpus there are 41.94% CTand 66.06% PS+ documents with different sentence-level event factuality values, but these CT+ documents only cover 11.13%. While for Chinese corpus, these CTand PS+ documents cover 63.41% and 62.15%, but these CT+ documents only make up 14.23%. Table 2 indicates that sentence-level factuality usually agrees with document-level factuality in CT+ documents, making them straightforward to be identified. However, in those non-CT+ documents with non-factual document-level values, sentence-level factuality is likely to have different values from documents, making them more difficult to be identified. In general, English and Chinese corpus have 25.64% and 37.84% documents with different sentence-level event factuality values, indicating this corpus is suitable for the document-level event factuality identification. Table 3 shows the statistics of the DLEF corpus. CT+ document-level events are in the majority, because information reported by news texts is usually real. Kappa (Cohen, 1960) is employed to measure the inter-annotator agreement of annotating documentand sentence-level event factuality between the two independent annotators who annotate the entire corpus, just as shown in Table 4. These two annotators are postgraduate stu-Corpus Statistics English Documents CT-279/16.16% PS+ 274/15.87% PS-12/0.69% Uu 12/0.69% CT+ 1150/66.59% Total 1727 Sentence-LevelEvents CT-662/11.52% PS+ 574/9.99% PS-37/6.44% Uu 71/1.24% CT+ 4401/76.61% Total 5745 Avg. dents who major in NLP. In addition, the Kappa of events on English and Chinese corpus are 0.83 and 0.85, respectively. All the Kappa values are larger than 0.75, proving the effectiveness and meaningfulness of our DLEF corpus. This section describes the LSTM neural network for document-level event factuality identification in detail. As shown in Figure 2, to extract feature representations of events from the view of documents, we consider both intraand inter-sequence attention for dependency paths and sentences. In addition, due to the diversity of contents of doc-softmax ( W 1 h e + b 1 ) Sentences ( S 0 , S 1 ,  , S j1 ) output Input Layer Softmax Layer Dependency Syntactic Paths ( P 0 , P 1 ,  , P i -1 ) LSTM_1 (Intra-Sequence) Embedding Layer LSTM Layer InterSequence Attention Layer P 0 P 1 P 2 P i1 S 0 S 1 S 2 S 3 S j1 0 1 2 i1 LSTM_2 (Intra-Sequence)0 1 2 3 j -1 h e  T  T h sp h ss Figure 2: Neural network architecture for document-level event factuality identification. uments in DLEF corpus, we consider adversarial training to ensure the robustness of our model. For our task, we use the specified events that have been annotated, and utilize the Chinese cues in CNeUn corpus (Zou et al., 2015) and the English cues in BioScope corpus (Vincze et al., 2008) that also considers multi-word cues, e.g., rule out . We do not use any annotated sentence-level event factuality. For one event, we consider all the sentences containing it, and mainly employ the following two features in our model: 1) Syntactic Features : Previous studies (Saur and Pustejovsky, 2012; de Marneffe et al., 2012) have proved the effectiveness of dependency trees on event factuality identification tasks. Hence, we employ the dependency paths from speculative or negative cues to the event as syntactic features. In addition, we also consider the above features in contexts of each sentence containing the event as the input, and set the windows size as 3 , i.e., one sentence before and after the current one. If adjacent sentences contain speculative or negative cues, the dependency path is the concatenation of the path from the cue to the root and the path from the root to the event (Quirk and Poon, 2017). A dependency path or sentence can be represented as X 0 according to the embedding table. We employ LSTM with hidden units n h to model the sequences from both directions to produce the for-ward hidden sequence  H , the backward hidden sequence  H , and the output sequence H =  H +  H . We adopt the attention mechanism to capture the most important information from H , and obtain the output h : H m = tanh( H ) (1)  = softmax( v TH ) (2) h = tanh( H T ) (3) where v  R n h is the parameter. One event can have k sequences X 0 , X 1 , . . . , X k  1 , whose representation is H s = h 0 , h 1 , . . . , h k  1 according to the above equations, where H s  R k  n h . To extract the feature representation h s  R n h from the k sequences, we utilize an inter-sequence attention mechanism that is computed as: H ms = tanh( H s ) (4)  s = softmax( v Ts H ms ) (5) h s = tanh( H s  Ts ) (6) where v s  R n h is the parameter. Suppose that an event has i dependency paths P 0 , P 1 , . . . , P i  1 , and appears in j sentences S 0 , S 1 , . . . , S j  1 . Considering that dependency paths and sentences contain syntactic and semantic information, respectively, we employ two LSTM neural networks defined above to learn vector representations h sp and h ss of dependency paths and sentences, and concatenate them into the feature representation of the event h e : h e = h sp  h ss (7) where  is the concatenation operator. o = softmax( W 1 h e + b 1 ) where W 1  R c  dim( h e ) and b 1  R c are parameters, and c = 5 is the number of categories of factuality values (CT+, CT-, PS+, PS-, Uu). The objective function of the proposed neural network is designed as: LD (  ) =  1 m m  1 (cid:88) i =0 log p ( y ( i ) j | x ( i ) ,  ) (9) where y ( i ) is the golden label of the instance x ( i ) and p ( y ( i ) j | x ( i ) ) is the probability, m is the number of instances, and  is the parameter set to learn. This model with TWO attention layers is denoted as Att 2 in the next section. As described in Section 2, documents in DLEF corpus cover various topics. To improve the robustness of our model, we consider Adversarial Training. Similar to previous work (Miyato et al., 2016; Wu et al., 2017), we add a small adversarial perturbation e adv to word embeddings, and employ the following objective function: L adv ( X |  ) = L ( X + e adv |  ) (10) e adv = arg max (cid:107) e (cid:107) (cid:54) (cid:15) L ( X + e |   ) (11) where   is a fixed copy value of the current  and X is the input. Due to the intractable nature in the computation of Eq. (11), Goodfellow et al. (2014) proposed Eq. (12) to linear L ( X |  ) near X to approximate Eq. (11): e adv = (cid:15) g / (cid:107) g (cid:107) (12) g =  TL ( X |   ) (13) where T is the embedding table. We introduce the experimental settings and the baselines, finally presenting the experimental results and analysis in detail. The PSand Uu documents only cover 1.39% and 1.20% in our English and Chinese corpus, respectively. Therefore, we mainly focus on the performance of CT+, CT-, and PS+. For fair comparison, we perform 10-fold cross-validation on English and Chinese corpora, respectively. In addition to Precision, Recall, and F1-measure for each category of factuality value, we consider macroand micro-averaging to obtain the overall performance of all the categories of factuality values. The hidden units of LSTM are set as n h = 50 . We initialize word embed-dings via Word2Vec (Mikolov et al., 2013), setting the dimensions as d 0 = 100 , and fine-tuning them during training. SGD with momentum is applied to optimize our models. Att 2 and Att 2+AT are the models proposed in Section 3 that consider the contexts, i.e., one sentence before and after the current sentence containing the event as the input. Compared to Att 2, Att 2+AT considers Adversarial Training (AT, the same below). We also consider the following baselines for the comparison with our models: MaxEntVote is a maximum entropy model that only considers the view of AUTHOR (de Marneffe et al., 2012). We use maximum entropy model to identify sentence-level event factuality, and consider voting mechanism, i.e., choose the value committed by the most sentences as the document-level factuality value. We also consider other machine learning models, e.g. Lee et al. (2015), but obtain lower micro-/macro-averaged F1 on English (59.38/33.36) and Chinese corpus (53.91/43.20). SentVote identifies sentence-level event factuality, and does not consider inter-sequence attention in the model proposed in Section 3. Similar to MaxEntVote model, voting mechanism is used to identify document-level event factuality in this SentVote model. MP 2 considers Max-Pooling instead of attention compared with Att 2. Att 1 considers only intra-sequence attention, but not the inter-sequence attention. For an event, we concatenate its i dependency paths and j sentences into one path and one sentence as the input, respectively. Table 5 presents the performances of our models and baselines. MaxEntVote gives relatively lower results than other models, especially on CT-and PS+. SentVote models are better than MaxEntVote, but still obtain lower results than Att 2, which can prove that inter-sequence attention is more useful than voting. Max-pooling only selects the most active information for each dimension of features, While attention takes into account all the features and assigns weights for them according their degrees of importance. Hence, Att 2 gets better results than MP L2. Att 1 only considers the intra-sequence attention and obtains lower results than Att 2, which proves the effectiveness of inter-sequence attention. Att 2 and Att 2+AT achieve better results than other baselines. Compared to Att 2, Att 2+AT considers the adversarial perturbation and training that can alleviate over-fitting. Therefore, Att 2+AT is superior to Att 2, which can prove the effectiveness of adversarial training. On both English and Chinese corpora, the performance of CT+ is better than those of PS+ and CT-. On one hand, it is easier to identify CT+ documents due to their majority. On the other hand, most news texts hardly contain bogus and false contents. Therefore, in most CT+ documents, sentence-level factuality values are consistent with the document-level value, just as in S10. However, in PS+ and CTdocuments with non-CT+ document-level values, sentence-level factuality values have different viewpoints with the corresponding document, varying among CT-, PS+, and CT+, making the task more difficult, e.g., S11. (S10) India successfully tested (CT+) a supersonic missile, capable of destroying an incoming ballistic missile at low altitude. ...... The test (CT+) was carried out from a test range in Odisha , offi-cial sources said. (S11) Argentine navy said it had not contacted (CT-) the SAN Juan submarine. ... ... Some media previously said the navy may have received signals from the submarine and contacted (PS+) it. For Att 2+AT, we also investigate the effects of contexts of the sentences containing events as the input on the performance. The results is given in Table 6, which shows that contexts can improve the performance more significantly on the Chinese corpus than the English corpus. We find that in the Chinese corpus these sentences are commonly in the same paragraph and have a strong semantic coherence. Therefore, information in adjacent sentences can contribute to the identification of the document-level factuality of the events in the current sentences. Sentences S12 and S13 are adjacent sentences in one paragraph. The document-level factuality value of the event  provided  in S12 is CT-. However, the sentence-level value of  provided  is PS+. If we consider S13, the negative cue  denied  can lead to the correct document-level factuality value of  provided . While in the English corpus, similar sentences are much fewer, because paragraphs in most English news texts only contain one or two sentences, and sentences in different paragraphs share less semantic correlation Corpus Systems CT-PS+ CT+ Micro-A Macro-A English MaxEntVote 58.17 35.89 75.14 68.42 56.40 SentVote 70.22 57.85 83.98 78.06 70.68 MP 2 70.57 56.39 83.72 77.65 70.23 Att 1 65.25 53.65 79.18 73.23 66.03 Att 2 73.88 59.29 88.59 81.84 73.92 Att 2+AT 76.87 62.14 89.84 83.56 76.28 Chinese MaxEntVote 62.44 58.29 72.22 67.72 64.32 SentVote 72.66 58.39 80.68 74.70 70.58 MP 2 74.34 65.17 78.91 75.22 72.81 Att 1 68.82 49.78 81.89 71.12 67.28 Att 2 81.41 73.35 86.58 82.79 80.45 Att 2+AT 83.35 74.06 87.52 84.03 81.64 Table 5: F1-measures of baselines and our model. than those in the same paragraph. Hence, performance improvement is less when considering adjacent sentences in the English corpus. (S12) ( It is doubted that the Mexican government provided vantage points for the enterprises involved during the bidding process. ) (S13)        7 ( The Mexican Foreign Ministry responded and denied the rumor on 7th. ) If we consider more adjacent sentences, e.g., two sentences before and after the current sentence, however, the results will be a bit lower. The micro-/macro-averaged F1 on English and Chinese corpus are 81.20/75.65 and 82.57/80.91, respectively. We think the reason is that some sentences are far away from the current sentence and have little effect on the current event, and considering more contexts may also lead to overfitting. Moreover, we explore the effects of considering only dependency path (Dpath) and only sentence (Sent) in Table 6. Att 2+AT achieves the best results when considering both paths and sentences as input, proving that both of them are effective features for our model. Att 2+AT obtains higher performance with only sentences than only paths as input, meaning that Att 2+AT is mainly benefi-cial from sentences that can offer semantic information. Error analysis shows that documents with incorrect identified values contains sentences with more speculative or negative cues: (S14) When asked if it might be arson , authorities said that no fire raiser has been found now, but the possibility of artificial arson should not been ruled out. S14 contains speculative cues  if ,  might ,  possibility  and negative cues  no ,  not ,  ruled out . It is difficult to identify whether the events are governed by the cues when only considering the dependency paths and ignoring the semantic information offered by sentences. S14 can demonstrate the importance of semantic features. As mentioned in Section 2.4, the document-level task becomes trivial if most documents have only one category of sentence-level factuality value that is the same as document-level value. Table 7 shows the performance of Att 2+AT on the documents with n different types of sentence-level factuality values. The microand macro-averaged F1 of n (cid:62) 2 are lower than those of n =1, indicating that the factuality of documents that have different types of sentence-level factuality are more difficult to identify due to the interference from sentence-level values. We notice that in the Chinese corpus, the performance of CTis much higher than that of PS+ and CT+ when n (cid:62) 2. According to the analysis on the Chinese corpus, we find that most CTdocuments are usually used to deny the rumors, i.e., those sentence-level events whose factuality values are not CT-. Therefore, the sentence-level CT-events are often in the topic sentences of the documents and dominate among sentences, which can contribute to the better results of document-level CTevents in Chinese corpus. Because document-level event factuality is related with sentence-level factuality information, we also consider the joint optimization model for them. For sentence-level task, we use the LSTM neural network in Section 3 and only consider the current sentence, i.e., do not consider information in adjacent sentences and the inter-sequence attention layer. The objective of documentand sentence-level task are denoted as LD (  ) and LS (  ) , and the objective of our joint optimization model is: LJ (  ) = L D (  ) + (1   ) LS (  ) (14) where  =0.6 is the trade-off. The performance of both sentence-level and document-level event factuality identification is shown in Table 8. The micro-/macro-averaged F1 of joint optimization model on English and Chinese corpus are 82.89/75.64 and 83.83/81.48, respectively. Although document-level event factuality is based on the factuality information in sentences, sentence-level factuality value of an event only depends on the current sentence, and is likely to have a different value compared to the current document-level factuality. Therefore, the joint model can not improve the performance of document-level task. Researchers have studied document-level tasks in many NLP applications, e.g., sentiment analysis (Xu et al., 2016; Dou, 2017), named entity recognition (Luo et al., 2018), and machine translation (Born et al., 2017). But related studies on event factuality are limited to the sentence-level task. Diab et al. (2009) and Prabhakaran et al. (2010) presented studies of belief annotation and tagging, and classified predicate events into Committed Belief (CB), Non-CB or Not Applicable using a supervised framework. For factuality assessment, Lee et al. (2015) employed dependency features, while Stanovsky et al. (2017) considered deep linguistic information, such as modality classes, syntactic re-ordering with PropS tree annotation structure (Lotan et al., 2013). Baly et al. (2018) considered a set of features and predicted the factuality of reporting and bias of news media. Saur (2008) and Saur and Pustejovsky (2012) proposed a rule-based model to identify event factuality on FactBank. de Marneffe et al. (2012) used a machine learning model and Qian et al. (2015) utilized a two-step framework combining machine learning and rule-based approaches on FactBank. In addition to FactBank, Prabhakaran et al. (2015) proposed a ongoing framework for a larger corpus based on LU, and Cao et al. (2013) constructed a Chinese corpus annotated with event factuality based on ACE 2005. However, no previous work annotated a document-level corpus. We construct DLEF corpus with document-level event factuality for the first time. Some studies focused on document-level event identification task. Choubey et al. (2018) designed a rule-based classifier to identify central events according to event coreference relations. Liu et al. (2018) utilized a kernel-based neural model that captured semantic relations between discourse units for event salience identification. However, they did not consider the document-level event factuality. To our best knowledge, this paper is the first work on document-level event factuality identification task. Previous studies (He et al., 2017; Rudinger et al., 2018; Qian et al., 2018) have tried neural network models on sentence-level factuality identification. Recent research has shown that neural networks with multi-level attention can extract meaningful information from heterogeneous input and improve the performance of NLP tasks, e.g., discourse relation (Liu and Li, 2016), relation classification (Wang et al., 2016), and question answering (Yu et al., 2017). Moreover, to improve the robustness of neural networks, related studies considered adversarial perturbation and training on text classification (Miyato et al., 2016) and relation extraction (Wu et al., 2017). This paper is in line in proposing an adversarial neural network with both intraand inter-sequence attention. We investigated document-level event factuality identification task by constructing a corpus annotated with documentand sentence-level event factuality based on both English and Chinese texts. To identify document-level event factuality, we proposed an LSTM neural network with both intraand inter-sequence attention, and consider adversarial training to improve the robustness. Experimental results showed that document-level event identification on our DLEF corpus is useful, and our adversarial training model outperforms several baselines. To our knowledge, this is the first paper for the document-level event factuality identification. In the future work, we will consider to detect events and their sentence-level and document-level factuality with a joint framework, and we will also continue to expand the scale of our DLEF corpus. The authors would like to thank the three anonymous reviewers for their comments on this paper. This work was partially supported by national Natural Science Foundation of China (NSFC) via Grant Nos. 61836007, 6177235, 61773276, 61673290.
abstain abstain objective method result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain objective objective method objective result objective result result other abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other method method method method abstain method method abstain abstain method abstain abstain method abstain method abstain abstain other abstain abstain abstain other abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other other other objective other other other other objective other other other objective objective objective result objective method other other