The goal of event detection (ED) is to detect the occurrences of events and categorize them. Previous work solved this task by recognizing and classifying event triggers, which is defined as the word or phrase that most clearly expresses an event occurrence. As a consequence, existing approaches required both annotated triggers and event types in training data. However, triggers are nonessential to event detection, and it is time-consuming for annotators to pick out the  most clearly  word from a given sentence, especially from a long sentence. The expensive annotation of training corpus limits the application of existing approaches. To reduce manual effort, we explore detecting events without triggers. In this work, we propose a novel framework dubbed as T ype-aware B ias N eural N etwork with A ttention M echanisms (TBN-NAM), which encodes the representation of a sentence based on target event types. Experimental results demonstrate the effectiveness. Remarkably, the proposed approach even achieves competitive performances compared with state-of-the-arts that used annotated triggers. This work tackles the task of event detection (ED), whose goal is to detect the occurrences of predefined events and categorize them. For example, consider the following sentence  In Baghdad, a cameraman died when an American tank fired on the Palestine Hotel. , an ideal event detection system should recognize two events, Death and Attack (suppose that both Death and Attack are in the predefined event set) . Previous work typically solved this task by recognizing and classifying event triggers. According to ACE (Automatic Context Extraction) event evaluation program, event trigger is defined as the word or phrase that most clearly expresses an event occurrence. died  is the trigger word of Death event, and  fired  is the trigger word of Attack event. The majority of existing approaches modeled this task as word classification (Ji and Grishman, 2008; Liao and Grishman, 2010; Hong et al., 2011; Li et al., 2013; Nguyen and Grishman, 2015; Liu et al., 2016b,a; Chen et al., 2017), which predicted whether each word in a given sentence is an event trigger and what type of event it triggered. As a consequence, these approaches required both annotated triggers and event types for training. However, event triggers are nonessential to this task. Remind that the goal of event detection is to recognize and categorize events, thus triggers could be viewed as intermediate results of this task. Furthermore, it is time-consuming for annotators to pick out the  most clearly  word from a given sentence, especially from a long sentence, which limits the application of existing ED approaches. To reduce manual effort, we explore detecting events without triggers. In this study, the only annotated information of each sentence is the types of events occurred in it. Consider the aforementioned example S again, its annotation is { Death, Attack } . On the contrast, previous work also required an annotated trigger for each event, which means the annotated information of S is { Death :died, Attack :fired } in previous work. Without event triggers, it is intuitive to model this task via text classification. However, there are two challenges: (1) Multi-label problem : each sentence may contain arbitrary number of events, which means it could have zero or multiple target labels. In machine learning, this problem is called multi-label problem. (2) Trigger absence problem : previous work illustrated that trigger words play important roles in event detection(Chen et al., 2015; Liu et al., 2016a). It is challenging to model this information without annotated triggers. To solve the first challenge, we transform multi-label classification to multiple binary classification problems. Specifically, a given sentence s attached each pre-defined event type t forms an instance, which is expected to be labeled with 0 or 1 according to whether s contains an event of type t . For example, suppose there totally are 3 predefined types of events(denoted by t 1 , t 2 and t 3 ), and sentence s contains two events of type t 1 and t 3 , then it could be transformed to the following three instances: instance label < s, t 1 > 1 < s, t 2 > 0 < s, t 3 > 1 Table 1: Example of instances in binary classifications for sentence s , which contains events of type t 1 and t 3 . In this paradigm, sentences that convey multiple events will yield multiple positive pairs, thus the multi-label problem could be well solved. Furthermore, each type of events are usually triggered by a set of specific words, which are called event trigger words. For example, Death events are usually triggered by die, passed away, gone, etc. Therefore, event trigger words are important clues to this task. Since existing work explicitly exploited annotated trigger words in their approaches, they can directly model this observation. However, in our case, annotated triggers are unavailable. To model this information, we propose a simple but effective model, called T ype-aware B ias N eural N etwork with A ttention M echanisms ( TBNNAM ). Figure 1 illustrates the framework of TBNNAM . The input is consisted of two parts: a tokenized sentence with NER tags and a target event type. The output o is expected to be 1 if the given sentence conveys an event of the target type, otherwise 0 (the output should be 1 for the example given in Figure 1). Specifically, given a sentence, the proposed model first transforms the input tokens into embeddings, and applies an LSTM layer to calculate a context-dependent representation for each token. Then it computes an attention vector,  , based on the target event type, where the trigger word is expected to obtain higher score. Finally, the sentence representation s att is calculated based on  . Here, s att is expected to focus on local information (trigger word). To capture global information, the final output, o , is also connected to the last LSTM units, which encodes the global information of the input sentence. Furthermore, to reinforce the influence of positive samples, we devise a bias objective function in our model . We call our model type-aware because the representation of a sentence, s att , is calculated based on the target event type. We have conducted experimental comparisons on a widely used benchmark dataset ACE2005 1 . The results illustrate that our approach outperforms all the compared baselines, and even achieves competitive performances compared with exiting approaches that used annotated triggers. We publish our code for further study by the NLP community. 2 In summary, the main contributions of this work are: (1) To the best of our knowledge, this is the first work that focuses on detecting events without triggers. Compared with existing approaches, the proposed method requires less manual annotations. (2) Without triggers, this task encounters two challenges: multi-label problem and trigger absence problem. We propose a simple but effective model, which even achieves competitive results compared with approaches that using annotated triggers. (3) Since this is the first work on detecting events without triggers, we implement a series of baseline models for this task, and systematically evaluate and analyze them. Event detection task requires that certain speci-fied types of events, which are mentioned in the annotated data, to be detected. The most common used benchmark dataset in previous work is ACE 2005 corpus. This corpus includes 8 types of events, with 33 subtypes. Following previous work(Ahn, 2006; Ji and Grishman, 2008; Liao and Grishman, 2010; Hong et al., 2011; Li et al., 1 https://catalog.ldc.upenn.edu/LDC2006T06 2 https://github.com/liushulinle/event detection without triggers Figure 1: The framework of type-aware bias neural network with attention mechanisms. 2013; Chen et al., 2015; Nguyen and Grishman, 2016), we treat them simply as 33 separate event types and ignore the hierarchical structure among them. Consider the following sentence  In Baghdad, a cameraman died when an American tank fired on the Palestine Hotel , an ideal event detector should detect two events from this sentence: a Die event and an Attack event. Event detection is one of important topics in NLP. Many approaches have been proposed for this task. Nearly all the existing methods on ACE event task follow supervised paradigm. We further divide them into feature-based methods and representation-based methods. In feature-based methods, a diverse set of strategies has been exploited to convert classification clues into feature vectors. Ahn (2006) us-es the lexical features(e.g., full word), syntactic features (e.g., dependency features) and external-knowledge features(WordNet (Miller, 1998)) to extract the event. Inspired by the hypothesis of One Sense Per Discourse (Yarowsky, 1995), Ji and Grishman (2008) combined global evidence from related documents with local decisions for the event extraction. To capture more clues from the texts, Gupta and Ji (2009), Liao and Grishman (2010) and Hong et al. (2011) proposed the cross-event and cross-entity inference for the ACE event task. Li et al. (2013) proposed a joint model to capture the combinational features of triggers and arguments. Liu et al. (2016b) proposed a global inference approach to employ both latent local and global information for event detection. In recent years, representation-based methods have dominated the research. In this paradigm, candidate event mentions are represented by embeddings, which typically are fed into neural networks. Chen et al. (2015) and Nguyen and Grishman (2015) are the first work in this paradigm. Their models are based on CNNs (Convolu-tional Neural Networks). To model the dependency of triggers and arguments, Nguyen and Grishman (2016) proposed a joint event extraction approach based RNNs(Recurrent Neural Networks). Liu et al. (2017) proposed to encode argument information in event detection via supervised attention mechanisms. Recently, Nguyen and Grishman (2018) and Sha et al. (2018) proposed to exploit syntactic information for event detection. All the existing approaches required annotated triggers. The expensive annotation of training data limits the application of these approaches. To reduce manual effort, we perform this task without event triggers. To deal the multi-label problem, we model this task via multiple binary classifications. Given a sentence, it will be fed into a binary classifier with each candidate event type. We add the label NA to sentences that do not contain any events. To capture the hidden trigger information, we propose a simple but effective model, called T ype-aware B ias N eural N etwork with A ttention M echanisms ( TBNNAM ). Our model is type-aware because it calculates the representation of a sentence based on the target event type. Figure 1 illustrates the framework of TBNNAM . The input is consisted of two parts: a tokenized sentence with NER tags and a target event type. The output o is expected to be 1 if the given sentence conveys an event of the target type, otherwise 0. Next, we describe the structure of this model in bottom-up order. Given a sentence, we use Stanford CoreNLP tools 3 (Manning et al., 2014) to convert texts into tokens. The ACE 2005 corpus annotated not only events but also entities for each given sentence. Following previous work, we exploit the annotated entity tags in our model(Li et al., 2013; Chen et al., 2015; Nguyen and Grishman, 2015, 2016; Liu et al., 2016b). Word embeddings learned from a large amount of unlabeled data have been shown to be able to capture the meaningful semantic regularities of word-s(Bengio et al., 2003; Erhan et al., 2010). Much work(Socher et al., 2012; Zeng et al., 2014) has shown its power in many NLP tasks. In this work, we use the Skip-gram mod-el(Mikolov et al., 2013) to learn word embeddings on the NYT corpus 4 . Furthermore, we randomly initialized an embedding table for each entity tags. All the input word tokens and entity tags will be transformed into low-dimensional vectors by looking up these embedding tables. In this work, we denote the dimension of word embeddings by d w , and that of entity embeddings by d e . As illustrated in Figure 1, an event type is transformed into two embedding vectors: t 1 and t 2 . The first one (colored with brown) is designed to capture local information (hidden trigger word), and the latter one (colored with red) is designed to capture global information. Both of them are randomly initialized. The dimension of event type embeddings is denoted by d evt . 3 http://stanfordnlp.github.io/CoreNLP 4 https://catalog.ldc.upenn.edu/LDC2008T19 3.4 LSTM Layer As shown in Figure 1, the LSTM layer is run over the sequence of concatenation of word and entity embeddings. LSTM has three gates(input i , forget f and output o ), and a cell memory vector c . The input gate can determine how incoming vectors x(t) alter the state of the memory cell. The output gate can allow the memory cell to have an effect on the outputs. Finally, the forget gate allows the cell to remember or forget its previous state. Each type of events are usually triggered by a set of specific words, which are called event trigger words. For example, Death events are usually triggered by die, passed away, gone, etc. Therefore, event trigger words are important clues to this task. However, this information is hidden in our task, because annotated triggers are unavailable. To model the hidden triggers, we introduce attention mechanisms in our approach. As illustrated in Figure 1, the attention vector  is calculated based on the target event type embedding t 1 and the hidden states h yielded by LSTM. Specifically, the attention score for the k -th token in a given sentence is calculated by the following equation:  k = exp( h k  t T 1 ) (cid:80) i exp( h i  t T 1 ) (1) In this model, trigger words of the target event type are expected to obtain higher scores than other words. Finally, the representation of the sentence, s att , is computed by the following equation: s att =  TH (2) where  = [  1 , ...,  n ] is the attention vector, H = [ h 1 , h 2 , ..., h n ] is a matrix, h k is the LSTM's output for the k -th token, and s att is the representation of the given sentence. As illustrated in Figure 1, the final output o is connected to two components: v att and v global . On one hand, v att is calculated by the dot product of s att and t 1 , which is designed to capture local features (specifically, features about hidden trigger words). On the other hand, the last output of the LSTM layer, h n , encodes global information of the whole sentence, thus v global = h n  t T 2 is expected to capture global features of a sentence. Finally, o is defined as the weighted sum of v att and v global : o =  (   v att + (1   )  v global ) (3) where  is the Sigmoid function,   [0 , 1] is a hyper-parameter for trade-off between v att and v global . We devise a bias loss function to reinforce the influence influence of positive samples because of the following reasons. 1) positive samples are much less than negative samples. In our approach, each training sample is a < sentence,event type > pair, whose label is 1 or 0 according to whether the given sentence conveys an event of type t. For example, we totally have 33 target event types, if a sentence only contains one event, then it will be transformed into 32 negative pairs and 1 positive pair. The majority of sentences convey at most two events, thus negative samples are much more than positive samples. 2) positive samples are more informative than negatives. A positive pair < s , t > means that s conveys an event of type t , whereas negative pair means s does not convey any event of type t . Apparently, the former is more informative. Given all of the (suppose T) training instances ( x ( i ) , y ( i ) ), the loss function is defined as follows: J (  ) = 1 TT (cid:88) i =1 ( o ( x i )  y ( i ) ) 2 (1+ y ( i )   )+  ||  || 2 (4) where x is a pair consisted of a sentence and a target event type, y  { 0 , 1 } ,  is the parameter of our model and  > 0 is the weight of L2 normalization term. (1 + y ( i )   ) is the bias term. Specifically, the value of this term is 1 for negative samples ( y ( i ) is 0) and 1 +  for positive samples ( y ( i ) is 1), where   0 . We train the model by using a simple optimization technique called stochastic gradient descent (S-GD) over shuffled mini-batches with the Adadelta rule (Zeiler, 2012). Regularization is implemented by a dropout and L2 norm. Given a instance x , the model assign it a label (cid:101) y according to the following equation: (cid:101) y = (cid:26) 0 o ( x ) < 0 . Since this is the first work to perform event detection without triggers, we implement a series of baseline systems for comparisons, which could be divided into two categories: binary classification based methods and multi-class classification based methods. Similar with the proposed approach, baseline systems in this group solved this task via binary classification. Figure 2 illustrates the framework of these methods. These models take a sentence and a target event type as input. Then all the inputs are transformed into embeddings by looking up embedding tables. These models have the same loss function as the proposed approach (see Equation 4). The key component of these models is sentence encoder. According to the strategy of encoding sentence, we implement three models for comparison: BC-CNN, BC-LSTM last , BC-LSTM avg . BC-CNN employs a CNN model to encode sentence. BC-LSTM last employs LSTM model, and use the hidden state of the last token as the representation of a given sentence. BC-LSTM avg also employs LSTM model, but use the average of all hidden states as the representation of a given sentence. All existing approaches model the task of event detection (with triggers) via multi-class classification 5 . Given a sentence, these methods predict whether each token is an event trigger and what type of event it triggered. We also implement several multi-class classification based systems for comparison. Since annotated triggers are unavailable in our task, the sentence is the input of our model. Figure 3 illustrates the framework of these models. Following existing work(Chen et al., 2015; Liu et al., 2017), we employ a negative log-likelihood loss function in the soft-max classifier: J (  ) =  1 T (cid:80) Ti =1 log( p ( y ( i ) | x ( i ) ,  )) , where ( x ( i ) , y ( i ) ) is a training sample, y ( i ) is a label from the valid label set (all the predefined event types plus a NA for none event), T is the total number of training instances,  is the parameters of the model. According to the strategy of encoding sentence, we implement three models: MC-CNN, MC-LSTM last and MC-LSTM avg . MC-CNN employs a CNN model to encode sentence. MC-LSTM last employs LSTM model, and use the hidden state of the last token as the representation of a given sentence. MC-LSTM avg also employs LSTM model, but use the average of all hidden states as the representation of a given sentence. 5 Multi-class classification means a classification task with more than two classes, but each sample belongs to only one class. multi-class is different from multi-label. In this section, we introduce the dataset, evaluation metrics and the settings of hyper parameters. Our experiments are conducted on ACE 2005 dataset. Following the evaluation of previous work(Li et al., 2013; Chen et al., 2015; Nguyen and Grishman, 2016; Liu et al., 2017), we randomly selected 30 articles from different genres as the development set, and subsequently conducted a blind test on a separate set of 40 ACE 2005 newswire documents. We used the remaining 529 articles as our training set. This work focuses on detecting events without triggers. Therefore, we remove trigger annotations from the corpus. Specifically, we employ Stanford CoreNLP Toolkit to split each document into sentences, and assign each sentence with a set of labels according to the original annotations in ACE 2005 corpus. If a sentence does not contain any event, we assign it with a special label, NA . If a sentence contains multiple events of the same type (less than 3% in ACE corpus), we only keep one label for each type. Table 2 shows several samples of the our corpus. Following previous work (Liao and Grishman, 2010; Li et al., 2013; Chen et al., 2015; Liu et al., 2017), we use precision (P), recall (R) and F 1 measure (F 1 ) to evaluate the results. Precision: the proportion of correctly predicted events in total predicted events. Recall: the proportion of correctly predicted events in total gold events of the dataset. Hyper parameters are tuned on the development dataset via grid search. In all experiments, we Figure 4: Experimental results on development dataset with different setting of  . set the dimension of word embeddings as 200, the dimension of entity type embeddings as 50, batch size as 100, the hyper parameter for the L 2 norm as 10  5 ,  in the bias term as 1.0. Furthermore, we also tune  in Equation 3 on the development dataset. Figure 4 illustrates experimental results with different settings of  , finally we set  as 0.25. And in all the CNN-based baseline systems, the sizes of filter windows are set to 1, 2, 3 with 100 feature maps each. Table 3 illustrates the experimental results, where methods with name MC-* are based on multi-class classification, and methods with name BC-* are based on binary classification. According to the strategy to encode a sentence, methods in Table 3 are grouped into three parts. From the table, we make the following observations:  In each group, binary classification based approach significantly outperforms multi-class classification based approach. The reason is that BC-* can solve the multi-label problem, but MC-* can not. Moreover, MC-* achieve much lower recall than BC-* , because they predict at most one event for each sentence. The reason is that trigger words are important clues to event detection, and CNN is good at extracting such local features. In this section, we illustrates the results of the proposed approach (see Table 4). The results of baseline systems are listed in the first group. Methods in the second group are the proposed approaches. They have the same model structure as Figure 1. In BC-LSTM att ,  (see Equation 3) is set as 1.0, which is designed to show the effects of the proposed attention strategy. In TBNNAM ,  is set as 0.25, which is designed to employ both local information (captured by the attention mechanism) and global information (captured by the last output of LSTM). Methods in the last group are state-of-the-art ED systems on ACE 2005 dataset. We give a brief introduction of them as follows: 1). Nguyen's CNN : the CNN model proposed by Nguyen and Grishman (2015) 2). Chen's DMCNN : the dynamic multi-pooling CNN model proposed by Chen et al. (2015) 3). Liu's PSL : the soft probabilistic soft logic model proposed by Liu et al. (2016b) 4). DS-DMCNN : the DMCNN model augmented with automatic labeled data, proposed by Chen et al. (2017) From the table, we make the following observations: Figure 5: Visualization of attention weight vector  of sample instances learned by our model. BC-LSTM att outperforms all the baseline systems with remarkable gains, which demonstrates the effectiveness of the proposed attention mechanism. TBNNAM achieves better performance than BC-LSTM att (69.9% vs. 66.3%), which means that global information captured by the last state of LSTM is also important to this task. Such global information and local information captured by the attention mechanisms are complementary to each other. All state-of-the-art ED systems require annotated triggers. Without trigger annotations, our approach achieves competitive results, even outperforms some of them. Figure 5 shows several examples of the attention vector  learned by our model. In the first case, died is the most significant keyword for the Death event, and our model succeeded to capture this feature by assigning it with a large attention score. Similarly, in the second case, fired is a key clue of Attack event, and our model also learned it and assigned it with a large attention score. Actually, died and fired are the trigger words of Death and Attack events, respectively. Therefore, we argue that, although annotated triggers are unavailable, our model still can exploit trigger information for this task. Moreover, our approach also could model the dependencies among different events, which has been demonstrated useful for this task(Liao and Grishman, 2010; Liu et al., 2016b). For example, Attack events often co-occur with Death events. In Case1 and Case2 (Figure 5), our approach models such information by paying attention on both words died and fired. Furthermore, the 3-rd case is a negative sample, thus methods P (%) R (%) F 1 (%) BC-LSTM att \ Bias 74.5 57.2 64.7 BC-LSTM att 68.3 64.5 66.3 TBNNAM \ Bias 76.6 59.8 67.2 TBNNAM 76.2 64.5 69.9 Table 5: Results of systems without/with bias term in loss function, where * \ Bias do not use bias term. In this section, we illustrates the effectiveness of the bias term in Equation 4. Table 5 shows experimental results. Methods named with  * \ Bias  do not use bias term. From the table, we observe that systems with bias term in loss function significantly outperform those without bias term, which demonstrates the correctness of our analysis in Section 3.7 that positive samples should be reinforced during training. Existing event detection approaches required annotated triggers, which limits their applications because of the expensive annotations. To reduce manual effort, we investigate performing this task without event triggers. In this setting, event detection task encounters two challenges: multi-label problem and trigger absence problem. We propose a simple but effective model to solve them, which computes the representation of a sentence according to the target event type. Experimental results demonstrate its effectiveness. Remarkably, the proposed approach even achieves competitive performances compared with state-of-the-arts that used annotated triggers.
abstain abstain abstain abstain abstain objective objective abstain abstain objective abstain abstain abstain abstain other abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain abstain abstain objective abstain other abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain abstain abstain objective abstain method result abstain objective abstain abstain objective objective other other other other abstain other other other other method other method other other other other other other other other other other other other other method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain method method method method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective abstain abstain