Representations of events described in text are important for various tasks. In this work, we present SWCC : a S imultaneous W eakly supervised C ontrastive learning and C lustering framework for event representation learning. SWCC learns event representations by making better use of co-occurrence information of events. Specifically, we introduce a weakly supervised contrastive learning method that allows us to consider multiple positives and multiple negatives, and a prototype-based clustering method that avoids semantically related events being pulled apart. For model training, SWCC learns representations by simultaneously performing weakly supervised contrastive learning and prototype-based clustering. Experimental results show that SWCC outperforms other baselines on Hard Similarity and Transitive Sentence Similarity tasks. In addition, a thorough analysis of the prototype-based clustering method demonstrates that the learned prototype vectors are able to implicitly capture various relations between events. Our code will be available at https://github. com/gaojun4ever/SWCC4Event . Distributed representations of events, are a common way to represent events in a machine-readable form and have shown to provide meaningful features for various tasks (Lee and Goldwasser, 2018; Rezaee and Ferraro, 2021; Deng et al., 2021; Martin et al., 2018; Chen et al., 2021). Obtaining effective event representations is challenging, as it requires representations to capture various relations between events. Figure 1 presents four pairs of events with different relations. Two events may share the same event attributes (e.g. event types and sentiments), and there may also be a causal or temporal relation between two events. Early works (Weber et al., 2018) exploit easily accessible co-occurrence relation of events to learn event representations. Although the use of co-occurrence relation works well, it is too coarse for deep understanding of events, which requires fine-grained knowledge (Lee and Goldwasser, 2019). Recent works focus on fine-grained knowledge, such as discourse relations (Lee and Goldwasser, 2019; Zheng et al., 2020) and commonsense knowledge (e.g. sentiments and intents) (Sap et al., 2019; Ding et al., 2019). Concretely, Lee and Goldwasser (2019) and Zheng et al. (2020) leverage 11 discourse relation types to model event script knowledge. Ding et al. (2019) incorporate manually labeled commonsense knowledge (intents and sentiments) into event representation learning. However, the types of fine-grained event knowledge are so diverse that we cannot enumerate all of them and currently adopted fine-grained knowledge fall under a small set of event knowledge. In addition, some manually labeled knowledge (Sap et al., 2019; Hwang et al., 2021) is costly and difficult to apply on large datasets. In our work, we observe that there is a rich amount of information in co-occurring events, but previous works did not make good use of such information. Based on existing works on event relation 3036 extraction (Xue et al., 2016; Lee and Goldwasser, 2019; Zhang et al., 2020; Wang et al., 2020), we find that the co-occurrence relation, which refers to two events appearing in the same document, can be seen as a superset of currently defined explicit discourse relations. To be specific, these relations are often indicated by discourse markers (e.g., because, capturing the casual relation) (Lee and Goldwasser, 2019). Therefore, two related events must exist in the same sentence or document. More than that, the co-occurrence relation also includes other implicit event knowledge. For example, events that occur in the same document may share the same topic and event type. To learn event representations, previous works (Granroth-Wilding and Clark, 2016; Weber et al., 2018) based on co-occurrence information usually exploit instance-wise contrastive learning approaches related to the margin loss, which consists of an anchor, positive, and negative sample, where the anchor is more similar to the positive than the negative. However, they share two common limitations: (1) such margin-based approaches struggle to capture the essential differences between events with different semantics, as they only consider one positive and one negative per anchor. (2) Randomly sampled negative samples may contain samples semantically related to the anchor, but are undesirably pushed apart in embedding space. This problem arises because these instance-wise contrastive learning approaches treat randomly selected events as negative samples, regardless of their semantic relevance. We are motivated to address the above issues with the goal of making better use of co-occurrence information of events. To this end, we present SWCC : a S imultaneous W eakly supervised C ontrastive learning and C lustering framework for event representation learning, where we exploit document-level co-occurrence information of events as weak supervision and learn event representations by simultaneously performing weakly supervised contrastive learning and prototype-based clustering. To address the first issue, we build our approach on the contrastive framework with the InfoNCE objective (van den Oord et al., 2019), which is a self-supervised contrastive learning method that uses one positive and multiple negatives. Further, we extend the InfoNCE to a weakly supervised contrastive learning setting, allowing us to consider multiple positives and multiple negatives per anchor (as opposed to the previous works which use only one positive and one nega-tive). Co-occurring events are then incorporated as additional positives, weighted by a normalized co-occurrence frequency. To address the second issue, we introduce a prototype-based clustering method to avoid semantically related events being pulled apart. Specifically, we impose a prototype for each cluster, which is a representative embedding for a group of semantically related events. Then we cluster the data while enforce consistency between cluster assignments produced for different augmented representations of an event. Unlike the instance-wise contrastive learning, our clustering method focuses on the cluster-level semantic concepts by contrasting between representations of events and clusters. Overall, we make the following contributions:  We propose a simple and effective framework ( SWCC ) that learns event representations by making better use of co-occurrence information of events. Experimental results show that our approach outperforms previous approaches on several event related tasks. We introduce a weakly supervised contrastive learning method that allows us to consider multiple positives and multiple negatives, and a prototype-based clustering method that avoids semantically related events being pulled apart. We provide a thorough analysis of the prototype-based clustering method to demonstrate that the learned prototype vectors are able to implicitly capture various relations between events. Event representation model. In the early works (Weber et al., 2018; Ding et al., 2019), Neural Tensor Networks (NTNs) (Socher et al., 2013b,a) are widely adopted to compose the representation of event constitutions, i.e., (subject, predicate, object) . However, such methods introduced strong compositional inductive bias and can not extend to events with more additional arguments, such as time, location etc. Several recent works (Zheng et al., 2020; Vijayaraghavan and Roy, 2021) replaced static word vector compositions with powerful pretrained language models, such as BERT (Devlin et al., 2019), for flexible event representations and achieved better performance. Following them, we also take the BERT as the backbone model. event text, which contains a sequence of tokens and the input format can be represented as follows: [CLS] , pred, subj, obj, [SEP] . Define x = [ x 0 , x 1 ,    , x L ] to be the input sequence of length L , where x 0 and x L are the [CLS] token and the [SEP] token respectively. Given x , the BERT returns a sequence of contextualized vectors: [ v [CLS] , v x 1 ,    , v x L ] = BERT( x ) , (2) where v [CLS] is the representation for the [CLS] token. In the default case, the final vector representation z of the event is the output representation of the [CLS] token: z = v [CLS] . Instance-wise contrastive learning. Event representation models learn representations with contrastive learning, which aims to pull related events together and push apart unrelated events. Margin loss (Schroff et al., 2015) is a widely used contrastive loss in most of the existing works on event representation learning (Weber et al., 2018; Ding et al., 2019; Zheng et al., 2020). Most recently, an alternative contrastive loss function, called InfoNCE (van den Oord et al., 2019), has been proposed and shown effective in various contrastive learning tasks (He et al., 2020; Hu et al., 2021; Gao et al., 2021). Chen et al. (2020a) further demonstrate that InfoNCE works better than the Margin loss. In this work, we explore the use of InfoNCE to train our event representation model. Formally, given a set of N paired events D = { x i , x + i } Ni =1 , where x + i is a positive sample for x i , the InfoNCE objective for ( x i , x + i ) is presented in a softmax form with in-batch negatives (Chen et al., 2020a; Gao et al., 2021): L =  log g ( z i , z + i ) g ( z i , z + i ) + (cid:80) k N ( i ) g ( z i , z k ) , (3) where z i and z + i are the augmented representations of x i and x + i obtained through a representation model , k  N ( i ) is the index of in-batch negatives. and g is a function: g ( z i , z k ) = exp( z (cid:62) i z k / ) , where   R + is a positive value of temperature. Data augmentation. One critical question in contrastive learning is how to obtain z + i . In language representation, z + i are often obtained by first applying data augmentation in the form of word deletion, reordering, or substitution on x i and then feeding it into the event representation model. Several recent works (Gao et al., 2021; Liang et al., 2021) exploit dropout noise as data augmentation for NLP tasks and find that this data augmentation technique performs much better than common data augmentation techniques. Specifically, given an input event x i , we obtain z i and z + i by feeding the same input to the BERT encoder with the parametric weights  twice, and each time we apply a different dropout mask: z i = f  ( x i ,  1 ) , z + i = f  ( x i ,  2 ) , (4) where  1 and  2 are two different random masks for dropout. As described in Sec.3.1, given an anchor event z i , we generate 3 positive samples z a 1 , z a 2 and z a 3 with different dropout masks. In this section, we will present technical details of our proposed approach and our goal is to learn event representations by making better use of co-occurrence information of events. Figure 2 presents an overview of our proposed approach, which contains two parts: the weakly-supervised contrastive learning method (left) and the prototype-based clustering method (right). In the following sections, we will introduce both methods separately. We build our approach on the contrastive framework with the InfoNCE objective (Eq.3) instead of the margin loss. To incorporate co-occurrence information into event representation learning, a straightforward way is to consider the co-occurring event of each input event as an additional positive sample, that is, the positive augmented representations of x i come not only from itself but also from its co-occurring event denoted as x p . However, The original InfoNCE objective cannot handle the case where there exists multiple positive samples. Inspired by Khosla et al. (2020), we take a similar formulation to tackle this problem. More than that, we also introduce a weighting mechanism to consider co-occurrence frequency of two events, which indicates the strength of the connection between two events. Co-occurrence as weak supervision. Formally, for each input pair ( x i , x p ) , where x i and x p refer to the input event and one of its co-occurring events, we first compute an augmented representation z i of x i as an anchor event, through the event representation model mentioned in  2. How the method differs from InfoNCE is in the construction of the positive set A ( i ) for x i . In InfoNCE, A ( i ) only contains one positive. In our method, we generalize Eq. 3 to support multiple positives learning: L = (cid:88) a A ( i )  log g ( z i , z a ) g ( z i , z a ) + (cid:80) k N ( i ) g ( z i , z k ) , (5) where A ( i ) and N ( i ) refer to the positive set and the negative set for the event x i . Note that we support arbitrary number of positives here. In our work, considering the limited GPU memory, we use A ( i ) = { z a 1 , z a 2 , z a 3 } , where z a 1 and z a 2 are two augmented representations of the same event x i , obtained with different dropout masks, and z a 3 is an augmented representation of its co-occurring event. Here z a 1 and z a 2 will then be used in the prototype-based clustering method (See Fig. 2 for example) as detailed later ( 3.2). Incorporating co-occurrence frequency. The co-occurrence frequency indicates the strength of the connection between two events. To make better use of data, we introduce a weighting mechanism to exploit the co-occurrence frequency between events as instance weights and rewrite the Eq. 5: L cl = (cid:88) a A ( i )  log  a  g ( z i , z a ) g ( z i , z a ) + (cid:80) k N ( i ) g ( z i , z k ) . (6) Here  a is a weight for the positive sample z a . In our work, the two weights  a 1 and  a 2 of the positive samples ( z a 1 and z a 2 ) obtained from the input event, are set as  a 1 =  a 2 = 1 |A ( i ) | 1 , where |A ( i ) | is its cardinality. To obtain the weight  a 3 for the augmented representation z a 3 of the co-occurring event, we create a cooccurrence matrix, V with each entry corresponding to the co-occurrence frequency of two distinct events. Then V is normalized to  V with the Min-Max normalization method, and we take the entry in  V as the weight  a 3 for the co-occurrence event. In this way, the model draws the input events closer to the events with higher co-occurrence frequency, as each entry in  V indicates the strength of the connection between two events. To avoid semantically related events being pulled apart, we draw inspiration from the recent approach (Caron et al., 2020) in the computer vision domain and introduce a prototype-based clustering method, where we impose a prototype, which is a representative embedding for a group of semantically related events for each cluster. Then we cluster the data while enforce consistency between cluster assignments produced for different augmented representations of an event. These prototypes essentially serve as the center of data representation clusters for a group of semantically related events (See Figure 1 for example). Unlike the instance-wise contrastive learning, our clustering method focuses on the cluster-level semantic concepts by contrasting between representations of events and clusters. Cluster prediction. This method works by comparing two different augmented representations of the same event using their intermediate cluster assignments. The motivation is that if these two representations capture the same information, it should be possible to predict the cluster assignment of 3039 one augmented representation from another augmented representation. In detail, we consider a set of M prototypes, each associated with a learnable vector c i , where i  (cid:74) M (cid:75) . Given an input event, we first transform the event into two augmented representations with two different dropout masks. Here we use the two augmented representations z a 1 and z a 2 of the event x i . We compute their cluster assignments q a 1 and q a 2 by matching the two augmented representations to the set of M prototypes. The cluster assignments are then swapped between the two augmented representations: the cluster assignment q a 1 of the augmented representation z a 1 should be predicted from the augmented representation z a 2 , and vice-versa. Formally, the cluster prediction loss is defined as: L cp = (cid:96) ( z a 1 , q a 2 ) + (cid:96) ( z a 2 , q a 1 ) , (7) where function (cid:96) ( z , q ) measures the fit between the representation z and the cluster assignment q , as defined by: (cid:96) ( z , q ) =  q log p . Here p is a probability vector over the M prototypes whose components are: p ( j ) = exp( z (cid:62) c j / ) (cid:80) Mk =1 exp(exp( z (cid:62) c k / ) , (8) where  is a temperature hyperparameter. Intuitively, this cluster prediction method links representations z a 1 and z a 2 using the intermediate cluster assignments q a 1 and q a 2 . Computing cluster assignments. We compute the cluster assignments using an Optimal Transport solver. This solver ensures equal partitioning of the prototypes or clusters across all augmented representations, avoiding trivial solutions where all representations are mapped to a unique prototype. In particular, we employ the Sinkhorn-Knopp algorithm (Cuturi, 2013). The algorithm first begins with a matrix   RM  N with each element initialized to z (cid:62) b c m , where b  (cid:74) N (cid:75) is the index of each column. It then iteratively produces a doubly-normalized matrix, the columns of which comprise q for the minibatch. Our approach learns event representations by simultaneously performing weakly supervised contrastive learning and prototype-based clustering. The overall training objective has three terms: L overall = L cl +  L cp +  L mlm , (9) where  and  are hyperparameters. The first term is the weakly supervised contrastive learning loss that allows us to effectively incorporate co-occurrence information into event representation learning. The second term is the prototype-based clustering loss, whose goal is to cluster the events while enforcing consistency between cluster assignments produced for different augmented representations of the input event. Lastly, we introduce the masked language modeling (MLM) objective (Devlin et al., 2019) as an auxiliary loss to avoid forgetting of token-level knowledge. Following common practice in event representation learning (Weber et al., 2018; Ding et al., 2019; Zheng et al., 2020), we analyze the event representations learned by our approach on two event similarity tasks ( 4.2) and one transfer task ( 4.4). The event triples we use for the training data are extracted from the New York Times Gigaword Corpus using the Open Information Extraction system Ollie (Mausam et al., 2012). We filtered the events with frequencies less than 3 and ended up with 4,029,877 distinct events. We use the MCNC dataset adopted in Lee and Goldwasser (2019) 1 for the transfer task. Our event representation model is implemented using the Texar-PyTorch package (Hu et al., 2019). The model starts from the pre-trained checkpoint of BERT-based-uncased (Devlin et al., 2019) and we use the [CLS] token representation as the event representation. We train our model with a batch size of 256 using an Adam optimizer. The learning rate is set as 2e-7 for the event representation model and 2e-5 for the prototype memory. We adopt the temperature  = 0 . 3 and the numbers of prototypes used in our experiment is 10. Similarity task is a common way to measure the quality of vector representations. Weber et al. (2018) introduce two event related similarity tasks: (1) Hard Similarity Task and (2) Transitive Sentence Similarity . push away representations of dissimilar events while pulling together those of similar events. Weber et al. (2018) created a dataset (denoted as Original), where each sample has two types of event pairs: one with events that should be close to each other but have very little lexical overlap, and another with events that should be farther apart but have high overlap. This dataset contains 230 event pairs. After that, Ding et al. (2019) extended this dataset to 1,000 event pairs (denoted as Ex-tended). For this task, we use Accuracy as the evaluation metric, which measures the percentage of cases where the similar pair receives a higher cosine value than the dissimilar pair. Transitive Sentence Similarity. The transitive sentence similarity dataset (Kartsaklis and Sadrzadeh, 2014) contains 108 pairs of transitive sentences that contain a single subject, object, and verb (e.g., agent sell property ) and each pair in this dataset is manually annotated by a similarity score from 1 to 7. A larger score indicates that the two events are more similar. Following previous work (Weber et al., 2018; Ding et al., 2019; Zheng et al., 2020), we evaluate using the Spear-man's correlation of the cosine similarity predicted by each method and the annotated similarity score. We compare our proposed approach with a variety of baselines. These methods can be categorized into three types: (1) Co-occurrence : Event-comp (Weber et al., 2018), Role-factor Tensor (Weber et al., 2018) and Predicate Tensor (Weber et al., 2018) are models that use tensors to learn the interactions between the predicate and its arguments and are trained using co-occurring events as supervision. (2) Discourse Relations : This line of work exploits discourse relations. UniFA-S (Zheng et al., 2020) adopt discourse relations. (3) Commonsense Knowledge : Several works have shown the effectiveness of using commonsense knowledge. KGEB (Ding et al., 2016) incorporates knowledge graph information. NTN-IntSent (Ding et al., 2019) leverages external commonsense knowledge about the intent and sentiment of the event. Results. Table 1 reports the performance of different methods on the hard similarity tasks and the transitive sentence similarity task. The result shows that the proposed SWCC achieves the best performance among the compared methods. It not only outperforms the Role-factor Tensor method that based on co-occurrence information, but also has better performance than the methods trained with additional annotations and commonsense knowledge, e.g. NTN-IntSent and UniFA-S. This implies the co-occurrence information of events is effective but underutilized by previous works, and the proposed SWCC makes better use of the co-occurrence information. Ablation study. To investigate the effect of each component in our approach, we conduct an ablation study as reported in Table 2. We remove a certain component of SWCC and examine the corresponding performance of the incomplete SWCC on the similarity tasks. We first explore the impact of our prototype-based clustering method by removing the loss term L cp in Eq. 9. We find that this component has a significant impact on the transitive sentence similarity task. Removing this component causes a 0.05 (maximum) point drop in performance on the transitive sentence similarity task. And for the weakly supervised contrastive learning method, we find that it has a strong impact on both hard simi-3041 Model Hard similarity (Accuracy %) Transitive sentence Original Extended similarity (  ) SWCC 80.9 72.1 0.82 w/o Prototype-based Clustering 77.4 (-3.5) 67.4 (-4.7) 0.77 (-0.05) w/o Weakly Supervised CL 75.7 (-5.2) 65.1 (-7.0) 0.78 (-0.04) w/o MLM 77.4 (-3.5) 70.4 (-1.7) 0.80 (-0.02) BERT (InfoNCE) 72.1 63.4 0.75 BERT (Margin) 43.5 51.4 0.67 Table 2: Ablation study for several methods evaluated on the similarity tasks. larity tasks, especially the extended hard similarity task. Removing this component causes an 7.0 point drop in performance of the model. We also study the impact of the MLM auxiliary objective. As shown in Table 2 the token-level MLM objective improves the performance on the extended hard similarity task modestly, it does not help much for the transitive sentence similarity task. Next, we compare the InfoNCE against the margin loss in Table 2. For a fair comparison, the BERT (InfoNCE) is trained using the InfoNCE objective only, with co-occurring events as positives and other samples in the minibatch as negatives, and the BERT (Margin) is trained using the margin loss, with co-occurring events as positives and randomly sampled events as negatives. Obviously, BERT (InfoNCE) achieves much competitive results on all tasks, suggesting that the InfoNCE with adjustable temperature works better than the margin loss. This can be explained by the fact that the InfoNCE weighs multiple different negatives, and an appropriate temperature can help the model learn from hard negatives, while the margin loss uses only one negative and can not weigh the negatives by their relative hardness. We test the generalization of the event representations by transferring to a downstream event related tasks, the Multiple Choice Narrative Cloze (MCNC) task (Granroth-Wilding and Clark, 2016), which was proposed to evaluate script knowledge. In particular, given an event chain which is a series of events, this task requires a reasoning system to distinguish the next event from a small set of randomly drawn events. We evaluate our methods with several methods based on unsupervised learning: (1) Random picks a candidate at random uniformly; (2) PPMI (Chambers and Jurafsky, 2008) uses co-occurrence information and calculates Positive PMI for event pairs; (3) BiGram (Jans et al., 2012) calculates bi-gram conditional probabilities based on event term frequencies; (4) Word2Vec (Mikolov et al., 2013) uses the word embeddings trained by Skipgram algorithm and event representations are the summation of word embeddings of predicates and arguments. Note that we did not compare with supervised methods (Bai et al., 2021; Zhou et al., 2021; Lv et al., 2020) since unsupervised ones are more suitable for purely evaluating event representations. Results. Table 3 reports the performance of different methods on the MCNC task. As shown in the table, SWCC achieves the best accuracy on the MCNC task under the zero-shot transfer setting, suggesting the proposed SWCC has better generalizability to the downstream tasks than other compared methods. Number of prototypes. Figure 3 displays the impact of the number of prototypes in training. As shown in the figure, the performance increases as the number M increases, but it will not further in-crease after 10. We speculate that because these evaluation data are too small and contain too few types of relations, a larger number of prototypes would not help much in performance improvement. Visualization of learned representation. We randomly sample 3000 events and embed the event representations learned by BERT (InfoNCE) and SWCC in 2D using the PCA method. The cluster label of each event is determined by matching its representation to the set of M prototypes. The resulting visualizations are given in Figure 4. It shows that the proposed SWCC yields significantly better clustering performance than the BERT (InfoNCE), which means, to a certain extent, the prototype-based clustering method can help the event representation model capture various relations of events. Overall, the class separation in the visualizations qualitatively agrees with the performance in Table 1. Case study. We also present sampled events from two different prototypes in Table 4 (see Appendix for more examples), to further demonstrate the ability of SWCC to capture various relations of events. We can see that the events belonging to Prototype1 mainly describe financial stuff, for example, earnings be reduced, while the events belonging to Prototype2 are mainly related to politics. Clearly, the events in the same cluster have the same topic. And we also find that there are also causal and temporal relations between some of these events. For example, earnings be reduced led to company cut costs. Event representation learning. Effectively representing events and their relations (casual, temporal, entailment (Ning et al., 2018; Yu et al., 2020)) becomes important for various downstream tasks, such as event schema induction (Li et al., 2020), event narrative modeling (Chambers and Jurafsky, 2008; Li et al., 2018; Lee and Goldwasser, 2019), event knowledge graph construction (Sap et al., 2019; Zhang et al., 2020) etc. Many efforts have been devoted into learning distributed event representation. Though driven by various motivations, the main idea of these methods is to exploit explicit relations of events as supervision signals and these supervision signals can be roughly categorized into three types: (1) discourse relations (e.g. casual and temporal relations) obtained with automatic annotation tools (Zheng et al., 2020); (2) manually annotated external knowledge (e.g. sentiments and intents) (Lee and Goldwasser, 2018; Ding et al., 2019) and (3) co-occurrence information (Weber et al., 2018). Existing work has focused on the first two supervision signals, with less research on how to better utilize co-occurrence information. Though, discourse relations and external knowledge are fine-grained relations that can provide more accurate knowledge, the current explicitly defined fine-grained relations fall under a small set of event relations. Co-occurrence information is easily accessible but underutilized. Our work focus on exploiting document-level co-occurrence information of events to learn event representations, without any additional annotations. Instance-wise contrastive learning. Recently, a number of instance-wise contrastive learning methods have emerged to greatly improve the performance of unsupervised visual and text representations (He et al., 2020; Chen et al., 2020b,a; Chen and He, 2021; Grill et al., 2020; Zbontar et al., 2021; Chen et al., 2020a; Hu et al., 2021; Gao et al., 2021; Yang et al., 2021). This line of work aims at learning an embedding space where samples from the same instance are pulled closer and 3043 samples from different instances are pushed apart, and usually adopt InfoNCE (van den Oord et al., 2019) objective for training their models. Unlike the margin loss using one positive example and one negative example, the InfoNCE can handle the case where there exists multiple negative samples. In our work, we extend the InfoNCE, which is a self-supervised contrastive learning approach, to a weakly supervised contrastive learning setting, allowing us to effectively leverage co-occurrence information. Deep unsupervised clustering. Clustering based methods have been proposed for representation learning (Caron et al., 2018; Zhan et al., 2020; Caron et al., 2020; Li et al., 2021; Zhang et al., 2021). Caron et al. (2018) use k-means assignments pseudo-labels to learn visual representations. Later, Asano et al. (2020) and Caron et al. (2020) cast the pseudo-label assignment problem as an instance of the optimal transport problem. Inspired by Caron et al. (2020), we leverage a similar formulation to map event representations to prototype vectors. Different from Caron et al. (2020), we simultaneously perform weakly supervised contrastive learning and prototype-based clustering. In this work, we propose a simple and effective framework ( SWCC ) that learns event representations by making better use of co-occurrence information of events, without any addition annotations. In particular, we introduce a weakly supervised contrastive learning method that allows us to consider multiple positives and multiple negatives, and a prototype-based clustering method that avoids semantically related events being pulled apart. Our experiments indicate that our approach not only outperforms other baselines on several event related tasks, but has a good clustering performance on events. We also provide a thorough analysis of the prototype-based clustering method to demonstrate that the learned prototype vectors are able to implicitly capture various relations between events. This work was partially supported by the National Natural Science Foundation of China (61876053, 62006062, 62176076), the Shenzhen Foundational Research Funding (JCYJ20200109113441941, JCYJ20210324115614039), Joint Lab of Lab of HITSZ and China Merchants Securities.
abstain method abstain method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain result result abstain abstain abstain abstain abstain abstain abstain abstain objective method abstain objective abstain abstain method method objective objective result method objective abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain other method method abstain method method method abstain abstain method method abstain method abstain abstain method other method method abstain abstain abstain method abstain abstain method method method abstain method method abstain method abstain abstain abstain method method method method abstain other abstain abstain abstain method abstain method abstain abstain method abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain result abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other method other other other other objective other other other other abstain method objective method result objective other other