While significant progress has been made on the task of Legal Judgment Prediction (LJP) in recent years, the incorrect predictions made by SOTA LJP models can be attributed in part to their failure to (1) locate the key event information that determines the judgment, and (2) exploit the cross-task consistency constraints that exist among the subtasks of LJP. To address these weaknesses, we propose EPM , an Event-based Prediction Model with constraints, which surpasses existing SOTA models in performance on a standard LJP dataset. Legal Judgment Prediction (LJP) is a crucial task in the legal judgment decision making process. Given the facts of a legal case, the goal is to predict the court's outcome. So far, English LJP has focused on predicting law articles (Chalkidis et al., 2019a, 2021) and court decisions (Malik et al., 2021) while French LJP (Sulea et al., 2017b) has focused on predicting court rulings. In this paper, we examine LJP in the context of Chinese via the widely used CAIL dataset (Zhong et al., 2018; Xu et al., 2020), which involves three subtasks: predicting (1) law articles, (2) charges and (3) terms of penalty, as shown in Figure 1. While state-of-the-art (SOTA) LJP models have several fundamental limitations (Binns, 2019), one of the technical issues they face concerns their failure to locate the key event information that determines the judgment results. Consider Figure 2, where the fact statement of a robbery case involves the illegal break-in description. Existing models wrongly predict that the law article is about illegal search since many words describe the break-in process even though the main point is about robbery. How can we address this problem? Recall that in the continental judicial system, a law article consists of two parts: (1) the event pattern, which stipulates the behavior that violates the law, and (2) Law Article Fact Statement Article 263: [Crime ofRobbery] Anyonewho robs public or private property is guilty of the crime of robbery. the judgment, which describes the corresponding penalties. In the law article related to Robbery in Figure 1, the event pattern is Anyone robs public or private property and the judgment is be sentenced to imprisonment of not less than three years and not more than ten years . The event pattern and the corresponding judgment defined by each law article can be viewed as a causal pair: if an event pattern is detected, the corresponding judgment can be inferred from the causal pair. In other words, it is the event information described in the case facts on which the reasoning judgment for the case is based. If we use the fine-grained key event information extracted from the facts to match the event pattern defined in the law articles, the law articles that are applicable to the case could be retrieved accurately and the penalty could be inferred with the judgment in the law article. For example, if we could compress the fact statement in Figure 1 into the fine-grained event in Table 1, we could easily match it with the event pattern defined in Article 263 (see Figure 1). Then the penalty defined in this article can be used as the predicted judgment. by (1) extracting the fine-grained key event of the case and then (2) predict the judgment based on the extracted event information (instead of the whole fact statement). To this end, we propose a hierarchical event definition referring to the hierarchy of law articles. Since there is no public LJP dataset that is annotated with event information, we manually annotate a legal event dataset on the top of CAIL (a public LJP dataset widely used by SOTA methods) (Xiao et al., 2018). Nevertheless, event extraction is challenging. So, to guide the learning process, we design output constraints on event extraction (e.g., what role types are compulsory for a given trigger type) and employ them in our model. Another weakness associated with SOTA methods concerns their failure to exploit the consistency constraints among the three LJP subtasks. Specifically, each law article imposes constraints on what charge and term penalty are possible. However, SOTA methods typically frame LJP as a multi-task learning problem in which the three tasks are jointly learned in a model via a shared representation, without guaranteeing that the aforementioned cross-task constraints are satisfied. To address this problem, we introduce consistency constraints. In sum, our contributions are three-fold. First, we present the first study on leveraging event extraction from case facts to solve LJP tasks. Second, we define a hierarchical event structure for legal cases and collect a new LJP dataset with event annotations. Finally, we propose a model that learns LJP and event extraction jointly subject to two kinds of constraints. Experiments show that our model surpasses the existing SOTA models in performance. Legal judgment prediction. LJP has been investigated in the context of different jurisdictions, such as China (Luo et al., 2017; Zhong et al., 2018; Yue et al., 2021; Feng et al., 2021), the U.S. (Katz Argument Role Who is the criminal? Mike Criminal Who is the victim Jessica Victim What happened? robbed Trigger-Rob What were robbed? gold ring Property What is the price of swag? 1,535 RMB Quantity Judgment Results : Article 263, Robbery, three-year imprisonment Table 1: An example of the judging process of a real case based on event information. et al., 2017), Europe (Chalkidis et al., 2019a, 2021), French (Sulea et al., 2017b,a), India (Malik et al., 2021; Paul et al., 2020). While early works relied on rule-based approaches (Kort, 1957; Segal, 1984; Nagel, 1963), later approaches use classification techniques (Aletras et al., 2016; Liu et al., 2015; Sulea et al., 2017a,b; Katz et al., 2017). More recently, neural models are learned to predict judgment results jointly by sharing parameters in a unified framework (Zhong et al., 2018; Xu et al., 2020; Dong and Niu, 2021; Yang et al., 2019; Feng et al., 2019), applying pre-trained language models (Chalkidis et al., 2020, 2021; Xiao et al., 2021; Niklaus et al., 2021; Zhong et al., 2019), exploiting label-attention mechanisms (Wang et al., 2018, 2019), or injecting legal knowledge (Hu et al., 2018; Gan et al., 2021; Zhong et al., 2020). Unlike our work, these works do not explore the use of case events for LJP. Though existing works exploit dependency between subtasks (Zhong et al., 2018; Yang et al., 2019), they merely utilize the subtasks' prediction results as auxiliary features to influence each other and therefore may still predict inconsistent results. In contrast, our cross-task consistency constraints can guarantee that the predictions are consistent. Event extraction in legal domain. Some works have defined legal events and built models to automatically extract legal events from fact statements using these definitions (Shen et al., 2020; Li et al., 2019; Chen et al., 2020). However, we cannot use these event-annotated legal datasets for two reasons. First, the legal documents in these datasets do not contain legal judgment predictions, so we cannot use them to jointly extract events and make legal judgment predictions. Second, there is a key difference between our work and previous work in terms of how legal events (i.e., the trigger types and argument roles) are defined: while existing works define legal events solely from the perspective of event extraction, we define legal events so that the trigger types and argument roles are useful for LJP. Dataset. We employ as our dataset CAIL (Xiao et al., 2018), a large-scale publicly available Chinese legal document dataset that has been widely used. In CAIL, each judgment document consists of a fact statement and judgment results (law articles, charges and term of penalty). We follow prior works (Xu et al., 2020; Yang et al., 2019) for preprocessing CAIL (see Appendix H). CAIL is composed of two subdatasets: CAIL-big and CAIL-small, and their statistics are shown in Table 2. LJP on CAIL is by no means trivial: there are 127, 140 and 11 categories for article, charge and penalty respectively on CAIL-big. Task definition. Given a fact statement, LJP on CAIL involves three prediction subtasks t a , t c , t p  T , which correspond to law article, charge and term of penalty respectively. Following previous works (Xu et al., 2020; Yang et al., 2019), we formalize each subtask t  T as a multi-class classification problem and predict for each t the corresponding result y t  Y t , where Y t is the label set of t . We begin by designing a multi-task legal judgment prediction model, which we will use as a baseline and augment with event extraction and constraints in subsequent sections. The framework of our model is shown in Figure 3. Token representation layer. Given a fact statement represented as a character sequence D = { x 1 , x 2 , ...x l f } , we first encode each character by passing them into a pretrained legal BERT encoder (Zhong et al., 2019). where H f = { h 1 , h 2 , ... h l f } is the hidden vector sequence of the fact statement and l f is the length of the fact statement. robbed a gold ring. Incorporating law article semantics. Using the aforementioned context representation to predict judgment essentially treats each law article as an atomic label, leaving its semantic information unexploited. Inspired by previous work (Chalkidis et al., 2019b; Rios and Kavuluru, 2018), we employ an attention mechanism to incorporate article semantics into the model. Specifically, we match  h with all candidate law articles. To do so, we first use the same encoder to encode the character sequence of each law article and obtain the hidden vector sequence H a = { h 1 , h 2 , ... h l a } , where l a is the length of a law article text. Then we apply a max-pooling layer to H a to get the context representation c . Next, we use the context representation of the fact statement,  h , to query all candidate articles in order to mine the most relevant semantics in the article texts. Specifically, we first obtain the relevance scores between  h and the j -th article c j :  j =  h TW c c j (3) where W c is a trainable matrix. where  c contains the integrated article semantics. Legal judgment prediction layer. To predict legal judgment, we input  h and  c into three task-specific classifiers as follows:  y t = softmax ( W t [ h ;  c ] + b t ) (5) where W t and b t are the learnable parameters and  y t is the prediction distribution of task t . Training. For each legal judgment prediction task t , we employ cross-entropy as the loss function to measure the distance between the predicted  y t and the ground-truth y t . where hyperparameters  determine the trade-off between all subtask losses. The model is trained to minimize L () . In this section, we propose a novel method to leverage event extraction to improve LJP. Event definition. Each law article stipulates what event violates this article, so it is easy to define legal events based on law articles. The Chinese law articles have been organized in a hierarchical manner. For example, robbery-related and theft-related articles belong to Property Infringement , which is the general name of robbery-related and theft-related articles. We define legal events following this hierarchy. As shown in Figure 4, Property Infringement is treated as a superordinate event type, whereas Robbery and Theft are treated as subordinate event types. This hierarchy can express the connections between different legal events. Trigger and role definitions. An event trigger is a word that realizes the occurrence of an event and has a type. There is a one-to-one correspondence between event type and trigger type. For example, the Robbery event has the trigger type Trigger-Rob . Next, we define the roles for each event such that they reflect the key elements of the event that would be useful for making legal judgments. For example, the Criminal and Victim roles specify the parties involved in a case, whereas the Quantity role measures the value of loot, based on which term penalty is derived. We define the roles in a hierarchical manner. As seen in Figure 4, the Party arguments are the people involved in the cases, and its subordinate roles include Criminal and Victim . 1 5.2 Dataset Collection To investigate the use of event extraction for LJP, we manually create an event-annotated LJP dataset since no such dataset is publicly available. Step 1: judgment document collection. We construct LJP-E, our event-annotated dataset, based on CAIL. Specifically, we first analyze the performance of the SOTA models (Xu et al., 2020; Zhong et al., 2018) on the validation portion of CAIL-small and identify the 15 law articles for which they achieved poor performance, and then select a subset of the cases that can be judged by these 15 law articles for annotation. This subset consists of 1367 documents (957 as training set, 136 as validation set and 274 as test set). We henceforth refer to this set of judgment documents as D o . Step 2: event trigger and argument role annotation. Next, we hire two annotators to manually produce event triggers and argument roles for each case in D o after giving them a three-hour tutorial on how to annotate events. The annotators are native speakers of Chinese who are graduate students in NLP with significant experience with working on legal problems (none of them are the authors). The annotation process. Given the fact statement and the gold law article of a case, each annotator is asked to independently highlight the salient words in the fact statement that reflect the core event of the case and correlate well with the event pattern of the law article. Then each of them is asked to (1) select a trigger word and assign it a subordinate trigger type, and (2) assign a subordinate role type to each of its arguments from a predefined role list. The trigger type and role type inventories were designed by the authors after having read a large number of fact statements and corresponding articles. Inter-annotator agreement numbers can be found in Appendix E. After the above steps, each case in D o is annotated with a trigger, its type, its arguments and roles. The average number of arguments per event is 4.13. 1 Details of the event and role definitions together with their explanations can be found in Appendix A and B. 651 There are 16 distinct subordinate roles and 15 distinct subordinate trigger types. 2 Each disagreement between the annotators is resolved via discussion. To make use of the event annotations, we augment our baseline model with a hierarchical event extraction layer that detects event triggers and arguments and determines trigger types and arguments roles (see Figure 3). The resulting model simultaneously learns event extraction and LJP. We formalize event extraction as a token labeling problem. Given the hidden vectors of the fact tokens H l f = { h 1 , h 2 , ... h l f } , we assign each token a subordinate trigger (if it is part of a trigger) or a subordinate role type (if it is part of an argument). The hierarchical event extraction layer consists of two modules: (1) a superordinate module that attends each hidden vector to all superordinate types/roles for obtaining their correlations, and (2) a subordinate module that computes the subordinate type/role probability distribution based on hierarchical information, as described below. For a specific superordinate type/role j , we represent its semantic features with a trainable vector p j . We adopt a fully-connected layer to calculate the correlation score between hidden vector h i and superordinate type/role p j . where u ij represents the correlation score and [; ] denotes the concatenation of two vectors. Then, we apply a softmax operation to get the superordinate type/role feature for each token x i . ij = exp ( u ij ) (cid:80) k =1 exp ( u ik ) (9) o i = (cid:88) j =1  ij p j (10) where o i is the integrated superordinate type/role feature, which provides superordinate-oriented information useful for predicting subordinate types/ roles. Next, we concatenate each h i with o i as the input feature for the trigger type and argument role classifier and estimate the probability that token x i belongs to subordinate type/role r j as follows: s r ( x i ,  y r j ( i ) ) = exp ( q Tj [ h i ; o i ]) (cid:80) k =1 exp ( q Tk [ h i ; o i ]] (11) 2 Statistics of LJP-E can be found in Appendix D. where q j is the trainable vector of r j . After obtaining the type/role probability distribution of x i , we apply a CRF (Lafferty et al., 2001) to produce the sequence of types/roles with the highest score, where the score of a sequence of types/roles  y r = {  y r (1) ,  y r (2) ... } is computed as: score ( D,  y r ) = l f (cid:88) i =1 T  y r ( i  1) ,  y r ( i ) + l f (cid:88) i =1 s r ( x i ,  y r ( i ) ) (12) Here, T is the score of transitioning from one tag to another tag. Instead of predicting LJP based on the context fact representation  h , we replace it with the detected event features. Specifically, we input the extracted trigger word and arguments with their type/role embeddings into the three task-specific classifiers. Denote an extracted span as H s = { h 1 , h 2 , ... h l s } . We apply a max-pooling layer to each span and concatenate with the corresponding subordinate type/role embedding q : g i = [ maxpooling ( H s ); q ] (13) where g i denotes the representation of span i , which contains both semantics and subordinate type/role features. Based on g i , we can calculate the context span representation  g as follows:  g = maxpooling ( g 1 , g 2 ... ) (14) which is used to replace the context fact representation  h in Equation 2. Training. L () =  t a L t a +  t c L t c +  t p L t p +  r L r (16) To improve model performance, we explore two types of constraints, as described below. Absolute constraint. For a legal event, the trigger must appear exactly once and certain roles are compulsory (e.g., subordinate role Criminal should appear at least once). If the trigger is missing, we impose the following penalty: P t = (cid:88)  r T G l f (cid:88) i =1 s r ( x i ,  y  r ( i ) )  max i,  r T G [ s r ( x i ,  y  r ( i ) )] + | 1  max i,  r T G [ s r ( x i ,  y  r ( i ) )] | (17) where T G is the trigger inventory. 3 If a required role r is missing, we impose the following penalty: P r = | 1  max i [ s r ( x i ,  y r ( i ) )] | (18) Event-Based consistency constraint. If a trigger type is detected, all and only its related roles should be detected. For example, if a Illegal Doctoring event is detected, no roles related to Illegal Logging should be predicted. If a trigger r is predicted, we impose the following penalty: P e = (cid:88)  r R + | 1  max i [ s r ( x i ,  y  r ( i ) )] | + (cid:88)  r R  l f (cid:88) i =1 s r ( x i ,  y  r ( i ) ) (19) where R + is the set of roles that should occur if r is predicted, and R  is the set of roles that cannot occur. We sum all the penalty terms and incorporate them into the total loss as follows: L () =  t a L t a +  t c L t c +  t p L t p +  r L r +  p (cid:88) P i (20) 6.2 Cross-Task Consistency Constraints While the multi-task learning setup employed by our model allows subtasks of LJP to benefit each other via the shared representation layer, it fails to exploit the dependency explicitly that exist among them. Below we exploit two such dependencies, one between law article and charge and the other between law article and term of penalty. Each law article states the allowable charges and range of term of penalty. Hence, we can utilize these dependencies to constrain (and hopefully improve) the prediction of charge and term of penalty 3 An explanation of the penalty functions in Equation 17, 18 and 19 can be found in Appendix K using the predicted law article. More specifically, we make the model learn how to predict charge and term of penalty based on the predicted article during training by modifying the cross entropy loss as follows. If the law article is predicted correctly by the model, then when calculating L t c (i.e., the cross-entropy loss associated with the charge prediction task), we mask each term in the loss corresponding to a charge that is not allowed according to the predicted article: L t c =  (cid:88) (cid:88) mask  y t c log  y t c (21) where mask is equal to 0 if the charge is not allowed according to the predicted article and 1 otherwise. However, if the article is predicted incorrectly by the model, L t c is the standard cross entropy loss. Intuitively, through masking, the model is forced to predict a charge that is allowed according to the predicted article. During testing, since we do not know whether the law article is predicted correctly or not, we always mask the charge probability distribution according to the predicted article. We adopt the same strategy to compute L t p when enforcing the consistency constraint between law article prediction and term of penalty prediction. We train our model EPM using the pre-training and fine-tuning strategy. Specifically, we pre-train EPM without event components on the training portion of CAIL (Table 2), and then fine-tune EPM on the training portion of LJP-E, our event-annotated LJP dataset, to learn from the event annotations. As for the encoder, the maximum fact length is set to 512. For training, we utilize the Adam optimizer with learning rate of 10  4 and the batch size is 32. The warmup step is 3000. For the hyperparameters,  , in the loss function, the best setting is {0.5, 0.5, 0.4, 0.2, 0.1} for {  t a ,  t c ,  t p ,  r ,  p }. Models are trained for a maximum of 20 epochs. 4 LJP results are reported in terms of Accuracy (Acc), Macro-Precision (MP), Macro-Recall (MR) and Macro-F1 (F1). We compare EPM with SOTA models on the test portion of our annotated dataset LJP-E in Table 3. and the official test portion of the CAIL dataset in 4 Details of experimental setup can be found in Appendix I 653 Law Article Charge Term of Penalty Acc % MP % MR % F1 % Acc % MP % MR % F1 % Acc % MP % MR % F1 % 1 MLAC 83.75 71.49 71.79 70.05 73.20 52.82 55.93 52.19 23.57 17.92 17.22 16.38 2 TOPJUDGE 86.46 75.51 75.07 73.97 75.16 56.04 58.96 55.34 23.82 18.59 18.43 17.63 3 MBPFN 86.72 86.72 75.60 74.28 73.95 73.95 56.48 54.00 27.53 27.53 17.97 19.65 4 LADAN 89.92 78.13 78.01 77.06 79.12 58.54 61.87 58.35 26.06 20.86 18.03 16.58 5 NeurJudge 87.87 81.17 82.68 80.41 76.04 61.95 60.07 59.46 27.88 20.99 16.81 18.51 6 EPM 93.85 91.15 89.14 89.37 79.37 61.14 63.15 60.79 28.51 28.27 23.58 23.23 7 w/ gold 97.05 95.63 93.42 93.82 86.98 70.45 73.08 70.92 33.16 30.28 23.52 24.11 8 TOPJUDGE+Event 88.84 78.03 79.72 77.39 77.24 60.70 60.57 57.40 27.49 22.38 18.17 18.10 Table 3: Comparisons with the SOTA models on LJP-E. Table 4 and 5. Since LJP-E only contains the 15 case types of CAIL, when applying EPM on the CAIL test set we use the pretrained version of EPM (i.e., without fine-tuning) to predict samples that do not belong to the 15 types and use the fine-tuned version of EPM to predict samples that belong to one of the 15 types. In order to determine whether a sample belongs to one of the 15 types, we train a binary classifier using legal BERT on the training set of CAIL. We refer to this model as the Switch . 5 We compare EPM with four SOTA neural models: (1) MLAC (Luo et al., 2017), which jointly 5 Details of the Switch can be found in Appendix J. modeled charge prediction and the relevant article extraction task in a unified framework. Here, we add a fully-connected layer in order to predict the term of penalty; (2) TOPJUDGE (Zhong et al., 2018), which formalized the subtasks of LJP in a joint framework as a directed acyclic graph in which the subtasks share parameters; (3) MPBFN (Yang et al., 2019), which proposed a multi-perspective forward and backward prediction framework to make the sharing of parameters by different subtasks effectively, as well as a number embedding method for term of penalty prediction; (4) LADAN (Xu et al., 2020), which developed a 654 graph networks to learn the subtle differences between law articles in order to extract compelling discriminative features from fact statements; and (5) NeurJudge (Yue et al., 2021), which utilized the results of intermediate subtasks to separate the fact statement into different circumstances and exploits them to make the predictions of other subtasks. As shown in Tables 3, 4 and 5, EPM (row 6) achieves the best results, substantially outperforming not only MLAC but also TOPJUDGE, MPBFN, LADAN and NeurJudge, which further leverage extensions like number embedding and graph networks , particularly on law article prediction. Next, we conduct two oracle experiments involving EPM . First, we use gold rather than predicted event annotations to make predictions for the three subtasks. 6 The results, which are show in row 7 of Table 3, show that considerably better results can be obtained when gold event annotations are used. These results suggest that existing LJP results can be substantially improved by improving event extraction. Next, we assume that the Switch is perfect when obtaining the EPM results on CAIL. Perhaps not surprisingly, results, which are shown in row 7 of Table 4 and 5, are better w.r.t. all subtasks. Further, we apply EPM to MLAC , TOPJUDGE , MPBFN , LADAN and NeurJudge on CAIL, the five SOTA models following the same scheme (i.e., use fine-tuned EPM to classify when the Switch says the sample belongs to the 15 types and use the SOTA model to classify otherwise), showing the results in rows 8 to 12 in Table 4 and 5. We see that EPM can also improve the performance of the four SOTA models, yielding new SOTA results. Finally, we examine whether modifying a SOTA model, TOPJUDGE , by having it jointly perform event extraction and the LJP tasks can improve its performance. To do so, we replace its CNN encoder by an LSTM and feed the LJP classifiers with the extracted events rather than the case facts in the same way as in EPM . We can see that TOPJUDGE+Event outperforms TOPJUDGE , which shows the usefulness of event information. However, TOPJUDGE+Event underperforms TOP-JUDGE+EPM . This suggests that better LJP results can be achieved by treating TOPJUDGE as a black box (by exploiting event information using the Switch ) rather than a glass box (by modifying 6 Event extraction results in EPM are as follows: 53.75% (R), 47.52% (P), and 50.37% (F1) for trigger detection and 55.69% (R), 49.88% (P), and 52.59% (F1) for role prediction. the model to learn from event annotations). We conduct experiments on the ablated versions of EPM . Ablation results on LJP-E and CAIL are shown in Tables 6 and 7, 8. Event extraction. To test the usefulness of event extraction, we delete all event components from EPM . Results are shown in row 2. As we can see, performance degrades substantially on all three subtasks in terms of both Acc and F1. Event-based constraints. Next, we evaluate the usefulness of the two event-based constraints (Sec-tion 6.1) on the outputs of event extraction. Removing the absolute constraint ( w/o CSTR1 , row 3) or the event-based consistency constraint ( w/o CSTR2 , row 4) generally yields worse results in terms of both Acc and F1. In particular, removing the consistency constraint generally provides bigger deterioration than the absolute constraint. Cross-task consistency constraints. We also evaluate the cross-task consistency constraints. Removing the article-charge constraint ( w/o DEP1 , row 5) or the article-term constraint ( w/o DEP2 , row 6) negatively impacts performance, with the largest negative impact observed on charge prediction. While these constraints are intended to employ the predicted law article results to improve charge prediction and term prediction, we see that law article performance also deteriorates. Superordinate types. So far, we have assumed that hierarchical event extraction would be beneficial to LJP. To better understand whether the hierarchy is indeed useful, we evaluate a version of EPM without using superordinate features. In other words, the model predicts the subordinate types/roles directly. Results are shown in row 7. Comparing rows 1 and 7, we see that Acc and F1 scores drop across all subtasks when superordinate features are not used, indicating their usefulness. Event extraction as an auxiliary task. In EPM , we use the predicted event features as inputs for the three LJP task classifiers. Another way to exploit event information would be to treat event extraction as an auxiliary task in the model by having it share encoders with the LJP tasks. Results of treating event extraction as an auxiliary task are shown in row 8. As we can see, these results are worse than those of EPM (row 1), which means that EPM 's way of exploiting event information is better, but they are better than those when event information is 655 Law Article Charge Term of Penalty Acc % MP % MR % F1 % Acc % MP % MR % F1 % Acc % MP % MR % F1 % 1 EPM 93.85 91.15 89.14 89.37 79.37 61.14 63.15 60.79 28.51 28.27 23.58 23.23 2 w/o event 86.17 76.49 75.66 75.19 73.21 56.94 56.46 55.46 26.25 18.78 15.71 15.06 3 w/o CSTR1 88.44 83.22 80.77 80.00 74.21 57.06 59.37 56.54 27.85 18.01 18.16 16.87 4 w/o CSTR2 86.96 77.95 76.17 76.84 74.69 57.29 56.98 55.99 27.77 19.18 17.55 16.99 5 w/o DEP1 91.15 86.29 85.23 84.96 73.62 56.84 58.69 56.71 23.35 16.29 16.06 15.23 6 w/o DEP2 91.89 90.98 87.41 88.30 78.38 60.48 61.43 59.31 23.31 17.30 14.12 14.49 7 w/o hierarchy 87.21 79.20 76.52 76.38 73.95 58.23 59.50 57.36 23.59 19.42 16.77 16.32 8 w/ auxiliary 92.62 85.24 83.95 83.90 76.18 55.08 59.05 56.39 25.31 21.06 16.58 15.88 Table 6: Ablation results on LJP-E. not use (row 2), which means that using predicted events for LJP is still better than not using them. Next, we perform a qualitative analysis of EPM to better understand the role played by event information and constraints. In CAIL, the data distribution of term penalty for the same law article is skewed towards larger penalty values, thus causing EPM to inherit this bias in its prediction of term penalty when cross-task consistency constraints are not used. However, when constraints are used, EPM was forced to only predict those term penalties that are allowed by the predicted law article and was thus more robust to the skewed data distribution. As for events, the use of event information prevents EPM from focusing on certain words in a case fact that could trigger the prediction of wrong law articles. A detailed analysis can be found in Appendix F. 8 Conclusion We proposed the first model that uses event extraction and hand-crafted constraints to improve LJP, achieving SOTA results. To facilitate future research, we make our codes and annotations publicly available at https://github.com/WAPAY/EPM. We thank the three anonymous reviewers for their comments on an earlier draft of this paper. This work was supported in part by the National Natural Science Foundation of China (No. 61802167), the US National Science Foundation (Grant IIS1528037). Chuanyi Li is the corresponding author. Any opinions, findings, conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views or official policies, either expressed or implied, of the funding agencies. Automatic legal judgment prediction is a sensitive research area. The proposed EPM model is a preliminary work and is not ready to be productized. The goal of designing the EPM model is to surpass the performances of existing SOAT approaches which are not ready to be productized either. Legal cases contain personal privacy information. Therefore, we use a public dataset that has been anonymized, i.e., CAIL, which is collected from the official website for disclosing legal case information. The private information of the cases has been anonymized when the cases are published on the official website.
abstain objective abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain method result method abstain objective method abstain method abstain abstain abstain abstain objective objective objective objective result other other other other objective other objective other other method objective method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain method abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain other other method method abstain method method abstain abstain abstain abstain abstain method method abstain abstain method method abstain abstain abstain method method abstain abstain method method method method abstain method abstain abstain abstain abstain method other abstain method method abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain other abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain method method abstain abstain abstain abstain method abstain abstain other abstain method abstain abstain abstain abstain other other other other other abstain abstain abstain abstain method abstain