We examine the extent to which supervised bridging resolvers can be improved without employing additional labeled bridging data by proposing a novel constrained multi-task learning framework for bridging resolution, within which we (1) design cross-task consistency constraints to guide the learning process; (2) pretrain the entity coreference model in the multitask framework on the large amount of publicly available coreference data; and (3) integrate prior knowledge encoded in rule-based resolvers. Our approach achieves state-of-the-art results on three standard evaluation corpora. Bridging (Clark, 1975) plays an important role in establishing entity coherence in a text. In contrast to direct anaphors , which indicate the coreference relation between a nominal expression and its antecedent, bridging anaphors or associative anaphors link to their antecedents via non-identical relations. Bridging resolution is the task of recognizing and resolving bridging anaphors in a text. Bridging resolution and coreference resolution are closely related to Information Status (IS henceforth) classification, the goal of which is to assign an IS to each discourse entity that indicates how these entities are referred to in a text (Prince, 1981; Nissim et al., 2004; Markert et al., 2012). In general, an entity is old if it is coreferent with an entity that has been mentioned before (e.g., [The busi-ness] and [its] in Figure 1). Bridging anaphors are discourse-new but hearer-old. They have not been introduced in the discourse directly, but are inferrable from previously mentioned entities (e.g., [the customers] in Figure 1). New entities are introduced into the discourse for the first time and are not known to the hearer before (e.g. [The Bakersfield Supermarket] in Figure 1). data. While one of the largest annotated entity coreference resolution datasets, OntoNotes, is composed of 2802 English documents in its training split, the two most commonly used English corpora for bridging resolution research, ISNotes (Mark-ert et al., 2012) and BASHI (Rsiger, 2018), are composed of 50 WSJ documents each. Perhaps the most straightforward way to mitigate this data scarcity problem is to combine existing annotated bridging datasets to create a larger training set (Yu and Poesio, 2020). While it makes sense to combine corpora that are created using the same annotation guidelines (e.g., ISNotes and BASHI), attempting to combine corpora created using different guidelines (e.g., ARRAU (Poesio and Artstein, 2008) and ISNotes) will likely confuse the learner, thus limiting the applicability of this method. Some researchers have instead attempted to create automatically labeled data via lexico-syntactic patterns (Hou, 2018) and distant supervision (Hou, 2020), but a manual analysis of the resulting data instances reveals that they may be too noisy for training: on average only one-fourth of them are correctly labeled (Hou, 2020). By contrast, we aim to investigate the extent to which supervised bridging resolvers can be improved without increasing the amount of labeled bridging data. To this end, we begin by proposing a novel constrained multi-task learning (MTL) framework for bridging resolution. While Yu and Poesio (2020) develop a standard MTL model for 759 bridging resolution and use coreference resolution as the only auxiliary task, we propose to (1) exploit the close connection between IS and bridg-ing/coreference resolution by introducing IS classification as the third task into the MTL framework and (2) guide the learning process by designing cross-task consistency constraints. For instance, in Figure 1, the prediction from the coreference resolution module indicating that both [The business] and [The murder] are old entities can help the bridging resolution module to avoid misclassifying these two mentions as bridging anaphors. Similarly, if the IS classification module predicts [the customers] as a bridging anaphor, then the bridging resolution module should find an antecedent for it. We hypothesize that such constraints can guide the training of a complex model to produce a more coherent output across different tasks, thereby improving bridging resolution performance. While the cross-task consistency constraints could improve performance, they could also hurt performance. Returning to our example in Figure 1, if the IS classification module misclassifies "[the customers]" as non-bridging, the constraints will propagate this error to the bridging resolution module, causing it not to resolve the mention. To address this problem, we (1) formulate these constraints as soft rather than hard constraints, and (2) improve entity coreference resolution performance by leveraging the large amount of publicly-available coreference-annotated data in OntoNotes to pre-train the coreference module. Finally, since previous work (Hou et al., 2014; Roesiger et al., 2018) has shown that manually defined rules based on various syntactic and semantic properties are valuable to recognize and resolve bridging anaphors, we integrate such prior knowledge about bridging into our MTL framework. Note that the only hybrid rule-based and learning-based approach to bridging resolution (Kobayashi and Ng, 2021) merely applies the rule-based resolver and the learning-based resolver in a sequential manner, without combining them into a single model. In sum, our contributions are two-fold. First, we propose a novel constrained MTL framework that jointly learns three tasks, bridging resolution, coreference resolution, and IS classification, via the use of soft cross-task consistency constraints, prior knowledge provided by rule-based approaches, and pre-training on coreference data. Second, experimental results demonstrate that our framework achieves new state-of-the-art results for full bridging resolution on three datasets (ISNotes, BASHI, and ARRAU). The rest of the paper is structured as follows. Section 2 describes related work on bridging resolution and constrained multi-task learning with deep neural networks. Section 3 describes our model, including our multi-task framework for jointly learning IS classification, entity coreference resolution and bridging resolution, our cross-task consistency constraints, and how we integrate rule knowledge into the framework. We present evaluation results in Section 4 and our conclusions in Section 5. 2 Related Work Bridging resolution. Bridging resolution is composed two sub-tasks: bridging anaphora recognition and antecedent selection . Most previous work tackles them separately. One line of research models bridging recognition as part of IS classification (Rahman and Ng, 2011; Markert et al., 2012; Cahill and Riester, 2012; Rahman and Ng, 2012; Hou, 2021), while others have focused on antecedent selection based on gold bridging anaphors (Poesio et al., 2004; Lassalle and Denis, 2011; Hou et al., 2013; Hou, 2020). There are a few studies tackling the challenging task of full bridging resolution (i.e., bridging anaphor recognition and resolution). Hou et al. (2014) and Roesiger et al. (2018) develop rules to identify bridging links based on syntactic and semantic constraints. Hou et al. (2018) propose a pipeline system built on top of complex manually designed features. Yu and Poesio (2020) design a MTL neural model for bridging resolution that uses coreference resolution as an auxiliary task. Recently, Kobayashi and Ng (2021) show the effectiveness of a hybrid rule-based and MTL approach for bridging resolution. For a detailed overview of these approaches, we refer the reader to a recent survey by Kobayashi and Ng (2020). Constrained multi-task learning with deep neural networks. Multi-task learning has been widely adopted in various NLP applications to improve the performance of individual tasks (Ruder, 2017). Recently, several studies have demonstrated that multi-task training in neural networks can be further improved by integrating logical constraints to enforce a coherent output across different tasks (Li et al., 2019; Wang et al., 2020; Lu and Ng, 760 2021). However, for a complex task like bridging resolution, it is non-trivial to choose auxiliary tasks and model the relationships between these tasks in deep neural networks. In this work, we (1) jointly train three tasks (i.e., bridging resolution, coreference resolution, and IS classification); (2) design five soft cross-task consistency constraints to guide the training process; and (3) integrate prior knowledge about bridging into our MTL model. In this section, we present our constrained MTL framework for bridging resolution. Inspired by Yu and Poesio's (2020) span-based model for bridging resolution, which employs an unconstrained MTL framework that jointly learns bridging and coreference, our model takes as input a document D represented as a sequence of word tokens and gold mentions M , from which we create span representations. Our model simultaneously learns three tasks, namely IS classification, bridging, and coreference, as defined below. The IS classification task aims to assign each span i an IS y is taken from an IS inventory. The model predicts the IS of i to be y  is = arg max y is s is ( i, y is ) , where s is is a function suggesting i 's likelihood of having y is as its IS. The bridging resolution task involves determining an antecedent for each bridging anaphor. Formally, it assigns span i an antecedent y b , where y b  Y ( i ) = { 1 , ..., i  1 ,  } . In other words, the value of each y b is the id of its antecedent, which can be one of the preceding spans or a dummy antecedent  (if the mention underlying i is not a bridging anaphor) in the associated document. We define the following scoring function: s b ( i, j ) = (cid:40) 0 j =  s a ( i, j ) j  =  (1) where s a ( i, j ) is a pairwise bridging score computed over i and a preceding span j . The model predicts the antecedent of i to be y  b = arg max y b Y ( i ) s b ( i, y b ) . The entity coreference resolution task involves determining an antecedent for each identity anaphor. Formally, it aims to assign span i an antecedent y c based on a scoring function s c that can be defined in an analogous manner as the s b function in the bridging resolution task. Figure 2 shows the structure of our constrained MTL framework. Below we describe the details. Span Representation Layer Following Yu and Poesio (2020), we use BERT embeddings as the input to a bidirectional LSTM (Hochreiter and Schmidhuber, 1997) to encode tokens and their contexts. Then, we set g i , the representation of span i , to [ h start ( i ) ; h end ( i ) ; h head ( i ) ; f i ] , where h start ( i ) and h end ( i ) are the hidden vectors of the start and end tokens of i , h head ( i ) is an attention-based head vector and f i is a span width feature embedding. 1 IS Prediction Layer For each span i , we pass its representation g i to FFNN is , a standard feed-forward neural network. FFNN is outputs a vector oi i of dimension of S , where S is the number of possible IS labels. Specifically: oi i = FFNN is ( g i ) (2) s is ( i, y is ) = oi i ( y is ) (3) where oi i ( y is ) , the y is -th element of oi i , is a score that indicates i 's likelihood of belonging to IS y is . This score is then used to compute s is . Bridging Prediction Layer To predict bridging links, we define the pairwise score between span i and span j as follows: s a ( i, j ) = FFNN b ([ g i ; g j ; g i  g j ; u ij ]) (4) where  denotes element-wise multiplication, g i  g j encodes the similarity between span i and span j , u ij is a feature embedding encoding the distance between two spans 1 , and FFNN b is the FFNN used in the bridging prediction layer. This pairwise score is then used to compute s b (see Equation (1)). Coreference Prediction Layer The coreference prediction layer is defined in the same way as the bridging prediction layer, with the coreference pairwise score s c ( i, j ) between two spans i and j computed by another FFNN, FFNN c . Note that the first few layers of FFNN c and FFNN b are shared. As noted before, we propose to guide the learning process by incorporating consistency constraints on the three tasks involved in our model. Below we design five cross-task consistency constraints and show how they can be incorporated into our model in a soft manner. 1 This feature embedding is originally proposed by Clark and Manning (2016). See their paper for details. To enforce P1 in a soft manner in our model, we define a penalty function p 1 , which imposes a penalty on span i if it violates the constraint, as shown below: p 1 ( i ) =   0 argmax y is Y s is ( i,y is )  = brid s is ( i, brid )  max y is Y\{ brid } s is ( i,y is ) otherwise (5) where Y is the set of possible IS labels. Intuitively, p 1 estimates the minimum amount that needs to be adjusted so that span i 's IS type is not BRIDGING . In particular, p 1 returns 0 (i.e., no penalty) if i 's IS type is not BRIDGING . s b ( i,  ) = s b ( i,  )   1 p 1 ( i ) where  1 is a positive constant that controls the hardness of the constraint. The smaller  1 is, the softer the constraint is. Intuitively, if P1 is violated, s b ( i,  ) will be lowered by the penalty term, and the dummy antecedent will less likely be selected as the antecedent of i . Constraint P2: If a span i has OLD as its IS value, then its coreference antecedent must not be the dummy antecedent. Similar to P1 , we define a penalty function p 3 to enforce P3 : p 3 ( i ) =   0 argmax y Y s is ( i,y ) = brid max y Y\{ brid } s is ( i,y )  s is ( i, brid ) otherwise (7) We employ p 3 to update s b as follows: s b ( i, j ) = s b ( i, j )   3 p 3 ( i ) (8) where  3 , like  1 , is the hardness coefficient. This penalty is applied only when P3 is violated. Specifically, if IS task predicts a span i as non-B RIDGING but its antecedent selected in the bridging task is not the dummy antecedent, then the penalty term will lower the s b score for each of i 's non-dummy antecedents, which in turn makes it more likely for the dummy antecedent to be selected as the antecedent of i . Constraint P4: If a span i does not have OLD as its IS value, then its coreference antecedent must be the dummy antecedent. Constraint P5: If a span i has a non-dummy antecedent as its coreference antecedent, then its bridging antecedent must be the dummy antecedent. The penalty function p 5 used to enforce P5 is defined as follows: p 5 ( i ) =    0 arg max j Y ( i ) s c ( i, j ) =  max j Y ( i ) \{  } s c ( i, j ) otherwise (9) 762 where Y ( i ) is the set of candidate antecedents of span i . where  5 is the hardness coefficient. Next, we incorporate the prior knowledge provided by rule-based resolvers into our model. Specifically, we employ the set of corpus-specific rules designed by Rsiger et al. (2018). Recall that the output of a rule-based bridging resolver is a set of links between a bridging anaphor and one of its antecedents. We incorporate these bridging links into our model by encoding them as a binary feature, r ij , whose value is 1 if and only if the rule-based resolver posits a bridging link between span i and span j . This feature will be used as an additional feature for FFNN b and FFNN c . As noted by Rsiger et al. (2018), rule-based resolvers are precisionrather than recall-oriented. The reason is that these hand-crafted rules are designed to resolve specific (rather than all) categories of bridging anaphors. For instance, one rule is designed to resolve a building part (e.g., "the door") to the building of which it is a part (e.g., "the house"). Because of the low-recall nature of rule-based resolvers, the feature r ij , which we compute based on the rule-based outputs, could be perceived as not particularly useful by our model. Consequently, to encourage the model to seriously take into consideration the potentially useful information encoded in r ij , we design a rule loss (see Section 3.4), which imposes a penalty on the model during training if the antecedent selected by the model is a non-dummy antecedent that is neither a correct antecedent of i nor the one selected by the rules (as encoded in r ij ). The loss function, L () , consists of the losses of the three tasks and the rule loss as follows: where d is the number of training documents and the hyperparameters (i.e., the  's), which determine the trade-off between the task losses, are tuned using grid search to maximize the average resolution F-scores on development data. Defining the bridging loss is tricky since the antecedents for each bridging anaphor are evaluated in the form of coreference clusters. We adopt the entity coreference loss function originally defined by Wiseman et al. (2015). Specifically, let GOLD b ( i ) denote the set consisting of span i 's bridging antecedent as well as the spans preceding i that are coreferent with the antecedent, and y lb be arg max y  GOLD b ( i ) s b ( i, y ) . In other words, y lb is the highest scoring (latent) antecedent of i according to s b among all the antecedents of i . The loss function for bridging is defined as: L b () = (cid:80) ni =1 max j Y ( i ) ( b ( i, j )(1 + s b ( i, j )  s b ( i, y lb ))) (12) where  b ( i, j ) is a mistake-specific cost function that returns the cost associated with a particular type of error if an error exists and 0 otherwise (Dur-rett and Klein, 2013). 2 Intuitively, the loss function penalizes a span i if the predicted antecedent j has a higher score than the correct latent antecedent y lb . The task loss for coreference, L c , is defined in the same way as the bridging loss, having an analogous mistake-driven cost function  c ( i, j ) . 3 The task loss for the IS prediction task, L is , is the weighted softmax cross entropy loss, where misclassified bridging mentions and non-bridging mentions are weighted according to a mistake-driven cost function  is ( i, j ) . 4 The rule loss is motivated by the bridging loss. Specifically, the model will be penalized if there exists an incorrect non-dummy candidate antecedent whose s b score is higher than the score of the antecedent chosen by the rules, as shown below: L r () = (cid:80) i  N  max j Y ( i ) \  ( r ( i, j )(1 + s b ( i, j )  s b ( i, y r ))) (13) where N  is the set of candidate anaphors for which the rule-based system found a (non-dummy) an-2 In  b ( i, j ) , there are three error types: (1) false link (incorrectly resolved anaphoric mentions); (2) false new (anaphoric mentions misclassified as non-anaphoric); and (3) wrong link (non-anaphoric mentions misclassified as anaphoric). We use hyperparameters  b 1 ,  b 2 , and  b 3 to determine their trade-offs. 3 In  c ( i, j ) , the error types are the same as those in  b ( i, j ) . We use hyperparameters  c 1 ,  c 2 , and  c 3 to determine their trade-offs. 4 In  is ( i, j ) , there are two error types: (1) false new (bridging mentions misclassified as non-bridging); and (2) false bridging (non-bridging mentions misclassified as bridg-ing). We use hyperparameters  is 1 and  is 2 to determine their trade-offs. tecedent, y r is the antecedent selected by the rules, and  r ( i, j ) is an indicator function that returns 0 if j is the correct antecedent and 1 otherwise. As mentioned in the introduction, we pre-train the coreference module in our MTL framework on the English portion of OntoNotes 5.0 5 , excluding those documents that appear in ISNotes or BASHI. To do so, we pre-train the full model shown in Figure 2, setting  b to 1 and the remaining  's to 0 in the loss function so that only the network weights associated with the coreference module will be updated. Note that we follow Yu and Poesio (2020) and use the softmax cross entropy loss rather than the max-margin loss for L b during pre-training, the reason being that this could simplify pre-training by obviating the need to tune the hyperparameters associated with the mistake-specific cost functions. We use three English corpora that are arguably the most widely used corpora for bridging evaluation, namely ISNotes (composed of 50 WSJ articles in OntoNotes) (Markert et al., 2012) , BASHI (The Bridging Anaphors Hand-annotated Inventory, composed of another 50 WSJ articles in OntoNotes) (Rsiger, 2018), and ARRAU (composed of articles from four domains, RST, GNOME, PEAR, and TRAINS) (Poesio and Artstein, 2008; Uryupina et al., 2020). Following previous work, we report results only on RST, the most comprehensively annotated segment of ARRAU. Table 1 shows the statistics on these corpora. For ARRAU RST, we use the standard train-test split. For ISNotes and BASHI, we divide the documents in each corpus into 10 folds (8 folds for training, 1 fold for development, and 1 fold for testing) and report 10-fold cross-validation results. Following previous work (Hou et al., 2014; Roesiger et al., 2018), we report results for full bridging resolution based on gold mentions. In this setting, a system is given as input both a document and its the gold mentions. The goal is to identify bridging anaphors from the gold mentions and resolve them 5 https://catalog.ldc.upenn.edu/ LDC2013T19 Corpora Docs Tokens Mentions Anaphors ISNotes 50 40,292 11,272 663 BASHI 50 57,709 18,561 459 ARRAU RST 413 228,901 72,013 3,777 Table 1: Statistics on different corpora. the gold mentions. There is a caveat in this evaluation setting, however. In ISNotes and BASHI, some bridging antecedents correspond to events (see Example (4) in Table 5), and previous studies differ in terms of how event antecedents should be handled. The reason is that while these event antecedents are annotated, they are not annotated as gold mentions. When reporting results on resolving gold mentions, some previous work (e.g., Hou et al. (2014), Hou et al. (2018)) chose not to include these event antecedents in the list of candidate antecedents and others (e.g., Roesiger et al. (2018), Yu and Poesio (2020)) did. Obviously, the setting in which gold event antecedents are not included in train-ing/evaluation is harsher because it implies that anaphors with event antecedents will always be resolved incorrectly. We believe that including gold event antecedents during evaluation does not represent a realistic setting, and will only report results using the "harsh" setting in this paper. Following Yu and Poesio (2020), we report results for bridging recognition and resolution in terms of precision (P), recall (R), and F-score (F). For recognition, recall is the fraction of gold bridging anaphors that are correctly identified, whereas precision is the fraction of bridging anaphors identified by the system that is correct. For resolution, recall and precision are defined in a similar fashion. In addition, we report IS classification results in terms of accuracy and coreference results in terms of CoNLL score (Pradhan et al., 2014), which is the unweighted average of the F-scores provided by three metrics, MUC (Vilain et al., 1995), B 3 (Bagga and Baldwin, 1998), and CEAF e (Luo, 2005). To train the neural models in our experiments, we use ADAM (Kingma and Ba, 2014) as the optimizer and set all model parameters that originated in Yu and Poesio's (2020) model to the same values as those reported in their paper. Each model is trained for up to 150 epochs in ISNotes and BASHI 764 Model Bridging IS Coreference Recognition Resolution Classification Resolution P R F P R F Accuracy CoNLL ISNotes Roesiger et al. (2018) 46.8 17.7 25.6 32.0 12.1 17.5 -Y&P-MTL 51.8 27.2 36.7 (  1 . 6) 25.3 12.5 17.4 (  1 . 3) -62.6 Hybrid 44.8 35.5 39.6 (  0 . 2) 24.7 19.6 21.9 (  1 . 6) -62.6 MM-MTL 45.5 41.6 43.4 (  0 . 8) 21.1 19.3 20.2 (  0 . 7) -64.5 Full model 54.1 48.0 50.9 (  0 . 2) 27.6 24.5 26.0 (  0 . 0) 78.0 76.3 BASHI Roesiger et al. (2018) 33.5 22.9 27.2 17.3 11.8 14.0 -Y&P-MTL 35.7 15.2 21.3 (  1 . 5) 19.3 8.2 11.5 (  0 . 8) -57.2 Hybrid 32.4 32.3 32.3 (  0 . 7) 16.3 16.3 16.0 (  0 . 4) -57.2 MM-MTL 37.9 27.7 32.0 (  0 . 3) 15.6 11.4 13.2 (  0 . 6) -57.0 Full model 40.7 35.3 37.5 (  0 . 7) 20.1 17.5 18.6 (  0 . 1) 85.3 72.6 ARRAU RST Roesiger et al. (2018) 18.3 33.9 23.7 11.7 21.7 15.2 -Y&P-MTL 27.6 23.1 25.2 (  0 . 3) 20.5 17.2 18.7 (  0 . 1) -55.9 Hybrid 16.8 43.2 24.2 (  0 . 1) 11.3 29.1 16.3 (  0 . 1) -55.9 Full model 26.1 45.6 33.2 (  1 . 2) 17.1 29.8 21.7 (  0 . 0) 84.5 61.2 Table 2: Results of different resolvers on bridging resolution and related tasks. and up to 200 epochs in ARRAU, with early stopping based on the development set. For our model, we pre-train the coreference model for 15 epochs, and the remaining parameters are chosen jointly using grid search to maximize resolution F-score on development data. Specifically, the weights associated with each task and the rule in the loss function (i.e., the  i 's) are searched out of { 0 . 1 , 0 . 5 , 1 , 5 , 10 , 20 , 30 } . The weights associated with the mistake-driven cost functions (i.e., the  i 's) are searched out of { 0 . 1 , 0 . 5 , 1 , 5 , 10 , 15 , 20 } . The hardness coefficients of the consistency constraints (i.e., the  i 's) are searched out of { 0 . 05 , 0 . 1 , 0 . 5 , 1 , 5 , 10 , 20 , 30 } . 6 4.2 Baseline Systems We employ three baselines. The first one is Rsiger et al.'s (2018) rule-based approach, which consists of rules that are built on top of Hou et al. (2014). 7 The second one, Y&P-MTL, is Yu and Poesio's (2020) MTL system. 8 The third one is the Hybrid rule-based and learning-based system proposed by Kobayashi and Ng (2021) in which the rules are first applied and then Y&P-MTL is used to resolve the remaining bridging anaphors. 6 See Appendix A for the final hyperparameters chosen for the full model. Results are shown in Table 2. A few points about the baseline results deserve mention. First, in terms of bridging recognition and resolution performance, the best baselines are Hybrid for both ISNotes and BASHI and Y&P-MTL for ARRAU RST. Hence, these two baselines can be viewed as the prior state of the art. Second, while Rsiger et al.'s rule-based model never achieves the best results on any of the three datasets, it is not always the worst performer: Y&P-MTL is the worst baseline on BASHI in terms of resolution. 9 Third, Hybrid fails to improve the performance of Y&P-MTL in ARRAU RST, meaning that the rules fail to provide additional benefits to Hybrid. This could be attributed to the fact that the rules in ARRAU RST have much lower recognition and resolution precision scores than those in ISNotes and BASHI (Roesiger et al., 2018). While Y&P-MTL uses undersampling (to reduce the number of negative examples used to train the bridging module) and a likelihood loss, we additionally experiment with a max-margin loss (see Section 3.4) without undersampling in our model. To see how these two changes impact performance, we create another model, MM-MTL, which is simply a max-margin version of Y&P-MTL without 9 The baseline results in Table 2 are lower than those reported in the original papers because (1) we report results using the "harsh" setting (see Section 4.1.2); (2) Roesiger et al. (2018) and Kobayashi and Ng (2021) postprocess the system output with gold coreference information, and (3) Yu and Poesio (2020) and Kobayashi and Ng (2021) use additional labeled data for model training. undersampling. Results on the development set are mixed: while MM-MTL outperforms Y&P-MTL on ISNotes and BASHI, the reverse is true on ARRAU RST. Consequently, we use the max-margin loss without undersampling when training our model on ISNotes and BASHI, but fall back on the likelihood loss with undersampling for ARRAU RST. To better understand the impact of using a max-margin loss with undersampling, we show in Table 2 the test results of MM-MTL. As we can see, MM-MTL outperforms Y&P-MTL by 6.710.7% points in F-score for bridging recognition and 1.7 2.8% points in F-score for bridging resolution in ISNotes and BASHI. The last row of each section of Table 2 shows the results of our full model, which outperforms the best baseline by 5.211.3% points in F-score for bridging recognition and 2.64.1% points in F-score for bridging resolution. Hence, the full model establishes new state-of-the-art results on these three datasets. For bookkeeping purposes, we also report the scores for each component of our model in terms of IS classification accuracy and coreference CoNLL score. To evaluate the contribution of the different components in our full model, we show in Tables 3 and 4 ablation results on ISNotes, which we obtain by removing one component at a time from the model and retraining it. Note that for coreference we show the anaphor recognition results as they are affected by the consistency constraints. Consistency constraints. Ablating the consistency constraints means removing all the penalty terms from s b and s c . The resulting system resembles a typical multi-task learning setup, where the different tasks only interact via a shared representation. As we can see in Table 3, bridging resolution F-score drops by 1.7% points, coreference recognition F-score drops by 0.5% points, and IS bridging recognition F-score drops by 1.2% points. These results suggest the effectiveness of using consistency constraints in a multi-task setup. Soft  Hard. Next, we replace soft constraints with hard constraints. Comparing with the results in row 2, bridging resolution F-score drops by 1.2% points. This indicates that having hard constraints is worse than having no constraints at all. rule loss and by 2.4% points when ablating both the rule loss and the rule feature. These results suggest that the rule feature is useful and that the rule loss enhances the effectiveness of the rule feature. Pre-training. Next, we do not pre-train the coreference component in the multi-task framework. This causes bridging resolution F-score and coreference recognition F-score to drop abruptly by 5.8% points and 3.9% points respectively, suggesting the important role played by pre-training. Coreference resolution and IS classification tasks. Next, we ablate one of the tasks in the multi-task framework. Bridging resolution F-score drops by 3.4% points when ablating coreference and by 3.3% points when ablating IS classification. These results suggest that both tasks contribute considerably to bridging resolution performance. Individual soft constraints. Finally, we ablate one soft constraint at a time from the full model. Results are shown in Table 4. Bridging resolution F-score drops by 1.22.3% points, suggesting the positive contribution of each soft constraint. While our discussion of these results has focused on bridging resolution, the same trends can be observed for bridging recognition for the most part. Overall, these results suggest that each component contributes positively to bridging resolution. Although our full model outperforms all previous models for bridging resolution, it is still far from perfect. To better understand what areas of improvement are required, we discuss some common 766 errors made by our full model in this subsection. Bridging anaphora recognition errors. Recall errors in bridging anaphora recognition are the result of a system's failure in identifying bridging anaphors. We find that on the three datasets, the highest proportion of the recall errors (57% on ISNotes, 61% on ARRAU, and 82% on BASHI) is due to the fact that a large number of bridging anaphors are misclassified as new or other 10 mentions in the IS classification module, such as  income  in Example (1) in Table 5. Precision errors in bridging anaphora recognition are the result of a system's misclassification of non-bridging mentions as bridging anaphors. Similar to the recall errors described above, most precision errors are new or other mentions being misclassified as bridging, which account for 50%, 74% and 82% of the precision errors in ISNotes, ARRAU, and BASHI, respectively. In Example (2),  service  is misclassified by both the bridging and IS components as a bridging anaphor. In general, it seems that our system struggles to distinguish bridging anaphors from generic new mentions with simple syntactic structures, an observation that has also been reported in previous work (Hou, 2021; Kobayashi and Ng, 2021). Note that most of these bridging or new mentions are relational nouns (de Bruin and Scha, 1988). Normally, whether additional implicit arguments are required to interpret such relational nouns depends on the surrounding context. In Example (1),  the industry  is necessary to fully understand the meaning of  income ; while in Example (2), no additional implicit arguments are required to understand the meaning of  service . Precision errors in bridging anaphora resolution appear when a system selects the wrong antecedent for a bridging anaphor. A major reason for this error is that our model largely fails to exploit contextual information. In Example (3), the model links the bridging anaphor  a spokesman  to the wrong antecedent [the state], which is reasonable if one does not look into the context. However, according to the context, the correct antecedent should be  Gov. Deukmejian , which requires a system to know that Gov. is the abbreviation for 10 Unlike ISNotes and ARRAU, BASHI does not have IS annotations. We use heuristics to derive four IS types: old , mediated/bridging , mediated/comparative and other . A men-tion's IS is other if it is not annotated as mediated and is not coreferent with any previous mentions. (1) In 1984, an attempt was made to crack down on the industry with tougher restrictions. Then, in 1988, a proposal to keep better track of income by selling prepaid cards for pachinko was fielded in parliament. (2) The Bay Area Rapid Transit system, which runs subway trains beneath the bay, is braced for a doubling of its daily regular ridership to 300,000. BART has increased service to 24 hours a day in preparation for the onslaught. (3) Both Mr.Brown, the state's most influential legislator, and Gov. Deukmejian favor a temporary sales tax increase  should more money be needed than [the state] can raise from existing sources and the federal government. According to a spokesman , the governor is also studying the possibility of raising state gasoline taxes. (4) ... the drug still lacks federal approval for use in the youngest patients. As a result , many youngsters have been unable to obtain the drug ... Governor and that normally a governor will have a spokesman. In addition, on ISNotes, 6% of the bridging anaphors have a non-mention antecedent (see  a result  in Example (4)) and 12% of the bridging anaphors have antecedents that are more than five sentences away. Currently our system does not handle these difficult cases. We proposed the first neural model for full bridging resolution that (1) exploits the connection between information status classification, entity coreference resolution, and bridging resolution in a multi-task learning framework, (2) employs soft cross-task consistency constraints to guide the learning process, (3) pre-trains the entity coreference model, and (4) integrates prior knowledge encoded in handcrafted bridging resolution rules into the learning framework. Our model outperformed several strong baselines and achieved state-of-the-art results on three evaluation datasets. Ablation results provided suggestive evidence that each component of our model contributed positively to bridging resolution performance. We thank the four anonymous reviewers for their insightful comments on an earlier draft of the paper. This work was supported in part by NSF Grants IIS-1528037 and CCF-1848608. Any opinions, findings, conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views or official policies, either expressed or implied, of the NSF.
other result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective abstain abstain abstain objective abstain abstain result result abstain objective objective objective abstain abstain abstain result abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain other abstain abstain abstain abstain other method abstain method abstain method abstain method abstain method method abstain abstain method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain result result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other result objective other other other