Sequence tagging models for constituent parsing are faster, but less accurate than other types of parsers. In this work, we address the following weaknesses of such constituent parsers: (a) high error rates around closing brackets of long constituents, (b) large label sets, leading to sparsity, and (c) error propagation arising from greedy decoding. To effectively close brackets, we train a model that learns to switch between tagging schemes. To reduce sparsity, we decompose the label set and use multi-task learning to jointly learn to predict sublabels. Finally, we mitigate issues from greedy decoding through auxiliary losses and sentence-level fine-tuning with policy gradient. Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further. On the SPMRL datasets, we observe even greater improvements across the board, including a new state of the art on Basque, Hebrew, Polish and Swedish. 1 1 Introduction Constituent parsing is a core task in natural language processing ( NLP ), with a wide set of applications. Most competitive parsers are slow, however, to the extent that it is prohibitive of downstream applications in large-scale environments (Kummerfeld et al., 2012). Previous efforts to obtain speed-ups have focused on creating more efficient versions of traditional shift-reduce (Sagae and Lavie, 2006; Zhang and Clark, 2009) or chart-based parsers (Collins, 1997; Charniak, 2000). Zhu et al. (2013), for example, presented 1 After this paper was submitted, Kitaev and Klein (2018b) have improved our results using their previous self-attentive constituent parser (Kitaev and Klein, 2018a) and BERT representations (Devlin et al., 2018) as input to their system. We will acknowledge these results in the Experiments section. a fast shift-reduce parser with transitions learned by a SVM classifier. Similarly, Hall et al. (2014) introduced a fast GPU implementation for Petrov and Klein (2007), and Shen et al. (2018) significantly improved the speed of the Stern et al. (2017) greedy top-down algorithm, by learning to predict a list of syntactic distances that determine the order in which the sentence should be split. In an alternative line of work, some authors have proposed new parsing paradigms that aim to both reduce the complexity of existing parsers and improve their speed. Vinyals et al. (2015) proposed a machine translation-inspired sequence-to-sequence approach to constituent parsing, where the input is the raw sentence, and the transla-tion' is a parenthesized version of its tree. Gomez-Rodrguez and Vilares (2018) reduced constituent parsing to sequence tagging, where only n tagging actions need to be made, and obtained one of the fastest parsers to date. However, the performance is well below the state of the art (Dyer et al., 2016; Stern et al., 2017; Kitaev and Klein, 2018a). Contribution We first explore different factors that prevent sequence tagging constituent parsers from obtaining better results. These include: high error rates when long constituents need to be closed, label sparsity, and error propagation arising from greedy inference. We then present the technical contributions of the work. To effectively close brackets of long constituents, we combine the relative-scale tagging scheme used by Gomez-Rodrguez and Vilares (2018) with a secondary top-down absolute-scale scheme. This makes it possible to train a model that learns how to switch between two encodings, depending on which one is more suitable at each time step. To reduce label sparsity, we recast the constituent-parsing-as-sequence-tagging problem as multi-task learning ( MTL ) (Caruana, 1997), to decompose a large label space and also obtain speed ups. Finally, we mitigate error propagation using two strategies that come at no cost to inference efficiency: auxiliary tasks and policy gradient fine-tuning. We briefly introduce preliminaries that we will build upon in the rest of this paper: encoding functions for constituent trees, sequence tagging, multi-task learning, and reinforcement learning. Notation We use w = [ w 0 , w 1 , ..., w n ] to refer to a raw input sentence and bold style lower-cased and math style upper-cased characters to refer to vectors and matrices, respectively (e.g. x and W ). Gomez-Rodrguez and Vilares (2018) define a linearization function of the form  | w | : T | w |  L ( | w | 1) to map a phrase structure tree with | w | words to a sequence of labels of length | w |  1 . 2 For each word w t , the function generates a label l t  L of the form l t = ( n t , c t , u t ) , where:  n t encodes the number of ancestors in common between between w t and w t +1 . To reduce the number of possible values, n t is encoded as the relative variation in the number of common ancestors with respect to n t  1 . c t encodes the lowest common ancestor between w t and w t +1 . u t contains the unary branch for w t , if any. Figure 1 explains the encoding with an example. 2 They (1) generate a dummy label for the last word and (2) pad sentences with a beginningand end-of-sentence tokens. Sequence tagging is a structured prediction task that generates an output label for every input token. Long short-term memory networks ( LSTM ) (Hochreiter and Schmidhuber, 1997) are a popular architecture for such tasks, often giving state-of-the-art performance (Reimers and Gurevych, 2017; Yang and Zhang, 2018). Tagging with LSTM s In LSTM s, the prediction for the i th element is conditioned on the output of the previous steps. Let LSTM  ( x 1: n ) be a parametrized function of the network, where the input is a sequence of vectors x 1: n , its output is a sequence of hidden vectors h 1: n . To obtain better contextualized hidden vectors, it is possible to instead use bidirectional LSTMS (Schuster and Paliwal, 1997). First, a LSTM l processes the tokens from left-to-right and then an independent LSTM r processes them from right-to-left. The i th final hidden vector is represented as the concatenation of both outputs, i.e. BILSTM  ( x , i ) = LSTM l ( x [1: i ] )  LSTM r ( x [ | x | : i ] ) . BILSTM s can be stacked in order to obtain richer representations. To decode the final hidden vectors into discrete labels, a standard approach is to use a feed-forward network together with a softmax transformation, i.e. P ( y | h i ) = softmax ( W  h i + b ) . We will use the BILSTM -based model by Yang and Zhang (2018), for direct comparison against Gomez-Rodrguez and Vilares (2018), who use the same model. As input, we will use word embeddings, PoS-tag embeddings and a second word embedding learned by a character-based LSTM layer. The model is optimized minimizing the categorical cross-entropy loss, i.e. L =  (cid:80) log ( P ( y | h i )) . The architecture is shown in Figure 2. 2.3 Multi-task Learning Multi-task learning is used to solve multiple tasks using a single model architecture, with task-specific classifier functions from the outer-most representations (Caruana, 1997; Collobert and Weston, 2008). The benefits are intuitive: sharing a common representation for different tasks acts as a generalization mechanism and allows to address them in a parallel fashion. The hard-sharing strategy is the most basic MTL architecture, where the internal representation is fully shared across all tasks. The approach has proven robust for a number of NLP tasks (Bingel and Sgaard, 2017) and comes with certain guarantees if a common, op-(1,S,NP) (1,VP,) (3,NP,) (-1,NP,) (1,PP,) (-2,S,) (-2,S,ADJP) I find your lack of faith disturbing . timal representation exists (Baxter, 2000). Dong et al. (2015) use it for their multilingual machine translation system, where the encoder is a shared gated recurrent neural network (Cho et al., 2014) and the decoder is language-specific. Plank et al. (2016) also use a hard-sharing setup to improve the performance of BILSTM -based PoS taggers. To do so, they rely on auxiliary tasks , i.e, tasks that are not of interest themselves, but that are co-learned in a MTL setup with the goal of improving the network's performance on the main task(s). We will introduce auxiliary tasks for sequence tagging constituent parsing later on in this work. AMTL architecture can also rely on partial sharing when the different tasks do not fully share the internal representations (Duong et al., 2015; Rei, 2017; Ruder et al., 2019) and recent work has also shown that hierarchical sharing (e.g. low-level task outputs used as input for higher-level ones) could be beneficial (Sgaard and Goldberg, 2016; Sanh et al., 2018). Policy gradient ( PG ) methods are a class of reinforcement learning algorithms that directly learn a parametrized policy, by which an agent selects actions based on the gradient of a scalar performance measure with respect to the policy. Compared to other reinforcement learning methods, PG is well-suited to NLP problems due to its appealing convergence properties and effectiveness in high-dimensional spaces (Sutton and Barto, 2018). Previous work on constituent parsing has employed PG methods to mitigate the effect of exposure bias, finding that they function as a model-agnostic substitute for dynamic oracles (Fried and Klein, 2018). Similarly, Le and Fokkens (2017) apply PG methods to Chen and Manning (2014)'s transition-based dependency parser to reduce error propagation. In this work, we also employ PG to fine-tune models trained using supervised learning. However, our setting (sequence tagging) has a considerably larger action space than a transition parser. To deal with that, we will adopt a number of variance reduction and regularization techniques to make reinforcement learning stable. We describe the methods introduced in this work, motivated by current limitations of existing sequence tagging models, which are first reviewed. The source code can be found as a part of https: //github.com/aghie/tree2labels . For brevity, we limit this analysis to the English Penn Treebank ( PTB ) (Marcus et al., 1993). We reproduced the best setup by Gomez-Rodrguez and Vilares (2018), which we are using as baseline, and run the model on the development set. We below show insights for the elements of the output tuple ( n t , c t , u t ) , where n t is the number of levels in common between w t and w t +1 , c t is the nonterminal symbol shared at that level, and u t is a leaf unary chain located at w t . High error rate on closing brackets We first focus on predicting relative tree levels ( n t ). See Figure 3 for F-scores over n t labels. The sparsity on negative n t s is larger than for the positive ones, and we see that consequently, the performance is also significantly worse for negative n t values, and performance worsens with higher negative values. This indicates that the current model cannot effectively identify the end of long constituents. This is a known source of error for shift-reduce or chart-based parsers, but in the case of sequence tagging parsers, the problem seems particularly serious. Sparsity The label space is large and sparse: the output labels are simply the possible values in the tuple ( n t , c t , u t ) . An analysis over the PTB training set shows a total of 1423 labels, with 58% of them occurring 5 or less times. These infrequent cases might be difficult to predict, even if some of the elements of the tuple are common. Greedy decoding Greedy decoding is prone to issues such as error propagation. This is a known source of error in transition-based dependency parsing (Qi and Manning, 2017); in contrast with graph-based parsing, in which parsing is reduced to global optimization over edge-factored scores (McDonald et al., 2005). In the case of BILSTM -based sequence tagging parsers, for a given word w t , the output label as encoded by Gomez-Rodrguez and Vilares (2018) only reflects a relation between w t and w t +1 . We hypothesize that even if the hidden vector representations are globally contextualized over the whole sequence, the intrinsic locality of the output label also turns into error propagation and consequently causes a drop in the performance. These hypotheses will be tested in  4. In particular, we will evaluate the impact of the different methods intended to perform structured inference (  3.4). Gomez-Rodrguez and Vilares (2018) encode the number of common ancestors n t , from the output tuple ( n t , c t , u t ) , as the variation with respect to n t  1 . We propose instead to encode certain elements of a sentence using a secondary linearization function. The aim is to generate a model that can dynamically switch between different tagging schemes at each time step t to select the one that represents the relation between w t and w t +1 in the most effective way. On the one hand, the relative-scale encoding is effective to predict the beginning and the end of short constituents, i.e. when a short constituent must be predicted ( | n t |  2 ). On the other hand, with a relative encoding scheme, the F-score was low for words where the corresponding n t has a large negative value (as showed in Figure 3). This matches a case where a long constituent must be closed: w t is located at a deep level in the tree and will only (probably) share a few ancestors with w t +1 . These configurations are encoded in a more sparse way by a relative scheme, as the n t value shows a large variability and it depends on the depth of the tree in the current time step. We can obtain a compressed representation of these cases by using a top-down absolute scale instead, as any pair of words that share the same m top levels will be equally encoded. The absolute scale becomes however sparse when predicting deep levels. Figure 4 illustrates the strengths and weaknesses of both encodings with an example, and how a dynamically encoded tree helps reduce variability on n t values. In our particular implementation, we will be using the following setup:   | w | : T | w |  L | w | 1 , the relative-scale encoding function, is used by default. | w | : T | w |  L (cid:48)| w | 1 is the secondary linearization function that maps words to labels according to a top-down absolute scale. is used iff: (1) ( w [ t : t +1] ) = ( n (cid:48) t , c (cid:48) t , u (cid:48) t ) with n (cid:48) t  3 , i.e. w t and w t +1 share at most the three top levels, and (2) ( w [ t : t +1] ) = ( n t , c t , u t ) with n t   2 , i.e. w t is at least located two levels deeper in the tree than w t +1 . 3 a b c d e f g h i j k l m Relative: 2 1 1 1 -4 1 1 -2 1 1 1 -3  Absolute: 2 3 4 5 1 2 3 1 2 3 4 1  Dynamic: 2 r 1 r 1 r 1 r 1 a 1 r 1 r 1 a 1 r 1 r 1 r 1 a  Figure 4: A synthetic constituent tree where n t is encoded using a relative scheme, a top-down absolute scale, and an ideal dynamic combination. We showed that labels of the form ( n t , c t , u t )  L are sparse. An intuitive approach is to decompose the label space into three smaller sub-spaces, such that n i  N , c i  C and u i  U . This reduces the output space from potentially | N |  | C |  | U | labels to just | N | + | C | + | U | . We propose to learn this decomposed label space through a multitask learning setup, where each of the subspaces is considered a different task, namely task N , task C and task U . The final loss is now computed as L = L n + L c + L u . We relied on a hard-sharing architecture, as it has been proved to reduce the risk of overfitting the shared parameters (Baxter, 1997). A natural issue that arises is that the prediction of labels from different label sub-spaces could be interdependent to a certain extent, and therefore a hierarchical sharing architecture could also be appropriate. To test this, in preliminary experiments we considered variants of hierarchical sharing architectures. We fed the output of the task U as input to task N and/or task C . Similarly, we tested whether it was beneficial to feed the output of task N into task C , and viceversa. However, all these results did not improve those of the hard-sharing model. In this context, in addition to a generalization mechanism, the shared representation could be also acting as way to keep the model aware of the potential interdependencies that might exist between subtasks. We propose two ways to mitigate error propagation arising from greedy decoding in constituent parsing as sequence tagging: auxiliary tasks and policy gradient fine-tuning. Note that we want to optimize bracketing F-score and speed. For this reason we do not explore approaches that come at a speed cost in testing time, such as beam-search or using conditional random fields (Lafferty et al., 2001) on top of our LSTM . Auxiliary tasks Auxiliary tasks force the model to take into account patterns in the input space that can be useful to solve the main task(s), but that remain ignored due to a number of factors, such as the distribution of the output label space (Rei, 2017). In a similar fashion, we use auxiliary tasks as a way to force the parser to pay attention to aspects beyond those needed for greedy decoding. We propose and evaluate two separate strategies: 1. Predict partial labels n t + k that are k steps from the current time step t . This way we can jointly optimize at each time step a prediction for the pairs ( w t , w t +1 ) , . . . , ( w t + k , w t + k +1 ) . In particular, we will experiment both with previous and upcoming n k 's, setting | k | = 1 . 2. Predict the syntactic distances presented by Shen et al. (2018), which reflect the order a sentence must be split to obtain its constituent tree using a top-down parsing algorithm (Stern et al., 2017). The algorithm was initially defined for binary trees, but its adaptation to n -ary trees is immediate: leaf nodes have a split priority of zero and the ancestors' priority is computed as the maximum priority of their children plus one. In this work, we use this algorithm in a sequence tagging setup: the label assigned to each token corresponds to the syntactic distance of the lowest common ancestor with the next token. This is illustrated in Figure 5. I find your lack of faith disturbing . On the one hand, the encoding of the n t s by Gomez-Rodrguez and Vilares (2018) only needs to know about w t and w t +1 paths to generate the label for the time step t . On the other hand, to compute the syntactic distance of a given non-terminal symbol, we need to compute the syntactic distances of its subtree, providing a more global, but also sparser context. For training, the loss coming from the auxiliary task(s) is weighted by  =0.1, i.e, the final loss is computed as L = L n + L c + L u +  (cid:80) a L a . Policy gradient fine-tuning Policy gradient training methods allow us to fine-tune our models with a tree-level objective, optimizing directly for bracketing F-score. We start off with a converged supervised model as our initial policy. The sequence labeling model can be seen as a functional approximation of the policy  parametrized by  , which at timestep t selects a label l t = ( n t , c t , u t ) 4 given the current state of the model's parameters, s t . The agent's reward, R tree , is then derived from the bracketing F-score. This can be seen as a variant of the REINFORCE algorithm (Williams, 1992) where the policy is updated by gradient ascent in the direction of:   log ( l t | s t ;  ) R tree (1) Baseline and Variance Reduction We use as baseline a copy of a pre-trained model where the parameters are frozen. The reward used to scale the policy gradient can then be seen as an estimate of the advantage of an action l t in state s t over the baseline model. This is equivalent to R tree  B tree , where R tree is the bracketing F-score of a sequence sampled from the current policy and B tree is the the tree-level F-score of the sequence greedily predicted by the baseline. To further reduce the variance, we standardize the gradient estimate   using its running mean and standard deviation for all candidates seen in training so far. In initial experiments without these augmentations, we observed that fine-tuning with vanilla PG often led to a deterioration in performance. To encourage exploration away from the converged supervised model's policy, we add the entropy of the policy to the objective function (Williams and Peng, 1991). Moreover, following Lillicrap et al. (2015), we optionally add noise sampled from a noise process N to the policy. The gradient of our full fine-tuning objective function takes the following form:   ( log ( l t | s t ;  ) + N )( R tree  B tree ) +    H (  ( s t ;  ) + N ) (2) where H is the entropy and  controls the strength of the entropy regularization term. Datasets We use the English Penn Treebank ( PTB ) (Marcus et al., 1993) and the Chinese Penn Treebank ( CTB ) (Xue et al., 2005). For these, we use the same predicted PoS tags as Dyer et al. (2016). We also provide detailed results on the SPMRL treebanks (Seddah et al., 2014), 5 a set of datasets for constituent parsing on morphologically rich languages. For these, we use the predicted PoS tags provided together with the corpora. To the best of our knowledge, we provide the first evaluation on the SPMRL datasets for sequence tagging constituent parsers. Setup We use NCRF pp (Yang and Zhang, 2018), for direct comparison against Gomez-Rodrguez and Vilares (2018). We adopt bracketing F-score instead of label accuracy for model selection and report this performance as our second baseline. After 100 epochs, we select the model that fared best on the development set. We use GloVe embeddings (Pennington et al., 2014) for our English models and zzgiga embeddings (Liu and Zhang, 2017) for the Chinese models, for a more homogeneous comparison against other parsers (Dyer et al., 2016; Liu and Zhang, 2017; Fernandez-Gonzalez and Gomez-Rodrguez, 2018). ELMo (Peters et al., 2018) or BERT (Devlin et al., 2018) could be used to improve the precision, but in this paper we focus on keeping a good speed-accuracy tradeoff. For SPMRL , no pretrained embeddings are used, following Kitaev and Klein (2018a). As a side note, if we wanted to improve the performance on these languages we could rely on the CoNLL 2018 shared task pretrained word embeddings (Zeman et al., 2018) or even the multilingual BERT model 6 . Our models are run on a single CPU 7 (and optionally on a consumer-grade GPU for further comparison) using a batch size of 128 for testing. Additional hyperparameters can be found in Appendix A. 4.1 Results Table 1 contrasts the performance of our models against the baseline on the PTB development set. 5 Except for Arabic, for which we do not have the license. 6 https://github.com/google-research/ bert/blob/master/multilingual.md 7 Intel Core i7-7700 CPU 4.2 GHz Model F-score (+/-) Sents/s Gomez and Vilares (2018) 89.70 109 Our baseline 89.77 (+0.07) 111 + DE 90.22 (+0.52) 111 + MTL 90.38 (+0.68) 130 aux( n t +1 ) 90.41 (+0.71) 130 aux( n t  1 ) 90.57 (+0.87) 130 aux(distances) 90.55 (+0.85) 130 + PG 90.70 (+1.00) 130 Table 1: Results on the PTB dev set, compared against Gomez-Rodrguez and Vilares (2018). To show that the model which employs dynamic encoding is better (+0.52) than the baseline when it comes to closing brackets from long constituents, we compare their F-scores in Figure 6. When we recast the constituent-parsing-as-sequence-tagging problem as multi-task learning, we obtain both a higher bracketing F-score (+0.68) and speed (1.17x faster). Fusing strategies to mitigate issues from greedy decoding also leads to better models (up to +0.87 when adding an auxiliary task 8 and up to +1.00 if we also fine-tune with PG ). Note that including auxiliary tasks and PG come at a time cost in training, but not in testing, which makes them suitable for fast parsing. 8 We observed that adding more than one auxiliary task did not translate into a clear improvement. We therefore chose the auxiliary task that performed the best in the development set. large treebanks, e.g. German, French or Korean, but causes some drops in the smaller ones, e.g. Swedish or Hebrew. Overall, casting the problem as multitask learning and the strategies used to mitigate error propagation lead to improvements. For the experiments on the test sets we select the models that summarize our contributions: the models with dynamic encoding and the multi-task setup, the models including the best auxiliary task, and the models fine-tuned with policy gradient. Tables 3, 4 and 5 compare our parsers against the state of the art on the PTB , CTB and SPMRL test sets. Gomez-Rodrguez and Vilares (2018) also run experiments without character embeddings, to improve speed without suffering from a big drop in performance. For further comparison, we also include them as additional results (shadowed). In a related line, Smith et al. (2018) show that for dependency parsing two out of three embeddings (word, postag and characters) can suffice. The results across the board show that the dynamic encoding has a positive effect on 6 out of 10 treebanks. Casting the constituent-parsing-as-sequence-labeling problem as MTL surpasses the baseline for all tested treebanks (and it leads to better parsing speeds too). Finally, by mitigating issues from greedy decoding we further improve the performance of all models that include dynamic encodings and multi-task learning. On the PTB , our models are both faster and more accurate than existing sequence tagging or sequence-to-sequence models, which already were among the fastest parsers (Gomez-Rodrguez and Vilares, 2018; Vinyals et al., 2015). We also outperform other approaches that were not surpassed by the original sequence tagging models in terms of F-score (Zhu et al., 2013; Fernandez-Gonzalez and Martins, 2015). On the CTB our techniques also have a positive effect. The baseline parses 70 sents/s on the CTB , while the full model processes up to 120. The speed up is expected to be larger than the one obtained for the PTB because the size of the label set for the baseline is bigger, and it is reduced in a greater proportion when the constituent-parsing-as-sequence-labeling problem is cast as MTL . On the SPMRL corpora, we provide the first evaluation of sequence labeling constituent parsers, to verify if these perform well on mor-Model CTB Basque French German Hebrew Hungarian Korean Polish Swedish Our baseline 88.57 87.93 81.09 87.83 89.27 88.85 83.51 92.60 80.11 + DE 88.37 87.91 81.16 88.81 89.03 88.70 83.92 93.35 79.57 + MTL 88.57 89.41 81.70 88.52 92.72 89.73 84.10 93.81 82.83 aux( n t +1 ) 88.73 89.65 81.95 88.64 92.65 89.69 84.09 93.86 82.82 aux( n t  1 ) 88.48 89.47 81.77 88.58 92.53 89.71 84.13 93.87 82.74 aux(distances) 88.51 89.48 82.02 88.68 92.66 89.80 84.20 93.83 83.12 + PG 89.01 89.73 82.13 88.80 92.66 89.86 84.45 93.93 83.15 Table 2: Results on the CTB and SPMRL dev sets Model Sents/s Hardware F-score Vinyals et al. (2015) 120 Many CPU 88.30 Coavoux and Crabbe (2016) 168 1 CPU 88.60 Fernandez and Martins (2018) 41 1 CPU 90.20 Zhu et al. (2013) 90 1 CPU 90.40 Dyer et al. (2016) 17 1 CPU 91.20 Stern et al. (2017) 76 16 CPU 91.77 Shen et al. (2018) 111 1 GPU 91.80 Kitaev and Klein (2018a) 213 2 GPU 93.55 (single model) Kitaev and Klein (2018a) 71 2 GPU 95.13 (with ELM o) Kitaev and Klein (2018b) -95.77 (ensemble and BERT ) Gomez and Vilares (2018) 115 1 CPU 90.00 Our baseline 115 1 CPU 90.06 + DE 115 1 CPU 90.19 + MTL 132 1 CPU 90.36 + best aux 132 1 CPU 90.59 + PG 132 1 CPU 90.60 + PG 942 1 GPU 90.60 + PG (no char emb) 149 1 CPU 90.50 + PG (no char emb) 1267 1 GPU 90.50 Table 3: Comparison on the PTB test set. phologically rich languages. We then evaluated whether the proposed techniques can generalize on heterogeneous settings. The tendency observed for the original tagging models by Gomez-Rodrguez and Vilares (2018) is similar to the one Model F-score Zhu et al. (2013) 83.2 Dyer et al. (2016) 84.6 Liu and Zhang (2017) 86.1 Shen et al. (2018) 86.5 Fern  andez and G  omez-Rodr  guez (2018) 86.8 G  omez and Vilares (2018) 84.1 Our baseline 83.90 + DE 83.98 + MTL 84.24 +best aux 85.01 + PG 85.61 + PG (no char emb) 83.93 Table 4: Comparison on the CTB test set for the PTB and CTB : they improve other fast parsers, e.g. Coavoux and Crabbe (2016), in 5 out of 8 treebanks and Fernandez-Gonzalez and Martins (2015) in 7 out of 8, but their performance is below more powerful models. When incorporating the techniques presented in this work, we outperform the original sequence tagging models on all datasets. We outperform the current best model for Basque, Hebrew and Polish (Kitaev and Klein, 2018a) and for Swedish (Bjorkelund et al., 2014), which corresponds to the four smallest treebanks among the SPMRL datasets. This indicates that even if sequence tagging models are conceptually simple and fast, they can be very suitable when little training data is available. This is also of special interest in terms of research for low-resource languages. Again, casting the problem as MTL reduces the parsing time for all tested treebanks, as reflected in Table 6. Finally, for treebanks such as French, designing methods to handle multi-word expressions could lead to better results, getting closer to other parsers (Coavoux and Crabbe, 2017). We have explored faster and more precise sequence tagging models for constituent parsing. We proposed a multitask-learning architecture that employs dynamic encodings, auxiliary tasks, and policy gradient fine-tuning. We performed experiments on the English and Chinese Penn Treebanks, and also on the SPMRL datasets. Our models improve current sequence tagging parsers on all treebanks, both in terms of performance and speed. We also report state-of-the-art results for the Basque, Hebrew, Polish, and Swedish datasets. The methods presented in this work are specifi-cally designed for constituent parsing. However, it seems natural to apply some of these to other NLP tagging tasks, e.g. using multi-task learning to predict sub-level morphological information for morphologically-rich part-of-speech tagging. DV has received support from the European Research Council (ERC), under the European Union's Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150), from the TELEPARES-UDC project (FFI2014-51978-C2-2-R) and the ANSWER-ASAP project (TIN2017-85160-C2-1-R) from MINECO, and from Xunta de Galicia (ED431B 2017/01). MA and AS are funded by a Google Focused Research Award.
abstain method abstain abstain abstain method method method result objective abstain abstain abstain result result abstain abstain abstain abstain abstain abstain objective abstain objective method abstain result method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain method abstain method method other method method method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain method abstain abstain abstain abstain abstain method abstain abstain method abstain other other method abstain abstain method abstain method abstain method method method abstain abstain method method method abstain method method abstain method method abstain abstain method abstain abstain abstain method abstain method method abstain abstain method abstain abstain method method method method method abstain abstain result abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain other result result abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain result abstain abstain abstain other abstain abstain result result result abstain abstain abstain abstain objective objective method result result method abstain other other