We introduce a transductive model for parsing into Universal Decompositional Semantics (UDS) representations, which jointly learns to map natural language utterances into UDS graph structures and annotate the graph with decompositional semantic attribute scores. We also introduce a strong pipeline model for parsing into the UDS graph structure, and show that our transductive parser performs comparably while additionally performing attribute prediction. By analyzing the attribute prediction errors, we find the model captures natural relationships between attribute groups. A structured account of compositional meaning has been longstanding goal for both natural language understanding and computational semantics. To this end, a number of efforts have focused on encoding semantic relationships and attributes in a semantic graphe.g. Abstract Meaning Representation (AMR; Banarescu et al., 2013), Universal Conceptual Cognitive Annotation (UCCA; Abend and Rappoport, 2013), and Semantic Dependency Parsing (SDP; Oepen et al., 2014, 2015, 2016). In these formalisms, semantic information is typically encoded discretely, using nominal category labels for nodes and edges. This categorical encoding can make such formalisms brittle when presented with non-prototypical instances, and leads to challenges in coping with changing label ontologies and new datasets (White et al., 2019). Furthermore, they are difficult to annotate, often requiring trained linguists and large annotation manuals. The Decompositional Semantics framework presents an alternative to categorical formalisms that encodes semantic information in a feature-based schemeusing continuous scales rather than categorical labels. Starting with a feature-based semantic role representation rooted in Dowty 1991's (1991) proto-role theory (Reisinger et al., 2015; White et al., 2016), this framework has expanded to cover a wide variety of phenomena: event factuality (Rudinger et al., 2018b), genericity (Govindarajan et al., 2019), entity types (White et al., 2016), and temporal relations (Vashishtha et al., 2019). While this rich array of annotation types has been separately modeled, no system yet exists for its joint prediction, which has only recently been made feasible by the introduction of Universal Decompositional Semantics v1.0 (UDS1.0). Presented by White et al. (2019), UDS1.0 normalizes all of these annotations, and incorporates them as nodeand edge-level attributes in a single semantic graph whose structure is deterministically extracted from Universal Dependencies (UD; Nivre et al., 2015) syntactic parses via the PredPatt tool (White et al., 2016; Zhang et al., 2017). 1 An example graph can be seen in Fig. 1. We present the first joint UDS parser, which learns to extract both graph structures and attributes from natural language input. This parser is a sequence-to-graph transductive model which takes as input a sentence and outputs a UDS graph complete with nodeand edge-level annotations. In contrast to the traditional semantic parsing paradigm, which shares its roots with syntactic parsing and rests on the assumption that the nodes in the graph correspond to tokens in the input i.e. the graph is lexicalized the parsing-as-transduction paradigm treats parsing as a sequence-to-graph problem. Rather than generating one sequence conditional on another sequence (sequence-to-sequence), we generate the nodes in a graph conditional on an input sequence, dynamically adding their edges during generation. As in sequence-to-sequence modeling, the supports of the input and output distributionsi.e. the input and output 1 Available at http://decomp.io . vocabulariesare not constrained to be identical. This has two benefits: first, post-hoc methods of obtaining alignments between input sequences and graphscommon especially in AMR parsing are no longer required; and second, we are able to produce semantic graphs from arbitrary input vocabulariesallowing for future extensions to cross-lingual parsing (Zhang et al., 2018). The parsing-as-transduction paradigm thus lends itself perfectly to UDS parsing, since the UDS protocol allows non-lexicalized (as well as cross-lingual) graphs, and these graphs may have nodes with multiple parentsi.e. re-entrant nodeswhich pose problems for traditional tree-based methods but are handled natively by the transductive paradigm. We compare our end-to-end transductive parser against a strong pipeline system, finding that the parser slightly outperforms the pipeline while additionally learning to produce decompositional attribute scores. Our results are reflected in the UDS1.0 leaderboard at http://decomp.io/ leaderboards/ . Datasets Reisinger et al. (2015) introduce the Decompositional Semantics framework in the context of a corpus-based verification of Dowty's seminal proto-role theory of semantic roles. This work was substantially expanded by White et al. (2016), who annotate for semantic proto-roles (SPR), word-sense, and temporal properties on top of semantic graphs extracted from English Web Treebank (EWT; Bies et al., 2012) UD parses using PredPatt (White et al., 2016; Zhang et al., 2017). White et al.'s EWT annotations are modeled by Teichert et al. (2017), who present a CRF-based multi-label classifier for proto-role labelling, and Rudinger et al. (2018a), who make use of an event-driven neural model. More recently, the annotation coverage for the same EWT data was expanded by Vashishtha et al. (2019) who annotate and model fine-grained temporal distinctions, and Govindarajan et al. (2019), who add annotations and models for genericityi.e. the degree of generality of events and entities in linguistic expressions. All of these efforts coalesce in White et al. (2019), which presents the first unified Decompositional Semantics-aligned datasetUniversal Decompositional Semantics v1.0 (UDS1.0) containing all properties annotated on top of EWT parses with standardized train, validation, and testing splits and a native reader and query interface. Parsing In most work on decompositional semantics, models are tasked with learning to predict attribute values, but not the structure of the graph. Zhang et al. (2018) develop the first model for performing both graph parsing and UDS attribute prediction in a cross-lingual setting, where Chinese input sentences were transduced into UDS graphs derived from UD parses of the input's English translation. This represents the first application of the parsing-as-transduction paradigm to a subset of UDS data as well as the introduction of a novel graph evaluation metric, S which we describe in further detail in Section 5. In contrast to the end-to-end approach presented here, Zhang et al. take a pipeline approach to parsing. Andreas et al. (2013) recast semantic parsing in a tree formalism as a sequence-to-sequence problem. Parsing-as-transduction, which extends this approach to directed acyclic graphs, has proven to be applicable in a variety of settings: Zhang et al. (2019a) use it to achieve state-of-the-art results in AMR parsing. These results are improved upon and shown to generalize to two other semantic formalisms (UCCA and SDP) by Zhang et al. (2019b), which set new state-of-the-art benchmarks for AMR and UCCA. The former result was subsequently surpassed by Cai and Lam (2020), which applies a similar transductive approach, while the latter was surpassed by Jiang et al. (2019). Having both been subjects of SemEval tasks (May, 2016; May and Priyadarshi, 2017; Oepen et al., 2019; Hershcovich et al., 2019), there are a number of contrasting methods for both AMR and UCCA parsing. These include transition-based parsing system for AMR (Wang et al., 2018; Goodman et al., 2016; Damonte et al., 2017; Ballesteros and Al-Onaizan, 2017) and for UCCA (Her-shcovich et al., 2017). In a similar vein to Zhang et al. (2019b), Hershcovich et al. (2018a) convert multiple formalisms into a unified formalism and use multitask learning for improved UCCA parsing; however, the latter does so at a loss to performance on the other formalisms, while Zhang et al. achieve state-of-the-art results in AMR and UCCA simultaneously. UCCA has also been shown to transfer to syntactic parsing: by converting UD parse trees into a format resembling UCCA, Hershcovich et al. (2018b) are able to apply a UCCA parser to both standard UD parses as well as enhanced UD parses, which contain re-entrant nodes. The UDS1.0 dataset is built on top of the UD-EWT data with three layers of annotations: UD parses, PredPatt graph structure, and decompositional semantic annotations on the edge and node level. In addition to specifying the syntactic head and head relation of every token in the input, UD parses include lexical features, such as word form, word lemma, and part-of-speech (POS) tag. This forms the syntactic graph , which is lexicalized (each token is tied to a node in the graph). From these pieces of information, PredPatt outputs a set of predicates and their arguments. Each predicate and argument is tied via an instance edge to a particular node in the syntactic graph. Because both predicates and arguments can consist of multi-word spans, there can be multiple instance edges leaving a semantic node. The semantic graph contains edges between predicates and arguments; in the case of clausal embedding, there can also be argument-argument edges. UDS1.0 asked Hiller Bush (1) name leaders the of Che. includes performative speaker/author and addressee nodes, which model discourse properties of the sentence. These nodes are structural place-holders for future discourse-level annotations; as these properties have not yet been annotated, we have opted to remove them from the graphs. 2 The crowdsourced decompositional annotations tied to the semantic subgraph can be divided into node-level annotations and edge-level annotations. On the node level, annotations were collected for factuality, genericity, time, and entity type. Edge-level annotations are in the space of semantic proto-roles, which are designed to provide a nuanced higher-dimensional substrate for notions of agency and patienthood. These are summarized in Table 1, where purple indicates a high attribute score, while orange indicates a low score. For further details on attribute types and data annotation, see White et al. (2019) and the references therein. Arborescence Recall that the lowest level of the UDS graph (Fig. 1) is a syntactic dependency parse. Modeling this level is out of scope for this work, as we are interesting in modeling the semantic structure and attributes. In order to train a parsing-as-transduction model, an arborescencea hierarchical tree structure which has only edge and node annotationsis required. From the full UDS graph, we construct the arborescence by: (a) Assigning each semantic node a lexical label; this label is taken from the syntactic head that the semantic node dominates. The only exception to this is in the case of embedded clauses, where an argument node dominates an embedded predicate. Here, we follow PredPatt, assigning the label SOMETHING to the embedded argument (c.f. Fig. 2). 2 Since these placeholder nodes are currently added deterministically, recovering them is also a deterministic operation. (b) Retaining all edges between semantic nodes as argument edges, duplicating nodes in cases of re-entrancy (e.g.  Bush (1)  in Fig. 2). (c) Converting the deep syntactic structure into a shallow representation, where we introduce non-head edges from the syntactic head (attached to a semantic node) to each node it dominates, and remove all other syntax-semantics edges. This effectively linearizes the yield of each semantic node (see Fig. 2). Our model is based on the transductive broad-coverage parsing model presented in Zhang et al. (2019b), which can be consulted for further details on the encoder, decoder, and pointer-generator modules. The original parser is composed of six major modules: the encoder, the decoder embedding module, the target node module, the target label module, the head module, and the relation module. In this work we introduce two new modules: the node attribute module and the edge attribute module, as well a loss function for attributes. Encoder The encoder module takes a concatenation of multiple input features: GloVe token embeddings (Pennington et al., 2014), POS tag embeddings, character CNN embeddings, and BERT (Devlin et al., 2019) contextual embeddings (mean-pooled over subwords). These representations are passed through a stacked bidirectional LSTM encoder, which has the following definition: s lt = (cid:20)  s lt  s lt (cid:21) = (cid:34)  LSTM ( s l  1 t , s tt  1 )  LSTM ( s l  1 t , s tt +1 ) (cid:35) where arrows denote the LSTM direction, t denotes the timestep, and l denotes the layer of the stack. Decoder embedding module In order to generate new semantic nodes and relationships, a method of embedding categorical semantic information is required. More formally, a semantic relation is given by a tuple (cid:104) u i , d ui , r i , v i , d vi (cid:105) , where u i denotes the head token of index i and v i denotes the token at index i . Note that these tokens are the labels of nodes in the arborescence (see Fig 2.) d ui and d vi are the indices of u i and v i , while r i is the relationship type between v i and u i . The decoder embedding module embeds these categorical variables into real space, producing a tuple of vectors (cid:104) u i , d ui , r i , v i , d vi (cid:105) . For node labels u i and v i , we take the concatenation of GloVe and CharCNN features. r i , d vi and d ui are randomly initialized. Target Node Module From the continuous embedding of a semantic relation (cid:104) u i , d ui , r i , v i , d vi (cid:105) we want to obtain a latent node representation z i . We initialize the hidden states of the 0 th layer and the hidden states of the 0 th state in each layer to h 0i = [ v i ; d vi ] h l0 = [  s l 1 ;  s ln ] respectively. Further, let c i be a context vector over encoder states s l 1: n , defined as a (enc) i = softmax (cid:0) MLP (enc) ([ h li ; s l 1: n ]) (cid:1) c i = a Ti s l 1: n Let h li and z i be defined as follows: z i = MLP (relation) ([ h li ; c i ; r i ; u i ; d ui ]) h li = LSTM ( h l  1 i , h li  1 ) where z i can be thought as a representation of node i in the graph, conditioned on previous nodes (via h li as well as the input text via c i , the graph token (via u i and d ui ) and the relation type (via r i ). Using this representation z i , Zhang et al. (2019b) introduce an extended pointer-generator network (See et al., 2017) which computes the distribution over the next node label v i +1 : [ p gen , p enc ,p dec ] = softmax (cid:0) MLP (switch) ( z i ) (cid:1) a dec i = softmax (cid:0) MLP dec ([ z 1: i ]) (cid:1) p (vocab) i = softmax (cid:0) MLP ( vocab ) ( z i ) (cid:1) P ( v i +1 ) = p gen p (vocab) i  p enc a (enc) i  p dec a (dec) i From this last equation, we have that the generation of a new node is decomposed into three options: (1) generate a new node from a vocabulary of node labels, (2) copy a node label directly from the input sequence (lexicalization), or (3) copy a node label from a previously generated node (re-entrancy). Parsing modules To obtain a parse from the node states h 1: n , a head node and relation type must be assigned to each node 1 : n . In order to assign a head node, we instantiate two multilayer perceptrons (MLPs): MLP (start) and MLP (end) , where (start) denotes the starting node of the edge and (end) denotes its target. Using these MLPs, for node i + 1 we obtain h (start) i +1 = MLP (start) ( h li +1 ) h (end) 1: i = MLP (end) ( h l 1: i ) P ( u i +1 ) = softmax (cid:0) BIAFFINE ( h (start) i +1 , h (end) 1: i ) (cid:1) The next relationship r i +1 is computed in a similar fashion, also using two MLPs: h (rel-src) i +1 = MLP (rel-src) ( h lj ) h (rel-tgt) i +1 = MLP (rel-tgt) ( h li +1 ) P ( r i +1 ) = softmax (cid:0) BILINEAR ( h (rel-src) i +1 , h (rel-tgt) i +1 ) (cid:1) where j is the index of the head assigned to the node indexed by i + 1 . 3 3 BIAFFINE is defined in Dozat and Manning (2016). BILINEAR ( x 1 , x 2 ) = x 1 Ax 2 + b where A and b are learned parameters. Node attribute module As noted in previous UDS projects, an important step in decompositional attribute annotation is determining whether a property applies in a given context. For example, factuality typically applies only to predicate nodes. Since all nodes (predicate and argument) are treated identically w.r.t. their semantic relations z i , this work introduces a two-fold node attribute model, which predicts whether a property j applies to a node i via a binary mask  ji as well as its value  ji . This module defines  ji and  ji as follows: P (  ji ) = sigmoid (cid:0) MLP (node-mask) ( z i ) (cid:1)  ji = MLP (node-attr) ( z i ) Edge attribute module As in the case of node attributes, edge attributes do not apply in all cases. Therefore, a similar bifurcation strategy is pursued with edge attribute prediction: we predict a binary attribute mask  js,e for attribute j on edge s  e as well as an attribute value  js,e . These are given by: m (mask) s,e = BILINEAR (mask) ( h ls , h le ) m (attr) s,e = BILINEAR (attr) ( h ls , h le ) P (  js,e ) = sigmoid (cid:0) MLP (edge-mask) ( m (mask) s,e ) (cid:1)  js,e = MLP (edge-attr) ( m (attr) s,e ) Training The nodes in the graph are linearized in a pre-order traversal over the arborescence, which ensures that at prediction time, we have seen the potential antecendent of a node for target-side copying (e.g. Bush (1) in Fig. 2), determining the order of semantic nodes in the graph. The syntactic children of these nodes are presented in the order they appear in the text. The loss functions for the node, head, and relation prediction modules are cross-entropy loss, while for the masks  and  binary cross-entropy loss is used, since each position in the mask is a separate classification decision. The loss function used for K attributes  1: K on N nodes/edges is given by:  ( x ) = (cid:40) 0 if x  0 1 otherwise LMSE ( ,   ) = 1 NKN (cid:88) i =1 K (cid:88) j =1 c ji (  ji   j  i ) 2 LBCE ( ,   ) = 1 NKN (cid:88) i =1 K (cid:88) j =1 (cid:18)  (  j  i ) log(  (  ji )) + (cid:0) 1   (  j  i ) (cid:1) log (cid:0) 1   (  ji ) (cid:1)(cid:19) L ( ,   ) =  2  LMSE ( ,   )  LBCE ( ,   ) LMSE ( ,   ) + LBCE ( ,   ) where  is a scaling factor, c ji is the annotator confidence for annotation j on token i ,  is the set of predicted attributes, and   is the set of true attributes. Note that inclusion of the confidence mask c ji means the model only incurs loss on attributes annotated for a given node, since c ji = 0 when an annotation is missing (i.e. no MSE loss is incurred for attributes which do not apply to a node or edge); in the binary experimental setting, we replace c ji with  ( c ji ) , removing the weighting but still masking out loss on un-annotated nodes. Also note than in the case of edges, the form of the loss is identical, but  is replaced by  , and  by  . This loss encourages the predicted attribute  ji value to be close in value to the true value  j  i via the mean-squared error criterion while concomitantly encouraging the predicted and reference values to share a sign via the thresholded cross-entropy criterion. Both node and edge attribute models are trained to predict attribute values independently, and that parameters are shared across attributes. This is central to our analysis in  7. Following Zhang et al. (2019b) we train the structural parsing modules with coverage loss (See et al., 2017). All models were trained to convergence using the Adam optimizer (Kingma and Ba, 2014) with a learning rate of 0 . 001 . Pipeline Model Recall from Section 3 that the semantic graph structure in UDS graphs is deterministically generated from PredPatt, which takes as input a UD parse and outputs a semantic graph structure. This leads to a strong pipeline model for the graph structure alone: running a high-performing UD parserthe Stanford UD parser (Chen and Manning, 2014)and passing its output through PredPatt to create a structure. 4 For this baseline, 4 This structure is missing the core decompositional attributes but has both predicate and argument nodes. Additionally, the pipeline model fails to capture nominal heads of copular predicates (e.g. Jo is a doctor ), which are not returned by PredPatt but are added to the dataset as a preprocessing step in the genericity annotation task. the only source of error is the UD parsing model, which for English performs very highly. S Metric For evaluating the quality of output graph structures, Smatch (Cai and Knight, 2013), a hill-climbing approach to approximating the optimal matching between variables in two graphs, is commonly used. While Smatch can match categorial variables such as those found in meaning representations like AMR, it lacks a matching function for continuous variables such as decompositional attributes. To remedy this, Zhang et al. (2018) introduce the S metric, an extension to Smatch that allows for attribute matching. Using hill-climbing, we are able to match instance and attribute nodes and edges; instance nodes are matched via string match, while attribute similarity is given by 1  (cid:16)  i   j  (cid:17) 2 where  = 6 is the maximum possible difference between attributes, which are bounded on [  3 , 3] . 5 6 Results Table 5 shows the Pearson's correlation coefficient (  ) and the F1 score computed on binarized responses for each node and edge attribute under the oracle decoding setting, where a gold graph structure is provided to the model. An asterisk denotes that p < 0 . 05 , where p is determined by a Student's t-test. F1 scores are obtained by binarizing continuous attribute predictions into positive and negative, following from the original UDS motivation found in Dowty (1991), where binary proto-role features were introduced. The binarization threshold was tuned per attribute on the validation set. The baseline column in Table 5 shows the binarized F1 score for the baseline attribute model, given by predicting the median attribute value for each attribute type at each position. Pearson's  is undefined for this approach, as the variance of the predicted distribution is 0. The thresholds were similarly tuned on validation data for this baseline. Table 2 shows S metric (c.f.  5) precision, recall, and F1 score as computed on full arborescences 5 This function was found to produce more matches on UDS1.0 than the e  MAE function used by Zhang et al. (2018). with both semantics and syntax nodes. Our parser slightly outperforms the pipeline, with higher performance in the binary setting, where we exclude annotator confidence from the loss. Table 3 shows precision, recall, and F1 score on semantics nodes alone. The first parser setting (syn-tax) reflects a parsing model trained on full graphs, and evaluated only on the semantic subgraphs of the produced graphs. The second parser (seman-tics) is directly trained on semantic subgraphs, with no syntactic nodes in the training graphs. The full parser performs comparably to the pipeline, while the parser trained specifically on semantics-only graphs outperforms the pipeline. However, the mean attribute  of the syntactic parser ( 0 . 3433 ) exceeded that of the semantics-only parser ( 0 . 3151 ). Table 4 gives the S metric results on full graphs predicted by the model, including attribute matching. The pipeline model is unable to perform this task because it predicts structure alone, without attributes. We see that training the parser with shared MLP and BILINEAR modules (i.e. MLP (mask) = MLP (attr) and BILINEAR (mask) = BILINEAR (attr) ) for both the attribute mask and attribute value heavily degrades the performance, while removing annotator confidence increases it slightly. Table 2 suggests that the structural quality of the parses obtained by the parsing model presented here is slightly superior to that of pipeline model's parses, with Table 3 indicating that the semantic component of the graph can be parsed significantly more accurately by our model. Taken together with Table 5, we can conclude that the model is able to learn to jointly predict the graph structure and attributes. This is further reinforced by Table 4. Note that the numbers reported in Tables 2 and 4 are not directly comparable, as the scores in Table 4 Method P R F1 Shared 79.52 32.48 46.12 Separate 83.46 82.27 82.86 Separate (binary) 84.19 84.19 84.19 Table 4: Test set precision, recall, and F1 computed via S score with attributes (syntactic nodes included) Property Pearson's  F1 F1 (model) (baseline) (model) nod e l e v e l      factuality-factual 0.6479* 75.15 84.46 g e n e r i c it y   arg-abstract 0.3392* 40.04 48.05 arg-kind 0.2145* 67.61 67.54 arg-particular 0.3347* 83.10 84.62 pred-dynamic 0.2469* 72.49 71.19 pred-hypothetical 0.3442* 44.16 50.21 pred-particular 0.1887* 77.47 78.16 ti m e   dur-centuries 0.1336* 10.14 12.30 dur-days 0.1802* 68.72 68.21 dur-decades 0.2383* 29.89 34.19 dur-forever 0.2524* 37.93 38.58 dur-hours 0.2227* 73.66 73.61 dur-instant 0.1761* 55.98 51.90 dur-minutes 0.3409* 86.28 87.05 dur-months 0.3204* 63.25 64.42 dur-seconds 0.2751* 65.33 64.75 dur-weeks 0.2475* 54.02 55.41 dur-years 0.4239* 65.03 66.19 w o r d s e n s e    supersense-noun.Tops 0.4660* 7.34 40.00 supersense-noun.act 0.6007* 27.37 56.39 supersense-noun.animal 0.3773* 5.60 25.64 supersense-noun.artifact 0.5617* 23.12 52.79 supersense-noun.attribute 0.4505* 10.81 29.27 supersense-noun.body 0.4543* 1.53 42.86 supersense-noun.cognition 0.5692* 21.17 50.56 supersense-noun.communication 0.6182* 30.60 62.12 supersense-noun.event 0.4233* 5.80 33.61 supersense-noun.feeling 0.2404* 2.74 5.45 supersense-noun.food 0.6773* 7.15 67.72 supersense-noun.group 0.5650* 15.57 55.22 supersense-noun.location 0.5118* 7.81 55.64 supersense-noun.motive 0.3447* 0.62 50.00 supersense-noun.object 0.2276* 2.04 19.05 supersense-noun.person 0.6091* 15.74 61.25 supersense-noun.phenomenon 0.2955* 2.04 8.85 supersense-noun.plant 0.0358 0.21 13.33 supersense-noun.possession 0.5247* 6.67 47.62 supersense-noun.process 0.1292* 1.13 3.96 supersense-noun.quantity 0.4403* 4.92 36.11 supersense-noun.relation 0.2089* 2.34 11.94 supersense-noun.shape 0.0659* 0.31 1.55 supersense-noun.state 0.4877* 11.36 36.17 supersense-noun.substance 0.2411* 1.43 3.64 supersense-noun.time 0.5175* 10.99 51.43 e dg e l e v e l   p r o t o r o l e s   awareness 0.6715* 68.20 81.99 change-of-location 0.1061* 38.98 36.90 change-of-possession 0.0452 14.93 20.00 change-of-state 0.0448 42.59 37.21 change-of-state-continuous 0.0793 31.47 27.69 existed-after 0.3910* 93.33 95.58 existed-before 0.4802* 91.60 92.31 existed-during 0.3247* 98.31 98.61 instigation 0.3820* 74.48 76.77 partitive 0.0213 31.91 34.64 sentient 0.6494* 64.67 82.81 volition 0.5501* 63.79 79.86 was-for-benefit 0.2389* 59.87 62.11 was-used 0.1608* 86.64 89.00 macro-average 0.3433 37.20 50.66 Table 5: Pearson's  , baseline F1, and model F1 for each UDS attribute given gold test-set graph structures. Table 3 shows that a parser trained on semantic subgraphs better recovers the subgraphs than a parser trained on full graphs whose outputs are postprocessed to remove syntactic nodes. However, the fact that the parser trained on full graphs achieves a higher Pearson's  score indicates that the inclusion of syntactic nodes may provide additional information for predicting UDS attributes. In examining instances with an S score below 50, we observe two trends: the input sentences are often ungrammatical, and for 63.82% (on the validation set) the model predicts no output nodes. While the pipeline system does well on modeling semantic graph structure, it is by its definition unable to perform attribute parsing. In contrast, the results presented in Tables 4 and 5 show that the parser can jointly learn to produce semantic graphs and annotate them with attributes. Finally, we find that while weighting the loss with the confidence scores has a small benefit in the semantics-only setting, it hurts overall attribute and structure prediction performance. This may be due to the relatively small size of the UDS dataset, which makes a strategy that is effectively weakening the loss signal at training time less effective. 6 Figs. 3a-3c show the correlational strength coefficient between the true and predicted attributes under a forced decode of the graph structure. It is defined over property types indices j , k with predicted attribute values  ji and true values  j  i as:  ( j, k ) = tanh (cid:0) 1  | corr (  j   j  ,  k   k  ) | | corr (  j  ,  k  ) | (cid:1) where corr (  j  ,  k  ) is Pearson's correlation coefficient. Further details are given in Appendix A.  ( i, j ) reflects how well the model captures the strength of the correlations (either positive or negative) between two attribute types in the dataset: a positive value indicates that the model captures the correlation to some extent, with values closer to 1 implying better performance; a value of 0 indicates that the model does not capture the correlation at all, or that no significant interaction was present; a negative value indicates that the model makes systematic mistakes while predicting the two variables, e.g. when the model under-predicts the value of property i , it also under-predicts property j 's value. A Bonferroni-corrected non-parametric bootstrap test ( 1000 replicants) was used for significance testing, with failing pairs being said to not be reliably different from 0 correlation. (a)  between argument-node attribute pairs. Subset of wordsenses used for readability. of argument-node attributes, with most  values close to -1. However, we do see positive correlations between some of the genericity annotations, Sentences Property (A) Ours (A) (B) Ours (B) (C) Ours (C) (A) She was untrained and, awareness 3 3.04 1 3.69 5 3.68 in one botched job, killed a client. as well as between genericity-arg-abstract , which rates how conceptually abstract an argument is, and the cognition wordsense, which applies to abstract terms such as doubts and thoughts. In Fig. 3b, we again observe several negative  values; however, some positive correlations can be seen between certain time properties, such as duration-days , duration-weeks , and duration-months , as well as more strongly positive  's between certain genericity annotations. The positive  between factuality and genericity-hypothetical indicates the model has captured the commonalities between predicates with these annotations. In contrast to the node attributes, Fig. 3c shows stronger results for edge attribute prediction, with all significant  's being positive, and related attributes falling into clusters (e.g. volition , awareness , sentience , or the existed attributes) Qualitative examples Table 6 lists three sentences from Reisinger et al. (2015) along with a relevant subset of their original SPR properties and values; the scale in Reisinger et al. was ordinal from 1-5, with 1 corresponding to very unlikely, 5 to very likely, and 3 to neutral. Our model's predictions for the same sentences and properties are given as well, mapped onto [1 , 5] . We first note that the structural component of the model is suffi-ciently strong that the correct predicate-argument edges were extracted during parsing, allowing for a direct comparison between the annotations by Reisinger et al. and the parser's predictions. We see that while for sentence (C), the model captures at least the correct direction of the protorole annotations, it overgeneralizes these results to (B), where a more nuanced analysis is required. For (A), we see that on most attributes the model captures the desired binary direction of the inferences, but that it fails on sentience . Overall, the model's predictions are weaker than the desired output, even when the prediction is on the correct side of the midpoint, 3. This might help explain the disparity between Pearson and F1 scores in Table 5, and represents a direction for future work. Note that to obtain attributes for (A) and (B), the threshold for the masks  was dropped; ideally, this would not be required. The scalar valued, multi-attribute nature of UDS provides for a distinct structured prediction problem as compared to other existing representations. We have demonstrated how a transductive parsing paradigm that has achieved state-of-the-art results on other representations can be adapted to UDS1.0 structures and attributes, and have provided procedures for analysis, with the fine-grained nature of UDS allowing for investigating novel correlations and aspects of meaning. While UDS structures and various attribute types have been modeled separately (Vashishtha et al., 2019; Govindarajan et al., 2019; White et al., 2016; Rudinger et al., 2018a,b; Zhang et al., 2018), this work represents the first time all of these attributes and structures have been modeled jointly, and establishes a baseline for future efforts on UDS1.0. We envision future efforts exploring the interactions between improving the underlying graph-structure prediction and ever-better correlations to human judgements on individual properties. This work was supported by NSF Awards #1749025 and #1763705, DARPA LORELEI and AIDA, and IARPA BETTER. We thank the anonymous reviewers for their constructive feedback. The views and conclusions expressed herein are those of the authors and should not be interpreted as representing official policies or endorsements of DARPA, IARPA, or the U.S. Government.
abstain result result abstain abstain abstain abstain abstain abstain other abstain abstain abstain objective abstain abstain method other abstain abstain abstain result other other method other other other other other other objective other other other other other other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain method abstain abstain abstain other abstain abstain abstain abstain method abstain abstain method other other abstain abstain other abstain abstain abstain abstain method abstain method other abstain abstain other method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain other other abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective other other other