Abstract Meaning Representation (AMR) annotations are often assumed to closely mirror dependency syntax, but AMR explicitly does not require this, and the assumption has never been tested. To test it, we devise an expressive framework to align AMR graphs to dependency graphs, which we use to annotate 200 AMRs. Our annotation explains how 97% of AMR edges are evoked by words or syntax. Previously existing AMR alignment frameworks did not allow for mapping AMR onto syntax, and as a consequence they explained at most 23%. While we find that there are indeed many cases where AMR annotations closely mirror syntax, there are also pervasive differences. We use our annotations to test a baseline AMR-to-syntax aligner, find-ing that this task is more difficult than AMR-to-string alignment; and to pinpoint errors in an AMR parser. We make our data and code freely available for further research on AMR parsing and generation, and the relationship of AMR to syntax. Abstract Meaning Representation (AMR; Banarescu et al., 2013) is a popular framework for annotating whole sentence meaning. An AMR annotation is a directed, usually acyclic graph in which nodes represent entities and events, and edges represent relations between them, as on the right in figure 1. 1 AMR annotations include no explicit mapping between elements of an AMR and the corresponding elements of the sentence that evoke them, and this presents a challenge to developers of machine learning systems that parse sentences to AMR or generate sentences from AMR, since they must 1 For clarity of presentation, we have constructed the sentences and AMRs shown in figuresexcept for figure 3, which is a simplified version of a sentence in the corpus. first infer this mapping in the training data (e.g. Flanigan et al., 2014; Wang et al., 2015; Artzi et al., 2015; Flanigan et al., 2016; Pourdamghani et al., 2016; Misra and Artzi, 2016; Damonte et al., 2017; Peng et al., 2017, inter alia). 2 This AMR alignment problem was first formalized by Flanigan et al. (2014), who mapped AMR nodes or connected subgraphs to words or sequences of words under the assumption of a one-to-one mappingwe call this JAMR alignment . Pourdamghani et al. (2014) then re-formalized it so that any AMR node or edge can map to any word without a one-to-one assumptionwe call this ISI alignment . In ISI alignments, edges often align to syntactic function words: for example, :location aligns to in in figure 1. So edge alignments allow ISI to explain more of the AMR structure than JAMR, but in a limited way: only 23% of AMR edges are aligned in the ISI corpus. This may be be-2 Some recent neural AMR sytems require minimal or no explicit alignments (Konstas et al., 2017; van Noord and Bos, 2017). But they implicitly learn them in the form of soft attention, and we believe that a clearer understanding of alignment will benefit modeling and error analysis even in these systems. cause edges are often evoked by syntactic structure rather than words: for instance, the :ARG1 edge in figure 1 is evoked by the fact that cat is the subject of lies and not by any particular word. Although it seems sensible to assume that all of the nodes and edges of an AMR are evoked by the words and syntax of a sentence, the existing alignment schemes do not allow for expressing that relationship. We therefore propose a framework expressive enough to align AMR to syntax (2) and use it to align a corpus of 200 AMRs to dependency parses. We analyse our corpus and show that the addition of syntactic alignments allows us account for 97% of the AMR content. Syntactic-semantic mappings are often assumed by AMR parsing models (e.g. Wang et al., 2015; Artzi et al., 2015; Damonte et al., 2017), which is understandable since these mappings are well-studied in linguistic theory. But AMR explicitly avoids theoretical commitment to a syntax-semantics mapping: Banarescu et al. (2013) state that AMR is agnostic about how we might want to derive meanings from strings. If we are go-ing to build such an assumption into our models, we should test it empirically, which we can do by analysing our corpus. We observe some pervasive structural differences between AMR and dependency syntax (3), despite the fact that a majority of AMR edges map easily onto dependency edges. Since syntactic alignment can largely explain AMRs, we also develop a baseline rule-based aligner for it, and show that this new task is much more difficult than lexical alignment (4). We also show how our data can be used to analyze errors made by an AMR parser (5). We make our annotated data and aligner freely available for further research. 3 2 Aligning AMR to dependency syntax Our syntactic representation is dependency grammar, which represents the sentence as a rooted, directed graph where nodes are words and edges are grammatical relations between them (Kruijff, 2006). We use Universal Dependencies (UD), a cross-lingual dependency annotation scheme, as implemented in Stanford CoreNLP (Manning et al., 2014). Within the UD framework, we use enhanced dependencies (Schuster and Manning, 2016), in which dependents can have more than one head, 3 https://github.com/ida-szubert/amr_ud resulting in dependency graphs (DGs). 4 Our alignment guidelines generalize ideas present in the existing frameworks. We want to allow many-to-many alignments, which we motivate by the observation that some phenomena cause an AMR graph to have one structure expressing the same information as multiple DG structures, and vice versa. For instance, in figure 2 the AMR subgraph representing Cruella de Vil aligns to two subgraphs in the dependency graph because of pronominal coreference. In the other direction, in figure 3 the capabilities node aligns to both capable nodes in the AMR, which is a result of the AMR treating conjoined adjectival modifiers as a case of ellipsis. The alignments we propose hold between subgraphs of any size. By aligning subgraphs we gain expressiveness needed to point out correspondences between semantic and syntactic structure. If AMR and DG were very similar in how they represent information, such correspondences would probably hold between subgraphs consisting of a single edge, as in figure 1 cat nmod:poss  my  cat poss  I . However, AMR by design abstracts away from syntax and it should not be assumed that all mappings will be so clean. For example, the same figure has lies nmod-in  sun case  in  lies location  sun . Moreover, AMR represents the meaning of particular words or phrases with elaborate structures, the result of which might be that the same information is expressed by a single word and a complex AMR subgraph, as in figure 3 where AMR represents general as person ARG0-of  have-org-role ARG2  general . An alignment is a link between subgraphs in an AMR and a DG which represent equivalent information. Given a sentence's DG and AMR we define an alignment as a mapping between an AMR subgraph and a DG subgraph. Lexical alignments (2.2) hold between pairs of nodes, and nodes from either graph may participate in multiple lexical alignments. Structural alignments (2.3) hold between pairs of connected subgraphs where at least one of the subgraphs contains an edge. 4 We chose UD because it emphasises shallow and semantically motivated annotation, by the virtue of which it can be expected to align relatively straightforwardly to a semantic annotation such as AMR. Aligning AMR with different versions of dependency grammar (e.g. Prague) or different syntactic frameworks (e.g. CCG, TAG) would be an interesting extension of our work. In the following two sections we discuss the types of alignments that our framework allows. More detailed guidelines regarding how to align particular linguistic constructions can be found in appendix A. 2.2 Lexical alignments A lexical alignment should hold between a word and an AMR concept if the latter is judged to express the lexical meaning of the former. Node labels usually reflect their lexically aligned word or its lemma, including derivational morphology (e.g. thirsty  thirst-01 ). Thus, string similarity is a useful heuristic for lexical alignment. 5 Most AMR nodes align lexically to a single word. Cases of one-to-many alignments include coreference , when an entity is mentioned multiple times in the sentence, and multiword expressions such as a verb-particle constructions ( pay off  pay-off-02 ) and fixed grammatical expressions ( instead of  instead-of-91 ). Occasionally an AMR node does not lexically align to any DG node. This is true for constants indicating sentence mood such as imperative , implicit uses of and to group list items, inferred concept nodes such as entity 5 Exceptions include: pronouns with noun antecedents in the sentence; the indicating negative polarity, which lexically aligns to no , not , and negative prefixes; modal auxiliaries, e.g., can  possible ; normalized dates and values such as February  2 in a date-entity ; and amr-unknown , which aligns to wh-words. types , name in named entities, and -91 frames like have-org-role-91 . Most words are lexically aligned to a single AMR node, if they are aligned at all. A word may align to multiple AMR nodes if it is duplicated in the AMR due to ellipsis or distributive coordination ( capabilities aligns to c2 / capable and c3 / capable in figure 3), or if it is morphologically decomposed in the AMR ( evildoer aligns to evil and do-02 in figure 2). Many words are not lexically aligned to any AMR node, including punctuation tokens, articles , copulas , nonmodal auxiliaries , expletive subjects, infinitival to , complementizer that , and relative pronouns. Structural alignments primarily reflect compositional grammatical constructions , be they syntactic or morphological. Note that the structural alignments build upon the lexical ones. Structural alignments hold between two subgraphs, at least one of which is larger than a single node. If a subgraph includes any edges, it automatically includes nodes adjacent to those edges. Structural alignments need not be disjoint: an edge can appear in two or more distinct alignments. Nodes and edges in both AMR and DG may be unaligned. can be interpreted. We establish the following principles to guide the specification of alignment: Connectedness Principle. In an alignment d  a , d must be a connected subgraph of the DG, and a must be a connected subgraph of the AMR. Minimality Principle. If two alignments, d  a and d   a  , have no dependency or AMR edges in common, then their union d  d   a  a  is redundant, even if it is valid. Individual alignments should be as small as possible; we believe compo-sitionality is best captured by keeping structures minimal. Therefore, in figure 1 there is no alignment between subgraphs spanning My , cat , lies and i , cat , lie . Such subgraphs do express equivalent information, but the alignment between them decomposes neatly into smaller alignments and we record only those. Subsumption Principle. This principle expresses the fact that our alignments are hierarchical. Structural alignments need to be consistent with lexical alignments: for subgraph a to be aligned to subgraph d , all nodes lexically aligned to nodes in a must be included in d , and vice versa. Moreover, structural alignments need to be consistent with other structural alignments. A structural alignment d  a is valid only if, for every connected AMR subgraph a <  a which is aligned to a DG subgraph, d   a < , we also have that d  is a subgraph of d and vice versa for every d <  d . Further, if a contains a node n which is not lexically aligned but which is part of a structurally aligned subgraph a  such that d   a  , it needs to be the case that a   a  d   d or a   a  d   d . (And vice versa for nodes in d .) For example, conceal nsubj-xsubj  Cruella  conceal ARG0  person name  name op1  Cruella is not a valid alignment, because the AMR side contains nodes person and name , which are not lexically aligned but which are both parts of a structural alignment marked in blue. Coordination Principle. If an alignment contains a dependency edge between two conjuncts, or between a conjunct and a coordinating conjunction, then it must also include all conjuncts and the conjunction. This preserves the integrity of coordinate structures in alignments. For example, in figure 2 there is no alignment glee cc  and  and op1  glee ; only the larger structure which includes the greed nodes is aligned. Named Entity Principle. Any structural alignment containing an AMR name node or any of the strings under it must contain the full subgraph rooted in the name plus the node above it specifying the entity type. This means that for example, in figure 2 there is no alignment conceal nsubj-xsubj  Cruella  conceal ARG0  person name  name op1  "Cruella" . Such an alignment would also be stopped by the Subsumption Principle provided that the blue alignment of the whole name was present. The Named Entity Principle is superfluous, but is provided to explicitly describe the treatment of such constructions. The smallest structure which can participate in a structural alignment is a single node, provided that it is aligned to a subgraph containing at least one edge. A DG node may align to an AMR subgraph if the word is morphologically decomposed or otherwise analyzed in the AMR (e.g. in figure 2, evildoer  person ARG0-of  do-02 ARG1  thing mod  evil ). Examples of DG structures whose meaning is expressed in a single AMR node include light verb constructions, phrasal verbs, and various other multiword expressions (e.g. in figure 2, makes dobj  attempt  attempt-01 ). Conceptually the simplest case of structural alignment is one edge to one edge, as in the blue and green alignments in figure 1. For such an alignment to be possible, two requirements must be satisfied: nodes which are endpoints of those edges need to be aligned one-to-one; and the AMR relation and the syntactic dependency must map cleanly in terms of the relationship they express. A one edge to multiple edges alignment arises when either of those requirements is not met. To see what happens in absence of one-to-one endpoint alignments let's look at the relation between confident and general in figure 3. The DG general node is aligned to an AMR subgraph: general  person ARG0-of  have-org-role ARG2  general . All alignments which involve the general node on the DG side need to include its aligned subgraph on the AMR side. It necessarily follows that the AMR subgraphs in those alignments will contain more edges that the DG ones; in this case the yellow subgraph in DG has 1 edge, and in AMR 3 edges. As for the second requirement, it is possible for one graph to use multiple edges to express a relationship when the other graph needs only one. This is the case for lie nmod-in  sun case  in  lie location  sun in figure 1. An example which combines both the nodeand edge-related issues is marked in red in figure 2. Finally, we also allow for many edges to many edges alignments. This may seem counterintuitive considering the assumption that we want to capture mappings between relations expressed in DG and AMR, and that we want to align minimal subgraphs. There are cases where an alignment is actually capturing a single relation, but we need to treat a subgraph as an endpoint of the edge both in DG and AMR. For instance, consider in figure 2 the relationship that holds between Cruella de Vil and concealing, expressed syntactically as an nsubj-xsubj edge and semantically as an ARG0 edge. One of the entities involved in that relationship, Cruella, is represented by a 2-edge DG subgraph and a 4-edge AMR subgraph. Consequently, the alignment covering the DG and AMR edges that relate Cruella to concealing must link subgraphs consisting respectively of 3 and 5 edges. A more difficult case of many edges to many edges alignment arises when relationships between nodes are expressed so differently in the DG and AMR that given an edge in one graph it is not possible to find in the other graph a subgraph that would convey the same information without also including some other information. Coordination has this property: e.g. in figure 2 the conj-and dependency between glee and greed has no counterpart in the AMR. There is no edge between AMR nodes aligned to those words, and the smallest AMR subgraph which contains them also contains and , which is itself lexically aligned. We cannot align glee conj-and  greed  glee op1  and op2  greed because of the rule that all lexically aligned nodes in one subgraph must be aligned to nodes in the other subgraph. Therefore we need to extend the DG side to and cc  glee conj-and  greed . We annotated a corpus of 200 AMR-sentence pairs (3813 aligned structures) using the guidelines of 2 and appendix A. 6 Data selection. To create the corpus we drew a total of 200 AMR-sentence pairs: 135 from the training split of the AMR Annotation Release 1.0 (Knight et al., 2014), 55 from the training split of The Little Prince Corpus v1.6, 7 and 10 sentences from the Adam part of the CHILDES Brown corpus (Brown, 1973), for which AMRs were produced by an experienced annotator. Seventy items were selected to illustrate particular linguistic phenomena. 8 The remaining 130 were selected at random. 6 We followed the precedent of previous AMR-to-sentence alignment corpora (see 4.2) in including 200 sentences in our gold standard, though ours was a different sample. 8 Namely: relative clauses, reflexive and non-reflexive pronominal anaphora, subject and object control, raising, exceptional case marking, coordination, wh-questions, do-support questions, ellipsis, expletives, modal verbs, light verbs, comparison constructions, and quantification. Preprocessing. Dependency parses were obtained using Stanford CoreNLP neural network parser 9 (Chen and Manning, 2014) and manually corrected. The final parses conform to the enhanced UD guidelines, 10 except they lack enhancements for ellipsis. Inter-annotator agreement. The corpus was created by one annotator. To assess inter-annotator agreement, a second annotator deeply familiar with UD and AMR annotated a random sample of sentences accounting for 10% of alignments in the corpus. The overall inter-annotator F 1 -score was 88%, with 96% agreement on lexical alignments and 80% on structural alignments. We take this as an indication that our richly structured alignment framework as laid out in 2 is reasonably well-defined for annotators. To assess our attempt to explain as much of the AMR as possible, we computed the proportion of AMR nodes and edges that participate in at least one alignment. Overall, 99.3% of nodes and 97.2% of edges in AMRs are aligned. We found that 81.5% of AMR graphs have full coverage, 18.5% have at least one unaligned edge, and 7.5% have one unaligned node (none had more than one; all unaligned nodes express mood or discourse-related information: interrogative , and , and say ). We conclude that nearly all information in an AMR is evoked by lexical items or syntactic structure. We expected coverage of DG to be lower because punctuation and many function words are unaligned in our guidelines (2.2). Indeed, only 71.4% of words and 65.2% of dependency edges are aligned. The similarity of AMR to syntax in examples like figure 1 invites the assumption of a close mapping, which often seems to be made in AMR parsers (Wang et al., 2015; Artzi et al., 2015; Misra and Artzi, 2016; Damonte et al., 2017) and aligners (Chu and Kurohashi, 2016; Chen and Palmer, 9 The corpus is annotated with UD v1; a release of the dataset converted to UD v2 is planned for the future. We used the pretrained dependency parsing model provided in CoreNLP with depparse.extradependencies set to MAXIMAL and used collapsed CCprocessed dependencies. 10 http://universaldependencies.org/u/overview/ enhanced-syntax.html configurations is max config. 2017). 11 Such an attitude reflects decades of work in the syntax-semantics interface (Partee, 2014) and the utility of dependency syntax for other forms of semantics (e.g., Oepen et al., 2014; Reddy et al., 2016; Stanovsky et al., 2016; White et al., 2016; Zhang et al., 2017; Hershcovich et al., 2017). However, this assumption has not been empirically tested, and as Bender et al. (2015) observe, it is an assumption not guaranteed by the AMR annotation style. Having aligned a corpus of AMR-DG pairs, we are in a position to provide empirical evidence. Are AMRs and dependency graphs structurally similar? We approach the question by analyzing the sizes of subgraphs used to align the two representations of the sentence. We define the size of a subgraph as the number of edges it contains. If a structure consists of a single node, we say its size is 0. The configuration of an alignment is then the pair of sizes for its AMR and DG sides; for example, an alignment with 1 AMR edge and 2 DG edges has configuration 1:2. We call an alignment configuration simple if at least one of the subgraphs is a single edge, indicating that there is a single relation which the alignment captures. Complex configurations cover multiple relations. By principle of minimality we infer that some structural difference between the graphs prevented those relations from aligning individually. One measure of similarity between AMR and DG graphs is the configuration of the most complex subgraph alignment between them. Configuration a:b is higher than c:d if a + b > c + d . However, all configurations involving 0 are lower than those which do not. A maximum of 1:1 means the graphs have only node-to-node, node-to-edge, and edge-to-edge alignments, rendering the graphs isomorphic (ignoring edge directions and unaligned nodes). In 11 In particular, Chen and Palmer (2017) align dependency paths to AMR edges. However, their evaluation only considers node-to-node alignment, and their code and data are not available for comparison at the time of this writing. general, if the maximum alignment configuration is a simple one, the graphs could be made isomorphic by collapsing the larger side of the alignment (e.g., in figure 2, the AMR side of the alignment evildoer  person ARG0-of  do ARG1  thing mod  evil could be collapsed into a node). In contrast, complex configurations imply serious structural dissimilarity, as in figure 3, where the cyan alignment has configuration 4:4. sentences are simple . Table 2 provides a detailed breakdown of alignment configurations in the corpus. Phenomena which often trigger complex configurations include coordination, named entities, semantically decomposed words, attachment of negation, and preposition-based concepts encoding location, time, and quantity. 12 We observe, comparing tables 1 and 2, that while simple configurations are most frequent in the corpus, the majority of sentences have at least one alignment which is complex. It should not be assumed that AMR and DG representations of a sentence are, or could trivially be made to be, isomorphic. It is worth noting that our analysis suggests that DG and AMR could be made more similar by applying simple transformations targeting problematic constructions like coordination and named entities. We use our annotations to measure the accuracy of AMR aligners on specific phenomena that were inexpressible in previous annotation schemes. Our experiments evaluate the JAMR heuristic aligner (Flanigan et al., 2014), the ISI statistical aligner (Pourdamghani et al., 2014), and a heuristic rule-based aligner that we developed specifically for 12 An AMR concept evoked by a preposition usually dominates the structure ( after op1  date-entity decade  nineties ), which is at odds with UD's prepositions-as-case-markers policy ( nineties case  after ). Lexical alignment algorithm. AMR concepts are cognate with English words, so we align them by lexical similarity. This algorithm does not make use of the DG. Before alignment, we remove sense identifiers on AMR node labels, and lemmatize DG node labels. Then for every pair of nodes a from the AMR and d from the DG we align them if any of the following conditions holds: 1. The Levenshtein distance of a and d is 15% or less of the length of the longer word. 13 2. The label of a is the morphological negation of d (e.g. prudent  imprudent ). 14 5. The label of a consists of multiple words, and the label of d matches any one of them under rule 3. The label of a is  (AMR's annotation of negation) and the parent of a aligns to d via rule 2. 4. The label of a is  and d is one of no , none , not , or never . 1. (e.g. sit  sit-down , war-torn  war ). 15 6. Labels of a and d likely have the same morphological root. We determine this by segmenting each word with Morfessor (Grnroos et al., 2014) trained on Wiki data and applying rule 1 to the first morpheme of each word. Note that if a word type is repeated in a sentence, each repetition is aligned to the same AMR nodes under the above rules. Structural alignment algorithm. We align subgraphs using the procedure below, first from AMR to DG, then from DG to AMR. For clarity, the explanation refers to the first case. 13 Threshold was determined empirically on a 10% sample from the dataset. Local phase. For every AMR edge e a whose endpoints are lexically aligned nodes a 1 (aligned to d 1 ) and a 2 (aligned to d 2 ), we attempt to align minimal and connected AMR and dependency subgraphs, a  and d  : 1. If there is a DG edge e d whose endpoints are d 1 and d 2 , then a   e a and d   e d . 2. Otherwise, let  d be the shortest undirected path between d 1 and d 2 . If all lexically aligned nodes in  d are aligned to a 1 or a 2 , then a   e a and d    d . 3. Otherwise, let a  be the smallest subgraph covering all AMR nodes that are lexically aligned to nodes in  d . If all the nodes in a  are aligned only to nodes in  d , then a   a  and d    d . 4. Otherwise, the attempt is abandoned. 5. Finally, if the top node of a  has a parent node labeled with an entity type concept, extend a  to include the parent. (This step is performed only in the AMR-to-DG step.) Global phase. The local phase might produce alignments that violate the Subsumption Principle (2.3.1), so we filter them out heuristically. For every pair of structural alignments,  d   a and   d    a where  a overlaps with   a , or  d with   d , if the region of overlap is not itself an aligned subgraph, we prune both alignments. 16 4.2 Experiments We evaluate JAMR, ISI, and our aligner on two distinct tasks. Lexical alignment. Lexical alignment involves aligning AMR nodes to words, a task all three systems can perform. We evaluate against three datasets: our own, the JAMR dataset (Flanigan et al., 2014), and the ISI dataset (Pourdamghani et al., 2014). 17 Results (table 3) suggest that this task is already well-addressed, but also that there exist marked differences between how lexical alignment is defined in each dataset and that aligners are 16 This could be order-dependent since the removal of one alignment could trigger the removal of others, but our aligner does not account for this. fine-tuned to their dataset. For our aligner, errors are due to faulty morphological analysis, duplicated words, and both accidental string similarity between AMR concepts and words and occasional lack of similarity between concepts and words that should be aligned. Structural alignment. An important goal of our experiments is to establish baselines for the structural alignment task. While we cannot evaluate the JAMR and ISI aligners directly on this task, we can use the lexical alignments they output in place of the first pass of our aligner. The only dataset for this task is our own. The results (table 4) evaluate accuracy of structural alignments only and do not count lexical alignments. The automatic alignments have lower coverage of AMRs than the gold alignments do: our best aligner leaves 13.3% of AMR nodes and 30.0% of AMR edges unaligned, compared to 0.07% and 2.8% in the gold standard. The aligner also leaves 39.2% of DG nodes and 47.7% of DG edges unaligned, compared to 28.6% and 34.8% in the gold standard. The relatively low F-score for the gold standard lexical alignments and DGs condition suggests that substantial improvements to our structural alignment algorithm are possible. The two most common reasons for low recall were missing one of the conjuncts in a coordinate structure and aligning structures that violate the principle of minimality. Our corpus gives alignments between AMRs and gold standard dependency parses. To see how much performance degrades when such parses are not available we also evaluate on automatic parses. 18 Both precision and recall are substantially worse when the aligner relies on automatic syntax. Our corpus of manually aligned AMRs can be used to identify linguistic constructions which cause 18 We use the CoreNLP dependency parser with settings as described in 3. problems for an AMR parser. We parsed the sentences from our corpus with the parser of Damonte et al. (2017). 19 We map the nodes of the resulting automatic AMRs to the gold AMRs using the smatch evaluation tool (Cai and Knight, 2013), and on the basis of this mapping identify those nodes and edges of the gold AMRs which are missing or mislabeled in the automatic AMRs. We then measured the number and rate of erroneous AMR fragments associated with each UD relation or construction (table 5). The largest proportion of recall errors were for fragments associated with the subject relation, prepositional phrases, and nominal compounds. Focusing on the subject relation, we can further say that 69% of the missing or mislabeled edges have the gold label ARG0 , 19% ARG1 , and the rest are distributed amongst domain , ARG2 , purpose and mod . Inspecting the errors we see that phenomena underlying them include pronominal coreference, sharing arguments between conjoined predicates, auxiliary verb constructions, and control and raising. 20 Our corpus facilitates fine-grained error analysis of AMR parsers with respect to individual syntactic constructions. We release the code for the above analysis in order to encourage syntactically-informed comparison and improvement of systems. We have presented a new framework and corpus for aligning AMRs to dependency syntax. Our data and analysis show that the vast majority of the semantics in AMR graphs can be mapped to the lexical and syntactic structure of a sentence, though current alignment systems do not fully capture this correspondence. The syntaxsemantics 19 The overall smatch score of the parser on this dataset was 0.65. correspondences are often structurally divergent (non-isomorphic). Simple algorithms for lexical and structural alignment establish baselines for the new alignment task; we expect statistical models will be brought to bear on this task in future work. Our framework also facilitates syntactically-based analysis of AMR parsers. We release our data and code for the benefit of the research community. This work was supported in part by EU ERC Advanced Fellowship 249520 GRAMPLUS and EU ERC H2020 Advanced Fellowship GA 742137 SE-MANTAX. We thank Sameer Bansal, Marco Damonte, Lucia Donatelli, Federico Fancellu, Sharon Goldwater, Andreas Grivas, Yova Kementchedjhieva, Junyi Li, Joana Ribeiro, and the anonymous reviewers for helpful discussion of this work and comments on previous drafts of the paper.
abstain method abstain abstain result result abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain objective result abstain abstain method result objective result abstain abstain method other abstain objective abstain abstain objective abstain abstain abstain abstain abstain abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method other abstain abstain abstain abstain method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method other abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain method objective result abstain abstain objective abstain abstain other other