In lexicalist linguistic theories, argument structure is assumed to be predictable from the meaning of verbs. As a result, the verb is the primary determinant of the meaning of a clause. In contrast, construction grammarians propose that argument structure is encoded in constructions (or form-meaning pairs) that are distinct from verbs. Decades of psycholinguistic research have produced substantial empirical evidence in favor of the construction view. Here we adapt several psycholinguistic studies to probe for the existence of argument structure constructions (ASCs) in Transformer-based language models (LMs). First, using a sentence sorting experiment, we find that sentences sharing the same construction are closer in embedding space than sentences sharing the same verb. Furthermore, LMs increasingly prefer grouping by construction with more input data, mirroring the behaviour of non-native language learners. Second, in a Jabberwocky priming-based experiment, we find that LMs associate ASCs with meaning, even in semantically nonsensical sentences. Our work offers the first evidence for ASCs in LMs and highlights the potential to devise novel probing methods grounded in psycholinguistic research. Pretrained Transformer-based language models (LMs) such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019b) have recently achieved impressive results on many natural language tasks, spawning a new interdisciplinary field of aligning LMs with linguistic theory and probing the linguistic capabilities of LMs (Linzen and Baroni, 2021). Most probing work so far has investigated the linguistic knowledge of LMs on phenomena such as agreement, binding, licensing, and movement (Warstadt et al., 2020a; Hu et al., 2020) Transitive Bob cut the bread S V OS acts on O Ditransitive Bob cut Joe the bread S V O1 O2 S transfers O2 to O1 Caused motion Bob cut the bread into the pan S V O Path S causes O to move via Path Resultative Bob cut the bread apart S V O State S causes O to become State Figure 1: Four argument structure constructions (ASCs) used by Bencini and Goldberg (2000), with example sentences (top right). with a particular focus on determining whether a sentence is linguistically acceptable (Schtze, 1996). Relatively little work has attempted to determine whether the linguistic knowledge induced by LMs is more similar to a formal grammar of the sort postulated by mainstream generative linguistics (Chomsky, 1965, 1981, 1995), or to a network of form-meaning pairs as advocated by construction grammar (Goldberg, 1995, 2006). One area where construction grammar disagrees with many generative theories of language is in the analysis of the argument structure of verbs, that is, the specification of the number of arguments that a verb takes, their semantic relation to the verb, and their syntactic form (Levin and Rappaport Hovav, 2005). Lexicalist theories were long dominant in generative grammar (Chomsky, 1981; Kaplan and Bresnan, 1982; Pollard and Sag, 1987). In lexicalist theories, argument structure is assumed to be encoded in the lexical entry of the verb: for 7410 example, the verb visit is lexically specified as being transitive and as requiring a noun phrase object (Chomsky, 1986). In contrast, construction grammar suggests that argument structure is encoded in form-meaning pairs known as argument structure constructions (ASCs, Figure 1), which are distinct from verbs. The argument structure of a verb is determined by pairing it with an ASC (Goldberg, 1995). To date, a substantial body of psycholinguistic work has provided evidence for the psychological reality of ASCs in sentence sorting (Bencini and Goldberg, 2000; Gries and Wulff, 2005), priming (Ziegler et al., 2019), and novel verb experiments (Kaschak and Glenberg, 2000; Johnson and Goldberg, 2013). Here we connect basic research in ASCs with neural probing by adapting several psycholinguistic studies to Transformer-based LMs and show evidence for the neural reality of ASCs. Our first case study is based on sentence sorting (Bencini and Goldberg, 2000); we discover that in English, German, Italian, and Spanish, LMs consider sentences that share the same construction to be more semantically similar than sentences sharing the main verb. Furthermore, this preference for constructional meaning only manifests in larger LMs (trained with more data), whereas smaller LMs rely on the main verb, an easily accessible surface feature. Human experiments with non-native speakers found a similarly increased preference for constructional meaning in more proficient speakers (Liang, 2002; Baicchi and Della Putta, 2019), suggesting commonalities in language acquisition between LMs and humans. Our second case study is based on nonsense Jabberwocky sentences that nevertheless convey meaning when they are arranged in constructional templates (Johnson and Goldberg, 2013). We adapt the original priming experiment to LMs and show that RoBERTa is able to derive meaning from ASCs, even without any lexical cues. This finding offers counter-evidence to earlier claims that LMs are relatively insensitive to word order when constructing sentence meaning (Yu and Ettinger, 2020; Sinha et al., 2021). Our source code and data are available at: https://github.com/SPOClab-ca/ neural-reality-constructions . Construction grammar is a family of linguistic theories proposing that all linguistic knowledge consists of constructions : pairings between form and meaning where some aspects of form or meaning are not predictable from their parts (Fillmore et al., 1988; Kay and Fillmore, 1999; Goldberg, 1995, 2006). Common examples include idiomatic expressions such as under the weather (meaning to feel un-well), but many linguistic patterns are constructions, including morphemes (e.g., -ify ), words (e.g., apple ), and abstract patterns like the ditransitive and passive. In contrast to lexicalist theories of argument structure, construction grammar rejects the dichotomy between syntax and lexicon. In contrast to transformational grammar, it rejects any distinction between surface and underlying structure. We focus on a specific family of constructions for which there is an ample body of psycholinguistic evidence: argument structure constructions (ASCs). ASCs are constructions that specify the argument structure of a verb (Goldberg, 1995). In the lexicalist, verb-centered view, argument structure is a lexical property of the verb, and the main verb of a sentence determines the form and meaning of the sentence (Chomsky, 1981; Kaplan and Bresnan, 1982; Pollard and Sag, 1987; Levin and Rappaport Hovav, 1995). For example, sneeze is intransitive (allowing no direct object) and hit is transitive (requiring one direct object). However, lexicalist theories encounter difficulties with sentences like he sneezed the napkin off the table since intransitive verbs are not permitted to have object arguments. Rather than assuming multiple implausible senses for the verb sneeze with different argument structures, Goldberg (1995) proposed that ASCs operate on an arbitrary verb, altering its argument structure while at the same time modifying its meaning. For example, the caused-motion ASC adds a direct object and a path argument to the verb sneeze , with the semantics of causing the object to move along the path. Other ASCs include the transitive, ditransitive, and resultative (Figure 1), which specify the argument structure of a verb and interact with its meaning in different ways. Sentence sorting. Several psycholinguistic studies have found evidence for argument structure con-7411 Transitive Ditransitive Caused-motion Resultative Throw Anita threw the hammer. structions using experimental methods. Among these, Bencini and Goldberg (2000) used a sentence sorting task to determine whether the verb or construction in a sentence was the main determinant of sentence meaning. 17 participants were given 16 index cards with sentences containing 4 verbs ( throw, get, slice , and take ) and 4 constructions ( transitive, ditransitive, caused-motion , and resultative ) and were instructed to sort them into 4 piles by overall sentence meaning (Table 1). The experimenters measured the deviation to a purely verb-based or construction-based sort, and found that on average, the piles were closer to a construction sort. Non-native sentence sorting. The same set of experimental stimuli was used with L2 (non-native) English speakers. Gries and Wulff (2005) ran the experiment with 22 German native speakers, who preferred the construction-based sort over the verb-based sort, showing that constructional knowledge is not limited to native speakers. Liang (2002) ran the experiment on Chinese native speakers of 3 different English levels (46 beginner, 31 intermediate, and 33 advanced), and found that beginners preferred a verb-based sort, while advanced learners produced construction-based sorts similar to native speakers (Figure 2). Likewise, Baicchi and Della Putta (2019) found the same result in Italian native speakers with B1 and B2 English proficiency levels. Overall, these studies show evidence for ASCs in the mental representations of native and L2 English speakers alike, and furthermore, preference for constructional over verb sorting increases with increasing English proficiency. Multilingual sentence sorting. Similar sentence sorting experiments have been conducted in other languages, with varying results. Kirsch (2019) ran a sentence sorting experiment in German with 40 participants and found that they mainly sorted by verb but rarely by construction. Baicchi and Della Putta (2019) ran an experiment with non-native learners of Italian (15 participants of B1 level and 10 participants of B2 level): both groups preferred the constructional sort, and similar to Liang (2002), the B2 learners sorted more by construction than the B1 learners. Vzquez (2004) ran an experiment in Spanish with 16 participants, and found approximately equal proportions of constructions and verb sort. In Italian and Spanish, some different constructions were substituted as not all of the English constructions had an equivalent in these languages; see the appendix for the complete set of stimuli in each language. Priming. Another line of psycholinguistic evidence comes from priming studies. Priming refers to the condition where exposure to a (prior) stimulus influences the response to a later stimulus (Pickering and Ferreira, 2008). Bock and Loebell (1990) found that participants were more likely to produce sentences of a given syntactic structure when primed with a sentence of the same structure; Ziegler et al. (2019) argued that Bock and Loebell (1990) did not adequately control for lexical overlap, and instead, they showed that the construction must be shared for the priming effect to occur, not just shared abstract syntax. Novel verbs. Even with unfamiliar words, there is evidence that constructions are associated with meaning. Kaschak and Glenberg (2000) constructed sentences with novel denominal verbs and found that participants were more likely to interpret a transfer event when the denominal verb was used in a ditransitive sentence ( Tom crutched Lyn an apple ) than a transitive one ( Tom crutched an apple ). Johnson and Goldberg (2013) used a Jabberwocky priming task to show that abstract constructional templates are associated with meaning. Participants were primed with a nonsense sentence of a given construction (e.g., He daxed her the norp for the ditransitive construction), followed by a lexical decision task of quickly deciding if a string of 7412 characters was a real English word or a non-word. The word in the decision task was semantically congruent with the construction ( gave ) or incongruent ( made ); furthermore, they experimented with target words that were high-frequency ( gave ), low-frequency ( handed ), or semantically related but not associated with the construction ( transferred ). They found priming effects (faster lexical decision times) in all three conditions, with the strongest effect for the high-frequency condition, followed by the low-frequency and the semantically nonas-sociate conditions. We adapt several of these psycholinguistic studies to LMs: the sentence sorting experiments in Case study 1, and the Jabberwocky priming experiment in Case study 2. We choose these studies because their designs allow for thousands of stimuli sentences to be generated automatically using templates, avoiding issues caused by small sample sizes from manually constructed sentences. Many studies have probed for various aspects of syntax in LSTMs and Transformer-based LMs. Linzen et al. (2016) tested LSTMs on their ability to capture subject-verb agreement, using templates to generate test data. This idea was extended by BLiMP (Warstadt et al., 2020a), a suite encompassing 67 linguistic phenomena, including filler-gap effects, NPI licensing, and ellipsis; Hu et al. (2020) released a similar test suite. Template generation is a convenient method to construct stimuli exhibiting specific linguistic properties, but alternative approaches include CoLA (Warstadt et al., 2019), which compiled an acceptability benchmark of sentences drawn from linguistic publications, and Gulordava et al. (2018), who perturbed natural sentences to study LMs' knowledge of agreement on nonsense sentences. We refer to Linzen and Baroni (2021) for a comprehensive review of the linguistic probing literature. So far, relatively few papers approached LM probing from a construction grammar perspective. Madabushi et al. (2020) probed for BERT's knowledge of constructions via a sentence pair classi-fication task of predicting whether two sentences share the same construction. Their probe was based on data from Dunn (2017), who used an unsupervised algorithm to extract plausible constructions from corpora based on association strength. However, the linguistic validity of these automatically induced constructions is uncertain, and there is currently no human-labelled wide-coverage construction grammar dataset in any language suitable for probing. Other computational work focused on a few specific constructions, such as identifying caused-motion constructions in corpora (Hwang and Palmer, 2015) and annotating constructions related to causal language (Dunietz et al., 2015). Lebani and Lenci (2016) is the most similar to our work: they probed distributional vector space models for ASCs based on the Jabberwocky priming experiment by Johnson and Goldberg (2013). Some recent probing studies adapted methods and data from psycholinguistic research, treating LMs as psycholinguistic participants. Using a cloze completion task, Ettinger (2020) found that BERT was less sensitive than humans at commonsense inferences and detecting role reversals, and fails completely at understanding negation. Michaelov and Bergen (2020) compared LM surprisals with the N400 (a measure of human language processing difficulty) across a wide range of conditions; Li et al. (2021) used psycholinguistic stimuli and found that LMs exhibit different layerwise surprisal patterns for morphosyntactic, semantic, and commonsense anomalies. Wilcox et al. (2021) compared LM and human sensitivities to syntactic violations using a maze task to collect human reaction times. Prasad et al. (2019); Misra et al. (2020) investigated whether LMs are sensitive to priming effects like humans. The advantage of psycholinguistic data is that they are carefully constructed by expert linguists to test theories of language processing in humans; however, their small sample size makes it challenging to make statistically meaningful conclusions when the (oft-sparse) experimental stimuli are used to probe a language model. This section describes our adaptation of the sentence sorting experiments to Transformer LMs. Models. To simulate varying non-native English proficiency levels, we use MiniBERTa models (Warstadt et al., 2020b), trained with 1M, 10M, 1 Bencini and Goldberg (2000) ran the sentence sorting experiment twice, so we take the average of the two runs. 100M, and 1B tokens. We also use the base RoBERTa model (Liu et al., 2019b), trained with 30B tokens. In other languages, there are no available pretrained checkpoints with varying amounts of pretraining data, so we use the mBERT model (Devlin et al., 2019) and a monolingual Transformer LM in each language. 2 We obtain sentence embeddings for our models by taking the average of their contextual token embeddings at the second-to-last layer (i.e., layer 11 for base RoBERTa). We use the second-to-last because the last layer is more specialized for the LM pretraining objective and less suitable for sentence embeddings (Liu et al., 2019a). Template generation. We use templates to generate stimuli similar to the 4x4 design in the Bencini and Goldberg (2000) experiment. To ensure an adequate sample size, we run multiple empirical trials. In each trial, we sample 4 random distinct verbs from a pool of 10 verbs that are compatible with all 4 constructions ( cut, hit, get, kick, pull, punch, push, slice, tear, throw ). We then randomly fill in the slots for proper names, objects, and complements for each sentence according to its verb, such that the sentence is semantically coherent, and there is no lexical overlap among the sentences of any construction. Table 3 in the ap-2 We use monolingual German and Italian models from https://github.com/dbmdz/berts , and the monolingual Spanish model from Caete et al. (2020). pendix shows a set of template-generated sentences. In English, we generate 1000 sets of stimuli using this procedure; for other languages, we use the original stimuli from their respective publications. Evaluation. Similar to the human experiments, we group the sentence embeddings into 4 clusters (not necessarily of the same size) using agglomerative clustering by Euclidean distance (Pedregosa et al., 2011). We then compute the deviation to a pure construction and pure verb sort using the Hungarian algorithm for optimal bipartite matching. This measures the minimal number of cluster assignment changes necessary to reach a pure construction or verb sort, ranging from 0 to 12. Thus, lower construction deviation indicates that constructional information is more salient in the LM's embeddings. Figure 2 shows the LM sentence sorting results for English. All differences are statistically significant ( p < . 001 ). The smallest 1M MiniBERTa model is the only LM to prefer verb over construction sorting, and as the amount of pretraining data grows, the LMs increasingly prefer sorting by construction instead of by verb. This closely mirrors the trend observed in the human experiments. The results for multilingual sorting are shown in Figure 3. Both mBERT and the monolingual LMs consistently prefer constructional sorting over verb 7414 0 2 4 6 8 10 12 mBERT mono mBERT mono mBERT mono CDev VDev German Italian Spanish Human data Language models 0 2 4 6 8 10 12 German Italian B1 Italian B2 Spanish V a l ue Figure 3: Multilingual sentence sorting results for German (Kirsch, 2019), Italian (Baicchi and Della Putta, 2019), and Spanish (Vzquez, 2004). Our results show that RoBERTa can generalize meaning from abstract constructions without lexical overlap. Only larger LMs and English speakers of more advanced proficiency are able to make this generalization, while smaller LMs and less proficient speakers derive meaning more from surface features like lexical content. This finding agrees with Warstadt et al. (2020b), who found that larger LMs have an inductive bias towards linguistic generalizations, while smaller LMs have an inductive bias towards surface generalizations; this may explain the success of large LMs on downstream tasks. A small quantity of data (10M tokens) is sufficient for LMs to prefer the constructional sort, indicating that ASCs are relatively easy to learn: roughly on par with other types of linguistic knowledge, and requiring less data than commonsense knowledge (Zhang et al., 2021; Liu et al., 2021). We note some limitations in these results, and reasons to avoid drawing unreasonably strong conclusions from them. Human sentence sorting experiments can be influenced by minor differences in the experimental setup: Bencini and Goldberg (2000) obtained significantly different results in two runs that only differed on the precise wording of instructions. In the German experiment (Kirsch, 2019), the author hypothesized that the participants were influenced by a different experiment that they had completed before the sentence sorting one. Given this experimental variation, we cannot attribute differences across languages to differences in their linguistic typology. Although LMs do not suffer from the same experimental variation, we cannot conclude statistical significance from the multilingual experiments, where only one set of stimuli is available in each language. We next adapt the Jabberwocky priming experiment from Johnson and Goldberg (2013) to LMs, and make several changes to the original setup to better assess the capabilities of LMs. Priming is a standard experimental paradigm in psycholinguistic research, but it is not directly applicable to LMs: existing methods simulate priming either by applying additional fine-tuning (Prasad et al., 2019), or by concatenating sentences that typically do not co-occur in natural text (Misra et al., 2020). Therefore, we instead propose a method to probe LMs for the same linguistic information using only distance measurements on their contextual embeddings. Template generation. We generate sentences for the four constructions randomly using the templates in Table 2. Instead of filling nonce words like norp into the templates as in the original study, we take an approach similar to Gulordava et al. (2018) and generate 5000 sentences for each construction 7415 She traded her the epicenter gave made put took Figure 4: In our adapted Jabberwocky experiment, we measure the Euclidean distance from the Jabberwocky verb ( traded ) to the 4 prototype verbs, of which 1 is congruent ( (cid:51) ) with the construction of the sentence, and 3 are incongruent ( (cid:55) ). by randomly filling real words of the appropriate part-of-speech into construction templates (Table 2). This gives nonsense sentences like She traded her the epicenter ; we refer to these random words as Jabberwocky words . By using real words, we avoid any potential instability from feeding tokens into the model that it has never seen during pretraining. We obtain a set of singular nouns, past tense verbs, and adjectives from the Penn Treebank (Marcus et al., 1993), excluding words with fewer than 10 occurrences. Verb embeddings. Our probing strategy is based on the assumption that the contextual embedding for a verb captures its meaning in context. Therefore, if LMs associate ASCs with meaning, we should expect the contextual embedding for the Jabberwocky verb to contain the meaning of the construction. Specifically, we measure the Euclidean distance to a prototype verb for each construction (Figure 4). These are verbs that Johnson and Goldberg (2013) selected whose mean-Congruent Incongruent High frequency Low frequency 11.0 11.5 12.0 12.5 13.0 Congruent Incongruent E u c li dean d i s t an c e Figure 5: Euclidean distance between Jabberwocky and prototype verbs for congruent and incongruent conditions. ing closely resembles the construction's meaning: gave , made , put , and took for the ditransitive, resultative, caused-motion, and removal constructions, respectively. 3 We also run the same setup using lower frequency prototype verbs from the same study: handed , turned , placed , and removed . 4 As a control, we measure the Euclidean distance to the prototype verbs of the other three unrelated constructions. The prototype verb embeddings are generated by taking the average across their contextual embeddings across a 4M-word subset of the British National Corpus (BNC; Leech (1992)). We use the second-to-last layer of RoBERTa-base, and in cases where a verb is split into multiple subwords, we take the embedding of the first subword token as the verb embedding. We find that the Euclidean distance between the prototype and Jabberwocky verb embeddings is significantly lower ( p < . 001 ) when the verb is congruent with the construction than when they are incongruent, and this is observed for both high and low-frequency prototype verbs (Figure 5). Examining the individual constructions and verbs (Figure 6), we note that in the high-frequency scenario, the lowest distance prototype verb is always the congruent one, for all four constructions. In the low-frequency scenario, the result is less consis-3 The reader may notice that the four constructions here are slightly different from Bencini and Goldberg (2000): the transitive construction is replaced with the removal construction in Johnson and Goldberg (2013). 4 Johnson and Goldberg (2013) also included a third experimental condition using four verbs that are semantically related but not associated with the construction, but one of the verbs is very low-frequency ( ousted ), so we exclude this condition in our experiment. tent: the congruent verb is not always the lowest distance one, although it is always still at most the second-lowest distance out of the four. The main result holds for both high and low-frequency scenarios, but the correct prototype verb is associated more consistently in the high-frequency case. This agrees with Wei et al. (2021), who found that LMs have greater difficulty learning the linguistic properties of less frequent words. We also note that the Euclidean distances are higher overall in the low-frequency scenario, which is consistent with previous work that found lower frequency words to occupy a peripheral region of the embedding space (Li et al., 2021). In any experiment, one must be careful to ensure that the observed patterns are due to the phenomenon under investigation rather than confounding factors. We discuss potential confounds arising from lexical overlap, anisotropy of contextual embeddings, and neighboring words. Lexical overlap . The randomized experiment design ensures that the Jabberwocky words cannot be lexically biased towards any construction, since each verb is equally likely to occur in every construction. Technically, the lexical content in the four constructions are not identical: i.e., words such as from (occurring only in the removal construction) or on (in the caused-motion construction) may provide hints to the sentence meaning. However, the ditransitive and resultative constructions do not contain any such informative words, yet RoBERTa still associates the correct prototype verb for these constructions, so we consider it unlikely to be relying solely on lexical overlap. There is substantial evidence that RoBERTa is able to associate abstract constructional templates with their meaning without lexical cues. This result is perhaps surprising, given that previous work found that LMs are relatively insensitive to word order in compositional phrases (Yu and Ettinger, 2020) and downstream inference tasks (Sinha et al., 2021; Pham et al., 2021), where their performance can be largely attributed to lexical overlap. Anisotropy . Recent probing work have found that contextual embeddings suffer from anisotropy, where embeddings lie in a narrow cone and have much higher cosine similarity than expected if they were directionally uniform (Ethayarajh, 2019). Furthermore, a small number of dimensions dominate geometric measures such as Euclidean and cosine distance, resulting in a degradation of representation quality (Kovaleva et al., 2021; Timkey and van Schijndel, 2021). Since our experiments rely heavily on Euclidean distance, anisotropy is a significant concern. Following Timkey and van Schijndel (2021), we perform standardization by subtracting the mean vector and dividing each dimension by its standard deviation, where the mean and standard deviation for each dimension is computed from a sample of the BNC. We observe little difference after standardization: in both the high and low frequency scenarios, the Euclidean distances are lower for the congruent than the incongruent conditions, by a similar margin compared to the original experiment without standardization. We also run standardization on the first case study, and find that the 7417 results remain essentially unchanged: smaller LMs still prefer verb sorting while larger LMs prefer construction sorting. Thus, neither of our experiments appear to be affected by anisotropy. Neighboring words. A final confounding fac-tor is our assumption that RoBERTa's contextual embeddings represent word meaning, when in reality, they contain a mixture of syntactic and semantic information. Contextual embeddings are known to contain syntax trees (Hewitt and Manning, 2019) and linguistic information about neighboring words in a sentence (Klafka and Ettinger, 2020); although previous work did not consider ASCs, it is plausible that our verb embeddings leak information about the sentence's construction in a similar manner. If this were the case, the prototype verb embedding for gave would contain not only the semantics of transfer that we intended, but also information about its usual syntactic form 5 of S gave NP1 NP2 , and both would be captured by our Euclidean distance measurement. Controlling for this syntactic confound is difficult  one could alternatively probe for transfer semantics without syntactic confounds using a natural language inference setup (e.g., whether the sentence entails the statement NP1 received NP2 ), but we leave further exploration of this idea to future work. We find evidence for argument structure constructions in Transformer language models from two separate angles: sentence sorting and Jabberwocky construction experiments. Our work extends the existing body of literature on LM probing by taking a constructionist instead of generative approach to linguistic probing. Our sentence sorting experiments identified a striking resemblance between hu-mans' and LMs' internal language representations as LMs are exposed to increasing quantities of data, despite the differences between neural language models and the human brain. Our two studies suggest that LMs are able to derive meaning from abstract constructional templates with minimal lexical overlap. Both sets of experiments were inspired by psycholinguistic studies, which we adapted to fit the capabilities of LMs  this illustrates the potential for future work on grounding LM probing methodologies in psycholinguistic research. YX is funded through an NSERC Discovery Grant, a SSHRC Insight Grant, and an Ontario ERA award.
abstain abstain abstain abstain abstain objective abstain result objective abstain other abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain other abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method other other other other abstain other other other other other abstain other other other other other other abstain abstain method abstain method method abstain abstain abstain method abstain method abstain other abstain method abstain method method abstain abstain abstain abstain abstain abstain abstain other result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method method method abstain abstain method method abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method method result objective method abstain abstain other