The dominant paradigm for high-performance models in novel NLP tasks today is direct specialization for the task via training from scratch or fine-tuning large pre-trained models. But does direct specialization capture how humans approach novel language tasks? We hypothesize that human performance is better characterized by flexible inference through composition of basic computational motifs available to the human language user. To test this hypothesis, we formulate a set of novel fragmentary text completion tasks, and compare the behavior of three direct-specialization models against a new model we introduce, GibbsComplete, which composes two basic computational motifs central to contemporary models: masked and autoregressive word prediction. We conduct three types of evaluation: human judgments of completion quality, satisfaction of syntactic constraints imposed by the input fragment, and similarity to human behavior in the structural statistics of the completions. With no task-specific parameter tuning, GibbsComplete performs comparably to direct-specialization models in the first two evaluations, and outperforms all direct-specialization models in the third evaluation. These results support our hypothesis that human behavior in novel language tasks and environments may be better characterized by flexible composition of basic computational motifs rather than by direct specialization. Representation learning has tremendously benefited engineering for language comprehension and generation systems. General frameworks such as encoder-decoder or auto-regressive model can be flexibly applied to a diverse range of problems. With scaled models, enormous data, and wellchosen training objective functions, generative pretraining extracts useful and transferable information from unlabeled text data (Howard and Ruder, 2018; Liu et al., 2019b). These generic representations can then be finetuned to quickly yield performant models directly specialized for downstream tasks (Howard and Ruder, 2018; Radford et al., 2019; Devlin et al., 2019, inter alia ). While the broad idea of exploiting rich statistical information in general-purpose representations has proven practically effective for deep learning models since a decade ago (Erhan et al., 2010; Collobert et al., 2011), it remains an open question how well this direct specialization approach yields models that behave similarly to humans in flexible linguistic behavior in novel situations. Here we take this direct specialization approach as a scientific hypothesis regarding the na-ture of linguistic knowledge and its flexible deployment in the human mind: that the human capacity of processing linguistic information in novel tasks arises from directly specializing and reshaping a generic yet versatile representation. We contrast it with an alternative hypothesis: that flexible knowledge deployment reflects algorithmic composition of a constrained repertoire of simple, reusable inference motifs. We depict these competing hypotheses in Figure 1. In the compositional inference hypothesis, the basic motifs arise from learning processes and could potentially involve distinct internal representations, but can be recombined into new inference routines guided by the principles of approximate probabilistic inference. In this hypothesis, the computational-level specifications (Marr, 1982) of motif functional forms play a crucial role. As a theory of how the human mind approaches new and potentially complex problems, the direct specialization hypothesis entails generic starting representations, task-specific supervision, and specialization. The compositional inference hypothesis, in contrast, entails novel combinations of solutions to old problems. ies of human behaviors involving various tasks, which goes beyond the scope of this work. However, as an initial step, we consider these two hypotheses as computational paradigms that inform and inspire the architecture of models that flexibly perform novel language tasks, and ask which paradigm gives rise to more human-like behaviors. Here we ground the general question in a specific setting: a novel set of fragmentary input completion challenges, inspired by the classic cloze task (Ebbinghaus, 1897) and its contemporary variants. Taking a reverse-engineering approach, we instantiate the direct specialization and compositional inference hypotheses as explicit computational models and compared the models' behaviors to that of humans in a behavioral task of fragmentary input completion. Our fragmentary input designs highlight various aspects of abstract reasoning involving subtle features of grammar and semantics. We find that the model of the compositional inference approach generates high-quality completions without direct training on the target task, achieves comparable performance to the models from the direct specialization approach in reasoning about constrained syntactic contexts, and better matches the fine-grained structural statistics of completions written by human subjects. For purposes of this paper, we define a fragmentary linguistic input , or simply a fragment , as a sequence of word strings. Completing a fragment involves adding a word string of any length between adjacent strings in the input to yield a single overall well-formed sentence. The fragment completion problem is formally equivalent to the text infilling problem studied in a number of recent papers (Fe-dus et al., 2018; Zhu et al., 2019; Liu et al., 2019a; Donahue et al., 2020; Shen et al., 2020; Huang et al., 2020), but here we study considerably more open-ended completion problems from potentially much briefer input than has been studied before. For example, many native English speakers find the simple fragmentary input published won . initially challenging yet solvable. 1 We find that carefully chosen simple fragmentary inputs can 1 The input requires building a nested center-embedding context such as  The most recent book she published won a prize . offer insight into the abilities of direct specialization and compositional inference models, and the similarity of model and human behavior. Formally, fragment completion involves generating, given input comprised of a sequence of k word strings C = { C 1 , . . . , C k } , a sequence of k  1 word strings B = { B 1 , . . . , B k  1 } , such that the resulting completion is C 1  B 1    C k  1  B k  1  C k . 2 In general, exact inference over the full conditional distribution P ( B | C ) will be intractable. The direct specialization and compositional inference paradigms offer differing model specifications and algorithmic options. With direct specialization, we learn or fine-tune representations under supervision for the task. We take the learning objective to be maximizing the likelihood p ( B | C ) for some sampled ( B, C ) pairs generated from a training corpus (see Section 2.2): p ( B | C ) = | B | (cid:89) i =1 p ( B i | B <i , C ) We ground this approach into two learning procedures: (a) fine-tuning pretrained language model to solve the target task, and (b) training on the target task from scratch. The fine-tuning procedure takes advantage of transferring knowledge from large pretrained models, while the training from scratch procedure allows us to further control the effect of pretrained representation. For both learning procedures, we use three existing models trained or fine-tuned for infilling: T5 (Raffel et al., 2019; In-fillT5), BART (Lewis et al., 2020; InfillBART), and GPT-2 (Radford et al., 2019) fine-tuned for text infilling, which was previously explored in Donahue et al. (2020) and which we will follow them in calling the Infilling Language Model (ILM) for short. Implementation details can be found in Section 2.2. For the compositional inference approach, we propose GibbsComplete , a neurally-guided approximate inference algorithm that combines two canonical computational motifs: masked word prediction and autoregressive word prediction. Consider a 2 We represent cases where material can be added at the beginning or the end of the input as c 1 or c k being the empty string, (cid:15) . candidate for the i -th completion string B i to consist of b i,<j  b i,j  b i,j< . GibbsComplete takes a masked language modelling motif as a proposal distribution p ( b i,j | B \ i , b i,<j , b i,j< , C ) and composes it with a global scoring function  ( B, C ) given by a unidirectional language modelling motif, in line with previous work on applying sampling-based methods to generative neural sequence models (Berglund et al., 2015; Su et al., 2018; Miao et al., 2019; Wang and Cho, 2019; He and Li, 2021). GibbsComplete can also be broadly viewed as an example of unsupervised language generation, among many other alternatives (Liu et al., 2019a; Qin et al., 2020; West et al., 2021). Our GibbsComplete algorithm first proposes a random uniform guess on [1 , 10] about the length of each { B i } , and initializes them as  [MASK]  sequences of the guessed lengths. It then proposes an edit to a randomly chosen position b i,j within the blanks by sampling from the sorted list of likely replacements according to the conditional probability p ( b i,j | B \ i , b i,<j , b i,j< , C ) given by a masked language model. We take 500 stochastic editing steps: for the first 250 steps as a burn-in period, the replacement is sampled from the top 50 likely words; for the remaining iterations, the most likely word is picked as the replacement, following the common practice of annealing temperature for better generation quality (Wang and Cho, 2019). The final output of the editing process is a candidate completion. We sample 1000 such candidates, and then rerank them with an autoregressive forward language model, using mean per-word conditional log-probability as a scoring function to promote fluency. or retrained on the target infilling task: we take them as basic computational motifs that are immediately available to a language-using agent for use in sampling-based probabilistic inference to facilitate novel behaviors. Completing fragmentary inputs of the type studied here is not a major form of everyday language use, yet as our experiments show, native speakers can perform even challenging fragment completions fairly well with little practice. Like the two learning settings considered in the direct specialization approach, we implement instances of GibbsComplete algorithm with (a) pretrained language models as the computational motifs, and (b) computational motifs of the same architecture as the pretrained counterparts but learned from the same corpus as those trained-from-scratch models from the direct optimization approach. We implement instances of the models under two learning settings: (a) transferring knowledge from pretrained models, and (b) training from scratch. Implementations are based on the Huggingface transformers package (Wolf et al., 2020). In the case of knowledge transfer with pretrained models, no fine-tuning or learning is needed for GibbsComplete. We simply use the small version of pretrained GPT-2 (Radford et al., 2019) and the base cased version of pretrained BERT (Devlin et al., 2019) as the corresponding computational motifs. For direct specialization models, we fine-tune pretrained GPT-2 small, T5 base, and BART large 3 to get ILM, InfillT5 and InfillBART respectively. The total number of parameters of these model architectures is listed in Table 1. All models were fine-tuned on a 10 million token subset of New York Times Corpus 2007 portion (Sand-haus, 2008), with a batch size of 32 and learning rate of 10  5 . The supervision signal is generated by randomly cropping some spans of words in a sentence to get the fragmentary context C and a plausible completion B (see Appendix C.2 for de-tails). We stopped fine-tuning when the validation loss increases for two epochs in a row. To generate 3 Although the pretraining tasks of T5 and BART do include modified versions of the sentence infilling problem or related text denoising tasks, our initial experimentation suggested that pretrained T5 and BART could not fully support the flexible generalization required in our studies, hence we fine-tuned them as above. completions from ILM, InfillT5, and InfillBART, we apply ancestral sampling from a list of top 50 mostly likely tokens at each time step. In experimenting with the fine-tuned models, we noticed lower diversity in InfillBART samples compared to the other models. Hence we set the sampling temperature as 1 for other models but 1.8 for the fine-tuned InfillBART to ensure that all models generate a good variety of completions. When training from scratch, we train all components to be learned in all models on two separate 42-million-token datasets: (1) part of the 2006 year portion of New York Times Corpus (Sand-haus, 2008; NYT ), and (2) part of the BLLIP corpus (Charniak et al., 2000) previously prepared by Hu et al. (2020). For GibbsComplete, we train an auto-regressive Transformer decoder language model of the same size as pretrained GPT-2 small and a masked language model of the same size as pretrained base cased BERT. For ILM, InfillT5, and InfillBART, we initialize the same architecture and tokenizer as their fine-tuned counterparts and train each model from scratch. We use a batch size of 16 for the masked language model in GibbsComplete and 32 for all the other models. The learning rate is set as 10  5 across all the models. Training is early stopped if either the validation loss increases for two epochs in a row or the total number of training epochs exceeds 100. We also collect human completions of our fragments on Mechanical Turk, to evaluate performance and for fine-grained comparison of human and model behavior. The visual layout of these experiments is shown in Appendix A. Participants were instructed to use as many or little words as they see appropriate to fill in the blanks in the fragmentary input, so that each completed sentence is coherent, grammatical, and meaningful. The interface required each blank to be filled in with at least one word. We imposed no time constraints. To address the question of whether the direct specialization or compositional inference paradigm gives rise to more humanlike behaviors, we focus on designing a series of linguistically-motivated experiments 4 . Our first experiment qualitatively confirms that each model respects basic bidirectional constraints, and quantitatively evaluates the fluency of each model's completions using human judgments. We design 30 two-fragment stimuli of the form   by adapting sentences from the Brown Corpus (Francis and Kucera, 1979) and British National Corpus (2001), choosing spans to crop out such that successful completion does not require outside-of-sentence information or much factual world knowledge, but does require non-trivial respect of grammatical constraints: we require that both  and  are multi-word fragments that cross conventional constituent boundaries. Table 4 in Appendix G.1 lists one randomly-generated completion from different models to a subset of the stimuli. Qualitatively, all models generate high-quality completions that fit the context and sound fluent, although coherence is sometimes lacking. For quantitative evaluation, we recruit human raters on Prolific to evaluate the quality of the completed sentences written by models as well as human writers previously recruited on Mechanical Turk. Human raters were presented with the fragmentary input together with a completion and asked to judge the grammaticality, coherence, interestingness, and overall quality of the presented completion. Ratings range from 1 to 100, with 1 the lowest score and 100 the highest. Each human rater judged 150 completions in total, with 30 completions from a human writer or each of the models. Raters did not know whether a completion comes from a human or a model. The results of these ratings are shown in Figure 2: GibbsComplete, InfillT5, and InfillBART achieve similar performance on grammaticality judgment on this set of stimuli. The same human evaluation procedure is also applied to the set of models trained from scratch on the NYT and BLLIP data; results are given in Figure 2. Overall performance is worse than with the pre-trained models, but the relative patterns from model to model are similar. Overall, the results of Evaluation 1 suggest that when extensive bidirectional context is given, all models are able to generate structurally well-formed completions. This success motivates our next experiment, which involves briefer input in syntactically constrained configurations that more strongly challenge models' grammatical abilities. The first fragment consists of a plural noun phrase with an incomplete relative clause postmodifier; the second fragment is a singular verb, is. The plural noun in the first fragment cannot be the subject of is due to number agreement in English, which forces a syntactically complex completion, such as The paintings that the artist gave to the museum are gorgeous and one of them is absolutely a masterpiece .. We design 26 novel fragment configurations to test models' syntactic behavior, ranging broadly across subject-predicate agreement, clausal structure, coordination, and filler-gap dependencies. For 8180 0.00 0.25 0.50 0.75 1.00 A cc u r a cy Human GibbsComplete InfillT5 InfillBART ILM Pretrain/Fine-tune From scratch (NYT) From scratch (BLLIP) Pretrain/Fine-tune From scratch (NYT) From scratch (BLLIP) Figure 3: Aggregated results on syntactic reasoning tests. each configuration we construct a set of semantically diverse fragmentary inputs that share the critical high-level syntactic structure for inducing similar grammatical constraints. The semantic diversity of the items facilitates more reliable estimation of the models' syntactic reasoning abilities. We provide descriptions and examples of each syntactic reasoning test in Appendix E. With each model we generated 35 completions for each stimulus in each of the 26 tests. To collect human judgments for every single completion would be laborious and difficult to scale, in particular because evaluating whether the syntactic constraint is satisfied often requires some linguistic expertise. Instead, we represent the key syntactic constraints imposed by the fragments as tree patterns to be expected in the constituency parse of a completed stimulus. For example, given a stimulus  published won . from the test (6) as shown in Appendix E, the linguistic intuition is that published should be part of a Verb Phrase embedded in a relative clause that modifies the subject of the predicate won, despite other possible structural variations. We express these tree patterns using the Tregex tool (Levy and Andrew, 2006), compute the average rate of hitting the desired syntactic patterns out of the 35 completions which are annotated with syntactic parses by an off-the-shelf neural constituency parser benepar (Kitaev and Klein, 2018), and average across all the stimuli in a test as the final accuracy score for that test. 5 Figure 3 shows the performance of humans and each model; Figure 8 in Appendix G.2 breaks down accuracy scores by each test separately. Human performance 6 is as good as or superior to all models 5 We also conducted manual evaluation of a sample of the pattern-matching results, which confirmed high accuracy of this automated evaluation procedure; see Appendix F for details. 6 We collected human completions from Mechanical Turk for two stimulus items of each test, with 56 responses for ( p < 0 . 05 for all models except fine-tuned InfillT5 and InfillBART, two-sided paired t -test). For the pretrained models, ILM performs the worst; the other three models' performance is comparable. When training from scratch, GibbsComplete outperforms all other models except on BLLIP , where its performance is matched by InfillBART. To address a potential concern about the ensembling effect of the reranking process in GibbsComplete, we also examine the performance obtained when composing the outputs of the directly-specialized models with the reranking process of GibbsComplete, generating 1000 candidate completions per fragment and selecting the top-ranked 35 completions using the same reranker as in GibbsComplete. Figure 10 in Appendix G.2 shows the results: InfillT5 and InfillBART now perform the best and even match human performance, further underscoring the value of a compositional approach even when dedicated training for direct specialization is available. Evaluation II showed that models can perform very well even on more grammatically challenging fragment completion tasks, in some cases matching human performance. But do these models complete fragments in a similar way as humans do? Evaluation III turns to this question, using still more open-ended fragments and fine-grained analysis of the structural similarity of human and model completions. To evaluate fine-grained similarity of human and model behavior, we define summary statistics of features of completions and assess the similarity of the summary statistics seen in human and model completions. We designed 120 stimuli in the form each item. Human completions were evaluated with the same tree pattern-based method as model-generated completions. we also examine the performance obtained when 413 composing the outputs of the directly-specialized 414 models with the reranking process of GibbsCom-415 plete, generating 1000 candidate completions and 416 outputs per fragment and reranking the top-ranked 417 35 completions using the same reranker as in Gibb-418 sComplete. of  w 1 w 2 ., where w 1 and w 2 are single-word fragments, allowing for a diverse range of plausible syntactic choices of the global context. For example, museum city . Figure 9 in Appendix F.2 shows the 419 results: InfillT5 and InfillBART now perform the 420 best and even match human performance, further 421 underscoring the value of a compositional approach 422 even when dedicated training for direct specializa-423 tion is available. We evaluate the performance of a model by its mean squared error (MSE) against human relative frequencies of the five syntactic category types for each item. Statistical significance of difference between model performances is tested with two-sided paired t -test. Figure 5 shows quantitative results for each model+training condition. Overall, GibbsComplete is the best-performing model. With pretrained models, GibbsComplete has significantly lower aggregated MSE than fine-tuned InfillT5 ( p = 0 . 012 ) and fine-tuned ILM ( p = 0 . 010 ), and is numerically lower than that of fine-tuned InfillBART ( p = 0 . 108 ). When training from scratch, GibbsComplete is not significantly better. The MSE of GibbsComplete with motifs trained from scratch is not significantly better than other models trained from scratch on NYT , but when training on BLLIP it significantly outperforms InfillT5 ( p = 0 . 017 ) and InfillBART ( p = 0 . 048 ) trained from scratch on BLLIP . Looking at specific LCA categories, we find that GibbsComplete outperforms all the other models ( p < 0 . 005 ) in matching the frequency of NP in human completions. Except when comparing ILM trained on NYT to GibbsComplete on VP ( p = 0 . 041 ), no other directly-specialized models significantly performs better than GibbsComplete for category S, VP, and ADJP. Overall, these results suggest that the statistics of LCA categories in the parsed completions by GibbsComplete with pretrained models better match those of humans than those fine-tuned models, and that the advantage of GibbsComplete may also extend to low-resource setting. may be completed using a variety of different structural configurations, as shown in Figure 4. We parse each completion with benepar (Kitaev and Klein, 2018) and use the syntactic category of the lowest common ancestor (LCA) of w 1 and w 2 in the parse tree as a feature of the completion. We choose 40 Noun-Noun, 40 Adjective-Adjective, and 40 Adjective-Noun combination as w 1 and w 2 respectively, once again with diverse semantic content. We recruited human subjects from Mechanical Turk, with 18 subjects for Noun-Noun condition, 18 subjects for Adjective-Adjective, and 18 subjects for Adjective-Noun. Each subject wrote one completion for every item in the assigned condition. For an item, the completions from the subjects provide the human data from which we estimate the summary statistics of interest. We estimate the LCA frequency distribution across five syntactic category types: S, NP, VP, ADJP, and Other (everything else). For models, we sample and parse 35 completions for each stimulus to estimate the LCA frequency distribution. We have contrasted this with a competing hypothesis, namely that human behavior in novel tasks is better captured by inferences resulting from flexibly composing existing basic computational motifs, which relates to the idea explored in compositional use of neural modules (Andreas et al., 2016). To compare these hypotheses, we have developed and tested new, more challenging, and more open-ended versions of the text infilling task, which we term fragmentary input completion, and evaluated the performance of different models instantiating the two 8182 0.00 0.01 0.02 0.03 0.04 M ean S qua r ed E rr o r P r e t r a i n / F i ne t une Aggregated Mean Squared Error S NP VP ADJP other 0.00 0.02 0.04 0.06 0.08 Mean Squared Error by LCA Category GibbsComplete InfillT5 InfillBART ILM Pretrain/Fine-tune From scratch (NYT) From scratch (BLLIP) Pretrain/Fine-tune From scratch (NYT) From scratch (BLLIP) Noun Noun Adj Adj Adj Noun 0.00 0.01 0.02 0.03 0.04 Mean Squared Error by Stimulus Condition 0.00 0.01 0.02 0.03 0.04 M ean S qua r ed E rr o r F r o m sc r a t c h ( NYT ) S NP VP ADJP other 0.00 0.02 0.04 0.06 0.08 Noun Noun Adj Adj Adj Noun 0.00 0.01 0.02 0.03 0.04 0.00 0.01 0.02 0.03 0.04 M ean S qua r ed E rr o r F r o m sc r a t c h ( BLLIP ) S NP VP ADJP other 0.00 0.02 0.04 0.06 0.08 Noun Noun Adj Adj Adj Noun 0.000.010.020.030.04 Figure 5: Comparing structural statistics in model's completions with that of the human-written completions. hypotheses against subjective human judgments, fixed success criteria, and fine-grained comparisons with human task performance. In the future this approach could be extended further for more comprehensive evaluation of conditioned language generation systems. Our results are generally favorable for the compositional inference hypothesis as exemplified by our novel GibbsComplete algorithm, which composes the two fundamental language modelling motifs central to today's language models, masked word prediction and autoregressive modeling. The motifs themselves need to be learned in the first place, but there is a strong case that these motifs reflect tasks fundamental to everyday language use: identifying an uncertain word using bidirectional context (Connine et al., 1991; Dilley and Pitt, 2010; Levy, 2008b) and predicting upcoming input (Hale, 2001; Levy, 2008a; Kutas et al., 2011; Kuperberg and Jaeger, 2016). The idea we advance here, that these fundamental motifs are pre-existing and flexibly deployed for novel tasks, echoes a long-standing perspective in cognitive science well-summarized by Bruner et al. (1986), that  Thinking is not the acquisition of knowledge, but the use of knowledge in the interest of solving problems . sampling-based approaches may capture important general features of human inferential patterns (Vul et al., 2014). Furthermore, the compositional inference approach to modelling flexible language generation by no means diminishes the value of learning or fine-tuningindeed, fine-tuned models can themselves be composed (see also our exploratory work in Appendix G.2). Rather, we hope that this work may help widen the perspective on the relationship between learning and inference in novel language tasks and contexts. Scaled learning and quick adaptation of linguistic representation have enabled huge progress in the engineering of high-performance NLP systems, but our results suggest that flexible redeployment of basic computational motifs may have advantages for capturing how humans flexibly use language in novel circumstances where they do not have extensive experience. Our studies offer systematic model comparisons with materials designed to highlight subtle features of grammatical knowledge and featural statistics of human completion preferences, and point to the need for longer-term efforts in understanding and modeling human cognitive flexibility in computational terms. As an initial step, we explored the compositional inference hypothesis by sketching out an inference algorithm based on the principles of approximate Bayesian 8183 inference. The results suggest certain advantages of an inference-oriented view of human language generation, and an alternative path towards building models that process linguistic information as flexibly as humans do. We thank the anonymous reviewers and members of the MIT Computational Psycholinguistics Lab for their helpful comments, and members of the Goals, Problems and Stories working group for discussions on early idea of the project. This work was supported by the MIT-IBM Watson AI Lab and by NSF award BCS-2121074.
abstain abstain objective objective method abstain objective abstain abstain abstain abstain abstain objective objective objective abstain abstain abstain abstain objective objective method result result abstain abstain method abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain method other abstain method method abstain other abstain method abstain method abstain method method method method abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain method method method method method method method method abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain result abstain objective result objective abstain other other