Recent work on controlled text generation has either required attribute-based fine-tuning of the base language model (LM), or has restricted the parameterization of the attribute discriminator to be compatible with the base autoregressive LM. In this work, we propose Mix and Match LM, a global score-based alternative for controllable text generation that combines arbitrary pre-trained black-box models for achieving the desired attributes in the generated text without involving any fine-tuning or structural assumptions about the black-box models. We interpret the task of controllable generation as drawing samples from an energy-based model whose energy values are a linear combination of scores from black-box models that are separately responsible for fluency, the control attribute, and faithfulness to any conditioning context. We use a Metropolis-Hastings sampling scheme to sample from this energy-based model using bidirectional context and global attribute features. We validate the effectiveness of our approach on various controlled generation and style-based text revision tasks by outperforming recently proposed methods that involve extra training, fine-tuning, or restrictive assumptions over the form of models. While large transformer-based autoregressive language models trained on massive amounts of data found on the internet exhibit exceptional capabilities to generate natural language text, effective methods for generating text that satisfy global constraints and possess holistic desired attributes remains an active area of research. These mechanisms for controlling the generation of language have the potential to mitigate undesirable biases encoded by the large language models and prevent the generation of hate speech and toxic language (Xu et al.; Gehman et al., 2020; Sap et al., 2021; Baheti et al., 2021; Mireshghallah and Berg-Kirkpatrick, 2021). Much of the prior work has approached controlled generation via either training domain-conditioned neural language models (Prabhumoye et al., 2020; He et al., 2020; Lample et al., 2018; Shen et al., 2017; Krishna et al., 2020; Reif et al., 2021; Ficler and Goldberg, 2017; Khalifa et al., 2021) or finetun-ing/modifying an underlying large pre-trained base model for generation on domain-specific data for attribute sensitive generation (Ziegler et al., 2019; Keskar et al., 2019; Mai et al., 2020; Gururangan et al., 2020; Chronopoulou et al., 2021). Not only do these approaches involve computational overhead and estimation errors associated with the training of language models, but they are also dependent on access to a large amount of attribute-specific language data which can be impractical in many scenarios and exacerbate privacy concerns (Brown et al., 2022; Mireshghallah et al., 2021; Kandpal et al., 2022). Our approach eschews training and focuses on generation-time control from pre-trained modules. Recent work in this space has used attribute discriminators (Dathathri et al., 2020; Krause et al., 2020; Yang and Klein, 2021; Holtzman et al., 2018) to steer the generation from a large autoregressive language model. These discriminators need to be separately trained on partial generations in order to be operationalized with step-wise autoregressive models. As a result, this approach also requires availability of data to train step-wise discriminators for attributes that are essentially global (at the sequence-level) in nature. Therefore, we focus on drawing samples from a test-time combination of pretrained blackbox experts that each score a desired property of output text  for example, fluency, attribute sensitivity, or faithfulness to the context. Specifically, we view the product of these black-box experts as a probabilistic energy model (Hinton, 2002)  i.e., a non-autoregressive , globally normalized language model  and then sample (without further training or fine-tuning) using a specialized Gibbs sampler with a Metropolis-Hastings correction step (Goyal et al., 2021). Our full framework, which we entitle Mix and Match LM (depicted in Figure 1), enables the generation of high-quality attribute-controlled samples by mixing and matching black-box models like off-the-shelf pre-trained attribute-sensitive discriminators (e.g., sentiment classifiers), large bidirectional pre-trained language models like BERT (Devlin et al., 2019), and other modules specializing in capturing desirable features pertaining to faithfulness to any additional context, like hamming distance, or BertScore distance (Zhang et al., 2020) between the sample and the conditioning context. We generate samples from the energy language model assembled from these component experts by using the recently proposed Gibbs-Metropolis-Hastings scheme (Goyal et al., 2021) for sampling from energy models using a masked language model as a proposal distribution. In this scheme, an expressive bidirectional language model like BERT is used to make a proposal at each transition step in the Gibbs chain to jump to a sequence x from the current sequence x . This proposal's fitness is judged by the change in the energy language model's score, with the sampler accepting proposals with larger energy reductions at a higher rate. While the MCMC nature of our sampler negatively impacts the runtime during decoding compared to autoregressive approaches with ancestral sampling, we find our approach to still be practical and yield high-quality diverse samples that respect the distribution induced by the product of expert black-box models. We demonstrate the flexibility of our approach by performing a variety of controlled generation tasks, such as aspect-based text revision, style transfer, and attribute grounded generation and compare it to recently proposed controlled generation approaches that are more resource/data intensive. We observe that our approach, which does not require any gradient optimization and is able to combine arbitrary heterogeneous black-box models, outperforms other approaches according to various automated metrics of fluency, quality, and control, as well as human evaluations. We have provided code, data, and sample generations in this GitHub repository: https://github. com/mireshghallah/mixmatch (see A.1 for details on reproducing the results). The approaches closest in spirit to our work involve steering generation from a base language model with external attribute-sensitive control mechanisms. Plug-and-Play LM (Dathathri et al., 2020) uses discriminators learned from an autoregressive LM's top-level hidden layer to modify the LM's states toward increasing the probability of the desired attribute via gradient ascent at each step. GeDi (Krause et al., 2020) and FUDGE (Yang and Klein, 2021) take a similar approach but train custom step-wise attribute-sensitive discriminators that decide whether the desired attribute is likely to be satisfied by the current generation path. GeDi trains class-conditional language models for these discriminators and hence additionally relies on access to attribute sensitive language data. Kumar et al. (2021) formulate the task of controlled generation as optimizing the base LM's likelihood subject to global differentiable attribute-based constraints by gradient descent over the position-wise simplexes over the vocabulary. DExperts (Liu et al., 2021) is another decoding-time controllable generation approach that modifies the step-wise softmax logits of an autoregressive pre-trained LM with softmax log-402 its of separately trained domain-specific expert autoregressive language models. These approaches require training of custom modules and do not readily enjoy the benefits of incorporating global attribute-based features into the generation mechanism in a simple probabilistic manner. In contrast, our energy-based formulation is not only optimization-free but also fully modular and able to easily incorporate global features, allowing for heterogeneous blackbox experts to be combined with each other. 3 Mix-and-match Language Models In this section, we describe our approach and motivation behind our method. Specifically, we frame the problem of performing controlled generation as a problem of sampling from a specialized energy-based (or globally normalized) sequence model that defines a probability distribution that satisfies the desired constraints we wish to impose in the controlled generation setting. As described below, this energy-based model is composed of pre-trained components and does not require any further optimization. An energy-based sequence model defines the probability distribution over the space of possible sequences X as: 1 p ( X ;  ) = e  E ( X ;  ) (cid:80) X X e  E ( X  ;  ) , where E ( X ;  ) refers to the scalar energy of a sequence X that is parametrized by  . Lower energy corresponds to the higher likelihood of X . In contrast to the common autoregressive sequence models, exact likelihood computation and efficient sampling from these models is challenging. Despite these challenges, we focus on this paradigm of sequence modeling because energy-based models offer increased flexibility via sequence-level features and constraints. As we discuss next, this capability lets us easily define expressive functions for controlled generation of sequences which is not readily offered by the autoregressive modeling paradigm. 3.1 Product of Experts Energy-based Models and Controlled Generation Our approach is motivated by the perspective that the task of controlled generation requires concentrating probability mass over a small subspace of sequences in X that satisfies various constraints pertaining to fluency, target attributes, and other control variables. Consider the task of generating positive sentiment sentences. This requires satisfaction of two major constraints: (1) The sequence X should be well-formed, (2) The sequence X should express 1 For simplicity, we are concerned with a finite set of sequences limited by some maximum length. positive sentiment. If we have access to two separate probability distributions over X , one for modeling well-formedness ( p 1 ( X ) ) and another for modeling positivity ( p 2 ( X ) ), then a natural solution for controlled generation in this setting would be to draw samples from a probability distribution that is a product of these two distributions i.e. p desire ( X )  p 1 ( X )  p 2 ( X ) . In our approach, we further relax this requirement by assuming access to expert black-boxes that yield scalar non-probabilistic energy scores E 1 and E 2 indicating fitness of a sequence w.r.t. well-formedness and positivity respectively. Under the product of experts framework above the desired probability distribution would take the form: log p desire ( X ) =  ( E 1 ( X ) + E 2 ( X ))  log Z . This expression shows that when working with scalar scores for the expert black-boxes, the product of expert models yields an energy model whose energy is simply the sum of the scalar energy values obtained from the expert models. Inspired by this, we propose a framework for controlled generation that involves linear combinations of various blackbox experts in order to obtain a distribution whose samples satisfy the requirements of a desired controlled generation task: E M&M ( X )= (cid:80) ki =1  i E i ( X ) , where our proposed mix-and-match energy is composed of k expert energy components, which are weighted by scalar hyperparameters  . As shown in Fig. 1, we use the following black-box experts in our experiments as modules that we can add or remove to produce desired behavior: E mlm ( X ) : Recent work has shown that large masked language models (MLM) like BERT can discriminate between well-formed and ill-formed sentences (Zhang et al., 2020) and induce an implicit energy function over the sequences (Goyal et al., 2021). Hence, we use BERT-base as a black-box to model the form and fluency of sentences. Specifically, we use an energy parametrization introduced in Goyal et al. (2021) which is negative of the sum of unnormalized logits iteratively computed at each position obtained via the forward pass of the MLM after masking the corresponding position. E disc ( X ) : This particular expert module refers to the energy obtained via the discriminator for the attributes of interest. What this module returns is the raw logits of the discriminator, for the target attribute. For instance, if we have a sentiment classifier, and want to produce positive sentiment, then E disc ( X )=  log p (+ | X ) . E hamm ( X ; X  ) : For a given sequence X  , this quantity refers to the hamming distance between the sequence X and X  . This penalizes token level deviation from X  which is useful if we are interested in only making minor edits to X  as described later. E fuzzy ( X ; X  ) : Similar to the hamming distance, this quantity refers to the BertScore (Zhang et al., 2020) computed between X and X  which can be viewed as a fuzzy hamming distance that takes semantic similarity into account. To sample from the energy parametrizations described in the previous section, we follow the Metropolis-Hastings (Hastings, 1970) MCMC scheme for sampling from the masked language models introduced by Goyal et al. (2021). While the proposal distribution we use is the same as Goyal et al. (2021) i.e. masked language model's (BERT's) conditionals, the energy parametrizations we use are more suitably designed for controlled generation. We briefly explain the sampling procedure, which involves forming long Markov chains of sequences starting with a random sequence, and following the MH scheme which uses a proposal distribution to propose a new sequence at each step in a chain which is either accepted or rejected based on its fitness to the energy function. The sequences at the end of these chains correspond to samples from the desired energy-based model. Operationally, at each MCMC step, we mask out a token at a random position in the current sequence X in the chain and propose a new sequence  X to transition to by sampling a token from the MLM conditional softmax at the masked position. This proposed sequence is evaluated by its ability to reduce the energy from the current sequence in the chain and is accepted with the probability p (  X ; X ) = min (cid:18) 1 , e  E M&M (  X ) p mlm ( X i | X \ i ) e  E M&M ( X ) p mlm (  X i | X \ i ) (cid:19) . EM & M ( X ) refers to the product of experts energy, i refers to the position chosen for masking, p mlm refers to the MLM's conditional distribution at the [MASK] position. Intuitively, this acceptance probability indicates that the proposed sequence  X is more acceptable if it has lower energy than the current sequence X in the chain and is rare or less likely to be proposed by the proposal distribution again. 3.4 Controlled generation Tasks We use the expert black-box factors and the sampling scheme described above in our framework to perform two kinds of controlled generation tasks. Prompted generation: This task focuses on generating well-formed sentences that start with a specified prompt and also satisfy a target attribute for which we have access to a discriminator. An example task would be to generate positive sentiment sequences starting with This movie . The energy function takes the form: E gen ( X )= E mlm ( X ) +  E disc ( X ) (1)  is a hyperparameter that controls the tradeoff between the MLM score and the discriminator's influence. For MH-based sampling for this task, we initialize the sequence with the starting prompt and the rest of the tokens masked out, which creates a seed text of shape the movie[MASK][MASK]... [MASK] , for the prompt example of the movie . The number of mask tokens depends on the target generation length, and we constrain the sampler to only produce proposals and revise non-prompt tokens, and mark the prompt tokens as frozen. Controlled text revision : This task involves editing a source sequence X  in order to satisfy the desired target attributes exhibited by the generated sequence X . The energy function for this task is: E rev ( X )= E gen ( X )+  E hamm ( X,X  )+  E fuzzy ( X,X  ) (2) This energy function in addition to valuing well-formedness and satisfying target attribute requirements also focuses on maintaining faithfulness to the source sequence X  . For sampling with this energy, we initialize the sequence with the sequence X  to be edited. This sets the length of the target sequence to be the same as the source. In this setup, the sampler can revise all tokens and is not constrained. For both these tasks, we run a separate MCMC chain for each generated sentence for 8 to 15 epochs, depending on the task. An epoch refers to one masking cycle over all the non-frozen positions (selected randomly) of the sequence. We provide full experimental details in appendix Section B, here we provide a brief overview of the tasks, datasets, baselines, and metrics used in the experiments. Controllable debiasing (ROC story cor-pus): We use the subset of the ROC story corpus (Mostafazadeh et al., 2016) test-set that is used by PowerTransformer (Ma et al., 2020) for their evaluations. We use this data for controllable debiasing, a text revision task which aims to correct the implicit and potentially undesirable agency biases in character portrayals, by replacing verbs such as wish" and dream", with pursue" and achieve". Sentiment transfer (Yelp): We use Yelp (Shen et al., 2017) dataset's test-set for the task of sentiment transfer. The test set comprises 1000 sentences, half with positive and half with negative sentiment. We also have a reference set of handwritten sentiment transferred sentences, provided by (He et al., 2020) that we use for reporting evaluation metrics. Formality transfer (GYAFC): We use 1051 sentences from the entertainment and music domain subset of the GYAFC (Rao and Tetreault, 2018) dataset, which contains formal and informal sentences for the task of formality transfer (both directions of formal to informal and informal to formal). Prompted generation: We evaluate our approach on two forms of prompted generation: 1) sentiment controlled generation and 2) topic controlled generation. For sentiment controlled generation, we set Mix and Match LM to generate text with positive or negative sentiment given prompts, by using a Yelp sentiment classifier as discriminator and compare against PPLM (Dathathri et al., 2020) which is a popular sentiment controlled generation method. For topic controlled generation, we compare against FUDGE (Yang and Klein, 2021), and follow their experimental setup consisting of 7 distinct topics and 20 prompts. We use a Huggingface pre-trained bert-base-uncased model as our MLM for yielding E mlm and also providing the proposal distribution in our MH MCMC sampler. For obtaining E disc , we train BERT-based classifiers on the training-set of our datasets to use as our attribute discriminators. We could have used any pre-trained attribute classifier from Huggingface for E disc , but we keep those aside to use as external attribute classifiers for fair evaluation against baselines. For experiments in which we add the BertScore (Zhang et al., 2020) component to the energy, we use the pre-trained roberta-large_L17 model. Finally, for agency score, we use the lexicon provided by (Sap et al., 2017) and check each generated sequence and count the number of target agency verbs that exist there. The count becomes the agency score. PowerTransformer. For the task of controllable debiasing (agency revision), we compare our work with PowerTransformer (Ma et al., 2020), an approach that uses paraphrasing and self-supervision based on a reconstruction loss, building on pre-trained language models, to re-write text and control agency level of sentences. He et al. For style transfer on sentiment an formality, we compare with He et al. (2020), a generative style transfer framework which uses a variational autoencoder (VAE) built using a sequence-to-sequence LSTM-based model to do unsupervised style transfer. This framework needs to be trained from scratch for each style transfer task. UNMT. As a second baseline for style transfer, we use UNMT (Lample et al., 2018), an unsupervised machine translation framework that demonstrates high performance for sentiment transfer. PPLM. For the task of sentiment controlled generation, we compare to Plug-and-Play LM (PPLM) Dathathri et al. (2020), which does attribute controlled generation using the flow of gradients from discriminators trained on the last hidden layer representations of the generator, to guide generation. FUDGE. This approach (Yang and Klein, 2021) trains step-wise discriminators on partial generations from GPT-2 to determine whether the constraints related to desired attributes will be satisfied by the future completion of the sequence or not. We compare against this on topic controlled generation as this approach was shown to be superior to PPLM on this task. We use a variety of evaluation metrics to compare our approach's performance on two major facets: (1) Quality of generated text, and (2) success on matching the target attribute used for control. GPT-2 PPL. We feed our generated test sentences to a Huggingface (Radford et al., 2019) pre-trained GPT-2 xl model, and report its perplexity (PPL), as an automatic measure of fluency. Although this measure is not a perfect indicator of fluency, we find it to be a useful metric alongside human judgements. 2 BLEU. For sentiment (Yelp) and formality (GYAFC) transfer where we have reference text, we 2 Due to the high variance in the PPL scores generated across sentences by GPT-2, we report the median score for each system under comparison. report the BLEU score. For controlled debiasing, we report BLEU between generated text and source and show it as BLEU (src). BertScore. As a measure of meaning preservation, we use the F1 BertScore metric (Zhang et al., 2020) to compare the semantic similarity of the provided reference sentence with the generated output. Internal Classifier Accuracy. We report the accuracy of the internal classifier (the discriminator used for generation) on the generated text, assuming the target attribute is the correct label. The higher this accuracy is, the better. External Classifier Accuracy. It is natural to get high accuracy on the internal classifier, since we are sampling from it. To have a fair comparison, we report accuracy using external classifiers from Huggingface ( textattack/bert-base-uncased-yelp-polarity (Morris et al., 2020) for sentiment and cointegrated/roberta-base-formality for formality). Agency Lexicon Accuracy. For controlled debiasing, we measure the accuracy of the change in agency by comparing the target agency level with that of the generated text, extracted using the connotation frames lexicon, and following the setup from Ma et al. (2020). Tables 1 and 2 show our results for the task of text revision for controlling agency bias which is introduced by PowerTransformer Ma et al. 2020, our Baseline for this task. PowerTransformer has a vanilla (no boost) variant and a variant with vocab boosting, which up-weights the logits of verbs that belong to the target agency lexicon so as to increase their probability and incentivize generation in that direction. We also measure our metrics on the original test-set, without revision, to provide a better sense of the changes made. We offer different variants of our framework, to provide a fair comparison and to better ablate our proposed method. Disc denotes our framework where we add the discriminator expert ( E disc ) which is trained to predict the agency level of a sentence, to the energy along with E mlm , and E hamm (Eq. 2). Hamming distance is computed between the generated proposals and the source sentence. The Agency Score variant adds an alternative term to E M&M instead of E disc , which is the number of target agency verbs according to the connotation frames lexicon (Sap et al., 2017) in the sentence. The Disc+Agency variant has both energy components. We also apply our method in two ways: Verb Replace which allows the sampler to propose revisions for only one pre-determined verb (pro-vided in the dataset). In this setup, all tokens remain frozen, except for the given verb. The conventional mode (M&M LM), however, proposes revisions for all tokens in the sentence and is not constrained. Table 2 shows that in the conventional setup, Mix and Match LM (Disc only) has performance similar to that of PowerTransformer, without boosting. With the Agency Score component, our method outperforms PowerTransformer in terms of accuracy of revision as per the agency lexicon accuracy metric, with negligible loss in meaning (BertScore). The reason behind this better performance in terms of applying target agency accuracy is that our method's sampling is guided by the energy that is directly built on the metrics we care about, as opposed to trying to apply them through paraphrasing and proxies such as vocab boosting, which are employed in the PowerTransformer method. Another important observation here is the difference between Verb Replace and conventional modes. This ablation shows that although our method makes few changes (the average Hamming distance between source and output sentences are between 1 . 37 and 2 . 45 ), it still outperforms a static method that has extra knowledge of the offending verb and focuses on changing only that verb, by a significant margin. In this section we experiment with sentiment and formality transfer, where Sentiment transfer needs fewer changes and formality transfer needs more structural change to the original sentence. We show sample sentences and transfers in Table 1 (we cannot show samples for formality as the dataset is not public). For this task, we include two components in our energy model, the attribute discriminator ( E disc ), to induce the target style, and the hamming distance ( E hamm ), to maintain the meaning of the sentence. We don't include the more complex semantic similarity-related component like E fuzzy , since sentiment transfer can normally be done by making only a few changes to the sentence. We report results with two different variants, one where the discriminator component has a higher coefficient in the energy (Discriminator  ) and one where the hamming distance has a higher coefficient (Hamming  ). In effect, these two show the trade-off between transfer quality and faithfulness to the source sentence. We see in Table 3 that our method, with the hamming component up-weighted, outperforms both the generative baselines in terms of transfer accuracy (Ext. Clsf.) and semantic similarity (BertScore). We can also see Mix and Match LM has higher BLEU score, with respect to the provided handwritten reference sentences. We hypothesize that this superiority is due to the tendency of our model 407 to make minimal revisions that satisfy the product of experts energy model. Therefore, our model can successfully change the style without changing the meaning of the sentence. The generative baselines, however, regenerate the sentence which imposes more change, as can be observed from the hamming distance column (Hamm.(src)) in Table 3. For this task, we include the formality classifier ( E disc ), Hamming distance ( E hamm ), and BertScore ( E fuzzy ) components in the energy formulation, to permit the transfer of style and also maintain the meaning of the sentence. E fuzzy helps with imposing semantic similarity between the source and generated sentences, since Hamming alone isn't sufficient for judging comparable formal and informal sentences. We show results for two setups of our framework, one where the discriminator coefficient is higher (Discriminator  ) and another where the BertScore coefficient is higher (BertScore  ). In Table 4 we have broken down the external classifier accuracy for the different transfer directions of formal to informal (  Inf.) and vice versa. We do this because the  Form. task is generally harder and therefore has lower accuracy. We observe that our method outperforms the baselines in terms of BertScore and BLEU, for similar levels of external classifier accuracy. However, we can see that the GPT-2 PPL of our method is higher than the baselines. The reason behind this is the format and noise in the data. The samples for this dataset are taken from the music and entertainment industry domain and contain some symbols and characters similar to emojis (e.g. :) and ***). This is where the tendency of our approach toward minimal revisions is hurtfulour revisions of text, often do not get rid of all of these symbols, while the baselines' generative methods successfully remove all the superfluous characters because they rewrite sentences from scratch. We generate 560 sequences of different lengths ( 12 , 20 and 50 tokens), given 14 prompts, 2 sentiments, and 20 sequences per sentiment, taken from Dathathri et al. (2020)'s experimental setup. The prompts and sample generations are in the appendix B.9 and A.2, and a full list of generations is in the supplementary material. Table 6 shows our results for this experiment. Here, we have an additional metric, the MLM energy (lower is better), which, like GPT-2, indicates the quality of generated sentences (Salazar et al., 2020) according to BERT. We report this extra metric here since PPLM uses a GPT model for generation, and it is natural that it would measure better on this metric. The table shows that for all lengths of generated sentences, our method is much better at inducing the target sentiment. However, we observe that PPLM performs better in terms of GPT-2 while our method performs better on the MLM energy metric. This suggests the tendency of model-based fluency metrics to be biased toward the corresponding models as the PPLM uses GPT-2 for generation and M & M LM uses BERT. To enable a more conclusive comparison of the text quality, we report results with human evaluations. For these evaluations, we randomly select 10 generated outputs for each prompt, per sentiment (240 overall), and asked three Amazon Turkers per sample pair, which sample they find more fluent. We report the majority vote of the Turkers in the table. The results show that for sequences with lengths 12 and 20, they found our generations more fluent. However, for length 50, the preference rate for M&M drops to 46 . 7% , which shows that our method is superior to PPLM for short/medium length generation, however, PPLM does better at generating longer sequences. We follow FUDGE's (Yang and Klein, 2021) experimental setup which covers 7 topics, given 20 prompts and generate 7  20 sequences of length 20 . To enforce topicality on our generations, we add a topic-based energy, E topic . This energy is essentially the negative count of the number of topic-related words (using the list provided by FUDGE). Table 7 shows the results of this experiment, generations are also provided in A.2. Topic-score (  ) is the usage rate of topic-related words that were used for training and evaluation of topic controlled generation by Yang and Klein in their paper. Grammaticality (  ) is the score of grammaticality given by a Roberta-based CoLA grammaticality model averaged over all outputs (Warstadt et al., 2019). The Div (  ) metrics show the diversity of generated text, over unigrams, bigrams and trigrams. Finally, the human evaluations show human preference, in terms of fluency of the sentences ( B.10). As shown by the table, the fluency of our method is comparable to that of FUDGE, even better in terms 408 Table 5: Samples of prompted sentiment controlled generations, using our Mix and Match LM and PPLM. of human preference and grammaticality judgment. FUDGE has a slightly higher topic score, which is expected since it trains a custom step-wise discriminator for each topic that is optimized for the task. But our approach shows competitive faithfulness to the topics especially considering the fact that prompted GPT-2 generations without the FUDGE discriminators only achieve a topic-score of 0 . 23 . Given that our model's inference procedure involves MCMC sampling, it's reasonable to expect its run-time to be slower than more traditional baselines. For sequences of length 20, we find that our un-optimized implementation requires 8 seconds per generation and 3 seconds per revision  while, in contrast, baseline system PPLM requires 16 seconds and FUDGE requires 0.4 seconds per generation. This is a substantial slowdown compared to FUDGE, but not one that renders the proposed approach impractical in offline settings. Further, faster sampling schemes are beyond the scope of this paper but might be explored in future work to speed up models like M&M LM. We present Mix and Match Language Models (M&M LM), a training-free framework for controlled text generation that can easily mix heterogeneous expert modules. We show that our framework outperforms prior methods on a suite of text revision and attribute-controlled generation tasks. Further, our results indicate that probabilistic energy language models, typically considered intractable, can be used for practical text generation tasks when combined with an appropriate sampling scheme. The authors would like to thank the anonymous reviewers and meta-reviewers for their helpful feedback. We also thank our colleagues at the UCSD/CMU Berg Lab for their helpful comments and feedback. The proposed approach takes steps towards a novel paradigm that might partially mitigate the need for energy-intensive GPU training  potentially leading to positive environmental impact down the line. The approach may also have positive impacts on accessibility as strong computational resources are not required when setting up a new controlled text generation system. We do however acknowledge that strong controlled generation methods that rely on discriminators have the potential to regurgitate sensitive training data and produce harmful outputs and toxic language (Xu et al.; Gehman et al., 2020; Wallace et al., 2020). However, if used properly and for good, we anticipate a positive impact on debiasing and safe generation.
abstain objective abstain method objective abstain abstain other abstain method abstain abstain abstain method method method abstain abstain abstain result objective result other abstain abstain other other other other other other objective objective method other other other other method method objective other method other method method other other other result abstain abstain result abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method method abstain abstain method method abstain abstain method abstain abstain method abstain method method abstain method method abstain method method method abstain abstain method method method method method method method abstain abstain method abstain method abstain abstain method abstain method abstain abstain method method abstain method method abstain method abstain method abstain method abstain method abstain abstain method method abstain method result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain result abstain abstain result abstain result result abstain abstain abstain abstain abstain abstain result abstain abstain abstain result result abstain abstain abstain abstain abstain result abstain abstain result result abstain result result abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain method result result other other abstain abstain method method