We propose a generative model of paraphrase generation, that encourages syntactic diversity by conditioning on an explicit syntactic sketch. We introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-VAE), a method for learning decompositions of dense encodings as a sequence of discrete latent variables that make iterative refinements of increasing granularity. This hierarchy of codes is learned through end-to-end training, and represents fine-to-coarse grained information about the input. We use HRQ-VAE to encode the syntactic form of an input sentence as a path through the hierarchy, allowing us to more easily predict syntactic sketches at test time. Extensive experiments, including a human evaluation, confirm that HRQ-VAE learns a hierarchical representation of the input space, and generates paraphrases of higher quality than previous systems. Humans use natural language to convey information, mapping an abstract idea to a sentence with a specific surface form. A paraphrase is an alternative surface form of the same underlying semantic content. The ability to automatically identify and generate paraphrases is of significant interest, with applications in data augmentation (Iyyer et al., 2018), query rewriting, (Dong et al., 2017) and duplicate question detection (Shah et al., 2018). While autoregressive models of language (in-cluding paraphrasing systems) predict one token at a time, there is evidence that in humans some degree of planning occurs at a higher level than individual words (Levelt, 1993; Martin et al., 2010). Prior work on paraphrase generation has attempted to include this inductive bias by specifying an alternative surface form as additional model input, either in the form of target parse trees (Iyyer et al., 2018; Chen et al., 2019a; Kumar et al., 2020), exemplars (Meng et al., 2021), or syntactic codes x syn x sem z syn z sem q 1 q 2 q 3 (b) Generative model (decoder) Figure 1: The generative models underlying our approach. (Shu et al., 2019; Hosking and Lapata, 2021). Most of these approaches suffer from an all or noth-ing' problem: the target surface form must be fully specified during inference. However, predicting the complete syntactic structure is almost as difficult as predicting the sentence itself, negating the benefit of the additional planning step. In this paper, we propose a generative model for paraphrase generation, that combines the diversity introduced by an explicit syntactic target with the tractability of models trained end-to-end. Shown in Figure 1, the model begins by assuming the existence of some semantic content z sem . Conditioned on this semantic information, the model predicts a syntactic sketch' in the form of a hierarchical set of discrete codes q 1: D , that describe the target syntactic structure with increasing granularity. The sketch is combined into an embedding z syn , and fed along with the original meaning z sem to a de-2489 coder that generates the final output utterance y . Choosing a discrete representation for the sketch means it can be predicted from the meaning as a simple classification task, and the hierarchical na-ture means that the joint probability over the codes admits an autoregressive factorisation, making prediction more tractable. The separation between z sem and z syn is induced by a training scheme introduced in earlier work (Hosking and Lapata, 2021; Huang and Chang, 2021) and inspired by prior work on separated latent spaces (Chen et al., 2019b; Bao et al., 2019), whereby the model must reconstruct a target output from one input with the correct meaning, and another input with the correct syntactic form. To learn the discretized sketches, we propose a variant of Vector-Quantized Variational Autoencoders (VQ-VAE, or VQ) that learns a hierarchy of embeddings within a shared vector space, and represents an input encoding as a path through this hierarchy. Our approach, which we call Hierarchical Refinement Quantized Variational Autoencoders or HRQ-VAE , leads to a decomposition of a dense vector into embeddings of increasing granularity, representing high-level information at the top level before gradually refining the encoding over subsequent levels. Our contributions are summarized as follows:  We propose a generative model of natural language generation, HRQ-VAE, that induces a syntactic sketch to account for the diversity exhibited by paraphrases. We present a parameterization of our generative model that is a novel method for learning hierarchical discretized embeddings over a single latent encoding space. These embeddings are trained end-to-end and jointly with the en-coder/decoder. We use HRQ-VAE to induce hierarchical sketches for paraphrase generation, demonstrating that the known factorization over codes makes them easier to predict at test time, and leads to higher quality paraphrases. Let y be a sentence, represented as a sequence of tokens. We assume that y contains semantic content, that can be represented by a latent variable z sem . Types of semantic content might include the description of an image, or a question intent. However, the mapping from semantics to surface form is not unique: in general, there is more than one way to express the semantic content. Sentences with the same underlying meaning z sem but different surface form y are paraphrases . Standard approaches to paraphrasing (e.g., Bowman et al. 2016) map directly from z sem to y , and do not account for this diversity of syntactic structure. Following recent work on syntax-guided paraphrasing (Chen et al., 2019a; Hosking and Lapata, 2021), and inspired by evidence that humans plan out utterances at a higher level than individual words (Martin et al., 2010), we introduce an intermediary sketching step, depicted in Figure 1b. We assume that the output sentence y is generated as a function both of the meaning z sem and of a syntactic encoding z syn that describes the structure of the output. Moreover, since natural language displays hierarchical organization in a wide range of ways, including at a syntactic level (con-stituents may contain other consituents), we also assume that the syntactic encoding z syn can be decomposed into a hierarchical set of discrete latent variables q 1: D , and that these q d are conditioned on the meaning z sem . This contrasts with popular model architectures such as VAE (Bowman et al., 2015) which use a flat internal representation in a dense Euclidean vector space. Intuitively, our generative model corresponds to a process where a person thinks of a message they wish to convey; then, they decide roughly how to say it, and incrementally refine this decision; fi-nally, they combine the meaning with the syntactic sketch to spell out' the sequence of words making up the sentence. The graphical model in Figure 1b factorizes as p ( y , z sem ) = (cid:88) q 1: D , z syn p ( y | z sem , z syn )  p ( z syn | q 1: D )  p ( z sem )  p ( q 1 | z sem ) D (cid:89) d =2  p ( q d | q <d , z sem ) . (1) Although q 1: D are conditionally dependent on z sem , we assume that z sem may be determined from y without needing to explicitly calculate q 1: D or z syn . We also assume that the mapping from discrete codes q 1: D to z syn is a deterministic func-2490 tion f q  z (  ) . The posterior therefore factorises as  ( z sem , z syn | y ) =  ( z sem | y )   ( z syn | y )   ( q 1 | z syn )  D (cid:89) d =2  ( q d | q <d , z syn ) . (2) The separation between z sem and q 1: D , such that they represent the meaning and form of the input respectively, is induced by the training scheme. During training, the model is trained to reconstruct a target y using z sem derived from an input with the correct meaning (a paraphrase) x sem , and q 1: D from another input with the correct form (a syntactic exemplar) x syn . Hosking and Lapata (2021) showed that the model therefore learns to encode primarily semantic information about the input in z sem , and primarily syntactic information in q 1: D . Exemplars are retrieved from the training data following to the process described in Hosking and Lapata (2021), with examples in Appendix C. The setup is shown in Figure 1a; in summary, during training we set  ( z sem | y ) =  ( z sem | x sem ) and  ( q d | y , q <d ) =  ( q d | x syn , q <d ) . The final objective is given by ELBO = E  (cid:2)  log p ( y | z sem , q 1: D ))  log p ( q 1 | z sem )  D (cid:88) d =2 log p ( q d | q <d , z sem ) (cid:3) + KL (cid:2)  ( z sem | x sem ) || p ( z sem ) (cid:3) , (3) where q d   ( q d | x syn ) and z sem   ( z sem | x sem ) . We assume a Gaussian distribution for z sem , with prior p ( z sem )  N ( 0 , 1 ) . The encoders  ( z sem | x sem ) and  ( z syn | x syn ) are Transformers (Vaswani et al., 2017), and we use an autoregressive Transformer decoder for p ( y | z sem , z syn ) . The mapping f q  z (  ) from q 1: D to z syn and the posterior network  ( q d | q <d , z syn ) are more complex, and form a significant part of our contribution. Our choice of parameterization is learned end-to-end, and ensures that the sketches learned are hierarchical both in the shared embedding space and in the information they represent. Let z syn  RD be the output of the encoder network  ( z syn | y ) , that we wish to decompose as a sequence of discrete hierarchical codes. Recall that q d  [1 , K ] are discrete latent variables corresponding to the codes at different levels in the hierarchy, d  [1 , D ] . Each level uses a distinct codebook, C d  RK  D , which maps each discrete code to a continuous embedding C d ( q d )  RD . The distribution over codes at each level is a softmax distribution, with the scores s d given by the distance from each of the codebook embeddings to the residual error between the input and the cumulative embedding from all previous levels, s d ( q ) =  (cid:32)(cid:34) x  d  1 (cid:88) d (cid:48) =1 C d (cid:48) ( q d (cid:48) ) (cid:35)  C d ( q ) (cid:33) 2 . Illustrated in Figure 2, these embeddings therefore represent iterative refinements on the quantization of the input. The posterior network  ( q d | q <d , z syn ) iteratively decomposes an encoding vector into a path through a hierarchy of clusters whose centroids are the codebook embeddings. HRQ-VAE can be viewed as an extension of VQ-VAE (van den Oord et al., 2017), with two significant differences: (1) the codes are hierarchically ordered and the joint distribution p ( q 1 , . . . , q D ) admits an autoregressive factorization; and (2) the HRQ-VAE composition function is a sum, compared to concatenation in VQ or a complex neural network in VQ-VAE 2 (Razavi et al., 2019). Under HRQ, latent codes describe a path through the learned hierarchy within a shared encoding space. The form of the posterior  ( q d | q <d , z syn ) and the composition function f q  z (  ) do not rely on any particular properties of the paraphrasing task; the technique could be applied to any encoding space. Initialisation Decay Smaller perturbations in encoding space should result in more fine grained changes in the information they encode. Therefore, we encourage ordering between the levels of hierarchy (such that lower levels encode finer grained information) by initialising the codebook with a decaying scale, such that later embeddings have a smaller norm than those higher in the hierarchy. Specifically, the norm of the embeddings at level d is weighted by a factor (  init ) d  1 . Depth Dropout To encourage the hierarchy within the encoding space to correspond to hierarchical properties of the output, we introduce depth dropout , whereby the hierarchy is truncated at each level during training with some probability p depth . The output of the quantizer is then given by z syn = D (cid:88) d =1 (cid:32) C d ( q d ) d (cid:89) d (cid:48) =1  d (cid:48) (cid:33) , (6) where  h  Bernoulli (1  p depth ) . This means that the model is sometimes trained to reconstruct the output based only on a partial encoding of the input, and should learn to cluster similar outputs together at each level in the hierarchy. During training the decoder is driven using sketches sampled from the encoder, but at test time exemplars are unavailable and we must predict a distribution over syntactic sketches p ( q 1: D | z sem ) . Modelling the sketches as hierarchical ensures that this distribution admits an autoregressive factorization. We use a simple recurrent network to infer valid codes at each level of hierarchy, using the semantics of the input sentence and the cumulative embedding of the predicted path so far as input, such that q d is sampled from p ( q d | z sem , q <d ) = Softmax ( MLP d ( z sem , z <d )) , where z <d = d  1 (cid:80) d (cid:48) =1 C d (cid:48) ( q d (cid:48) ) . This MLP is trained jointly with the encoder/decoder model, using the outputs of the posterior network  ( q d | x syn , q <d ) as targets. To generate paraphrases as test time, we sample from the sketch prediction model p ( q d | z sem , q <d ) using beam search and condition generation on these predicted sketches. We use the Gumbel reparameterisation trick (Jang et al., 2016; Maddison et al., 2017; Snderby et al., 2017) for the discrete codes and the standard Gaussian reparameterisation for the semantic representation. To encourage the model to use the full codebook, we decayed the Gumbel temperature  , according to the schedule given in Appendix A. We approximate the expectation in Equation (3) by sampling from the training set and updating via backpropagation (Kingma and Welling, 2014). The full model was trained jointly by optimizing the ELBO in Equation (3). Datasets A paraphrase is an alternative surface form in the same language expressing the same semantic content as the original form' (Madnani and Dorr, 2010), but it is not always clear what counts as the same semantic content'. Our approach requires access to reference paraphrases; we evaluate on three English paraphrasing datasets which have clear grounding for the meaning of each sentence: Paralex (Fader et al., 2013), a dataset of question paraphrase clusters scraped from WikiAn-swers; Quora Question Pairs (QQP) 1 sourced from the community question answering forum Quora; and MSCOCO 2017 (Lin et al., 2014), a set of images that have been captioned by multiple annotators. For the question datasets, each paraphrase is grounded to the (hypothetical) answer they share. We use the splits released by Hosking and Lapata (2021). For MSCOCO, each caption is grounded by the image that it describes. We evaluate on the public validation set, randomly selecting one cap-1 https://www.kaggle.com/c/quora-question-pairs 2492 Figure 3: t-SNE visualisation of the syntactic encodings z syn for 10k examples from Paralex: colours indicate top-level codes q 1 , shapes indicate the second level, and patterns are used to label the third level. Model Configuration Hyperparameters were tuned on the Paralex development set, and reused for the other evaluations. We set the depth of the hierarchy D = 3 , and the codebook size K = 16 . The Transformer encoder and decoder consist of 5 layers each, and we use the vocabulary and token embeddings from BERT-Base (Devlin et al., 2018). We use an initialisation decay factor of  init = 0 . 5 , and a depth dropout probability p depth = 0 . 3 . A full set of hyperparameters is given in Appendix A, and our code is available at https://github.com/tomhosking/hrq-vae . Comparison Systems As baselines, we consider three popular architectures: a vanilla autoencoder (AE) that learns a single dense vector representation of an input sentence; a Gaussian Variational AutoEncoder (VAE, Bowman et al., 2015), which learns a distribution over dense vectors; and a Vector-Quantized Variational AutoEncoder (VQ-VAE, van den Oord et al., 2017), that represents the full input sentence as a set of discrete codes. All three models are trained to generate a sentence from one of its paraphrases in the training data, and are not trained with an autoencoder objective. We implement a simple tf-idf baseline (Jones, 1972), retrieving the question from the training set with the highest cosine similarity to the input. Finally, we include a basic copy baseline as a lower bound, that simply uses the input sentences as the output. We also compare to a range of recent paraphrasing systems. Latent bag-of-words (BoW, Fu et al., 2019) uses an encoder-decoder model with a discrete bag-of-words as the latent encoding. SOW/REAP (Goyal and Durrett, 2020) uses a two stage approach, deriving a set of feasible syntactic rearrangements that is used to guide a second encoder-decoder model. BTmPG (Lin and Wan, 2021) uses multi-round generation to improve diversity and a reverse paraphrasing model to preserve semantic fidelity. We use the results after 10 rounds of paraphrasing. Separator (Hosking and Lapata, 2021) uses separated, non-hierarchical encoding spaces for the meaning and form of an input, and an additional inference model to predict the target syntactic form at test time. All comparison systems were trained and evaluated on our splits of the datasets. As an upper bound, we select a sentence from the evaluation set to use as an oracle syntactic exemplar, conditioning generation on a sketch that is known to represent a valid surface form. Our experiments were designed to test two primary hypotheses: (1) Does HRQ-VAE learn hierarchical decompositions of an encoding space? and (2) Does our choice of generative model enable us to generate high quality and diverse paraphrases? Figure 3 shows a t-SNE (van der Maaten and Hinton, 2008) plot of the syntactic encodings z syn for 10,000 examples from Paralex. The encodings are labelled by their quantization, so that colours indicate top-level codes q 1 , shapes denote q 2 , and patterns q 3 . The first plot shows clear high level structure, with increasingly fine levels of substructure visible as we zoom into each cluster. This confirms that the discrete codes are ordered, with lower levels in the hierarchy encoding more fine grained information. To confirm that intermediate levels of hierarchy represent valid points in the encoding space, we generate paraphrases using oracle sketches, but truncate the sketches at different depths. Masking one level (i.e., using only q 1 , q 2 ) reduces performance by 2 . 5 iBLEU points, and two levels by 5 . 5 . (iBLEU is an automatic metric for assessing paraphrase quality; see Section 5.2). Although encodings using the full depth are the most informative, partial encodings still lead to good quality output, with a gradual degradation. This implies both that each level in the hierarchy contains useful information, and that the cluster centroids at each level are representative of the individual members of those clusters. Metrics Our primary metric is iBLEU (Sun and Zhou, 2012), that measures the fidelity of generated outputs to reference paraphrases as well as the level of diversity introduced. We use the corpus-level variant. Following the recommendations of Sun and Zhou (2012), we set  = 0 . 8 , with a sensitivity analysis shown in Appendix A. We also report BLEU ( outputs, references ) as well as Self-BLEU ( outputs, inputs ) . The latter allows us to examine the extent to which models generate paraphrases that differ from the original input. To evaluate the diversity between multiple candidates generated by the same system , we report pairwise-BLEU (Cao and Wan, 2020), P-BLEU = E i (cid:54) = j [ BLEU ( outputs i , outputs j )] . This measures the average similarity between the different candidates, with a lower score indicating more diverse hypotheses. Automatic Evaluation Shown in Table 1, the results of the automatic evaluation highlight the importance of measuring both paraphrase quality and similarity to the input: the Copy baseline is able to achieve high BLEU scores despite simply duplicating the input. The VAE baseline is competitive but tends to have a hi gh Self-BLEU score, indicating that the semantic preservation comes at the cost of low syntactic diversity. HRQ-VAE achieves both higher BLEU scores and higher iBLEU scores than the comparison systems, indicating that it is able to generate 2494 q 1 q 2 q 3 Output Input Two types of fats in body ? 0 3 6 What types of fats are in a body? 13 7 What types of fats are there in body? 2 1 2 How many types of fats are there in the body? 3 7 How many types of fats are there in a body? 5 3 6 What are the different types of fats in a body? 5 7 What are the different types of fats in body? 8 7 Types of fats are different from body fat? 14 Two types of fats in body? 13 0 2 What are the different types of fats in the body? 6 What are the different types of fats in a body? 3 7 What are two types of fats in a body? 5 7 What are the different types of fats in body? 8 What are the different types of fats? 14 What are the different types of fats in the body? The examples in Table 2 demonstrate that HRQ is able to introduce significant syntactic variation while preserving the original meaning of the input. However, there is still a gap between generation using predicted sketches and oracle' sketches (i.e., when the target syntactic form is known in advance), indicating ample scope for improvement. Worked Example Since the sketches q 1: D are latent variables, interpretation is difficult. However, a detailed inspection of example output reveals some structure. Table 3 shows the model output for a single semantic input drawn from Paralex, across a range of different syntactic sketches. It shows that q 1 is primarily responsible for encoding the question type, with q 1 = 13 leading to what' questions and q 1 = 2 how' questions. q 2 and q 3 encode more fine grained details; for example, all outputs shown with q 3 = 6 use the indefinite article a'. We also examine how using increasingly granular sketches refines the syntactic template of the output. Table 4 shows the model output for a single semantic input, using varying granularities of sketch extracted from the exemplar. When no sketch is specified, the model defaults to a canonical phrasing of the question. When only q 1 is specified, the output becomes a how many' question, and Input Two types of fat in body? Generating Multiple Paraphrases We evaluated the ability of our system to generate multiple diverse paraphrases for a single input, and compared to the other comparison systems capable of producing more than one output. For both HRQ-VAE and Separator, we used beam search to sample from the sketch prediction network as in the top-1 case, and condition generation on the top-3 hypotheses predicted. For BTmPG, we used the paraphrases generated after 3, 6 and 10 rounds. For the VAE, we conditioned generation on 3 different samples from the encoding space. The results in Table 5 show that HRQ-VAE is able to generate multiple high quality paraphrases for a single input, with lower similarity between the candidates than other systems. In addition to automatic evaluation we elicited judgements from crowdworkers on Amazon Mechanical Turk. They were shown a sentence and two paraphrases, each generated by a different system, and asked to select which one was preferred along three dimensions: the dissimilarity of the paraphrase compared to the original sentence; how 2495 Meaning Dissimilarity Fluency 20 0 20 R e l a t i v e p r e f e r e n c e % +36 -16 -24 +4 -33 +9 +27 -3 +22 -24 -5 +8 VAE Latent BoW Separator HRQ-VAE (ours) Figure 4: Results of our human evaluation. well the paraphrase reflected the meaning of the original; and the fluency of the paraphrase (see Appendix B). We evaluated a total of 300 sentences sampled equally from each of the three evaluation datasets, and collected 3 ratings for each sample. We assigned each system a score of +1 when it was selected,  1 when the other system was selected, and took the mean over all samples. Negative scores indicate that a system was selected less often than an alternative. We chose the four best performing models for our evaluation: HRQ-VAE, Separator, Latent BoW, and VAE. Figure 4 shows that although the VAE baseline is the best at preserving question meaning, it is also the worst at introducing variation to the output. HRQ-VAE better preserves the original question intent compared to the other systems while introducing more diversity than the VAE, as well as generating much more fluent output. To confirm that the hierarchical model allows for more expressive sketches, we performed two ablations. We compared to the full model using oracle sketches, so that code prediction performance was not a factor. We set the depth D = 1 and K = 48 , giving equivalent total capacity to the full model ( D = 3 , K = 16 ) but without hierarchy. We also removed the initialisation scaling at lower depths, instead initialising all codebooks with the same scale. Table 6 shows that a non-hierarchical model with the same capacity is much less expressive. We also performed two ablations against the model using predicted sketches; we removed depth dropout, so that the model is always trained on a full encoding. We confirm that learning the code-Variant Paralex QQP MSCOCO HRQ-VAE (oracle) 34.85 33.01 26.07 No initialisation scaling  3.06  2.48  3.02 No hierarchy  8.84  12.72  3.10 HRQ-VAE 24.93 18.42 19.04 No head dropout  0.62  0.74  0.81 Post-hoc k-means  3.30  5.35  2.83 Table 6: Changes in iBLEU score for a range of ablations from our full model. books jointly with the encoder/decoder leads to a stronger model, by first training a model with a continuous Gaussian bottleneck (instead of the HRQ-VAE); then, we recursively apply k -means clustering (Lloyd, 1982), with the clustering at each level taking place over the residual error from all levels so far, analogous to HRQ-VAE. The results of these ablations shown in Table 6 indicate that our approach leads to improvements over all datasets. Hierarchical VAEs VQ-VAEs were initially proposed in computer vision (van den Oord et al., 2017), and were later extended to be hierarchical' (Razavi et al., 2019). However, in vision the term refers to a stacked' version architecture, where the output of one variational layer is passed through a CNN and then another variational layer that can be continuous (Vahdat and Kautz, 2020) or quantized (Williams et al., 2020; Livin et al., 2019; Willetts et al., 2021). Unlike these approaches, we induce a single latent space that has hierarchical properties. Other work has looked at using the properties of hyperbolic geometry to encourage autoencoders to learn hierarchical representations. Mathieu et al. (2019) showed that a model endowed with a Poincar ball geometry was able to recover hierarchical structure in datasets, and Surs et al. (2021) used this property to deal with uncertainty in predicting events in video clips. However, their work was limited to continuous encoding spaces, and the hierarchy discovered was known to exist a priori. Syntax-controlled Paraphrase Generation Prior work on paraphrasing has used retrieval techniques (Barzilay and McKeown, 2001), Residual LSTMs (Prakash et al., 2016), VAEs (Bowman et al., 2016), VQ-VAEs (Roy and Grangier, 2019) and pivot languages (Mallinson et al., 2017). Syntax-controlled paraphrase generation has seen significant recent interest, as a 2496 means to explicitly generate diverse surface forms with the same meaning. However, most previous work has required knowledge of the correct or valid surface forms to be generated (Iyyer et al., 2018; Chen et al., 2019a; Kumar et al., 2020; Meng et al., 2021). It is generally assumed that the input can be rewritten without addressing the problem of predicting which template should be used, which is necessary if the method is to be useful. Hosking and Lapata (2021) proposed learning a simplified representation of the surface form using VQ, that could then be predicted at test time. However, the discrete codes learned by their approach are not independent and do not admit a known factorization, leading to a mismatch between training and inference. We present a generative model of paraphrasing, that uses a hierarchy of discrete latent variables as a rough syntactic sketch. We introduce HRQ-VAE, a method for mapping these hierarchical sketches to a continuous encoding space, and demonstrate that it can indeed learn a hierarchy, with lower levels representing more fine-grained information. We apply HRQ-VAE to the task of paraphrase generation, representing the syntactic form of sentences as paths through a learned hierarchy, that can be predicted during testing. Extensive experiments across multiple datasets and a human evaluation show that our method leads to high quality paraphrases. The generative model we introduce has potential application for any natural language generation task; z sem could be sourced from a sentence in a different language, from a different modality (e.g., images or tabular data) or from a task-specific model (e.g., summarization or machine translation). Furthermore, HRQ-VAE makes no assumptions about the type of space being represented, and could in principle be applied to a semantic space, learning a hierarchy over words or concepts. We thank our anonymous reviewers for their feedback. This work was supported in part by the UKRI Centre for Doctoral Training in Natural Language Processing, funded by the UKRI (grant EP/S022481/1) and the University of Edinburgh. Lapata acknowledges the support of the European Research Council (award number 681760, Trans-lating Multiple Modalities into Text).
objective abstain abstain method abstain abstain abstain abstain abstain other abstain abstain abstain abstain objective abstain abstain abstain abstain other objective abstain objective objective objective objective abstain method abstain abstain abstain abstain method method method abstain method abstain abstain method method abstain abstain abstain abstain other other method method abstain method abstain abstain abstain abstain other abstain abstain other abstain abstain abstain method abstain method abstain abstain method abstain method abstain method method method abstain abstain method abstain method abstain other abstain method method method abstain abstain other method abstain method method method abstain abstain abstain method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain result other other abstain other other other other other other other other other method objective method result abstain abstain other other other