Paraphrase generation has been widely used in various downstream tasks. Most tasks benefit mainly from high quality paraphrases, namely those that are semantically similar to, yet linguistically diverse from, the original sentence. Generating high-quality paraphrases is challenging as it becomes increasingly hard to preserve meaning as linguistic diversity increases. Recent works achieve nice results by controlling specific aspects of the paraphrase, such as its syntactic tree. However, they do not allow to directly control the quality of the generated paraphrase, and suffer from low flexibility and scalability. Here we propose QCPG, a quality-guided controlled paraphrase generation model, that allows directly controlling the quality dimensions. Furthermore, we suggest a method that given a sentence, identifies points in the quality control space that are expected to yield optimal generated paraphrases. We show that our method is able to generate paraphrases which maintain the original meaning while achieving higher diversity than the uncontrolled baseline. The models, the code, and the data can be found in https://github.com/IBM/quality-c ontrolled-paraphrase-generation . Paraphrase generation, namely rewriting a sentence using different words and/or syntax while preserving its meaning (Bhagat and Hovy, 2013), is an important technique in natural language processing, that has been widely used in various downstream tasks including question answering (Fader et al., 2014a; McCann et al., 2018), summarization (Rush et al., 2015), data augmentation (Yu et al., 2018) and adversarial learning (Iyyer et al., 2018). However, not all paraphrases are equally useful. For most real-world applications, paraphrases which are too similar to the original sentence are of limited value, while those with high linguistic diversity, High Quality Region ! i.e. with large syntactic/lexical differences between the paraphrase and the original sentence, are more beneficial to the robustness and accuracy of automatic text evaluation and classification, and can avoid the blandness caused by repetitive patterns (Qian et al., 2019). The quality of paraphrases is often evaluated using three dimensions, where high quality paraphrases are those with high semantic similarity as well as high lexical and/or syntactic diversity (McCarthy et al., 2009). Generating high quality paraphrases can be challenging (for both humans and automatic models) since it is increasingly difficult to preserve meaning with increasing linguistic diversity. Indeed, when examining the quality of paraphrases among paraphrase generation datasets, one can find a wide range of paraphrase qualities, where the area of high quality is often very sparse (see Figure 1). This in turn results in scarcity of supervised data for high-quality paraphrase generation. A recent approach aiming to produce high quality paraphrases is controlled paraphrase generation, which exposes control mechanisms that can be manipulated to produce diversity. While the controlled generation approaches have yielded impressive results, they require providing the model with very specific information regarding the target sentence, such as its parse tree (Iyyer et al., 2018) or the list of keywords it needs to contain (Zeng et al., 2019). However, for most downstream applications, the important property of the paraphrase is its overall quality, rather than its specific syntactic or lexical form. The over-specificity of existing control-based methods not only complicates their usage and limits their scalability, but also hinders their coverage. Thus, it would be desirable to develop a paraphrase generation model, which uses a simple mechanism for directly controlling paraphrase quality, while avoiding unnecessary complications associated with fine-grained controls. In this paper we propose QCPG, a Quality Controlled Paraphrase Generation model, that given an input sentence and quality constraints, represented by a three dimensional vector of semantic similarity, and syntactic and lexical distances, produces a target sentence that conforms to the quality constraints. Our constraints are much simpler than previously suggested ones, such as parse trees or keyword lists, and leave the model the freedom to choose how to attain the desired quality levels. Enabling the direct control of the three quality dimensions, allows flexibility with respect to the specific requirements of the task at hand, and opens a range of generation possibilities: paraphrases of various flavors (e.g. syntactically vs. lexically di-verse), quasi-paraphrases (with lower semantic sim-ilarity), and even non-paraphrases which may be useful for downstream tasks (e.g. hard negative examples of sentences that are linguistically similar but have different meanings (Guo et al., 2018; Reimers and Gurevych, 2020)). Furthermore, even though the training data is of mixed quality, and exhibits scarcity in the high quality area (see Figure 1), our model is able to learn high quality paraphrasing behavior, i.e. it increases the linguistic diversity of the generated paraphrases without decreasing the semantic similarity compared to the uncontrolled baseline. In this section we provide a general description of our approach. We first explain how the different quality dimensions are measured. We then describe the controlled paraphrase generation model, QCPG, and finally we suggest a method that given the task requirements, detects the input control values which maximize the quality of the generated paraphrases. Figure 2 summarizes our proposed solution for generating controlled paraphrases, which is detailed in the rest of the section. 2.1 Quantifying Paraphrase Quality The most common dimensions for measuring paraphrase quality are the semantic, syntactic and lexical dimensions. Several previous works used also a fluency evaluation metric (Siddique et al., 2020). However, since our focus is on the supervised setting, we rely on the gold paraphrases as fluency guidance for the model (Mc-Carthy et al., 2009). Thus, given a sentence s and a paraphrase s  , we define the paraphrase quality as a three dimensional vector q ( s, s  ) = ( q sem ( s, s  ) , q syn ( s, s  ) , q lex ( s, s  )) , where q sem is a measure of semantic similarity, and q syn and q lex are measures of syntactic and lexical variation, respectively. For the syntactic score, inspired by Iyyer et al. (2018) we choose q syn ( s, s  ) to be the normalized tree edit distance (Zhang and Shasha, 1989) between the third level constituency parse-trees of s and s  , after removing the tokens to increase the decoupling from the lexical distance metric. We define the lexical score q lex ( s, s  ) to be the normalized character-level minimal edit distance between the bag of words. This measure is independent of word order, and hence increases the decoupling from syntactic measures. Additionally, calculating the token distances on the character level enables to capture tokens that share the same stem/lemma. Character-level distance is also more robust to typos that may be found in noisy data. As for the semantic score, several strong metrics have been recently proposed for measuring semantic similarity between sentences. In order to select q sem ( s, s  ) , we studied the agreement between the candidate metrics and human judgments, using only development data, and found Bleurt (Sellam et al., 2020) to have the highest correlation with human judgments (see Appendix A). Thus, we define 597 Figure 2: Solution Architecture. For ease of presentation all metrics are presented on a 0  100 scale. The main component of our solution is a quality controlled paraphrase generation model (QCPG), which is an encoder-decoder model trained on the task of controlled paraphrase generation. Given an input sentence s and a control vector c = ( c sem , c syn , c lex ) , the goal of QCPG is to generate an output paraphrase QCP G ( s, c ) that conforms to c . We train QCPG using the training set pairs ( s, t ) , by setting c to be q ( s, t ) , and maximizing P ( t | s, c = q ( s, t )) over the training set via the autoregressive cross entropy loss. A major challenge in the research of controlled paraphrase generation, is selecting appropriate input control values that can be achieved by the model (Goyal and Durrett, 2020). Clearly, given a sentence, not all paraphrase qualities are achievable. Some sentences are more amenable to paraphrasing than others. For example, named entities and numbers are much harder to be replaced while keeping sentence meaning, and hence, the potential lexical diversity of paraphrases involving such terms is relatively limited. Forcing QCPG to conform to quality control values that are too high with respect to the input sentence, may lead to suboptimal quality of the resultant paraphrases. Thus, for a more effective use of QCPG, the control values should be determined with respect to the input sentence. Below we describe the second part of our solution, namely a method that given a sentence, predicts the input control values, c ( s ) , that optimize the expected quality of the paraphrases generated by QCPG. For simplicity we assume that the quality distribution p ( q | s ) of all paraphrases of sentence s , is approximately normally distributed around a sentence dependent mean q 0 ( s ) , and that the variance is approximately sentence-independent. We further assume that given an input sentence s , the difficulty to generate a paraphrase of a given quality, q , is dominated by p ( q | s ) rather than by the quality vector q itself. Following our assumptions, the level of difficulty can be expressed by the offset, o = ( o sem , o syn , o lex ) of q from q 0 ( s ) . Thus, the input control, c ( s ) , for QCPG, is the sum of q 0 ( s ) and an offset o . Our aim is to analyze the model results for varying levels of difficulty, namely under different offsets, o , from q 0 ( s ) . The Quality Predictor (QP): Since q 0 ( s ) is unknown, we introduce QP, a regressor whose output, termed the reference of s , r ( s ) = ( r sem ( s ) , r syn ( s ) , r lex ( s )) , approximates q 0 ( s ) . During training, QP aims to predict q ( s, t ) given s , where ( s, t ) are the input-output pairs of the training data. To summarize, we define sentence-aware quality control by decomposing the QCPG input control, c , into a sum of a sentence dependent reference 598 point, r ( s ) , and a sentence independent offset, o . To test the ability of our model to learn high quality behavior from mixed quality data we use weakly annotated datasets. These datasets are large but noisy, and contain only a relatively small amount of high quality paraphrases. MSCOCO : This dataset consists of 123K images, where each image contains at most five human-labeled captions (Lin et al., 2014). Similar to previous works we consider different captions of the same image as paraphrases. WikiAnswers (WikiAns for short) : The WikiAnswers corpus contains clusters of questions tagged by wiki-answers.com users as similar. There are 30 , 370 , 994 clusters with 25 question in each on average. In total, the corpus contains over 70 million question pairs (Fader et al., 2014b). ParaBank2.0 : A dataset containing clusters of sentential paraphrases, produced from a bilingual corpus using negative constraints, inference sampling, and clustering (Hu et al., 2019). The dataset is composed of avarage of 5 paraphrases in every cluster and close to 100 million pairs in total. To get comparable results across all datasets, we randomly sub-sampled ParaBank2.0 and WikiAns to the same size as MSCOCO, and split them to train, dev and test sets, of sizes 900 K , 14 K and 14 K respectively. We carefully made sure that there are no pairs from the same cluster in different splits of the data. The full data splits will be published with our code. All models are trained with batch size of 32 on 2 NVIDIA A100 GPUs for 6 epochs. Full details as well as train and dev results can be found in Appendix C.1. QCPG: We use the pre-trained T5-base (Raffel et al., 2020) as the encoder-decoder model. The control input vector to QCPG is quantized at every dimension into 20 equally spaced values ranging from 0 to 100 . Each value is assigned to a special saved-token. The three tokens corresponding to the quantized values of the control vector c , are concatenated to the head of the input sentence, and together used as input to the model. r ( s ) and o are also quantized in a similar way. QP: An Electra base model (Clark et al., 2020) finetuned with MSE loss to predict the typical quality values (see Section 2.3). For all the models, we adopt the experimental setup used in (Devlin et al., 2019), i.e. we train the model with several learning rates and choose the one that achieves the highest dev set performance (see appendix C.1). The aim of the following analysis is to study the level of control achieved by QCPG. To this end, we measure the model response to changes in the input offsets. We compute the expected difference in paraphrase quality, as a result of applying an input offset o compared to zero offset as a reference. More formally, we define the 3-dimensional responsiveness vector of QCPG at an offset o , R ( o ) as Q ( o )  Q ((0 , 0 , 0)) , where Q ( o ) is the expected quality of the paraphrases generated by QCPG at an offset o . We estimate Q ( o ) by averaging q ( QCP G ( s, r ( s ) + o )) over the input sentences s of the dev set, and denote this estimate by Q ( o ) = ( Q sem ( o ) , Q syn ( o ) , Q lex ( o )) , and the corresponding estimate of R ( o ) by R ( o ) . Specifically, in the following analysis we are interested in studying the model response to each of the dimensions separately, i.e. how changing the input offset along a given quality dimension dim  the controlled dimension  while keeping the two other dimensions constant, affects the responsiveness in each of the three dimensions. A good control mechanism would imply that increasing the input offset in one dimension will result in a monotonically increasing responsiveness in that dimension, with relatively small responsiveness in the other two dimensions. Figure 3 shows, for each of the three datasets, the responsiveness in the three quality dimensions, when changing the input offset along each of the three dimensions, while fixing the input offsets in the other two dimensions at 0 . Examining the actual values of quality in the paraphrases of the dev sets, reveals that the standard deviation is different in each dimension. Hence, for clarity of presentation, we present the input offset values and the responsiveness in units of standard deviation as measured in the respective dimension and dev set. For the range of offsets displayed in Figure 3, the responsiveness in the controlled dimension increases monotonically with the input offsets across all datasets and dimensions. As expected, the responsiveness in the uncontrolled dimensions does not zeros due to the inherent coupling between the dimensions. For example, many changes that increase syntactic diversity, also increase lexical diversity (e.g. a move from passive to active voice). Still, our control mechanism is able to increase the responsiveness in the controlled dimension with relative low responsiveness in the uncontrolled dimensions. Specifically, focusing on the relation between semantic similarity and expression diversity, the figure shows that there is a minor decrease in semantic similarity in response to an increase in lexical and syntactic diversity. In the next section, we will show that this does not prevent our model from generating paraphrases that are not only more lexically and syntactically diverse, but also more semantically similar to the source sentences, compared to the paraphrases generated by the uncontrolled baseline. Figure 3 focused on small to moderate input offsets, i.e. offsets up to 2 stds from the reference point. However, as we speculated before, with increasing offsets, i.e. the more the requested control value deviates from the typical value, it becomes increasingly difficult to generate a paraphrase that conforms to the requested control value. Figure 4 depicts the responsiveness in the syntactic and lexical dimensions for a larger range of offset values. For the semantic dimension, the typical values are too high to allow large positive offsets, which for most sentences result in exceeding the upper limit of the semantic score. Indeed, as can be seen in Figure 4, when moving to high offset values, the responsiveness in the syntactic and lexical dimensions starts to decrease. This behavior is in line with our aforementioned hypothesis, and reflects the detrimental effect of feeding QCPG with input control values that are too far from the typical paraphrase qualities of the input sentence. The non-monotonic behavior of the responsiveness implies that the input offsets should be selected carefully in order to optimize the quality of the resultant paraphrases. In Section 4.2 we suggest a method for identifying these optimal offsets. In this section, we suggest a method that given task requirements, selects the input offsets that are expected to yield the desired quality of paraphrases. The idea is to compute the estimated expected quality, Q ( o ) , for each input offset o , using the dev set as described in Section 4.1, and then search the 3D grid of input offsets to find the point for which Q ( o ) is best suited for the user's requirements. We envision this analysis as a preliminary step in which the user chooses the input control parameters that best achieve his desired paraphrasing operation point, and then uses the chosen values at inference  which is why we use the dev set. We study the behavior of Q ( o ) as a function of the 3D grid of offset points in the relevant range, i.e every o where o sem , o syn and o lex in 0 , 5 , 10 ... 50 . Figure 5 depicts Q ( o ) for WikiAns, on a slice of the full offset grid. The results for the full grid on all datasets are shown in Figure 6. The right-hand-side map depicts the estimated linguistic diversity (the average of Q syn ( o ) and Q lex ( o ) ) and the left-hand-side depicts the semantic similarity, Q sem ( o ) ). The maps are presented for o sem = 50 , and for different values of o syn and o lex . As expected, the two measures are anti-correlated, where areas with increased semantic similarity are characterized by decreased linguistic diversity. The QCPG results are compared to two reference points, which are invariant to o and are marked on the colorbars with black squares: 'Dataset' is the semantic-similarity/linguistic-diversity average value over the corresponding dev set paraphrases, and 'Baseline' is the average semantic-similarity/linguistic-diversity of the uncontrolled baseline over the corresponding dev set. Notice that the average diversity level achieved by the uncontrolled baseline is lower than that of the dev set mean, reflecting the difficulty of this model to generate diverse paraphrases. QCPG on the other hand, with suitable input offset values, is able to generate paraphrases which are on average higher than the baseline both in their linguistic diversity and in their semantic similarity, and in fact even higher in many cases than the values of the ground truth paraphrases in the dev-set. In general, the estimates of the expected quality achieved by QCPG at different input offsets, enable a user to generate paraphrases at different operation points, by manipulating the input offset control o to meet her desired quality values. Con-600 !"#$%&'()*+,-*' ./)0)0 1$2$3+4 /5+,&%,$%()*+,-*' /"6&+,$%()*+,-*' 7 ( 8 " 4 9*+ 4 $ : "+" 44 ; <&-&=&+2>?@ sider for example a typical use case, of aiming to maximize linguistic diversity under a constraint on semantic similarity. An example of such a case is an operation point, denoted by QCP G  , which aims to exemplify the advantage of QCPG over the baseline, by maximizing linguistic diversity under the constraint that the semantic similarity is at least 5 points higher than the baseline. The input offset values to obtain this operation point depend on the dataset, and can be found using heatmaps such as in Figure 5. For WikiAns the input offset for the QCP G  operation point values are (50 , 35 , 5) (entry marked by the black square). In the previous section we saw, using estimates based on the dev sets, that there are many operation points which generate paraphrases with higher quality than those achieved by the uncontrolled baseline. We now turn to evaluate one such operation point, namely QCP G  , using the source sentences of the test sets which were not used in the selection of the input offset values. Automatic Evaluation We use four quality measures to evaluate different aspects of generated paraphrases. The three quality measures used in the control of QCPG (Section 2.1) and Self-BLEU (Zhu et al., 2018) as adapted in Li et al. (2019); Liu et al. (2020a), which aims to measure the linguistic diversity in the generated paraphrases by penalizing copying from input sentences. As can be seen in Table 1, QCP G  outperforms the baseline in all metrics across all datasets, as predicted using the dev-set heatmaps. A clear advantage is obtained even for Self-BLEU, which was not part of the metrics used as input controls. Importantly, the quality of the paraphrases generated by our model is comparable to, or at times better than the quality of the paraphrases in the ground truth of the datasets. Examples of paraphrases generated by QCP G  compared to the ground truth paraphrases appear in Table 10. This is an important step towards the goal of obtaining paraphrases in the sparse area of high quality (recall the top right corner of Figure 1). Additionally, we examined QCPG from another perspective: the effect of the quality guidance on the model's ability to predict the ground truth paraphrases. Tables 5 and 6 show the BLEU scores (Papineni et al., 2002) obtained by QCPG and the uncontrolled baseline respectively. The results verify that the input quality vectors induced by the target sentences are effectively utilized by QCPG to achieve better prediction performance. Human Evaluation While linguistic diversity can be automatically measured by reliable metrics such as Self-BLEU, measuring semantic similarity is more challenging. We therefore rely on automatic metrics for evaluating the lexical and syntactic diversity, but use human annotation for validating the semantic evaluation. To this end, we selected a sample of 50 source sentences from each test set, and generated one paraphrase using the uncontrolled baseline and one using QCP G  . The 602 MSCOCO WikiAns ParaBank2 q sem  q syn  q lex  Self-BLEU  q sem  q syn  q lex  Self-BLEU  q sem  q syn  q lex  Self-BLEU  Gold 29.9 34.5 28.0 8.7 34.6 30.7 24.4 16.4 75.0 18.5 20.9 23.9 BL 50.0 27.8 23.0 18.8 46.6 24.7 20.9 23.4 77.8 16.8 18.6 29.4 QCPG  56.6 29.6 42.4 18.0 48.5 41.5 24.8 21.4 81.4 18.9 19.6 27.1 Table 1: Automatic evaluation of the QCPG model on the test set. annotators were shown the source sentence, along with the two generated paraphrases (randomly or-dered), and were asked which of the two better preserves the semantic meaning of the source sentence (ties are also allowed). In total, 150 triplets were evaluated by 5 judges. Table 2 demonstrates an advantage for QCP G  in all datasets, with a large margin in MSCOCO and WikiAns. This advantage is statistically significant ( p  value < 0 . 05 ) as obtained by applying the Wilcoxon signed-rank test to the difference between the number of annotators that voted for QCP G  and those voted for the baseline, across all datasets. Thus, the human evaluation is in line with the results of the automatic semantic similarity measure. We also verified, that the results of this sample, in terms of linguistic diversity, are very similar to those shown in Table 1. For examples of paraphrases generated by QCP G  see Table 10 in the Appendix. Many recent works on paraphrase generation have been focused on attempting to achieve high-quality paraphrases. These works can be divided into supervised and unsupervised approaches. Supervised Approaches To achieve diversity, some works focused on diverse decoding using heuristics such as Hamming distance or distinct n-grams to preserve diverse options during beam search (Vijayakumar et al., 2018). Other works generate multiple outputs by perturbing latent representations (Gupta et al., 2018; Park et al., 2019). or by using distinct generators (Qian et al., 2019). These methods achieve some diversity, but do not control generation in an interpretable manner. The works that are most similar to ours strive to gain diversity using controlled-paraphrase generation, by exposing control mechanisms that are manipulated to produce either lexically (Zeng et al., 2019; Thompson and Post, 2020) or syntactically (Chen et al., 2019; Goyal and Durrett, 2020) diverse paraphrases. One approach is to use an exemplar sentence for guiding the syntax of the generated paraphrase (Chen et al., 2019; Bao et al., 2019; Hosking and Lapata, 2021). An alternative is to directly employ constituency tree as the syntax guidance (Iyyer et al., 2018; Li and Choi, 2020). Goyal and Durrett (2020) promote syntactic diversity by conditioning over possible syntactic rearrangements of the input. Zeng et al. (2019) use keywords as lexical guidance for the generation process. Here we introduce a simple model for jointly controlling the lexical, syntactic and semantic aspects of the generated paraphrases. Unsupervised Approaches Niu et al. (2020) rely on neural models to generate high quality paraphrases, using a decoding method that enforces diversity by preventing repetitive copying of the input tokens. Liu et al. (2020b) optimize a quality oriented objective by casting paraphrase generation as an optimization problem, and searching the sentence space to find the optimal point. Garg et al. (2021) and Siddique et al. (2020) use reinforcement learning with quality-oriented reward combining textual entailment, semantic similarity, expression 603 diversity and fluency. In this paper, we propose a novel controlled paraphrase generation model, that leverages measures of paraphrase quality for encouraging the generation of paraphrases with desired quality. We demonstrate the high level of control achieved by the model, and suggest a method for coping with the challenging problem of finding suitable control values. Aside from offering a simple and effective way for controlling models' output quality, the quality control paradigm enables a holistic view of the data, the training process and the final model analysis. Namely: (I) Examination of the training data through the lens of data quality enables to characterize the data at hand, its strengths and limitations. (II) A quality-aware training process can be viewed as multi-task learning, where each quality level is a separate task with its own accurate supervision, as opposed to the standard quality-agnostic approach, where low quality data is in fact used as a poor supervision for a model which aims at generating higher quality output. (III) Analyzing the model behavior under different quality controls, allows finer understanding of the different model behaviors and the trade-offs between their output qualities. Better understanding the expected output quality of neural NLG models, for different input quality controls, can increase the trust in their output. Finally, our model analysis consistently shows that although the models generally follow the quality requirements, there is still room for improvement. A possible direction for future research is exploring methods, such as reinforcement learning, for further improving the ability of the model to satisfy the quality requirements.
abstain abstain abstain abstain abstain objective method result other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method other result method method method method abstain abstain method method method method abstain abstain abstain abstain method method abstain method abstain method abstain abstain abstain abstain abstain abstain method method method method abstain method method abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain method abstain abstain method abstain abstain abstain method method other abstain abstain abstain abstain abstain method abstain other other other other other other other other other other other method other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain