Most of privacy protection studies for textual data focus on removing explicit sensitive identifiers. However, personal writing style, as a strong indicator of the authorship, is often neglected. Recent studies, such as SynTF, have shown promising results on privacy-preserving text mining. However, their anonymization algorithm can only output numeric term vectors which are difficult for the recipients to interpret. We propose a novel text generation model with a two-set exponential mechanism for authorship anonymization. By augmenting the semantic information through a REINFORCE training reward function, the model can generate differentially private text that has a close semantic and similar grammatical structure to the original text while removing personal traits of the writing style. It does not assume any conditioned labels or paralleled text data for training. We evaluate the performance of the proposed model on the real-life peer reviews dataset and the Yelp review dataset. The result suggests that our model outperforms the state-of-the-art on semantic preservation, authorship obfuscation, and stylometric transformation. Privacy has become a vital issue in online data gathering and public data release. Various machine learning models and privacy preservation algorithms have been studied for relational data (John-son et al., 2018), network graph data (Chen et al., 2014), and transactional data (Li et al., 2012). Some of them have been successfully adopted in real-life applications such as telemetry collection (Corts et al., 2016). However, the studies on privacy protection for textual data are still preliminary. Most related works only focus on replacing the sensitive key phrases in the text (Vasudevan and John, 2014) without considering the author's writing style, which is indeed a strong indicator of a person's identity. Even though some textual data, such as double-blind academic reviews, is released anonymously, the adversaries may recover the author's identity using the personal traits in writing. Stylometric techniques (Koppel et al., 2011) can identify an author of the text from 10,000 candidates. They are effective across online posts, articles, emails, and reviews (Ding et al., 2015, 2017). Nevertheless, traditional text sanitization methods (Narayanan and Shmatikov, 2008) focus on anonymizing the contents, such as patient information, instead of the writing style, so they are ineffective against writing style analysis. The original author can be easily re-identified even if protected by these traditional approaches (Iqbal et al., 2008, 2010, 2013; Schmid et al., 2015). Only a few recent studies focus on authorship anonymization, aiming to hide the personal traits of writing style in the given textual data. Anonymouth (McDonald et al., 2012) is a semiautomatic framework that offers suggestions to users to change their writing style. Yet, this framework is not practical since it requires two datasets as a reference to compare the change in writing style. Also, the user has to make all the final modi-fication decisions. SynTF (Weggenmann and Kerschbaum, 2018) represents a line of research that protects the privacy of the numeric vector representation of textual data. It adopts the exponential mechanism for a privacy guarantee, but the output is only an opaque term frequency vector, not an interpretable text in natural language. Furthermore, its token substitution approach does not consider the grammatical correctness and semantic. Style transfer is another line of research that tries to generate text with controllable attributes (Shen et al., 2017; Hu et al., 2017; Sennrich et al., 2016). Representative models (Hu et al., 2017) can control the sentiment and tense of the generated text. However, they do not modify the personal traits in writing. Their applications on sentiment and word-reordering correspond to the content of the text more than the writing style. We argue that their definition of styles, such as sentiment or tense, is different from the personal linguistic writing characteristics that raise privacy concern. A4NT (Shetty et al., 2018) is a generative neural network that sanitizes the writing style of the input text. However, it requires text samples to be labeled with known author identities. It is not applicable to many textual data publishing scenarios. Additionally, according to the samples provided in the paper, it has difficul-ties keeping the same semantic meaning between the original and the generated text. Without using any privacy model, A4NT does not provide any privacy guarantee. To address the aforementioned issues, we propose an Embedding Reward Auto-Encoder (ER-AE) to generate differentially private text. Relying on differential privacy, it protects the author's identity through text indistinguishability without assuming any specific labels, any parallel data or any assump-tion on the attacker. It guards the privacy of the data against the worst information disclosure scenario. ER-AE receives the original text as input and generates a new text using the two-set exponential mechanism. We propose a REINFORCE (Sutton et al., 2000) embedding reward function to augment the semantic information during the text generation process. The model can keep the generated text a close semantic and sentiment similarity to the original while providing a guarantee that one can hardly recover the original author's identity. Unlike the aforementioned authorship anonymization works, ER-AE produces human-friendly text in natural language. Our key contributions are summarized as follows:  The first differentially private authorship anonymization model that can generate human-friendly text in natural language, instead of a numeric vector. A novel two-set exponential mechanism to overcome the large output space issue while producing meaningful results. A novel combination of a differential privacy mechanism with a sequential text generator, providing a privacy guarantee through a sampling process. A new REINFORCE reward function that can augment the semantic information through external knowledge, enabling better preservation of the semantic similarity in the data synthesis process. Comprehensive evaluations on two real-life datasets, namely NeurIPS & ICLR peer reviews and Yelp product reviews , show that ER-AE is effective in obfuscating the writing style, anonymizing the authorship, and preserving the semantics of the original text. Differential Privacy. Recently, differential privacy has received a lot of attention in the machine learning community. The deep private auto-encoder (Phan et al., 2016) is designed to preserve the training data privacy. Their purpose is to guarantee that publishing the trained model does not reveal the privacy of individual records. Our purpose is different. We publish the differentially private data generated by the model, rather than the model itself. Most existing models for differentially private data release, such as Chen et al. (2014) , focus on different types of data rather than text. One recent work (Weggenmann and Kerschbaum, 2018) aims to protect privacy in text data using the exponential mechanism. However, it releases the term frequency vectors instead of a readable text. This approach limits the utility of published data to only the applications that assume term frequency as features. In contrast, our goal is to generate differentially private text in a natural language without compromising individual privacy. Writing Style Transfer. Studies on writing style transferal try to change the writing style revealed from the text according to a given author. Shetty et al. (2018) design a GAN to transfer Obama's text to Trump's style. A sequence to sequence (seq2seq) model is proposed by Jham-tani et al. (2017) to transfer modern English into Shakespearean English. Shetty et al.(2017) design a model with a cross-alignment method to control the text sentiment while preserving semantic. These models can also be applied to writing style anonymization. However, these studies require the data to be labeled with authorship identity. They assume a number of known authors. In contrast, ours does not assume any label information. Writing Style Obfuscation. Writing style obfuscation studies try to hide the identity of an au-1 https://github.com/McGill-DMaS/Authorship-Anonymization thor. Anonymouth (McDonald et al., 2012) is a tool that utilizes JStylo to generate writing attributes. It gives users suggestions on which way they can anonymize their text according to two reference datasets. (Kacmarcik and Gamon, 2006) also propose a similar architecture to anonymize text. However, instead of directly changing the text, they all work on the term frequency vector, whose real-life utility is limited. Compared with semi-automatic methods that require users to make a decision, our approach provides an end-to-end solution that directly learns from data. Adjacency is a key notion in differential privacy. One of the commonly used adjacency definitions is that two datasets D 1 and D 2 are adjacent if D 2 can be obtained by modifying one record in D 1 (Dwork et al., 2010). Differential privacy (Dwork et al., 2006) is a framework that provides a rigorous privacy guarantee on a dataset. It demands inherent randomness of a sanitization algorithm or generation function: Definition 1. Differential Privacy. Two datasets are considered as adjacent if there is only one single element is different. Let privacy buget (cid:15) > 0 , a randomized algorithm A : D n  Z , and the image of A : im ( A ) . The algorithm A is said to preserve (cid:15) differential privacy if for any two adjacent datasets D 1 , D 2  D n , and for any possible set of output Z  im ( A ) : P r [ A ( D 1 )  Z ]  e (cid:15)  P r [ A ( D 2 )  Z ] (cid:3) It guarantees that the result from a given algorithm A is not sensitive to a change of any individual record in D . (cid:15) denotes the privacy budget, the allowed degree of sensitivity. A large (cid:15) implies a higher risk to privacy. However, (cid:15) is a relative value that implies different degrees of risk given different problems (Weggenmann and Kerschbaum, 2018). Some studies (Sala et al., 2011) use a large (cid:15) , while the others (Chen et al., 2014) use a smaller value. Adversary Scenario. Generally in an authorship identification problem, one assumes that the attacker holds an anonymous text authored by one of the suspects from the dataset. The attacker aims to infer the true author of the anonymous text based on a set of reference texts from each suspect. However, this scenario assumes certain information on the applicable dataset, such as author labels and the number of reference text samples. Therefore, following (Weggenmann and Kerschbaum, 2018), we define that any two pieces of text as adjacent datasets. Adjacency. Any two pieces of text can be considered adjacent in the strictest scenario that datasets D 1 and D 2 both have only one record, and D 2 can be obtained by editing one record in D 1 following Definition 1. With differential privacy, we can have text indistinguishability: one cannot distinguish the identity of any text to another. In our case, the identity of a text corresponds to the author who wrote the text. Along with this, the attacker would fail in the original authorship identification scenario since the anonymous text is indistinguishable from the rest of the dataset. Our definition follows Weggenmann and Kerschbaum (2018)'s idea, leading to the strictest and most conservative definition of adjacency. Definition 2. Differentially Private Text Generation. Let D denote a dataset that contains a set of texts where x  D is one of them. | x | , the length of the text, is bound by l . Given D with a privacy budget (cid:15) , for each x the model generates another text  x dp that satisfies (cid:15)l -differential privacy. (cid:3) Following the above definitions, any two datasets that contain only one record are probabilistically indistinguishable w.r.t. a privacy budget (cid:15) . It directly protects the identity of an individual record, disregarding whether some of the records belong to the same author or not. It assumes that every record is authored by a different author, which is the strictest situation. Technically, the proposed text generation approach protects the writing style by reorganizing the text, replacing tokens with different spelling, removing the lexical, syntactical and idiosyncratic features of the given text. The above definition is based on SynTF (2018), but our target is readable text rather than numeric vectors, which is more challenging. Figure 1 depicts the overall architecture of our proposed ER-AE model, which consists of an encoder and a generator. The encoder receives a sequence of tokens as input and generates a latent vector to represent the semantic features. The generator, which is incorporated with the two-set exponential mechanism, can produce differentially private text according to the latent vector. ER-AE is trained InputText Encoder LatentVector Generator Differentially-privateText Two-SetExponentialMechanism 0.1 0.6 0.05 0.1 0.01 0.6 0.7 0.0 0.2 Embedding Reward Reconstruction Loss Random K Top K + DataRelease Training ... ... ... by combining a reconstruction loss function and a novel embedding loss function. Our ER-AE model starts with a basic sequence-to-sequence (seq2seq) auto-encoder structure. Given a text x , its tokens (cid:104) x 1 . . . x l (cid:105) are firstly converted into a sequence of embedding vectors (cid:104) Em ( x 1 ) . . . Em ( x l ) (cid:105) by Em : V  R m 1 , where V is the vocabulary across the dataset and m 1 is the embedding dimension. On its top, we apply a bi-directional recurrent neural network with Gated Recurrent Unit (GRU) (Cho et al., 2014) that leverages both the forward and backward information. GRU achieves a comparable performance to LSTM with less computational overhead (Cho et al., 2014). Then, the produced final state vectors from both directions, s f and s b , are concatenated and linearly transformed to be a latent vector E ( x ) . m is the hidden state dimension for the GRU function. The generator is another recurrent neural network with GRU. It generates a text token-by-token. For each timestamp i , it calculates a logit weight z iv for every candidate token v  V , conditioned on the latent vector, last original token x i  1 , and the last hidden state s i  1 of the GRU function. Let  x i denote the random variable for the generated token at timestamp i . Its probability mass function is proportional to each candidate token's weight z ti . This is modeled through a typical softmax function: P r [ x i = v ] = exp ( z iv ) / (cid:88) v (cid:48) V exp ( z iv (cid:48) ) (2) For each timestamp i , a typical seq2seq model generates text by applying argmax v V P r [ x i = v ] . However, this process does not protect the privacy of the original data. To protect an individual's privacy and hide the authorship of the original input text, we couple differential privacy mechanism with the above sampling process in the generator. The exponential mechanism (McSherry and Talwar, 2007) can be applied to both numeric and categorical data (Fernandes et al., 2018). It is effective in various sampling process for discrete data. It guarantees privacy protection by injecting noise into the sampling process: Definition 3. Exponential Mechanism. Let M and N be two enumerable sets. Given a privacy budget (cid:15) > 0, a rating function  : M  N  R . The probability density function of the random variable  (cid:15), ( m ) , P r [  (cid:15), ( m ) = n ] is: exp (cid:0) (cid:15) 2  ( m, n ) (cid:1) (cid:80) n (cid:48) exp (cid:0) (cid:15) 2  ( m, n (cid:48) ) (cid:1) (3) where  , the sensitivity, means the maximum difference of rating function values between two adjacent datasets, and m  M , n  N . (cid:3) However, according to Weggenmann and Kerschbaum (2018), the exponential mechanism requires a large privacy budget to produce meaningful results while the output space is large, the vocabulary size in our case. It's nontrivial to randomly sample a good result directly among 20,000 candidates. To tackle the large output space issue, inspired by subsampled exponential mechanism (Lantz et al., 2015), we propose a two-set exponential mechanism to produce meaningful results with a better privacy protection. Instead of using a database in-dependent distribution, we use a model-based distribution to generate subsets of tokens. Definition 4. Two-Set Exponential Mechanism. Let V be a enumerable set with size s . Given the model-based probabilities of each item in V , P r [ v ] for v  V , an item set S of size k is built by repeatedly sampling proportional to P r [ v ] with replacement. Other items are denoted as set O , where V = O  S ,  = O  S . Let N = { S, O } . An item set C dp is chosen from N through the exponential mechanism with a rating function  : (cid:80) v  CP r [ v ] / (cid:80) C (cid:48) N ,v (cid:48)  C (cid:48) P r [ v (cid:48) ] . Given (cid:15) > 0 , N  N , the probability density function (PDF) of the random variable  (cid:15), ( C ) , P r [  (cid:15), ( C ) = N ] , is: exp (cid:0) (cid:15) 2  ( C, N ) (cid:1) (cid:80) N (cid:48) N exp (cid:0) (cid:15) 2  ( C, N (cid:48) ) (cid:1) (4) After choosing the set, C dp , an item is randomly picked from the chosen set: v  Random ( C dp ) . P r [ wS ]  P r [  (cid:15), ( C ) = S | wS ]  P r [ w | wS, S ]+ P r [ wO ]  P r [  (cid:15), ( C ) = O | wO ]  P r [ w | wO, O ] , where P r [ wS ] and P r [ wO ] are respectively the probability of w in set S and O , P r [ wO ] = (1  P r [ w ]) k . (cid:3) Theorem 1. Two-Set Exponential Mechanism. 2 Given a privacy budget (cid:15) > 0 and the size of output space s , two-set exponential mechanism is ( (cid:15) + ln ( s )) -differentially private. (cid:3) By plugging our model with this mechanism, we have the probability mass function for  (cid:15), i ( x i ) : P r [  (cid:15), i ( x i ) = tk ] . This function models the disturbed probability distribution for all the alternative token tk to replace the original variable. According to Theorem 4, sampling from  (cid:15), i ( x i ) for each 2 The proof is provided in Appendix B timestamp i is ( (cid:15) + ln ( s )) -differentially private. Recall that in Definition 1, the timestamp is bound by l . To generate text  x dp , the generator samples a token for timestamp i through the chosen set T i :  x dp [ i ]  Random ( T i ) for i  [1 , l ] (5) The composition theorem (Dwork et al., 2014) is an extension to differential privacy. By repeating n (cid:15) -differentially private algorithms, the complete process achieves an (cid:15)n -differential privacy. Algorithm 1 shows the differentially private text generation of ER-AE. As proved in Appendix A: Theorem 2. Differentially Private Text Sampling. Given a privacy budget (cid:15) > 0, a sequence length l > 0, the generator's sampling function in Eq. 5 is ( (cid:15) + ln ( s ))  l -differentially private. (cid:3) 4.2 Initial Grammar and Semantic Preservation To generate a human-friendly text that has a close semantic to the original one, we need to have a high-quality rating function  i for Eq. 4. This is achieved by training the ER-AE model's encoder to extract semantic information, and its generator to learn the relationships among the tokens for prediction. We follow an unsupervised learning approach since we do not assume any label information. First, we adopt the reconstruction loss function: L recon = (cid:88) x i  x,x  D  log P r [ x i = x i ] (6) It maximizes the probability of observing the original token x i itself for the random variable  x i . In the recent controllable text generation models, the reconstruction loss plays an important role to preserve grammar structure and semantics of input data (Shetty et al., 2018) when combined with the other loss. Diving into the optimization aspect of the softmax function, the reconstruction loss function above encourages the model to produce a higher probability on the original token while ignoring the rest candidates. It does not consider the other tokens that may have a similar meaning under a given context. This issue significantly limits the variety of usable alternative tokens. Additionally, this loss function relies on a single softmax function for multi-object learning, it cannot provide the expressiveness required by the language model (Yang et al., 2018). We inspect the candidates and in most of the cases, only the top-ranked token fits the context in the text. This is problematic because the mechanism for our sampling process also relies on the other candidates to generate text. To address the above issue, we propose a novel embedding reward function using the pre-trained word embeddings. Word representation learning models show that discrete text tokens' semantic can be embedded into a continuous latent vector space. The distance between word embedding vectors can be a reference to measure the similarity between different words. To encourage our rating function  i to learn richer and better substitute tokens, we propose a reward function that leverages the semantics learned from the other corpus. The text dataset to be anonymized and released can be small, and the extra semantic knowledge learned from the other corpus can provide additional reference for our rating function. This reward function is inspired by the Policy Gradient loss function (Sutton et al., 2000), L embed is:  (cid:88) x i  x,x  D (cid:16) (cid:88) v  E k ( x i ) log( P r [ x i = v ])  ( x i , v ) + (cid:88) w  V k log( P r [ x i = w ])  ( x i , w ) (cid:17) Generally, this reward function assigns credits to the under-rated tokens in the reconstruction loss function. Recall that D is the original dataset and x is one of its texts. At time step i , this reward function first assigns rewards to the topk selected tokens, denoted as E k ( x i ) , according to probability estimates for random variable  x i in Eq. 2. The rewards are proportional to their semantic relationship to the original token x i . It is defined as a function  : V  V  R ,  ( w, v ) is: min (cid:0) cosine( Em ( w ) , Em ( v )) , 0 . The min function avoids the generator focusing only on the original token. By assigning rewards to E k ( x i ) , it encourages the other candidates having a close semantic to the targeted one, but it may fail to reach infrequent tokens. Therefore, in the second part of the reward function, we encourage the model to explore less frequent tokens by random sampling candidates as V k . This design balances the exploitation (topk ) and the exploration ( V k ) in reinforcement learning. During training, the model is firstly pre-trained by minimizing the reconstruction loss in Eq. 6 through the Adam optimizer, and adopts the embedding reward loss later. The total loss is L =  recon  L recon +  embed  L embed (8) Specifically, the reconstruction loss can lead the model to generate grammatically correct text, and the embedding reward loss encourages the model to focus more on semantically similar tokens. The balance of the two loss functions are controlled by  recon and  embed . All the experiments are carried out on a Windows Server equipped with two Xeon E5-2697 CPUs (36 cores), 384 GB of RAM, and four NVIDIA TITAN XP GPU cards. We evaluate ER-AE on two different datasets with respect to its effectiveness for privacy protection and utility preservation. Yelp Review Dataset 3 : All the reviews and tips from the top 100 reviewers ranked by the number of published reviews and tips. It contains 76,241 reviews and 200,940 sentences from 100 authors. Academic Review Dataset : All the public reviews from NeurIPS (2013-2018) and ICLR (2017) based on the original data and the web crawler provided by (Kang et al., 2018). It has 17,719 reviews, 268,253 sentences, and the authorship of reviews is unknown. Each dataset is divided into 70/10/20 for train/dev/evaluation respectively. As mentioned in the related work discussion, most of the controllable text generation and style transferal studies rely on known authorship or other labels. Other generation models such as paraphrasing, however, hold an essentially different goal and cannot provide a privacy guarantee on the generated data. They are not applicable to our problem. Therefore, we pick SynTF (Weggenmann and Kerschbaum, 2018) and different generation and sampling models for evaluation:  Random Replacement (Random-R) : This method generates a new text by replacing each token in the text by randomly picking substitution from the vocabulary. AE with Differential Privacy (AE-DP) : Extended version of AE with the added two-set exponential mechanism for text generation. It does not include the embedding reward. SynTF (Weggenmann and Kerschbaum, 2018) : We directly generate the tokens through SynTF's differentially private sampling function, without further extraction of the frequency vector. SynTF is a state-of-the-art generation model that satisfies differential privacy property on textual data. The other two simple baselines are for ablation test purposes. For ER-AE, we adopted a two-layers stacked GRU network for both the encoder and the generator. There are 512 cells in each GRU layer. The vocabulary size is 20,000, separately built for each dataset. All the word embeddings in our model come from the pre-trained BERT embeddings provided by (Devlin et al., 2019), which has a dimension of 768 for each embedding. The maximum input length of our model is 50, the learning rate is 0.001, the k for embedding reward loss function is 5, the  recon is 1, the  embed is 0.5, and the batch size is 128. The k in two-set exponential mechanism is 5. ER-AE is implemented in TensorFlow (Abadi et al., 2016), and it uses the tokenizer in the NLTK library. Some traditional tricks for text generation, such as beam search, are not mentioned because they are incompatible with differential privacy. All the models are evaluated from three aspects: semantic preservation, privacy protection, and stylometric changes:  Semantic Preservation (USE) : A pre-trained Universal Sentence Embedding ( USE ) model 4 from Google. It can embed a sentence into a latent vector that represents its semantics. It is widely used for supervised NLP tasks such as sentiment analysis. We measure the degree of semantic preservation using the cosine similarity between the latent vector of the original text and 4 https://tfhub.dev/google/universal-sentence-encoder/1 Table 4: Sample sentences generated by models. one of the generated text. Privacy Protection (Authorship) : A state-of-the-art authorship identification neural network model (Sari et al., 2017) to identify the authorship of generated text. The model is firstly trained on the training dataset, and the performance is evaluated on the testing set. The author's privacy is protected if s/he cannot be iden-tified using authorship identification techniques. Stylometric Changes : Well-established stylistic context-free features such as text length and a number of function words. We adopt Stylo-Matrix (Ding et al., 2017) for an aggregation of features in (Iqbal et al., 2013; Zheng et al., 2006). The feature vector change is measured by the difference in L2 norm. Quantitative Evaluation (Table 1). With a low utility (USE) score around 0.2 for both datasets, SynTF, and Random-R generate grammatically incorrect text and completely change the meaning of the original one. In contrast, ER-AE without semantic augmentation through REINFORCE training, denoted as AE-DP, achieves a much higher utility score of around 0.61. The full model ER-AE, with an (cid:15) of 3, achieves the highest utility score of 0.75 for Yelp reviews and 0.74 for peer reviews. AE-DP, SynTF, and ER-AE all significantly reduce the chance of a successful authorship identification attack from 55% to lower than 10% in the Yelp data and introduce a variation in stylometric features of more than 10 in magnitude in the peer review dataset. They are all effective and competitive on removing the personal writing trait from the text data, but as mentioned above, AE-DP achieves the best and a much higher utility score. Although Random-R performs better on privacy protection, its generated texts are irrelevant to the original. Overall, with a competitive performance on anonymization, ER-AE performs significantly better than all of the other models on utility. Impact of Embedding Reward. Table 4 shows that the embedding reward plays an important role in selecting semantically similar candidates for substitution. AE-DP assigns a large probability to the original token and a tiny probability to the others. If applied with the mechanism, it is more likely to pick a semantically irrelevant token. ER-AE shows a smoother distribution and assigns higher probabilities to top-ranked semantically relevant tokens. Its generated candidates are better. Case Study. Table 4 shows that both SynTF and Random-R cannot generate human-friendly text. Due to the issue of reconstruction loss function [6], AE-DP cannot substitute token with similarly semantic tokens and destroys the semantic meaning. ER-AE, powered by embedding reward, can substitute some tokens with semantically similar ones: written" is replaced by editted", and the whole sentence still makes sense. Besides, it can preserve the grammatical structure of the input. However, due to some missing information from word embeddings, the model would fail to generate good candidates for sampling. The third sample replaces exactly" with  occult". ER-AE still performs way better than other models. Utility vs. Privacy. The privacy budget (cid:15) controls the trade-off between privacy and utility. A larger (cid:15) implies better utility but less protection on privacy. However, this is a relative value that implies different degrees of risk given different problems (Weggenmann and Kerschbaum, 2018; Fernandes et al., 2018). As proved by Weggenmann and Kerschbaum (2018) a higher (cid:15) is intrinsically necessary for a large output space, in our case the vocabulary, to generate relevant text. In fact, we have already significantly reduced the optimal (cid:15) value of 42.5 used by Weggenmann and Kerschbaum (2018) to around 13, given the same dataset. One possible way to lower the bound of (cid:15) is to directly factor in authorship and utility, such as topics, into the privacy model. However, it limits applicable to datasets. Exponential Mechanism vs. Two-Set Exponential Mechanism. In Table 3, we estimated the probability of a meaningful token (among top 5 semantically similar tokens) is sampled based on the intermediate probabilities in Table 2. Given a large output space of 20,000, the exponential mechanism is not likely to sample a meaningful token with a probability of 0.01 %. However, the two-set exponential mechanism dramatically improves it from 0.01 % to around 70%. Our generator has a much higher chance to generate meaningful results with a similar privacy budget. 6 Conclusion In this paper, we propose a novel model, ER-AE, to protect an individual's privacy for text data release. We are among the first to fuse the differential privacy mechanisms into the sequence generation process. We demonstrate the effectiveness of our model on the Yelp review dataset and two peer reviews datasets. However, we also find that ER-AE performs not very well on long texts due to the privacy budget accounting issue. Our future research will focus on improving long texts generation with better budget allocation scheme. 7 Ethical Considerations Our model outperforms others on authorship obscuration and semantic preservation. Similar to other text generation tools, this model may be abused to generate fake reviews, but this can be assuaged by using fake review detection methods. This research work directly contributes to the area of privacy protection and indirectly promotes freedom of speech and freedom of expression in cyberspace. Acknowledgments This research is supported in part by Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grants (RGPIN-2018-03872), Canada Research Chairs Program (950-232791), and Provost Research Fellowship Award (R20093) and Research Incentive Funds (R18055) from Zayed University, United Arab Emirates. The Titan Xp used for this research was donated by the NVIDIA Corporation. References Martn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI) . Rui Chen, Benjamin C. M. Fung, Philip S. Yu, and Bipin C. Desai. 2014. Correlated network data publication via differential privacy. The VLDB Jour-nalThe International Journal on Very Large Data Bases , 23(4). Kyunghyun Cho, Bart van Merrinboer, Dzmitry Bah-danau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoderdecoder approaches. Syntax, Semantics and Structure in Statistical Translation . Jorge Corts, Geir E Dullerud, Shuo Han, Jerome Le Ny, Sayan Mitra, and George J Pappas. 2016. Differential privacy in control and network systems. In IEEE 55th Conference on Decision and Control (CDC) . IEEE. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Steven H. H. Ding, Benjamin C. M. Fung, and Mourad Debbabi. 2015. A visualizable evidence-driven approach for authorship attribution. ACM Transactions on Information and System Security (TISSEC) , 17(3). Steven H. H. Ding, Benjamin C. M. Fung, Farkhund Iqbal, and William K Cheung. 2017. Learning stylometric representations for authorship analysis. IEEE transactions on cybernetics , (99). Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. Springer. Cynthia Dwork, Aaron Roth, et al. 2014. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science , 9(3 4). Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. 2010. Boosting and differential privacy. In 2010 IEEE 51st Annual Symposium on Foundations of Computer Science . IEEE. Natasha Fernandes, Mark Dras, and Annabelle McIver. 2018. Author obfuscation using generalised differential privacy. arXiv preprint arXiv:1805.08866 . Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning . JMLR. Farkhund Iqbal, Hamad Binsalleeh, Benjamin C. M. Fung, and Mourad Debbabi. 2010. Mining writeprints from anonymous e-mails for forensic investigation. Digital Investigation , 7(1-2). Farkhund Iqbal, Hamad Binsalleeh, Benjamin C. M. Fung, and Mourad Debbabi. 2013. A unified data mining solution for authorship analysis in anonymous textual communications. Information Sciences , 231. Farkhund Iqbal, Rachid Hadjidj, Benjamin C. M. Fung, and Mourad Debbabi. 2008. A novel approach of mining write-prints for authorship attribution in e-mail forensics. Digital Investigation , 5(1):S42S51. Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. 2017. Shakespearizing modern language using copy-enriched sequence to sequence models. In Proceedings of the Workshop on Stylistic Variation . Noah Johnson, Joseph P Near, and Dawn Song. 2018. Towards practical differential privacy for sql queries. Proceedings of the VLDB Endowment , 11(5). Gary Kacmarcik and Michael Gamon. 2006. Obfuscating document stylometry to preserve author anonymity. In Proceedings of the COLING/ACL on Main conference poster sessions . Association for Computational Linguistics. Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. A dataset of peer reviews (peerread): Collection, insights and nlp applications. In Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) , New Orleans, USA. Moshe Koppel, Jonathan Schler, and Shlomo Argamon. 2011. Authorship attribution in the wild. Language Resources and Evaluation , 45(1). Eric Lantz, Kendrick Boyd, and David Page. 2015. Subsampled exponential mechanism: Differential privacy in large output spaces. In Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security . Ninghui Li, Wahbeh Qardaji, Dong Su, and Jianneng Cao. 2012. Privbasis: Frequent itemset mining with differential privacy. Proceedings of the VLDB Endowment , 5(11). Andrew WE McDonald, Sadia Afroz, Aylin Caliskan, Ariel Stolerman, and Rachel Greenstadt. 2012. Use fewer instances of the letter i: Toward writing style anonymization.
abstain abstain abstain abstain objective abstain abstain objective result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain objective objective objective objective objective other other other other objective method other other other other objective other other other other other other other other other other other other other other other other method abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other