Neural machine translation systems tend to fail on less decent inputs despite its significant ef-ficacy, which may significantly harm the credibility of these systemsfathoming how and when neural-based systems fail in such cases is critical for industrial maintenance. Instead of collecting and analyzing bad cases using limited handcrafted error features, here we investigate this issue by generating adversarial examples via a new paradigm based on reinforcement learning. Our paradigm could expose pitfalls for a given performance metric, e.g., BLEU, and could target any given neural machine translation architecture. We conduct experiments of adversarial attacks on two mainstream neural machine translation architectures, RNN-search, and Transformer. The results show that our method efficiently produces stable attacks with meaning-preserving adversarial examples. We also present a qualitative and quantitative analysis for the preference pattern of the attack, demonstrating its capability of pitfall exposure. Neural machine translation (NMT) based on the encoder-decoder framework, such as RNN-Search (Bahdanau et al., 2014; Luong et al., 2015, RNNSearch) or Transformer (Vaswani et al., 2017, Transformer), has achieved remarkable progress and become a de-facto in various machine translation applications. However, there are still pitfalls for a well-trained neural translation system, especially when applied to less decent real-world inputs compared to training data (Belinkov and Bisk, 2017). For example, typos may severely deteriorate system outputs (Table 1). Moreover, recent studies show that a neural machine translation system can also be broken by noisy synthetic inputs (Belinkov and Bisk, 2017; Lee et al., 2018). Due to the black-box nature of a neural system, it has in            out suicide bombing in jerusalem in           out eastern jerusalem explores a case of eastern europe Table 1: Fragility of neural machine translation. been a challenge to fathom when and how the system tends to fail. Intuitively, researchers seek to apprehend such failures by the analysis of handcrafted error indicating features (Zhao et al., 2018; Karpukhin et al., 2019). This strategy is costly because it requires expert knowledge for both linguistics and the target neural architecture. Such features are also less applicable because some common errors in deep learning systems are hard to formulate, or very specific to certain architectures. Instead of designing error features, recent researchers adopt ideas from adversarial learning (Goodfellow et al., 2014) to generate adversarial examples for mining pitfalls of NLP systems (Cheng et al., 2018a; Ebrahimi et al., 2018; Zhao et al., 2017). Adversarial examples are minor perturbed inputs that keep the semantic meaning, yet yield degraded outputs. The generation of valid adversarial examples provides tools for error analysis that is interpretable for ordinary users, which can contribute to system maintenance. Though it has achieved success concerning continuous input, e.g., images, there are following major issues for NLP tasks. First, it is non-trivial to generate valid discrete tokens for natural language, e.g., words or characters. Cheng et al. (2018a) follow Goodfellow et al. (2014) to learn noised representation then sample tokens accordingly. However, there is no guaranteed correspondence between arbitrary representation and valid tokens. Therefore, it may genin Two man are playing on the street corner. erate tokens departing from learned representation, which undermines the generation. Ebrahimi et al. (2018) turns to a search paradigm by a brute-force search for direct perturbations on the token level. To lead the search, a gradient-based surrogate loss must be designed upon every token modification by given target annotations. However, this paradigm is inefficient due to the formidable computation for gradients. Furthermore, surrogate losses defined upon each token by targets requires high-quality targets, and risks being invalidated by any perturbation that changes tokenization. Another issue is to keep the semantics of original inputs. Different from the fact that minor noises on images do not change the semantics, sampling discrete tokens from arbitrary perturbed representation (Cheng et al., 2018a) may generate tokens with different semantics and lead to ill-perturbed samples (Table 2). Searching for the perturbed input also requires a semantic constraint of the search space, for which handcrafted constraints are employed (Ebrahimi et al., 2018). Though constraints can also be introduced by multitask modeling with additional annotations (Zhao et al., 2017), this is still not sufficient for tasks requiring strict semantic equivalence, such as machine translation. In this paper, we adopt a novel paradigm that generates more reasonable tokens and secures semantic constraints as much as possible. We summarize our contributions as the following:  We introduce a reinforcement learning (Sut-ton and Barto, 2018, RL) paradigm with a discriminator as the terminal signal in its environment to further constrain semantics. This paradigm learns to apply discrete perturbations on the token level, aiming for direct translation metric degradation. Experiments show that our approach not only achieves semantically constrained adversarial examples but also leads to effective attacks for machine translation. Our paradigm can achieve the adversarial example generation with outclassed efficiency by only given source data. Since our method is model-agnostic and free of handcrafted error feature targeting architectures, it is also viable among different machine translation models. We also present some analysis upon the state-of-the-art Transformer based on its attack, showing our method's competence in system pitfall exposure. The most popular architectures for neural machine translation are RNN-search (Bahdanau et al., 2014) and Transformer (Vaswani et al., 2017). They share the paradigm to learn the conditional probability P ( Y | X ) of a target translation Y = [ y 1 , y 2 , ..., y m ] given a source input X = [ x 1 , x 2 , ..., x n ] . A typical NMT architecture consists of an encoder, a decoder and attention networks. The encoder encodes the source embedding X emb = [ emb 1 , emb 2 , ...emb n ] into hidden representation H = [ h 1 , h 2 , ..., h n ] . Then a decoder f dec with attention network attentively accesses H for an auto-regressive generation of each y i until the end of sequence symbol (EOS) is generated: P ( y i | y <i , X ) = softmax ( f dec ( y i  1 , s t , c t ;  dec )) (1) where c t is the attentive result for current decoder state s t given H . Reinforcement learning (Sutton and Barto, 2018, RL) is a widely used machine learning technique following the paradigm of explore and exploit , which is apt for unsupervised policy learning in many challenging tasks (e.g., games (Mnih et al., 2015)). It is also used for direct optimization for non-differentiable learning objectives (Wu et al., 2018; Bahdanau et al., 2016) in NLP. Actor-critic (Konda and Tsitsiklis, 2000) is one of the most popular RL architectures where the agent consists of a separate policy and value networks called actor and critic. They both take in environment state s t at each time step as input, while softmax linear+dropout   { tgt bi-GRU src bi-GRU Tokens embedding actor linear linear+dropout Y N { src bi-GRU V ( s t ) critic linear Mean Mean Mean Sum Environment TGT SRC } x t x t + 1 x t 1   Y emb Y X X emb X X emb discriminator agent actor critic survival/terminal signals s t a t } & 2.3 adversarial examples in NLPA general adversarial example generation can be described as the learning process to find a perturbation  on input X that maximize system degradation L adv within a certain constraint C (  ) : argmax  L adv ( X +  )  C (  ) (6) where  denotes the constraint coefficient, L adv is determined by the goal of the attack. T i m e ec o n d s ) 30000 45000 60000 56467 31508 35671 18843 Ours Search(0.2) Search(0.4) 0.45 0.675 0.9 Search(0.4)Ours Figure 1: [a] Overview of our RL architecture. 1 (cid:13) Environment states are processed as inputs for agent; 2 (cid:13) agent yield modification upon SRC in environment; 3 (cid:13) determine survival and step reward of environment; 4 (cid:13) determine degradation with victim NMT as episodic reward; 5 (cid:13) update agent with total rewards. During a training episode, we loop 1 (cid:13) to 3 (cid:13) and accumulate step rewards until environment terminates. Dash line indicates execution at the end of an episode. [b] Architecture of discriminator. [c] Architecture of agent. actor determines an action a t among possible action set A and critic yields value estimation V t ( s t ) . In general, the agent is trained to maximize discounted rewards R t = (cid:80)  i =0  i r t + i for each state, where   (0 , 1] is the discount factor. Such goal can be further derived as individual losses applied to actor and critic. Thus the actor policy loss L  on step t is: L t (   ) = log P ( a t | s t ) A t ( s t , a t ); a t  A (2) where   denotes actor parameters, A t ( s t , a t ) denotes general advantage function (Schulman et al., 2015) on state s t for action a t given by (cid:80) k  1 i =0  i r t + i +  k V ( s t + k )  V ( s t ) , which can be further derived as: A t ( s t , a t ) = A t +1 ( s t +1 , a t +1 ) + r t + V t +1 ( s t +1 )  V t ( s t ) (3) Meanwhile, critic learns to estimate R t via minimizing a temporal difference loss L v on each step t : L vt (  v ) = 1 2( r t + R t +1  V t ( s t )) 2 (4) where  v denotes critic parameter. Usually, the training is regularized by maximizing policy entropy H  to avoid exploration failure before exploiting optimum policy (Ziebart, 2010). Thus the total loss becomes: L (  ) = (cid:88) t ( L v t  L t  H  ( | s t )) (5) where  and  are hyperparameters for value loss and entropy coefficients. However, currently effective adversarial generation for NLP is to search by maximizing a surrogate gradient-based loss: argmax 1  i  n,x (cid:48)  vocab L adv ( x 0 , x 1 , ...x (cid:48) i ...x n ) (7) where L adv is a differentiable function indicating the adversarial object. Due to its formidable search space, this paradigm simply perturbs on a small ratio of token positions and greedy search by brute force among candidates. Note that adversarial example generation is fundamentally different from noised hidden representation in adversarial training (Cheng et al., 2019; Sano et al., 2019), which is not to be concerned in this work. In this section, we will describe our reinforced learning and generation of adversarial examples (Figure 1) in detail. Overall, the victim model is a part of the environment (denoted as Env ), which yields rewards indicating overall degradation based on modified inputs. A reinforced agent learns to modify every source position from left to right sequentially. Meanwhile, a discriminator in Env provides every-step survival signals by determining whether SRC is ill-perturbed. We encapsulate the victim translation model with a discriminative reward process as an Env for a reinforced agent to interact. The state of the Env is described as s t = ( SRC, t ) , where SRC = [ src 0 , src 1 , ..., src N ] are N sequences processed by victim model's vocabulary and tokenization. Each sequence src i = [ x 1 , x 2 , ..., x n ] is concatenated with BOS, EOS , which indicate the begin and end of the sequence, then padded to same length. Time step t  [1 , n ] also indicates the token position to be perturbed by the agent. Env will consecutively loop for all token positions and update s t based on the agent's modification. Env also yields reward signals until the end or intermediately terminated. That is, all sequences in SRC are determined by D as ill-perturbed during the reward process. Once the Env terminates, it finishes the current episode and reset its state with a new batch of sequences as SRC . The reward process is only used during training. It consists of a survival reward r s on every step and a final degradation r d concerning an overall metric if the agent survives till the end. Overall, we have: r t =    1 , terminated 1 N (cid:80) N a  r s , survive & t  [1 , n ) 1 N (cid:80) N ( a  r s + b  r d ) , survive & t = n (8) where a, b are hyper parameters that keeps the overall r s and r d within similar magnitude. Instead of direct optimization of the constrained adversarial loss in Eq.6, we model discriminator D 's output as survival rewards similar to that in gaming (Mnih et al., 2015). That is, the agent must survive for its goal by also fooling D , which attempts to terminate ill-perturbed modifications. We define an ill-perturbed source by determining whether it still matches the original target tgt . Discriminator As it is shown in Figure 1(b), discriminator D consists of bi-directional GRU encoders for both source and target sequence. Their corresponding representation is averaged and concatenated before passed to a feedforward layer with dropout. Finally, the output distribution is calculated by a softmax layer. Once D determines the pair as positive, its corresponding possibility is regarded as the reward, otherwise 0: r s = (cid:40) P ( positive | ( src (cid:48) , tgt );  d ) , positive 0 , otherwise (9) As long as the environment survives, it yields averaged reward among samples from SRC (Eq.8) to mitigate rewards' fluctuation that destabilize training. Discriminator Training Similar to GAN training, the environment's D must update as the agent updates. During its training, the agent's parameter is freezed to provide training samples. For every D 's training epoch, we randomly choose half of the batch and perturb its source using the current agent as negative samples. During D 's updates, we randomly generate a new batch of pairs from parallel data likewise to test its accuracy. D is updated at most step D epochs, or until its test accuracy reaches acc bound . Env only 1 yields -1 as overall terminal rewards when all sequences in SRC are intermediately terminated. For samples classified as negative during survival, their follow-up rewards and actions are masked as 0. If the agent survives until the end, Env yields additional averaged r d as final rewards for an episode. We follow Michel et al. (2019) to adopt relative degradation: r d = score ( y, refs )  score ( y (cid:48) , refs ) score ( y, refs ) (10) where y and y (cid:48) denote original and perturbed output, refs are references, and score is a translation metric. If score ( y, refs ) is zero, we return zero as r d . To calculate score we retokenize perturbed SRC by victim models vocabulary and tokenizer before translation. 1 It is commonly accepted that frequent negative rewards result in agents' tendency to regard zero-reward as optimum and fail exploration, which further leads to training failure. As it is shown in Figure 1 (c), the agent's actor and critic share the same input layers and encoder, but later processed by individual feedforward layers and output layers. Actor takes in SRC and current token with its surrounding ( x t  1 , x t , x t +1 ) , then yields a binary distribution to determine whether to attack a token on step t , while critic emits a value V ( s t ) for every state. Once the actor decides to perturb a specific token, this token will be replaced by another token in its candidate set. Candidate Set We collect at most K candidates for each token in the victim's vocabulary within a distance of (cid:15) . (cid:15) is the averaged Euclidean distance of K -nearest embedding for all tokens in victim vocabulary. We note that there shall always be candidates for a token in test scenarios that are beyond victim's vocabulary, for those without a nearby candidate, we assign UNK as its candidate. Once the agent chooses to replace a token with UNK, we follow Michel et al. (2019) to present a valid token that is also UNK to the victim's vocabulary. Agent Training The agent is trained by algorithm in appendix A. Since the agent is required to explore with stochastic policy during training, it will first sample based on its actor's output distribution on whether to perturb the current position, then randomly choose among its candidates. The agent and discriminator take turns to update. We assume the training is converged when test accuracy for D does not reach over a certain value within certain continuous learning rounds of agent and discriminator. Agent Inference To generate adversarial examples, the agent will take in source sequences and perturb on each position based on the actor's output from left to right, then choose the nearest candidate. As the agent's critic learns to estimate expected future rewards for a step, only when it yields positive value will agent perturb, otherwise it indicates an undesirable perturbation; thus, the agent is muted. We test our adversarial example generations on Zh  En, En  Fr, and En  De translation tasks, which provide relatively strong baselines for victim victim models and mass test samples. We train our agent using only parallel data that is used for victims' training. we train on LDC Zh  En 2 (1.3M pairs), WMT14 En  De 3 (4.5M pairs) and WMT15 En  Fr 4 (2.2M pairs) for victim models respectively. For subword level translation, we apply byte pair encoding (Sennrich et al., 2015, BPE) for both source and target languages with the vocabulary size of 37k. We also use join-BPE for En-De and En-Fr experiments with 34k and 33k vocabulary size, respectively. For word-level translation, we use NLPIR-ICTCLAS and Moses tokenizer for Chinese and English tokenization, respectively. We adopt 30k as vocabulary size for both source and target language. We adopt NIST test sets 5 for Zh  En and WMT test sets for En  De and En  Fr, then generate adversarial examples for these sources for analysis. We choose the state-of-the-art RNN-search and Transformer as victim translation models. For RNN-search, we train subword level models and strictly follow the architecture in Bahdanau et al. (2014). As for Transformer, we train both word-level and subword-level model for Zh  En and only subword-level models for En  De and En  Fr with the architecture and the base parameter settings by Vaswani et al. (2017). For the above models, we apply the same batch scheme and Adam optimizer following Vaswani et al. (2017). We choose MT03, newsdiscuss2015 and new-stest2013 for Zh  En, En  Fr, En  De as validation set respectively. We first report attack results both in terms of char-level BLEU ( chrBLEU ) of perturbed source by the origin to indicate modification rate, and relative decrease in target BLEU ( RD ): We adopt sacreBLEU (Post, 2018) to test case-insensitive BLEU on detokenized targets. 2 ldc2002E18, ldc2003E14, ldc2004T08, ldc2005T06 3 https://nlp.stanford.edu/projects/nmt/ 4 Europarl-v7, news-commentary-v10 5 MT02,03,04,05,06 Zh-En MT02-06 BLEU chrBLEU RD  HE  Transformer-word 41.16 -RSNI (0.2)  29.68 0.892 2.580  1.39  RSNI (0.3)  19.94 0.781 2.350  1.10  GS (0.2) 33.46 0.749 0.746 3.23 GS (0.3) 29.86 0.676 0.847 2.49 Ours 33.72 0.804 0.952 3.73 Transformer-BPE 44.06 -RSNI (0.2)  34.44 0.892 2.019  1.45  RSNI (0.4)  25.78 0.781 1.891  1.08  GS (0.2) 35.52 0.823 1.094 3.88 GS (0.4) 28.18 0.675 1.004 2.90 Ours 35.48 0.807 1.009 3.79 RNN-search-BPE 40.90 -RSNI (0.2)  32.54 0.892 1.891  1.44  RSNI (0.4)  25.54 0.781 1.712  1.36  GS (0.2) 32.94 0.823 1.102 3.79 GS (0.4) 27.02 0.678 1.053 2.88 Ours 31.58 0.785 1.059 3.81 Table 3: Experiment results for Zh  En MT attack. As Michel et al. (2019) suggest, there is a tradeoff between achieving high RD and maintaining semantic. One can achieve rather high RD by testing with mismatched references, making degradation less meaningful. Therefore, we also test source semantic similarity with human evaluation ( HE ) ranging from 0 to 5 used by Michel et al. (2019) by randomly sampling 10% of total sequences mixed with baselines for a double-blind test. We implement state-of-the-art adversarial example generation by gradient search (Michel et al., 2019) (GS) as a baseline, which can be currently applied to various translation models. We also implemented random synthetic noise injection (Karpukhin et al., 2019) (RSNI) as an unconstrained contrast. Both baselines are required to provide a ratio for the amount of tokens to perturb during an attack, where we present the best results. Unlike our paradigm can generate on monolingual data, GS also requires target annotations, where we use one of the references to provide a strong baseline. Note that RSNI can significantly break semantics with distinctly lower HE to achieve rather high RD , which we do not consider as legit adversarial example generation and noted with * for exclusion. As it is shown in Table 3 and 4, our model En-De newstest13-16 BLEU chrBLEU RD  HE  RNN-search-BPE 25.35 -RSNI (0.2)  16.70 0.949 6.691  2.32  RSNI (0.4)  10.05 0.897 5.860  1.58  GS (0.2) 19.42 0.881 1.966 3.81 GS (0.4) 9.27 0.680 1.982 3.01 Ours 21.27 0.921 2.037 3.95 Transformer-BPE 29.05 -RSNI (0.2)  18.775 0.949 6.935  2.39  RSNI (0.4)  11.125 0.897 5.991  1.58  GS (0.2) 18.29 0.861 2.665 3.69 GS (0.4) 10.03 0.751 2.629 3.33 Ours 19.29 0.875 2.688 3.79 En-Fr newstest13-14 + newsdiscuss15 RNN-search-BPE 32.6 -RSNI (0.2)  21.93 0.947 6.175  2.23  RSNI (0.4)  14.3 0.894 5.271  1.56  GS (0.2) 22.7 0.833 1.818 3.80 GS (0.4) 15.2 0.708 1.828 3.25 Ours 22.3 0.843 2.009 3.87 Transformer-BPE 34.7 -RSNI (0.2)  24.0 0.947 5.774  2.34  RSNI (0.4)  15.8 0.894 5.114  1.67  GS (0.2) 23.01 0.830 1.982 3.74 GS (0.4) 19.6 0.788 2.053 3.68 Ours 21.33 0.798 1.907 3.78 Table 4: Experiment results for En  De and En  Fr MT attack. stably generate adversarial examples without significant change in semantics by the same training setting among different models and language pairs, achieving stably high HE ( > 3.7) without any handcrafted semantic constraints, while search methods (GS) must tune for proper ratio of modification, which can hardly strike a bal-ance between semantic constraints and degradation. Unlike search paradigm relying on reference and victim gradients, our paradigm is model-agnostic yet still achieving comparable RD with relatively high HE . As it is shown in Table 5, our method is less likely to perturb some easily-modified semantics (e.g. numbers are edited to other forms, but not different numbers), while search tends to generate semantically different tokens to achieve degradation. Thus our agent can lead to more insightful and plausible analyses for neural machine translation than search by gradient. As it is shown in Figure 2, given the same amount of memory cost, our method is significantly more efficient compared to the search paradigm. Gradient computation concerning every modified source sequence can cost considerably in time or space for a state-of-the-art system, which could be even worse for systems with recurrent units. When it comes to mass production of adversarial examples for a victim translation system, our method can also generate by given only monolingual inputs. In contrast, search methods must be provided the same amount of well-informed targets. NMT systems may have different robustness over different parts of the inputs, thus some researchers implement input preprocessing targeting certain empirically weak parts, e.g., named entities(Li et al., 2018). Since the agent's policy is to attack without handcrafted error features, we can further investigate vulnerability by its attack preferences of different parts of speech. We choose Chinese, for example, and adopt LTP POS tagger 6 to label NIST test sets, then check the modification rate for each POS. To ensure the reliability of our analysis, we run three rounds of experiments on both baselines and our agent with similar modification rate targeting state-of-the-art Transformer with BPE, and collect overall results. We also present random synthetic noise injection (Karpukhin et al., 2019) (RSNI), which is not intended for any preference as an additional baseline. As it is shown in Figure 3, our reinforced paradigm shows distinct preference upon certain POS tags, indicating pitfalls of a victim translation system. At the same time, RSNI distributed almost evenly upon different POS tags. Though the search paradigm (GS) does expose some types of pitfall, our method can further expose those omitted by the search. Note that unlike existing work relying on feature engineering to indicate errors, we have no such features implemented for an agent. However, our agent can still spot error patterns by favoring some of the POS, such as 6 https://github.com/HIT-SCIR/ltp softmax linear+dropout   { tgt bi-GRU src bi-GRU Tokens embedding actor linear linear+dropout Y N { src bi-GRU V ( s t ) critic linear Mean Mean Mean Sum Environment TGT SRC } x t x t + 1 x t 1   Y emb Y X X emb XX emb discriminator agent actor critic survival/terminal signals s t a t } & Ni (organization name), Nh (person name), Nl (lo-cation name), M (numbers), which are commonly accepted as hard-to-translate parts. Moreover, the agent also tends to favor K (suffix) more, which is less noticed. We additionally test agents by attacking different model architecture from the one that it's trained. As it is shown in Table 6, we perturb the inputs by agents trained to attack a different architecture, then test for degradation. The results show that our agent trained by targeting Transformer architecture can still achieve degradation on RNN-search, and vice-versa. Since the agent generates meaning-preserving adversarial examples efficiently, we can directly tune the original model with those samples. We choose Zh  En Transformer-BPE, for example, and generate the same amount of adversarial examples given original training sources(1.3M pairs), then paired with initial targets. We mix the augmented pairs with original pairs for a direct tuning. We test the tuned model on original test data and noisy test data generated by the attacking agent. We additionally test on IWSLT11-17 Zh  En test data, which is not used for training or tuning, to verified robustness improvement. As Table 7 shows, our agent can achieve substantial improvement(+4.83) on noisy tests with only minor loss on clean tests(-0.46). The improvement on the IWSLT test also indicates the adversarial tuning contributes to not only defending the agent's attack, but also overall robustness. We additionally switched the episodic rewards in the environment, then ignored all modifications that induce UNK tokens to train an agent, hoping to generate minor perturbed samples that can improve the translation metric. Though we failed to achieve overall improvements, we do succeed for quite a portion of samples, as shown in Table 8. Similar to adversarial examples, we call them reinforced examples . Such improvement is different from adversarial training that tunes model in for defense or strict text correction before the test phase. Reinforced examples are still noisy and can be directly applied for a test without any model updates to achieve improvements, which to our best knowledge is less investigated by researchers. Since we discovered that not all perturbed inputs are harmful, such an issue can be a good hint and alternative for better adversarial defense in NLP and should be further considered. Cheng et al. (2018a) and Cheng et al. (2018b) applied continuous perturbation learning on token's embedding and then manage a lexical representation out of a perturbed embedding. Zhao et al. (2017) learned such perturbation on the encoded representation of a sequence, and then decode it back as an adversarial example. These methods are applicable for simple NLP classification tasks, while failing machine translation which requires higher semantic constraints. Zhao et al. (2017) further attempted to constrain semantic in such paradigm by introducing multi-task modeling with accessory annotation, which further limits applicability. On the other hand, Ebrahimi et al. (2018), Chaturvedi et al. (2019) and Cheng et al. (2019) regarded it as a search problem by maximizing surrogate gradient losses. Due to the formidable gradient computation, such methods are less viable to more complex neural architectures. Cheng et al. (2019) introduced a learned language model to constrain generation. However, a learned language model is not apt for common typos or UNK. Another pitfall of this paradigm is that surrogate losses defined by a fixed tokenization for non-character level systems, risks being invalidated once the attack changes tokenization. Therefore, Ebrahimi et al. (2018) simply focused on char-level systems, while Michel et al. (2019) specially noted to exclude scenarios where attack changes tokenization in their paradigm. Other works turn to more sophisticated generation paradigms, e.g., Vidnerova and Neruda (2016) adopts a genetic algorithm for an evolutionary generation targeting simple machine learning models. Zang et al. (2019) consider adversarial generation as a word substitution-based combinatorial optimization problem tackled by particle swarm algorithm. Our paradigm shares some common ideology with Miao et al. (2019) and Xiao et al. (2018), which iteratively edit inputs constrained by generative adversarial learning. We propose a new paradigm to generate adversarial examples for neural machine translation, which is capable of exposing translation pitfalls without handcrafted error features. Experiments show that our method achieves stable degradation with meaning preserving adversarial examples over different victim models. It is noticeable that our method can generate adversarial examples efficiently from monolingual data. As a result, the mass production of adversarial examples for the victim model's analysis and further improvement of robustness become convenient. Furthermore, we notice some exceptional cases which we call as reinforced sam-ples, which we leave as the future work. We would like to thank the anonymous reviewers for their insightful comments. Shujian Huang is the corresponding author. This work is supported by National Science Foundation of China (No. 61672277, 61772261), the Jiangsu Provincial Research Foundation for Basic Research (No. BK20170074).
abstain objective abstain method result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective result result abstain result abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain result abstain result result abstain result result abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method abstain abstain method method other abstain abstain method abstain abstain method method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain method other other other other other other other other other other other other abstain objective result method abstain abstain other other other