Adversarial examples have been shown to be an effective way of assessing the robustness of neural sequence-to-sequence (seq2seq) models, by applying perturbations to the input of a model leading to large degradation in performance. However, these perturbations are only indicative of a weakness in the model if they do not change the semantics of the input in a way that would change the expected output. Using the example of machine translation (MT), we propose a new evaluation framework for adversarial attacks on seq2seq models taking meaning preservation into account and demonstrate that existing methods may not preserve meaning in general. Based on these findings, we propose new constraints for attacks on word-based MT systems and show, via human and automatic evaluation, that they produce more semantically similar adversarial inputs. Furthermore, we show that performing adversarial training with meaning-preserving attacks is beneficial to the model in terms of adversarial robustness without hurting test performance.
Summary: Proposes a framework for performing adversarial attacks on an NMT system in which perturbations to a source sentence aim to preserve its meaning, on the theory that an existing reference translation will remain valid if this is done. Given source and target metrics for measuring similarity, an attack is deemed successful if the source difference is smaller than the relative decrease in target similarity to the reference. A first experiment measures correlation with human judgements of similarity between original and perturbed sentences, and concludes that chrF is better than BLEU and METEOR for this purpose. Next, standard gradient-based adversarial attacks are carried out, replacing the three tokens that result in the biggest drop in (approximate) reference probability, either 1) with no constraints, 2) constrained to character swaps of the original token, or 3) constrained be among the 10 closest embeddings to the original token. In comparisons on three language pairs from IWSLT,  the constrained attacks are found to preserve meaning and yield more successful attacks according to the current framework. The Transformer architecture was also found to deal less well with attacks under the 10-closest embedding constraint. Finally, adversarial training with the character-swap constraint confers some robustness to this attack, without degrading performance on normal text.
I think it is a good idea to formalize a method for carrying out and assessing adversarial attacks, but the framework proposed here seems too narrow, as it excludes adversarial inputs that are sensible but not a close perturbation of an existing source/reference pair, or ones that contain varying amounts of noise. It is more difficult to measure output quality for such attacks, but that doesn’t seem like a good reason for excluding them from what is intended to be a general framework. Note also that “more difficult” doesn’t mean impossible, since good attacks can produce severely degraded output that is relatively easy to detect.
I found some of the methodology questionable. Limiting source perturbations to character swaps and neighbors in embedding space, then using automatic metrics to measure semantic distance seems both unnecessary and unlikely to succeed. Unnecessary because knowing the class of perturbation already gives you a lot of information about semantic distance. Unlikely to succeed because automatic metrics are too coarse to reliably distinguish among different perturbations. This is particularly obvious in the case of using character ngram distance (chrF) to determine which character swaps preserve meaning best. The experiments that support the viability of automatic metrics in 4.2 do so by measuring correlation with human judgment when the number of perturbed tokens varies from 1 to 3. I think the good correlation is likely due to the metrics being able to detect that, eg, changing 3 tokens makes things worse than changing only one. To be convincing, the experiments would have to be repeated with number of perturbations fixed at 3, to match the setting in the remaining experiments. 
Apart from the interesting observation about the Transformer’s performance on embedding-neighbor attacks mentioned above, it is difficult to know what conclusions to draw from the experiments. In 4.3 it seems obvious a priori that perturbations intended to be relatively meaning preserving would indeed preserve meaning better than unconstrained ones. Similarly, it is not surprising that character swaps that by design produce an OOV token will cause more damage than choosing a near neighbor in embedding space. In 5.3, training with OOVs (resulting from character swaps) is of course not likely to hurt performance on test sets containing few OOVs, and, as is known from previous work, it will improve robustness to the same kind of noise. A final comment about the experiments is that word-based systems are not state of the art, and it isn’t clear how much we could expect any conclusions to carry over to sub-word models.
To conclude, although this is an interesting initiative, both the framework and the methodology need to be tightened up.
Details:
End of 2.1: this would be easier to interpret if you had previously specified the allowed range for s_src.
3.2 For kNN, being semantically related doesn’t imply that the relationship is synonymy, as would be required for meaning preservation. It also doesn’t imply that the substitution will be grammatical, which could jeopardize meaning preservation even if the words are synonyms.
CharSwap seems odd. If you’re just going to replace a work with an OOV symbol in any case, why go to the trouble of swapping characters? No matter what actual semantic shift is caused by the swap, the model will always see exactly the same representation.
4.1 “Following previous work on adversarial examples for seq2seq models (Belinkov & Bisk, 2018; Ebrahimi et al., 2018a)” - this is misleading: Ebrahimi et al only work with classification, and don’t use IWLST.
4.1 Should mention the size of the training sets in this section.
Table 1, first sentence, CharSwap example omits “faire”.
4.3, “Adding Constraints Helps Preserve…” last sentence: but here you need to reason in the opposite direction.
5.2 It would be good to also give absolute scores for table 6, so we can judge how much the systems actually benefited, and whether these gains were statistically significant.