AdvCodec: Towards A Unified Framework for Adversarial Text Generation
Machine learning (ML) especially deep neural networks (DNNs) have been widely applied to real-world applications. However, recent studies show that DNNs are vulnerable to carefully crafted emph{adversarial examples} which only deviate from the original data by a small magnitude of perturbation. While there has been great interest on generating imperceptible adversarial examples in continuous data domain (e.g. image and audio) to explore the model vulnerabilities, generating emph{adversarial text} in the discrete domain is still challenging. The main contribution of this paper is to propose a general targeted attack framework advcodec for adversarial text generation which addresses the challenge of discrete input space and be easily adapted to general natural language processing (NLP) tasks. In particular, we propose a tree based autoencoder to encode discrete text data into continuous vector space, upon which we optimize the adversarial perturbation. With the tree based decoder, it is possible to ensure the grammar correctness of the generated text; and the tree based encoder enables flexibility of making manipulations on different levels of text, such as sentence (advcodecsent) and word (advcodecword) levels. We consider multiple attacking scenarios, including appending an adversarial sentence or adding unnoticeable words to a given paragraph, to achieve arbitrary emph{targeted attack}. To demonstrate the effectiveness of the proposed method, we consider two most representative NLP tasks: sentiment analysis and question answering (QA). Extensive experimental results show that advcodec has successfully attacked both tasks. In particular, our attack causes a BERT-based sentiment classifier accuracy to drop from to , and a BERT-based QA model*s F1 score to drop from to (with best targeted attack F1 score as ). Furthermore, we show that the white-box generated adversarial texts can transfer across other black-box models, shedding light on an effective way to examine the robustness of existing NLP models.
Motivated by recent development of attack/defense methods addressing the vulnerability of deep CNN classifiers for images, this paper proposes an attack framework for adversarial text generation, in which an autoencoder is employed to map discrete text to a high-dimensional continuous latent space, standard iterative optimization based attack method is performed in the continuous latent space to generate adversarial latent embeddings, and a decoder generates adversarial text from the adversarial embeddings. Different generation strategies of perturbing latent embeddings at sentence level or masked word level are both explored. Adversarial text generation can take either a form of appending an adversarial sentence or a form of scattering adversarial words into different specified positions. Experiments on both sentiment classification and question answering show that the proposed attack framework outperforms some baselines. Human evaluations are also conducted. Pros: This paper is well-written overall. Extensive experiments are performed. Many human studies comparing different adversarial text generation strategies and evaluating adversarial text for sentiment classification/question answering are conducted. Cons: 1) Although the studied problem in this paper is interesting, the technical innovation is very limited. All the techniques are standard or known. 2) There are two major issues: lacking a rigorous metric of human unnoticeability and lacking justification of the advantage of the tree-based autoencoder. I think the first issue is a major problem that renders all the claims in this paper questionable. The metrics used to define adversarial images for deep CNN classifiers are indeed valid and produce unnoticeable images for human observers. But in this paper, the adversarial attack is performed in the latent embedding space, and there is no explicit constraint enforced on the output text. It’s unconvincing that this approach will generate adversarial text that seems negligible to humans. Therefore, the studied problem in this paper has a completely different nature from the one for CNN image classifiers and it is hard to convince readers that the proposed framework generates adversarial text legitimate to human readers. 3) It is unclear why tree-structured LSTM instead of a standard LSTM/GRU should be chosen in this framework for adversarial text generation. If this architecture is preferred, sufficient ablation studies should be conducted. 4) In section 3.3, the description about adversarial attacks at word level is unclear. More detailed loss function and algorithms along with equations should be provided. 5) In section 5.2, it is unclear that the majority answers on the adversarial text will, respectively, match the majority answers on the original text. Moreover, it seems that there is a large performance drop from original text to adversarial text. Therefore, it is valid to argue that whether the proposed framework can generate legitimate adversarial text to human readers or not. 6) It’s better to include many examples of generated adversarial text in the appendix. 7) Missing training details: It is unclear how the model architectures are chosen, and learning rate, optimizer, training epochs etc. are also missing. All these training details should be included in the appendix. 8) Minor: Figure 1: *Append an initial sentence...*, section 3: *map discrete text into a high dimensional...*, section 3.2.2: *Different from attacking sentiment analysis...* .... In summary, the research direction of adversarial text generation studied in this paper is interesting and promising. However, some technical details are questionable, and the produced results without rigorous metrics seem to be unconvincing.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
AdvCodec: Towards A Unified Framework for Adversarial Text Generation
Machine learning (ML) especially deep neural networks (DNNs) have been widely applied to real-world applications. However, recent studies show that DNNs are vulnerable to carefully crafted emph{adversarial examples} which only deviate from the original data by a small magnitude of perturbation. While there has been great interest on generating imperceptible adversarial examples in continuous data domain (e.g. image and audio) to explore the model vulnerabilities, generating emph{adversarial text} in the discrete domain is still challenging. The main contribution of this paper is to propose a general targeted attack framework advcodec for adversarial text generation which addresses the challenge of discrete input space and be easily adapted to general natural language processing (NLP) tasks. In particular, we propose a tree based autoencoder to encode discrete text data into continuous vector space, upon which we optimize the adversarial perturbation. With the tree based decoder, it is possible to ensure the grammar correctness of the generated text; and the tree based encoder enables flexibility of making manipulations on different levels of text, such as sentence (advcodecsent) and word (advcodecword) levels. We consider multiple attacking scenarios, including appending an adversarial sentence or adding unnoticeable words to a given paragraph, to achieve arbitrary emph{targeted attack}. To demonstrate the effectiveness of the proposed method, we consider two most representative NLP tasks: sentiment analysis and question answering (QA). Extensive experimental results show that advcodec has successfully attacked both tasks. In particular, our attack causes a BERT-based sentiment classifier accuracy to drop from to , and a BERT-based QA model*s F1 score to drop from to (with best targeted attack F1 score as ). Furthermore, we show that the white-box generated adversarial texts can transfer across other black-box models, shedding light on an effective way to examine the robustness of existing NLP models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Motivated by recent development of attack/defense methods addressing the vulnerability of deep CNN classifiers for images, this paper proposes an attack framework for adversarial text generation, in which an autoencoder is employed to map discrete text to a high-dimensional continuous latent space, standard iterative optimization based attack method is performed in the continuous latent space to generate adversarial latent embeddings, and a decoder generates adversarial text from the adversarial embeddings. Different generation strategies of perturbing latent embeddings at sentence level or masked word level are both explored. Adversarial text generation can take either a form of appending an adversarial sentence or a form of scattering adversarial words into different specified positions. Experiments on both sentiment classification and question answering show that the proposed attack framework outperforms some baselines. Human evaluations are also conducted. Pros: This paper is well-written overall. Extensive experiments are performed. Many human studies comparing different adversarial text generation strategies and evaluating adversarial text for sentiment classification/question answering are conducted. Cons: 1) Although the studied problem in this paper is interesting, the technical innovation is very limited. All the techniques are standard or known. 2) There are two major issues: lacking a rigorous metric of human unnoticeability and lacking justification of the advantage of the tree-based autoencoder. I think the first issue is a major problem that renders all the claims in this paper questionable. The metrics used to define adversarial images for deep CNN classifiers are indeed valid and produce unnoticeable images for human observers. But in this paper, the adversarial attack is performed in the latent embedding space, and there is no explicit constraint enforced on the output text. It’s unconvincing that this approach will generate adversarial text that seems negligible to humans. Therefore, the studied problem in this paper has a completely different nature from the one for CNN image classifiers and it is hard to convince readers that the proposed framework generates adversarial text legitimate to human readers. 3) It is unclear why tree-structured LSTM instead of a standard LSTM/GRU should be chosen in this framework for adversarial text generation. If this architecture is preferred, sufficient ablation studies should be conducted. 4) In section 3.3, the description about adversarial attacks at word level is unclear. More detailed loss function and algorithms along with equations should be provided. 5) In section 5.2, it is unclear that the majority answers on the adversarial text will, respectively, match the majority answers on the original text. Moreover, it seems that there is a large performance drop from original text to adversarial text. Therefore, it is valid to argue that whether the proposed framework can generate legitimate adversarial text to human readers or not. 6) It’s better to include many examples of generated adversarial text in the appendix. 7) Missing training details: It is unclear how the model architectures are chosen, and learning rate, optimizer, training epochs etc. are also missing. All these training details should be included in the appendix. 8) Minor: Figure 1: *Append an initial sentence...*, section 3: *map discrete text into a high dimensional...*, section 3.2.2: *Different from attacking sentiment analysis...* .... In summary, the research direction of adversarial text generation studied in this paper is interesting and promising. However, some technical details are questionable, and the produced results without rigorous metrics seem to be unconvincing.
assistant
Motivated by recent development of attack/defense methods addressing the vulnerability of deep CNN classifiers for images, this paper proposes an attack framework for adversarial text generation, in which an autoencoder is employed to map discrete text to a high-dimensional continuous latent space, standard iterative optimization based attack method is performed in the continuous latent space to generate adversarial latent embeddings, and a decoder generates adversarial text from the adversarial embeddings. Different generation strategies of perturbing latent embeddings at sentence level or masked word level are both explored. Adversarial text generation can take either a form of appending an adversarial sentence or a form of scattering adversarial words into different specified positions. Experiments on both sentiment classification and question answering show that the proposed attack framework outperforms some baselines. Human evaluations are also conducted. Pros: This paper is well-written overall. Extensive experiments are performed. Many human studies comparing different adversarial text generation strategies and evaluating adversarial text for sentiment classification/question answering are conducted. Cons: 1) Although the studied problem in this paper is interesting, the technical innovation is very limited. All the techniques are standard or known. 2) There are two major issues: lacking a rigorous metric of human unnoticeability and lacking justification of the advantage of the tree-based autoencoder. I think the first issue is a major problem that renders all the claims in this paper questionable. The metrics used to define adversarial images for deep CNN classifiers are indeed valid and produce unnoticeable images for human observers. But in this paper, the adversarial attack is performed in the latent embedding space, and there is no explicit constraint enforced on the output text. It’s unconvincing that this approach will generate adversarial text that seems negligible to humans. Therefore, the studied problem in this paper has a completely different nature from the one for CNN image classifiers and it is hard to convince readers that the proposed framework generates adversarial text legitimate to human readers. 3) It is unclear why tree-structured LSTM instead of a standard LSTM/GRU should be chosen in this framework for adversarial text generation. If this architecture is preferred, sufficient ablation studies should be conducted. 4) In section 3.3, the description about adversarial attacks at word level is unclear. More detailed loss function and algorithms along with equations should be provided. 5) In section 5.2, it is unclear that the majority answers on the adversarial text will, respectively, match the majority answers on the original text. Moreover, it seems that there is a large performance drop from original text to adversarial text. Therefore, it is valid to argue that whether the proposed framework can generate legitimate adversarial text to human readers or not. 6) It’s better to include many examples of generated adversarial text in the appendix. 7) Missing training details: It is unclear how the model architectures are chosen, and learning rate, optimizer, training epochs etc. are also missing. All these training details should be included in the appendix. 8) Minor: Figure 1: *Append an initial sentence...*, section 3: *map discrete text into a high dimensional...*, section 3.2.2: *Different from attacking sentiment analysis...* .... In summary, the research direction of adversarial text generation studied in this paper is interesting and promising. However, some technical details are questionable, and the produced results without rigorous metrics seem to be unconvincing.