DATNet: Dual Adversarial Transfer for Low-resource Named Entity Recognition
We propose a new architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER). Specifically, two variants of DATNet, i.e., DATNet-F and DATNet-P, are proposed to explore effective feature fusion between high and low resource. To address the noisy and imbalanced training data, we propose a novel Generalized Resource-Adversarial Discriminator (GRAD). Additionally, adversarial training is adopted to boost model generalization. We examine the effects of different components in DATNet across domains and languages and show that significant improvement can be obtained especially for low-resource data. Without augmenting any additional hand-crafted features, we achieve new state-of-the-art performances on CoNLL and Twitter NER---88.16% F1 for Spanish, 53.43% F1 for WNUT-2016, and 42.83% F1 for WNUT-2017.
 <Summary> Authors propose the new “DATNet” for the NER task, which extends the base neural model for NER (Bi-LSTM+CRF sequence model with input represented with CharCNN-word embeddings) with the following two main components: (1) GRAD: a language (or resource) adversarial discriminator with adaptive weights that regularize source-target data imbalance, and (2) additional adversarial training approaches that perturb input samples in the embeddings space. The paper reports big improvement over their baseline approaches without having to rely on other auxiliary or hand-crafted features. The experiment is performed for various low-resource scenarios (varying training data size). <Comments> - While the idea of applying “dual adversarial” approaches is new in the context of NER, the technical novelty of each component is limited. GRAD, for example, is rather a minor modification of Language Adversarial Discriminator (Kim et al., 2017) with a scalar weight parameter on loss. The empirical superiority of the proposed method (GRAD) over normal AD approaches cannot be claimed either -- for this authors need to report quantitative results of the (Base + AT + F/P-transfer with AD -- or e.g. .alpha fixed at 0.5 or at some other rate), which I believe is missing in all figures and tables. Visualization of resulting feature distribution (Figure 3) is interesting to look at, but that alone does not suffice. There is no technical novelty in applying adversarial training either, except that it was used in the context of NER. - Authors use both source and target data to train their “base model” as well (“... we exploit all the source data and target data ...“), e.g. presumably by merging the source and target dataset as well as their embedding matrices, etc. It is important that authors report if there ever is a negative transfer case (e.g. a base model trained with just target data may outperform models trained with source+target data) at varying resource scenarios -- especially at sufficiently-resourced cases. If by any chance their “base model” refers to in-domain training / in-domain testing results on target, the aforementioned baseline (naive merge of source and target data) is obviously an important baseline to report. I suggest that authors provide these details or clarify. (The confusion comes mainly because some of the SOTA results the authors quote are in-domain training / in-domain evaluation results on respective languages, and some are cross-domain results -- yet they are all under “cross-lingual/domain” columns in Table 2). - It would be interesting to report the learned optimal .alpha value for each different setting (at varying r or training size) to see if the authors’ intuition is met. On a related note, from the manuscript alone, it is not entirely clear if .alpha is a learnable parameter or a tunable hyper parameter -- by context I believe it is a model parameter. If they are tuned, authors need to report these values. <Nit> - Section 3.2.3, “... (GRAD) to enable adaptive weights for [each sample]” → I think it reads better with [each resource] or [each source and target], unless you meant .alpha_i for each sentence. - Section 3.2.5, “... recently, adversarial samples are [wisely] incorporated” → [widely] - Fonts for figures could be bigger.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
DATNet: Dual Adversarial Transfer for Low-resource Named Entity Recognition
We propose a new architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER). Specifically, two variants of DATNet, i.e., DATNet-F and DATNet-P, are proposed to explore effective feature fusion between high and low resource. To address the noisy and imbalanced training data, we propose a novel Generalized Resource-Adversarial Discriminator (GRAD). Additionally, adversarial training is adopted to boost model generalization. We examine the effects of different components in DATNet across domains and languages and show that significant improvement can be obtained especially for low-resource data. Without augmenting any additional hand-crafted features, we achieve new state-of-the-art performances on CoNLL and Twitter NER---88.16% F1 for Spanish, 53.43% F1 for WNUT-2016, and 42.83% F1 for WNUT-2017.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 <Summary> Authors propose the new “DATNet” for the NER task, which extends the base neural model for NER (Bi-LSTM+CRF sequence model with input represented with CharCNN-word embeddings) with the following two main components: (1) GRAD: a language (or resource) adversarial discriminator with adaptive weights that regularize source-target data imbalance, and (2) additional adversarial training approaches that perturb input samples in the embeddings space. The paper reports big improvement over their baseline approaches without having to rely on other auxiliary or hand-crafted features. The experiment is performed for various low-resource scenarios (varying training data size). <Comments> - While the idea of applying “dual adversarial” approaches is new in the context of NER, the technical novelty of each component is limited. GRAD, for example, is rather a minor modification of Language Adversarial Discriminator (Kim et al., 2017) with a scalar weight parameter on loss. The empirical superiority of the proposed method (GRAD) over normal AD approaches cannot be claimed either -- for this authors need to report quantitative results of the (Base + AT + F/P-transfer with AD -- or e.g. .alpha fixed at 0.5 or at some other rate), which I believe is missing in all figures and tables. Visualization of resulting feature distribution (Figure 3) is interesting to look at, but that alone does not suffice. There is no technical novelty in applying adversarial training either, except that it was used in the context of NER. - Authors use both source and target data to train their “base model” as well (“... we exploit all the source data and target data ...“), e.g. presumably by merging the source and target dataset as well as their embedding matrices, etc. It is important that authors report if there ever is a negative transfer case (e.g. a base model trained with just target data may outperform models trained with source+target data) at varying resource scenarios -- especially at sufficiently-resourced cases. If by any chance their “base model” refers to in-domain training / in-domain testing results on target, the aforementioned baseline (naive merge of source and target data) is obviously an important baseline to report. I suggest that authors provide these details or clarify. (The confusion comes mainly because some of the SOTA results the authors quote are in-domain training / in-domain evaluation results on respective languages, and some are cross-domain results -- yet they are all under “cross-lingual/domain” columns in Table 2). - It would be interesting to report the learned optimal .alpha value for each different setting (at varying r or training size) to see if the authors’ intuition is met. On a related note, from the manuscript alone, it is not entirely clear if .alpha is a learnable parameter or a tunable hyper parameter -- by context I believe it is a model parameter. If they are tuned, authors need to report these values. <Nit> - Section 3.2.3, “... (GRAD) to enable adaptive weights for [each sample]” → I think it reads better with [each resource] or [each source and target], unless you meant .alpha_i for each sentence. - Section 3.2.5, “... recently, adversarial samples are [wisely] incorporated” → [widely] - Fonts for figures could be bigger.
assistant
 <Summary> Authors propose the new “DATNet” for the NER task, which extends the base neural model for NER (Bi-LSTM+CRF sequence model with input represented with CharCNN-word embeddings) with the following two main components: (1) GRAD: a language (or resource) adversarial discriminator with adaptive weights that regularize source-target data imbalance, and (2) additional adversarial training approaches that perturb input samples in the embeddings space. The paper reports big improvement over their baseline approaches without having to rely on other auxiliary or hand-crafted features. The experiment is performed for various low-resource scenarios (varying training data size). <Comments> - While the idea of applying “dual adversarial” approaches is new in the context of NER, the technical novelty of each component is limited. GRAD, for example, is rather a minor modification of Language Adversarial Discriminator (Kim et al., 2017) with a scalar weight parameter on loss. The empirical superiority of the proposed method (GRAD) over normal AD approaches cannot be claimed either -- for this authors need to report quantitative results of the (Base + AT + F/P-transfer with AD -- or e.g. .alpha fixed at 0.5 or at some other rate), which I believe is missing in all figures and tables. Visualization of resulting feature distribution (Figure 3) is interesting to look at, but that alone does not suffice. There is no technical novelty in applying adversarial training either, except that it was used in the context of NER. - Authors use both source and target data to train their “base model” as well (“... we exploit all the source data and target data ...“), e.g. presumably by merging the source and target dataset as well as their embedding matrices, etc. It is important that authors report if there ever is a negative transfer case (e.g. a base model trained with just target data may outperform models trained with source+target data) at varying resource scenarios -- especially at sufficiently-resourced cases. If by any chance their “base model” refers to in-domain training / in-domain testing results on target, the aforementioned baseline (naive merge of source and target data) is obviously an important baseline to report. I suggest that authors provide these details or clarify. (The confusion comes mainly because some of the SOTA results the authors quote are in-domain training / in-domain evaluation results on respective languages, and some are cross-domain results -- yet they are all under “cross-lingual/domain” columns in Table 2). - It would be interesting to report the learned optimal .alpha value for each different setting (at varying r or training size) to see if the authors’ intuition is met. On a related note, from the manuscript alone, it is not entirely clear if .alpha is a learnable parameter or a tunable hyper parameter -- by context I believe it is a model parameter. If they are tuned, authors need to report these values. <Nit> - Section 3.2.3, “... (GRAD) to enable adaptive weights for [each sample]” → I think it reads better with [each resource] or [each source and target], unless you meant .alpha_i for each sentence. - Section 3.2.5, “... recently, adversarial samples are [wisely] incorporated” → [widely] - Fonts for figures could be bigger.