DOMAIN ADAPTATION VIA DISTRIBUTION AND REPRESENTATION MATCHING: A CASE STUDY ON TRAINING DATA SELECTION VIA REINFORCEMENT LEARNING
Supervised models suffer from domain shifting where distribution mismatch across domains greatly affect model performance. Particularly, noise scattered in each domain has played a crucial role in representing such distribution, especially in various natural language processing (NLP) tasks. In addressing this issue, training data selection (TDS) has been proven to be a prospective way to train supervised models with higher performance and efficiency. Following the TDS methodology, in this paper, we propose a general data selection framework with representation learning and distribution matching simultaneously for domain adaptation on neural models. In doing so, we formulate TDS as a novel selection process based on a learned distribution from the input data, which is produced by a trainable selection distribution generator (SDG) that is optimized by reinforcement learning (RL). Then, the model trained by the selected data not only predicts the target domain data in a specific task, but also provides input for the value function of the RL. Experiments are conducted on three typical NLP tasks, namely, part-of-speech tagging, dependency parsing, and sentiment analysis. Results demonstrate the validity and effectiveness of our approach.
 == Originality == The idea of matching features/representations across the source domain and target domain is an old idea, but it is executed in an interesting new way in this paper. In this approach, feature representations are learned by training a neural classifier on the source domain, and an RL agent influences the feature representation by iteratively adding/removing examples from the source training data. The RL agent receives reward when the resulting feature representation causes the source domain data and target domain data to look more similar in distribution in feature space. To efficiently estimate the improvement in feature matching, a nice data bucketing strategy is used. The novelty of the approach is the main strength of this paper. == Quality of results == The experimental results seemed overall positive, but I felt that they could have been stronger. For POS tagging, the authors don*t compare against the domain adaptation methods mentioned in their related work section. Instead, they compare against Bayesian optimization using several heuristic criteria, and it was unclear where this baseline comes from. This made it hard to see whether the new approach represents a true improvement over existing techniques. For dependency parsing, it appears that the proposed approach is outperformed by simply training on all of the source domain data. It would be interesting to know whether this is because feature-matching is not a good proxy for target domain performance (objective mismatch) or whether the RL system converged to a poor local optima (optimization failure). == Clarity == I felt that the abstract and introduction were vague in describing the main conceptual contribution. However, Section 2 (The Approach) was clearly written and I came away understanding exactly what the authors are doing. == Minor comments == - Algorithm 1 seems to have a typo: the definition of .nabla .tilde{J}(W) on the second to last line seems to be missing .nabla .log .pi - Many citations throughout the paper need to be wrapped in parentheses == Conclusion == This paper presents an interesting new approach for dataset selection and learning domain-invariant representations. Pros: - originality of the approach Cons: - Experiments could have been more convincing: - should compare against at least one other state-of-the-art domain adaptation method - results on dependency parsing (the most challenging task they consider) were mostly negative - evaluation on other more recent multi-domain NLP tasks would have been nice (e.g. MultiNLI) - Abstract and intro could provide better description of the conceptual contribution, as well as motivation
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
DOMAIN ADAPTATION VIA DISTRIBUTION AND REPRESENTATION MATCHING: A CASE STUDY ON TRAINING DATA SELECTION VIA REINFORCEMENT LEARNING
Supervised models suffer from domain shifting where distribution mismatch across domains greatly affect model performance. Particularly, noise scattered in each domain has played a crucial role in representing such distribution, especially in various natural language processing (NLP) tasks. In addressing this issue, training data selection (TDS) has been proven to be a prospective way to train supervised models with higher performance and efficiency. Following the TDS methodology, in this paper, we propose a general data selection framework with representation learning and distribution matching simultaneously for domain adaptation on neural models. In doing so, we formulate TDS as a novel selection process based on a learned distribution from the input data, which is produced by a trainable selection distribution generator (SDG) that is optimized by reinforcement learning (RL). Then, the model trained by the selected data not only predicts the target domain data in a specific task, but also provides input for the value function of the RL. Experiments are conducted on three typical NLP tasks, namely, part-of-speech tagging, dependency parsing, and sentiment analysis. Results demonstrate the validity and effectiveness of our approach.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 == Originality == The idea of matching features/representations across the source domain and target domain is an old idea, but it is executed in an interesting new way in this paper. In this approach, feature representations are learned by training a neural classifier on the source domain, and an RL agent influences the feature representation by iteratively adding/removing examples from the source training data. The RL agent receives reward when the resulting feature representation causes the source domain data and target domain data to look more similar in distribution in feature space. To efficiently estimate the improvement in feature matching, a nice data bucketing strategy is used. The novelty of the approach is the main strength of this paper. == Quality of results == The experimental results seemed overall positive, but I felt that they could have been stronger. For POS tagging, the authors don*t compare against the domain adaptation methods mentioned in their related work section. Instead, they compare against Bayesian optimization using several heuristic criteria, and it was unclear where this baseline comes from. This made it hard to see whether the new approach represents a true improvement over existing techniques. For dependency parsing, it appears that the proposed approach is outperformed by simply training on all of the source domain data. It would be interesting to know whether this is because feature-matching is not a good proxy for target domain performance (objective mismatch) or whether the RL system converged to a poor local optima (optimization failure). == Clarity == I felt that the abstract and introduction were vague in describing the main conceptual contribution. However, Section 2 (The Approach) was clearly written and I came away understanding exactly what the authors are doing. == Minor comments == - Algorithm 1 seems to have a typo: the definition of .nabla .tilde{J}(W) on the second to last line seems to be missing .nabla .log .pi - Many citations throughout the paper need to be wrapped in parentheses == Conclusion == This paper presents an interesting new approach for dataset selection and learning domain-invariant representations. Pros: - originality of the approach Cons: - Experiments could have been more convincing: - should compare against at least one other state-of-the-art domain adaptation method - results on dependency parsing (the most challenging task they consider) were mostly negative - evaluation on other more recent multi-domain NLP tasks would have been nice (e.g. MultiNLI) - Abstract and intro could provide better description of the conceptual contribution, as well as motivation
assistant
 == Originality == The idea of matching features/representations across the source domain and target domain is an old idea, but it is executed in an interesting new way in this paper. In this approach, feature representations are learned by training a neural classifier on the source domain, and an RL agent influences the feature representation by iteratively adding/removing examples from the source training data. The RL agent receives reward when the resulting feature representation causes the source domain data and target domain data to look more similar in distribution in feature space. To efficiently estimate the improvement in feature matching, a nice data bucketing strategy is used. The novelty of the approach is the main strength of this paper. == Quality of results == The experimental results seemed overall positive, but I felt that they could have been stronger. For POS tagging, the authors don*t compare against the domain adaptation methods mentioned in their related work section. Instead, they compare against Bayesian optimization using several heuristic criteria, and it was unclear where this baseline comes from. This made it hard to see whether the new approach represents a true improvement over existing techniques. For dependency parsing, it appears that the proposed approach is outperformed by simply training on all of the source domain data. It would be interesting to know whether this is because feature-matching is not a good proxy for target domain performance (objective mismatch) or whether the RL system converged to a poor local optima (optimization failure). == Clarity == I felt that the abstract and introduction were vague in describing the main conceptual contribution. However, Section 2 (The Approach) was clearly written and I came away understanding exactly what the authors are doing. == Minor comments == - Algorithm 1 seems to have a typo: the definition of .nabla .tilde{J}(W) on the second to last line seems to be missing .nabla .log .pi - Many citations throughout the paper need to be wrapped in parentheses == Conclusion == This paper presents an interesting new approach for dataset selection and learning domain-invariant representations. Pros: - originality of the approach Cons: - Experiments could have been more convincing: - should compare against at least one other state-of-the-art domain adaptation method - results on dependency parsing (the most challenging task they consider) were mostly negative - evaluation on other more recent multi-domain NLP tasks would have been nice (e.g. MultiNLI) - Abstract and intro could provide better description of the conceptual contribution, as well as motivation