Semi-supervised Learning with Multi-Domain Sentiment Word Embeddings
Word embeddings are known to boost performance of many NLP tasks such as text classification, meanwhile they can be enhanced by labels at the document level to capture nuanced meaning such as sentiment and topic. Can one combine these two research directions to benefit from both? In this paper, we propose to jointly train a text classifier with a label-enhanced and domain-aware word embedding model, using an unlabeled corpus and only a few labeled data from non-target domains. The embeddings are trained on the unlabed corpus and enhanced by pseudo labels coming from the classifier, and at the same time are used by the classifier as input and training signals. We formalize this symbiotic cycle in a variational Bayes framework, and show that our method improves both the embeddings and the text classifier, outperforming state-of-the-art domain adaptation and semi-supervised learning techniques. We conduct detailed ablative tests to reveal gains from important components of our approach. The source code and experiment data will be publicly released.
 This paper jointly trains a sentiment classifier with a sentiment and domain-aware embedding model, using both labeled and unlabeled data. When sentiment label is observed, this model is trained with the usual cross entropy and maximum likelihood objectives; for unlabeled data, it uses pseudo labels produced by the sentiment classifier with variational Bayes objective. This idea is not novel but the authors report that there is no previous work that jointly trains sentiment aware embeddings with a sentiment classifier specifically, and makes use of an unlabeled corpus to improve both. However, there are general and broader methods such as *Toward Controlled Generation of Text* by Hu et al that apply semi-supervised techniques for generation (not classification) with specific constraints (sentiment, domain, etc). There are other recent methods such as *Improving Language Understanding by Generative Pre-Training* by Redford et al that use the idea of generative pre-training with discriminative fine-tuning that are task-agnostic and achieve very good performance - how does the paper compare to this approach? The experiments and analysis is very well written in the paper. Table 4 also shows very interesting, somewhat surprising results in the paper. The authors say that they will release the code and data for this technique which will be useful for the sentiment analysis research community.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Semi-supervised Learning with Multi-Domain Sentiment Word Embeddings
Word embeddings are known to boost performance of many NLP tasks such as text classification, meanwhile they can be enhanced by labels at the document level to capture nuanced meaning such as sentiment and topic. Can one combine these two research directions to benefit from both? In this paper, we propose to jointly train a text classifier with a label-enhanced and domain-aware word embedding model, using an unlabeled corpus and only a few labeled data from non-target domains. The embeddings are trained on the unlabed corpus and enhanced by pseudo labels coming from the classifier, and at the same time are used by the classifier as input and training signals. We formalize this symbiotic cycle in a variational Bayes framework, and show that our method improves both the embeddings and the text classifier, outperforming state-of-the-art domain adaptation and semi-supervised learning techniques. We conduct detailed ablative tests to reveal gains from important components of our approach. The source code and experiment data will be publicly released.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper jointly trains a sentiment classifier with a sentiment and domain-aware embedding model, using both labeled and unlabeled data. When sentiment label is observed, this model is trained with the usual cross entropy and maximum likelihood objectives; for unlabeled data, it uses pseudo labels produced by the sentiment classifier with variational Bayes objective. This idea is not novel but the authors report that there is no previous work that jointly trains sentiment aware embeddings with a sentiment classifier specifically, and makes use of an unlabeled corpus to improve both. However, there are general and broader methods such as *Toward Controlled Generation of Text* by Hu et al that apply semi-supervised techniques for generation (not classification) with specific constraints (sentiment, domain, etc). There are other recent methods such as *Improving Language Understanding by Generative Pre-Training* by Redford et al that use the idea of generative pre-training with discriminative fine-tuning that are task-agnostic and achieve very good performance - how does the paper compare to this approach? The experiments and analysis is very well written in the paper. Table 4 also shows very interesting, somewhat surprising results in the paper. The authors say that they will release the code and data for this technique which will be useful for the sentiment analysis research community.
assistant
 This paper jointly trains a sentiment classifier with a sentiment and domain-aware embedding model, using both labeled and unlabeled data. When sentiment label is observed, this model is trained with the usual cross entropy and maximum likelihood objectives; for unlabeled data, it uses pseudo labels produced by the sentiment classifier with variational Bayes objective. This idea is not novel but the authors report that there is no previous work that jointly trains sentiment aware embeddings with a sentiment classifier specifically, and makes use of an unlabeled corpus to improve both. However, there are general and broader methods such as *Toward Controlled Generation of Text* by Hu et al that apply semi-supervised techniques for generation (not classification) with specific constraints (sentiment, domain, etc). There are other recent methods such as *Improving Language Understanding by Generative Pre-Training* by Redford et al that use the idea of generative pre-training with discriminative fine-tuning that are task-agnostic and achieve very good performance - how does the paper compare to this approach? The experiments and analysis is very well written in the paper. Table 4 also shows very interesting, somewhat surprising results in the paper. The authors say that they will release the code and data for this technique which will be useful for the sentiment analysis research community.