A New Multi-input Model with the Attention Mechanism for Text Classification
Recently, deep learning has made extraordinary achievements in text classification. However, most of present models, especially convolutional neural network (CNN), do not extract long-range associations, global representations, and hierarchical features well due to their relatively shallow and simple structures. This causes a negative effect on text classification. Moreover, we find that there are many express methods of texts. It is appropriate to design the multi-input model to improve the classification effect. But most of models of text classification only use words or characters and do not use the multi-input model. Inspired by the above points and Densenet (Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708, 2017.), we propose a new text classification model, which uses words, characters, and labels as input. The model, which is a deep CNN with a novel attention mechanism, can effectively leverage the input information and solve the above issues of the shallow model. We conduct experiments on six large text classification datasets. Our model achieves the state of the art results on all datasets compared to multiple baseline models.
Summary This paper introduces a new model architecture for doing text classification. The main contribution is proposing a deeper CNN approach, using both word and character embeddings (as well as label embeddings with attention). The paper claims improved performance over baselines. Decision I reject the paper for 3 main reasons: 1) Very misleading claims regarding establishing a new state of the art. The baselines used for comparison don*t include any of the best existing published results. 2) Lack of positioning within the literature. In particular, no mention nor discussion of Transformers (self-attention) networks, including BERT and XLNet approaches, which are the state of the art in text classification. 3) Lack of justification/explanation for the proposed architecture. One key argument made is that current models are shallow, but it appears that only CNN models are considered for that comparison. More discussion is needed to understand why the new aspects of the proposed network are importantly different from other existing approaches. Additional details for decision The results from this paper are significantly inferior to the best results published. With a few quick searches, I found that there are several approaches performing better than the proposed model on every dataset considered in the analysis, as you can see below. http://nlpprogress.com/english/text_classification.html https://github.com/sebastianruder/NLP-progress/blob/master/english/sentiment_analysis.md https://paperswithcode.com/sota/text-classification-on-yahoo-answers Extra notes (not factoring in decision) - Consider spacing out the 3 rightmost blocks in Figure 1, I found the layout confusing and there*s space available. - In section 3, I would have liked more explanation for the motivation of the various design choices.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
A New Multi-input Model with the Attention Mechanism for Text Classification
Recently, deep learning has made extraordinary achievements in text classification. However, most of present models, especially convolutional neural network (CNN), do not extract long-range associations, global representations, and hierarchical features well due to their relatively shallow and simple structures. This causes a negative effect on text classification. Moreover, we find that there are many express methods of texts. It is appropriate to design the multi-input model to improve the classification effect. But most of models of text classification only use words or characters and do not use the multi-input model. Inspired by the above points and Densenet (Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708, 2017.), we propose a new text classification model, which uses words, characters, and labels as input. The model, which is a deep CNN with a novel attention mechanism, can effectively leverage the input information and solve the above issues of the shallow model. We conduct experiments on six large text classification datasets. Our model achieves the state of the art results on all datasets compared to multiple baseline models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Summary This paper introduces a new model architecture for doing text classification. The main contribution is proposing a deeper CNN approach, using both word and character embeddings (as well as label embeddings with attention). The paper claims improved performance over baselines. Decision I reject the paper for 3 main reasons: 1) Very misleading claims regarding establishing a new state of the art. The baselines used for comparison don*t include any of the best existing published results. 2) Lack of positioning within the literature. In particular, no mention nor discussion of Transformers (self-attention) networks, including BERT and XLNet approaches, which are the state of the art in text classification. 3) Lack of justification/explanation for the proposed architecture. One key argument made is that current models are shallow, but it appears that only CNN models are considered for that comparison. More discussion is needed to understand why the new aspects of the proposed network are importantly different from other existing approaches. Additional details for decision The results from this paper are significantly inferior to the best results published. With a few quick searches, I found that there are several approaches performing better than the proposed model on every dataset considered in the analysis, as you can see below. http://nlpprogress.com/english/text_classification.html https://github.com/sebastianruder/NLP-progress/blob/master/english/sentiment_analysis.md https://paperswithcode.com/sota/text-classification-on-yahoo-answers Extra notes (not factoring in decision) - Consider spacing out the 3 rightmost blocks in Figure 1, I found the layout confusing and there*s space available. - In section 3, I would have liked more explanation for the motivation of the various design choices.
assistant
Summary This paper introduces a new model architecture for doing text classification. The main contribution is proposing a deeper CNN approach, using both word and character embeddings (as well as label embeddings with attention). The paper claims improved performance over baselines. Decision I reject the paper for 3 main reasons: 1) Very misleading claims regarding establishing a new state of the art. The baselines used for comparison don*t include any of the best existing published results. 2) Lack of positioning within the literature. In particular, no mention nor discussion of Transformers (self-attention) networks, including BERT and XLNet approaches, which are the state of the art in text classification. 3) Lack of justification/explanation for the proposed architecture. One key argument made is that current models are shallow, but it appears that only CNN models are considered for that comparison. More discussion is needed to understand why the new aspects of the proposed network are importantly different from other existing approaches. Additional details for decision The results from this paper are significantly inferior to the best results published. With a few quick searches, I found that there are several approaches performing better than the proposed model on every dataset considered in the analysis, as you can see below. http://nlpprogress.com/english/text_classification.html https://github.com/sebastianruder/NLP-progress/blob/master/english/sentiment_analysis.md https://paperswithcode.com/sota/text-classification-on-yahoo-answers Extra notes (not factoring in decision) - Consider spacing out the 3 rightmost blocks in Figure 1, I found the layout confusing and there*s space available. - In section 3, I would have liked more explanation for the motivation of the various design choices.