Fully Quantized Transformer for Improved Translation
State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsolved. In this work, we propose a quantization strategy tailored to the Transformer architecture. We evaluate our method on the WMT14 EN-FR and WMT14 EN-DE translation tasks and achieve state-of-the-art quantization results for the Transformer, obtaining no loss in BLEU scores compared to the non-quantized baseline. We further compress the Transformer by showing that, once the model is trained, a good portion of the nodes in the encoder can be removed without causing any loss in BLEU.
This paper proposes a quantization strategy for Transformer models and demonstrate results on a neural machine translation benchmark. Although the results look promising, the experiments are a bit sparse in terms of details. In my opinion, an empirical performance-focused paper like this really needs to provide a lot of details. Specific recommendations: - Table 1 compares BLEU scores across different papers. But is is not clear exactly whether they are comparable. In particular, what is the model size for the model in each paper? It would be insightful to show more details such as model size, compression rate, inference time, etc. - Table 2: the compression rate is shown as x4. But it would be good to show the actual model size as well. Some parameters, such as the bias terms, are not quantized, correct? - If you are also arguing that quantization gives inference time improvements, it would be good to show timing numbers too. I would recommend dedicate significant portion of the paper to this. - The ablation study is good. For empirical papers like this, these are the kind of analyses that will be particularly exciting to the reader in my opinion. So more analysis on different variants and practical strategies will be helpful. - I would cut all the pruning results (e.g. Sec 4.4). I don*t see how that is related to quantization, which is the main contribution of the paper. The pruning results seem to be tacked on and make the paper a bit incoherent. - Finally, it is not very clear how does the proposed quantization method different from previous methods. The previous methods should be described with respect to your approach so it is easy to understand what is novel.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Fully Quantized Transformer for Improved Translation
State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsolved. In this work, we propose a quantization strategy tailored to the Transformer architecture. We evaluate our method on the WMT14 EN-FR and WMT14 EN-DE translation tasks and achieve state-of-the-art quantization results for the Transformer, obtaining no loss in BLEU scores compared to the non-quantized baseline. We further compress the Transformer by showing that, once the model is trained, a good portion of the nodes in the encoder can be removed without causing any loss in BLEU.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes a quantization strategy for Transformer models and demonstrate results on a neural machine translation benchmark. Although the results look promising, the experiments are a bit sparse in terms of details. In my opinion, an empirical performance-focused paper like this really needs to provide a lot of details. Specific recommendations: - Table 1 compares BLEU scores across different papers. But is is not clear exactly whether they are comparable. In particular, what is the model size for the model in each paper? It would be insightful to show more details such as model size, compression rate, inference time, etc. - Table 2: the compression rate is shown as x4. But it would be good to show the actual model size as well. Some parameters, such as the bias terms, are not quantized, correct? - If you are also arguing that quantization gives inference time improvements, it would be good to show timing numbers too. I would recommend dedicate significant portion of the paper to this. - The ablation study is good. For empirical papers like this, these are the kind of analyses that will be particularly exciting to the reader in my opinion. So more analysis on different variants and practical strategies will be helpful. - I would cut all the pruning results (e.g. Sec 4.4). I don*t see how that is related to quantization, which is the main contribution of the paper. The pruning results seem to be tacked on and make the paper a bit incoherent. - Finally, it is not very clear how does the proposed quantization method different from previous methods. The previous methods should be described with respect to your approach so it is easy to understand what is novel.
assistant
This paper proposes a quantization strategy for Transformer models and demonstrate results on a neural machine translation benchmark. Although the results look promising, the experiments are a bit sparse in terms of details. In my opinion, an empirical performance-focused paper like this really needs to provide a lot of details. Specific recommendations: - Table 1 compares BLEU scores across different papers. But is is not clear exactly whether they are comparable. In particular, what is the model size for the model in each paper? It would be insightful to show more details such as model size, compression rate, inference time, etc. - Table 2: the compression rate is shown as x4. But it would be good to show the actual model size as well. Some parameters, such as the bias terms, are not quantized, correct? - If you are also arguing that quantization gives inference time improvements, it would be good to show timing numbers too. I would recommend dedicate significant portion of the paper to this. - The ablation study is good. For empirical papers like this, these are the kind of analyses that will be particularly exciting to the reader in my opinion. So more analysis on different variants and practical strategies will be helpful. - I would cut all the pruning results (e.g. Sec 4.4). I don*t see how that is related to quantization, which is the main contribution of the paper. The pruning results seem to be tacked on and make the paper a bit incoherent. - Finally, it is not very clear how does the proposed quantization method different from previous methods. The previous methods should be described with respect to your approach so it is easy to understand what is novel.