CONTROLLING COVARIATE SHIFT USING EQUILIBRIUM NORMALIZATION OF WEIGHTS
We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs. The proposed technique keeps the contribution of positive and negative weights to the layer output in equilibrium. We validate our method on a set of standard benchmarks including CIFAR-10/100, SVHN and ILSVRC 2012 ImageNet.
 Authors present a new normalization technique called Equi-Norm and experimentally show its fast convergence properties over its popular competitors Batch-norm and Group-norm. The main idea in the paper is the EquiNorm method modifies the weights of a layer before forward propagating through it such that the contribution from positive and negative kernel weights to the output is same. In the experimental section, their approach is shown to be more accurate than Batch-norm and group norm on all datasets except, Imagenet. Is there any reasoning why this method performed slightly poorly on this particular dataset. Quality: Results are convincing and comprehensive; authors compare their proposed approach to major two baseline normalization frameworks and demonstrate improved performance. Clarity: The paper is well written for readers to understand however there are many grammar mistakes in multiple places. Originality: The work seems to be a logical new approach to covariate shift problem and as described above, I think the proposed method provides convincing results. I am personally not as familiar with normalization approaches so my confidence in the assessment is low; my main experience is in the computer vision for autonomous driving and sparse coding.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
CONTROLLING COVARIATE SHIFT USING EQUILIBRIUM NORMALIZATION OF WEIGHTS
We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs. The proposed technique keeps the contribution of positive and negative weights to the layer output in equilibrium. We validate our method on a set of standard benchmarks including CIFAR-10/100, SVHN and ILSVRC 2012 ImageNet.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Authors present a new normalization technique called Equi-Norm and experimentally show its fast convergence properties over its popular competitors Batch-norm and Group-norm. The main idea in the paper is the EquiNorm method modifies the weights of a layer before forward propagating through it such that the contribution from positive and negative kernel weights to the output is same. In the experimental section, their approach is shown to be more accurate than Batch-norm and group norm on all datasets except, Imagenet. Is there any reasoning why this method performed slightly poorly on this particular dataset. Quality: Results are convincing and comprehensive; authors compare their proposed approach to major two baseline normalization frameworks and demonstrate improved performance. Clarity: The paper is well written for readers to understand however there are many grammar mistakes in multiple places. Originality: The work seems to be a logical new approach to covariate shift problem and as described above, I think the proposed method provides convincing results. I am personally not as familiar with normalization approaches so my confidence in the assessment is low; my main experience is in the computer vision for autonomous driving and sparse coding.
assistant
 Authors present a new normalization technique called Equi-Norm and experimentally show its fast convergence properties over its popular competitors Batch-norm and Group-norm. The main idea in the paper is the EquiNorm method modifies the weights of a layer before forward propagating through it such that the contribution from positive and negative kernel weights to the output is same. In the experimental section, their approach is shown to be more accurate than Batch-norm and group norm on all datasets except, Imagenet. Is there any reasoning why this method performed slightly poorly on this particular dataset. Quality: Results are convincing and comprehensive; authors compare their proposed approach to major two baseline normalization frameworks and demonstrate improved performance. Clarity: The paper is well written for readers to understand however there are many grammar mistakes in multiple places. Originality: The work seems to be a logical new approach to covariate shift problem and as described above, I think the proposed method provides convincing results. I am personally not as familiar with normalization approaches so my confidence in the assessment is low; my main experience is in the computer vision for autonomous driving and sparse coding.