We introduce a new routing algorithm for capsule networks, in which a child capsule is routed to a parent based only on agreement between the parent's state and the child's vote. The new mechanism 1) designs routing via inverted dot-product attention; 2) imposes Layer Normalization as normalization; and 3) replaces sequential iterative routing with concurrent iterative routing. When compared to previously proposed routing algorithms, our method improves performance on benchmark datasets such as CIFAR-10 and CIFAR-100, and it performs at-par with a powerful CNN (ResNet-18) with 4× fewer parameters. On a different task of recognizing digits from overlayed digit images, the proposed capsule model performs favorably against CNNs given the same number of layers and neurons per layer. We believe that our work raises the possibility of applying capsule networks to complex real-world tasks.
Authors improve upon dynamic routing between capsules by removing the squash function (norm normalization) and apply a layerNorm normalization instead. Furthermore, they experiment with concurrent routing rather than sequential routing (route all caps layers once, then all layers concurrently again and again). This is an interesting development since provides better gradient in conjunction with layerNorm. They report results on Cifar10 and Cifar100 and achieve similar to CNN (resnet) performance.
First, I want to point out that inverted attention is exactly what happens in dynamic routing (sabour et al 2017), proc. 1 line 4,5, and 7. In dynamic routing the dot product with the next layer capsule is calculated and then normalized over all next layer capsules. The only difference that I notice between alg. 1 here and proc. 1 there is replacement of squash with layer norm. There is no "reconstructing the layer bellow" in Dynamic routing as authors suggest in intro. 
Second, the Capsules are promised to have better viewpoint generalizability than CNNs while having comparable performance. Replacing the 1 convolution layer with a ResNet backbone and replacing the activation with a classifier on top seems reducing the proposed CapsNet to the level of CNNs in terms of Viewpoint Generalization. Why should someone use this network rather than the ResNet itself? Fewer number of parameters by itself is not interesting, the reason it is reported usually is that it indicates lower memory consumption or fewer flops. Is that the case when comparing the baseline ResNet with the proposed CapsNet? Otherwise, a set of experiments showcasing the viewpoint generalizability of proposed CapsuleNetworks might only justify the switch between resnets to the proposed capsnets.
Thirdly, Fig. 4 top images seems to indicate all 3 routing procedures are following the same Learning Rate schedule. In the text it is said that optimization hyperparameters are tuned individually. Did authors tune learning rate schedule individually?
Forth, the proper baseline for the current study is the dynamic routing CapsNet. Why the multiMNIST experiment lacks comparison with dynamic routing capsnet?
For the reasons above, the manuscript in its current format is not ready for publication.
------------------------------------------------------rebuttal
Thank you for your response. I acknowledged the novel contributions of this work. My comment was that some claims in the paper are not right. i.e. "inverted dot-product attention" is not new and "reconstructing the layer bellow" does not happen in Sabour et al . Parallel execution + layer norm definitely is novel and significant.
Regarding the LR-schedule, I am not sure how fair it is to use same hyper-params tuned for the proposed method on the baselines. 
Regarding the viewpoint, the diverseMultiMNIST is two over lapping MNIST digits shifted 6 pixels. There is no rotation or scale in this dataset. An example experiment verifying the viewpoint generalizability of the proposed model is training on MNIST testing on AFFNIST.