Modern neural network architectures take advantage of increasingly deeper layers, and various advances in their structure to achieve better performance. While traditional explicit regularization techniques like dropout, weight decay, and data augmentation are still being used in these new models, little about the regularization and generalization effects of these new structures have been studied. Besides being deeper than their predecessors, could newer architectures like ResNet and DenseNet also benefit from their structures' implicit regularization properties? In this work, we investigate the skip connection's effect on network's generalization features. Through experiments, we show that certain neural network architectures contribute to their generalization abilities. Specifically, we study the effect that low-level features have on generalization performance when they are introduced to deeper layers in DenseNet, ResNet as well as networks with 'skip connections'. We show that these low-level representations do help with generalization in multiple settings when both the quality and quantity of training data is decreased.
The paper studies the effect of different network structures (plain CNN, ResNet and DenseNet). This is an interesting line of research to pursue, however, it gives an impression that a large amount of recent work in this direction has not been considered by the authors. The paper contains ONLY 4 references. 
Some references that might be useful to consider in the paper:
- K. Greff et. al. Highway and Residual Networks learn Unrolled Iterative Estimation.
- C. Zang et. al. UNDERSTANDING DEEP LEARNING REQUIRES RETHINKING GENERALIZATION
- Q. Liao el. al. Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
- A. Veit et. al. Residual Networks Behave Like Ensembles of Relatively Shallow Networks
- K. He at. Al Identity Mappings in Deep Residual Networks
The writing and the structure of the paper could be significantly improved. From the paper, it is difficult to understand the contributions. From the ones listed in Section 1, it seems that most of the contributions were shown in the original ResNet and DenseNet papers. Given, questionable contribution and a lack of relevant citations, it is difficult to recommend for acceptance of the paper. 
Other issues:
Section 2: “Skip connection …. overcome the overfitting”, could the authors comment on this a bit more or point to relevant citation?  
Section 2: “We increase the number of skip connections from 0 to 28”, it is not clear to me how this is done.
Section 3.1.1 “deep Linear model”, what the authors mean with this? Multiple layers without a nonlinearity? Is it the same as Cascade Net?
Section 3.2 From the data description, it is not clear how the training data was obtained. Could the authors provide more details on this?
Section 3.2 “…, only 3 of them are chosen to be displayed…”, how the selection process was done?
Section 3.2 “Instead of showing every layer’s output we exhibit the 3th, 5th, 7th, 9th, 11th, 13th and the final layer’s output”, according to the description in Fig. 7 we should be able to see 7 columns, this description does not correspond to Fig. 7.
Section 4 “This paper investigates how skip connections works in vision tasks…” I do not find experiments with vision datasets in the paper. In order to claim this, I would encourage the authors to run tests on a CV benchmark dataset (e. g. ImageNet)