Invariance and Stability of Deep Convolutional Representations
In this paper, we study deep signal representations that are near-invariant to groups of transformations and stable to the action of diffeomorphisms without losing signal information. This is achieved by generalizing the multilayer kernel introduced in the context of convolutional kernel networks and by studying the geometry of the corresponding reproducing kernel Hilbert space. We show that the signal representation is stable, and that models from this functional space, such as a large class of convolutional neural networks, may enjoy the same stability.
The primary focus of the paper is CKN (convolutional kernel network) [13, 14]. In this manuscript the authors analyse the stability [w.r.t. C^1 diffeomorphisms (such as translation), in the sense of Eq. (4)] of the representation formed by CKNs. They show that for norm-preserving and non-expansive kernels [(A1-A2) in line 193] stability holds for appropriately chosen patch sizes [(A3)]. Extension from (R^d,+) to locally compact groups is sketched in Section 4.
The paper is nicely organized, clearly written, technically sound, combining ideas from two exciting areas (deep networks and kernels). The stability result can be of interest to the ML community.  
-The submission would benefit from adding further motivation on the stability analysis. Currently there is only one short sentence (line 56-57: 'Finally, we note that the Lipschitz stability of deep predictive models was found to be important to get robustness to adversarial examples [7].') which motivates the main contribution of the paper.
-Overloading the \kappa notation in (A3) [line 193] might be confusing, it also denotes a function defining kernel K in Eq. (10).
-In the displayed equation between line 384 and 385, the second part ('and \forall v...') is superfluous; given the symmetry of kernel k, it is identical to the first constraint ('\forall u ...').
-Line 416: the definition of \phi is missing, it should be introduced in Eq. (10) [=<\phi(z),\phi(z')>_{H(K)}].
-Line 427-428: The inequality under '=' seems to also hold with equality, | ||z|| - ||z|| |^2 should be | ||z|| - ||z'|| |^2. 
References:
[3,6,8,13,14,18,25-27,30,31]: page information is missing.
[9]: appeared -> Amit Daniely, Roy Frostig, Yoram Singer. Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity. Advances in Neural Information Processing Systems (NIPS), pages 2253-2261, 2016.
[17]: appeared -> Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, Bernhard Sch{\"o}lkopf. Kernel Mean Embedding of Distributions: A Review and Beyond. Foundations and Trends in Machine Learning, 10(1-2): 1-141. 
[19]: appeared -> Anant Raj, Abhishek Kumar, Youssef Mroueh, Tom Fletcher, Bernhard Sch{\"o}lkopf. International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 54:1225-1235, 2017.
[32]: accepted (https://2017.icml.cc/Conferences/2017/Schedule?type=Poster) -> Yuchen Zhang, Percy Liang, Martin Wainwright. Convexified Convolutional Neural Networks. International Conference on Machine Learning (ICML), 2017, accepted.