In this paper we propose a generalization of deep neural networks called deep function machines (DFMs). DFMs act on vector spaces of arbitrary (possibly infinite) dimension and we show that a family of DFMs are invariant to the dimension of input data; that is, the parameterization of the model does not directly hinge on the quality of the input (eg. high resolution images). Using this generalization we provide a new theory of universal approximation of bounded non-linear operators between function spaces. We then suggest that DFMs provide an expressive framework for designing new neural network layer types with topological considerations in mind. Finally, we introduce a novel architecture, RippLeNet, for resolution invariant computer vision, which empirically achieves state of the art invariance.
This paper deals with the problem of learning nonlinear operators using deep learning. Specifically, the authors propose to extend deep neural networks to the case where hidden layers can be infinite-dimensional. They give results on the quality of the approximation using these operator networks, and show how to build neural network layers that are able to take into account topological information from data. Experiments on MNIST using the proposed deep function machines (DFM) are provided. 
The paper attempts to make progress in the region between deep learning and functional data analysis (FDA). This is interesting. Unfortunately, the paper requires significant improvements, both in terms of substance and in terms of presentation. My main concerns are the following:
1) One motivation of DFM is that in many applications data is a discretization of a continuous process and then can be represented by a function. FDA is the research field that formulated the ideas about the statistical data analysis of data samples consisting of continuous functions, where each function is viewed as one sample element. This paper fails to consider properly the work in its FDA context. Operator learning has been already studied in FDA. See for e.g. the problem of functional regression with functional responses. Indeed the functional model considered in the linear case is very similar to Eq. 2.5 or Eq. 3.2. Moreover, extension to nonparametric/nonlinear situations were also studied. The authors should add more information about previous work on this topic so that their results can be understood with respect to previous studies.
2) The computational aspects of DFM are not clear in the paper. From a practical computational perspective, the algorithm will be implemented on a machine which processes on finite representations of data. The paper does not clearly provide information about how the functional nature and the infinite dimensional can be handled in practice. In FDA, generally this is achieved via basis function approximations.
3) Some parts of the paper are hard to read. Sections 3 and 4 are not easy to understand. Maybe adding a section about the notation and developing more the intuition will improve the reading of the manuscript. 
4) The experimental section can be significantly improved. It will be interesting to compare more DFM with its discrete counterpart. Also, other FDA approaches for operator learning should be discussed and compared to the proposed approach.