Discussion of 'Deep learning for finance: deep portfolios'
Authors provided a rigorous and a clear introduction into the deep learning models. They described the problem of model estimation (back propagation) model quality assessment, and the issue of trade off between bias and variance of the model that can be addressed via regularization or drop out techniques. Because deep learning models are currently not widely used in econometrics or engineering applications, we present yet another point of view on those models that hopefully will make the content of the paper more accessible and less mysterious for some of the readers. The neural networks and deep learning models (networks with large number of links in the path from inputs to outputs) in particular have been widely used in the last three decades for the model free regression and classification problems. Deep learning models are also called black box models [1] in the sense that no domain insights (in our case financial principals) are used to build a model. A very flexible model structure is used to compensate for the lack of any assumption. It was shown in other applications that flexible black box models can outperform typically 'nice' approximations that dominated the application before. For example, deep learning methods outperform other parametric and nonparametric statistical methods on such tasks as speech recognition [2] and image processing [3,4] The main difficulty with the flexible of nonlinear approximations is that a very large set of possible models needs to be handled, in the context of deep learning that manifested in the network architecture. Although, for some applications, there are efficient methods for finding an optimal network architecture [5] Thus, the main advantage of deep learning models is that they do not make any assumption on a functional form of the map F Rp Rr to be learned. This trait of deep learners echoes a well studied in statistical literature class of nonparametric models [6] This is not a coincidence. We will show that in fact those approaches are very similar. Understanding this similarity helps us gaining some insights into deep learning methodology. This, hopefully, will make understanding of deep learning models easier for the readers unfamiliar with the technique. A statistical learning problem considers recovering the true regression function F(X) given N observations