The efficacy of the width of the basin of attraction surrounding a minimum in parameter space as an indicator for the generalizability of a model parametrization is a point of contention surrounding the training of artificial neural networks, with the dominant view being that wider areas in the landscape reflect better generalizability by the trained model. In this work, however, we aim to show that this is only true for a noiseless system and in general the trend of the model towards wide areas in the landscape reflect the propensity of the model to overfit the training data. Utilizing the objective Bayesian (Jeffreys) prior we instead propose a different determinant of the optimal width within the parameter landscape determined solely by the curvature of the landscape. In doing so we utilize the decomposition of the landscape into the dimensions of principal curvature and find the first principal curvature dimension of the parameter space to be independent of noise within the training data.
This paper targets at a deep learning theory contribution based on information geometry. This contribution is tightly based on Zhang et al. (2018) and explains the generalization of deep learning from a Bayesian perspective. The main contribution the authors claimed is an optimal degree of curvature exist which gives the best generalization guarantees, which is in contrast to the commonly perceived "the wider the better".
First of all, the writing (including language etc) is of poor quality, to the extent that the submission is very difficult to read and can be rejected merely based on this, with unusual expressions, missing  punctuations, super long sentenses, and wongly used words. The reviewer won't list example here because they are everywhere.
What is even worse is the conceptral errors and defected derivations. For example, in eq.(1), the authors equate the Fisher information matrix (which is an expected Hessian) to the Hessian matrix, this is subject to conditions which must be clearly given right before/after the equation. As their results are largely based on the correctness of eq.(2), let's examine the derivations in appendix A.1.  In the first equation in A.1, what is the subindex "j"?  "Utilizing Laplace Approximation of the integral": such approximations have conditions that must be clearly stated. It is not clear how one can get the last approximation in page 12 from the previous equations. In summary, their eq.(2) is a loose approximation which is subject to a set of conditions (that are not given), and the derivation is of poor quality.
As a theoreiritical contirbution, the authors did not manage to converge to some simple and clear statements (theorems or equvalent). Instead, the contribution is largely *explanatory*. It is hard to observe anything new, given the poor writing and organization. The first 4 pages are mainly introductions of previous works.
The authors used information geometry and minimum description length to explain the generalization of deep learning. This is a small area. It is hard to miss closely related works by simple searching. Instead, the authors only cited Rissanen (1978). On the other hand, as the authors used the spectrum properties of the Fisher information matrix, there are some recent works by Amari which can be cited.