A Multi-step Inertial Forward-Backward Splitting Method for Non-convex Optimization
We propose a multi-step inertial Forward-Backward splitting algorithm for minimizing the sum of two non-necessarily convex functions, one of which is proper lower semi-continuous while the other is differentiable with a Lipschitz continuous gradient. We first prove global convergence of the algorithm with the help of the Kurdyka-Łojasiewicz property. Then, when the non-smooth part is also partly smooth relative to a smooth submanifold, we establish finite identification of the latter and provide sharp local linear convergence analysis. The proposed method is illustrated on several problems arising from statistics and machine learning.
In this paper, a multi-step inertial forward-backward method is proposed   to minimise an objective function that can be written as the sum of a smooth (possibly non convex) function and a proper l.s.c. function. With respect to most common inertial schemes, the authors enlarge the possible choices for inertial parameters, allowing for combinations (also negative) of more than two previous iterates. Convergence to a critical point for arbitrary initialisation assuming a priori boundedness of the iterate is proved under KL property. Local linear convergence is established under partial smoothness. The results extend significantly what is known in the convex setting. The experimental results compare multi-step IFB with classical FB.
The paper provides new insight into the theory of inertial methods. The authors prove convergence to a critical point for non convex and non smooth objective functions.  Theoretical analysis: The theoretical results extend known results in two directions: to more general objective functions and to more general inertial schemes.  The more interesting aspect is the generalisation to non convex problems.  However, I am not sure of the correctness of the algorithm under the assumptions made in the paper.  Indeed, without additional conditions (e.g. boundedness from below) on the function R, the proximity operator can be empty, and in this case the iteration is not well-defined.  One of the main results, Theorem 2.2 establishes convergence of the iterates, assuming boundedness of the iterates. I think that a more deep analysis of the class of allowed inertial parameters would be useful. For instance, are classical choices included in the considered class (e.g. the FISTA ones)?   The local linear convergence result extend what was known in the convex setting to the non convex one.  Experiments: The experimental section is interesting, but to me the gains of using s\geq2 are limited.  Moreover, in the comparison with FB, why is the step-size so small?  FB should work with gamma\le 1/L. Does a choice of s\geq 1 require smaller step-sizes?  In view of the previous remarks, I think that the paper presents an interesting theoretical analysis, but the practical impact of the generalisation studied in the paper is not clear to me. In particular, the choice of several additional parameters with respect to standard inertial schemes is required, and the advantages shown in the experiments look  limited.