Geometric optimisation on positive definite matrices with application to elliptically contoured distributions
Hermitian positive definite (hpd) matrices recur throughout machine learning, statistics, and optimisation. This paper develops (conic) geometric optimisation on the cone of hpd matrices, which allows us to globally optimise a large class of nonconvex functions of hpd matrices. Specifically, we first use the Riemannian manifold structure of the hpd cone for studying functions that are nonconvex in the Euclidean sense but are geodesically convex (g-convex), hence globally optimisable. We then go beyond g-convexity, and exploit the conic geometry of hpd matrices to identify another class of functions that remain amenable to global optimisation without requiring g-convexity. We present key results that help recognise g-convexity and also the additional structure alluded to above. We illustrate our ideas by applying them to likelihood maximisation for a broad family of elliptically contoured distributions: for this maximisation, we derive novel, parameter free fixed-point algorithms. To our knowledge, ours are the most general results on geometric optimisation of hpd matrices known so far. Experiments show that advantages of using our fixed-point algorithms.
Summary of the paper The paper proposes a 
      framework for optimizing (potentially non-convex) functions on the space 
      of hermititian positive definite matrices. This framework makes use of 
      convexity, with respect to geodesics, to empirically estimate the 
      parameters of elliptically contoured distributions based on the 
      minimization of the negative log-likelihood. For this sake, a fixed point 
      algorithm is derived as well as its convergence guarantees. Theoretical 
      tools that help to recognize such geodesical convex functions are given. 
      Finally, numerical experiments illustrate proposed algorithm. 
      Comments This paper propose promising results for the 
      optimization of functions subject to Hermitian Positive Definite (HPD) 
      constraints. However the paper has a strong optimization and discrete 
      machine learning flavor and is a bit tedious to read. I would recommend to 
      modify the paper such that the machine learning aspect of this work is 
      pushed forward. Seemingly, the work fits more the topics of a pure 
      optimization conference. Some nice results about geodesic 
      convexity and log-nonexpansive functions are stated. Those results are 
      used to build a fixed-point algorithm. Even if the derivation seems 
      trivial to the authors, it would be helpful to make explicit the 
      optimization algorithms derived from the theoretical results. 
      Numerical experiments illustrating the effectiveness of proposed 
      algorithms need to be improved. Indeed, comparison with the matlab 
      function "fmincon" looks impressive, but it would be more interesting to 
      consider state of the art methods such as the gradient (or hessian) based 
      methods developed in the context of optimization over manifolds (see for 
      instance the work "Optimization algorithm on matrix manifolds" by Absil et 
      al.). Comparison to such competitive approaches in terms of convergence 
      speed of the objective function or of the iterates would be appreciable. 
      Besides these experimental evaluations the paper lacks an analysis of the 
      algorithm convergence rate. Finally, there are some comments about 
      the notations. It is somewhat confusing to refer to HPD matrices both as a 
      cone or as a manifold. Indeed, in those two cases, the implied geometrical 
      structures are different.