content_image/313621.jpg content_image/313622.jpg content_image/313623.jpg
Note the similarity of this gradient to the gradient from an L$_{2}$ -regularized multiple logistic regression: it consists of a contribution from the regularizer (the left-hand term), plus a residual-weighted sum of covariate vectors. Abbreviating w.GLYPH<22>$_{h}$; 6$_{h}$; x$_{ht}$ / to w$_{ht}$ , an argu- ment using matrix differentials (Magnus and Neudecker 2007) gives the Hessian 

<img src='content_image/313621.jpg'>

The 6$_{h}$ coordinate update is harder, because we need to insure that 6$_{h}$ GLYPH<23> 0. Using a reformulation, we can avoid making the constraint explicit, which would complicate the optimization. Let 6$_{h}$ D L$_{h}$ L > h for a lower-triangular matrix L$_{h}$ . Since 6$_{h}$ GLYPH<23> 0, one such L$_{h}$ always exists—the Cholesky factor. We replace each 6$_{h}$ in Q L with 6$_{h}$. L$_{h}$ / : D L$_{h}$ L > h , and optimize over the unconstrained set of lower-triangular matrices L$_{h}$ . 

The objective function (25) remains concave in L$_{h}$ . To see this, compare the terms depending on 6$_{h}$ D L$_{h}$ L > h to the function studied in Appendix C. We now give the gradient with respect to L$_{h}$ . Standard matrix differentiation of (25) leads to the 6$_{h}$ gradient 

<img src='content_image/313622.jpg'>

Again using matrix differentials and the Cauchy invariance rule, it is not hard to show that the gradient with respect to L$_{h}$ is 

<img src='content_image/313623.jpg'>

Note that this is the gradient with respect to a dense matrix L$_{h}$ . Since we optimize over lower-triangular matrices, i.e. vech( L$_{h}$ ), we need only use the lower triangular of the gra- dient. This is convenient for the term L GLYPH<0>> h : it is upper-triangular, so its lower triangle is a diagonal matrix. Furthermore, from a standard result of linear algebra, the diagonal entries 

