content_image/312395.jpg content_image/312396.jpg content_image/312397.jpg
are simply 1 =‘$_{ii}$ , where the ‘$_{ii}$ ’s form the diagonal of L$_{h}$ . 

In practice we do the GLYPH<22>$_{h}$ and 6$_{h}$ updates in a single step by optimizing jointly over GLYPH<22>$_{h}$ and L$_{h}$ , which remains a convex problem. 

## A.3 Empirical Bayes M-step 

In the M-step, we maximize (25) over GLYPH<16> and GLYPH<127> . Identifying the terms which depend on GLYPH<16> , we recognize the usual Gaussian mean estimation problem. Further, (25) is easily seen to be concave in GLYPH<127> GLYPH<0> $^{1}$, with a closed-form solution of the corresponding first-order condition. We obtain the M-step updates 

<img src='content_image/312395.jpg'>

Here ̂ Cov .GLYPH<22>$_{GLYPH<1>}$ / is the empirical covariance of the GLYPH<22>$_{h}$ vectors. 

## A.4 Variational hierarchical Bayes 

In the fully Bayesian MML model, GLYPH<16> and GLYPH<127> have prior distributions, with corresponding variational factors given in (14). The ELBO in this case has the same form as (20), with two differences. First, H . q / contains two new terms 

Second, there are two new cross-entropy terms 

<img src='content_image/312396.jpg'>

<img src='content_image/312397.jpg'>

Also, the middle term of (20) changes in the fully Bayesian case, because GLYPH<16> and GLYPH<127> are now averaged over rather than treated as constants. 

Using known formulas for normal and Wishart entropies, the two new entropy terms 

