content_image/310739.jpg content_image/310740.jpg content_image/310741.jpg
## A Variational inference and parameter estimation 

In this appendix we describe the variational inference and estimation procedures for the mixed multinomial logit model. A few words on notation: A GLYPH<31> 0 means the matrix A is positive definite; A GLYPH<23> 0 means positive semidefinite; j A j is the determinant of A . For a scalar function s : R ! R and a vector v 2 R $^{n}$, s .v/ means the n GLYPH<2> 1 vector . s .v$_{1}$/; : : : ; s .v$_{n}$ // $^{>}$. 

## A.1 The empirical Bayes ELBO 

For empirical Bayes in the MML model, the ELBO objective function (11) becomes 

<img src='content_image/310739.jpg'>

The first term in (20) is the Shannon entropy of the variational distribution. The second and third terms are (minus) an unnormalized cross entropy – the missing normalization constant is the marginal likelihood. Recall that the variational distribution q .GLYPH<12>$_{1:}$$_{H}$ j GLYPH<22>$_{1:}$$_{H}$ ; 6$_{1:}$$_{H}$ / is a product of normal distributions N .GLYPH<22>$_{h}$ ; 6$_{h}$ / . 

The first and second terms of (20) are straightforward to derive; the third term requires more attention. Using the multinomial logit mass function 

the third term becomes 

<img src='content_image/310740.jpg'>

<img src='content_image/310741.jpg'>

The expected log-sum-exp in (22) has no closed form. For variational inference, we there- 

