content_image/296900.jpg
scalability can be achieved because, under the conditional independence assumption, the Hessian of the log posterior is sparse, with a predictable sparsity pattern. 

## 4.1 Computing Log Posteriors, Gradients and Hessians 

Under the conditional independence assumption, the log posterior density is the sum of the logs of Equations 2 and 3, with a heterogeneous component 

<img src='content_image/296900.jpg'>

and a homogeneous component log $_{p}$($_{a}$ ) . This homogeneous component is the hyperprior on the population-level parameters, so its computation does not depend on N , while each additional household adds another element to the summation in Equation 17. Therefore, computation of the log posterior grows linearly in N . In the subsequent text, let k be the number of elements in each b i and let p be the number of elements in a . Using the notation from Section 2, q is a vector that concatenates all of the b i together, along with a . 

There are two reasons why we might need to compute the gradient and Hessian of the log posterior, namely to use in a derivative-based optimization algorithm to find the posterior mode, and for estimating the precision matrix of an MVN proposal distribution. 3 Ideally, we would derive the gradient and Hessian analytically, and write code to estimate it efficiently. For com- plicated models, the required effort for coding analytic gradients may not be worthwhile. An alternative would be numerical approximation through finite differencing. The fastest, yet least accurate, method for finite differencing for gradients, using either forward or backward differ- ences, requires Nk + p + 1 evaluations of the log posterior. Since the computational cost of the log posterior also grows linearly with N , computing the gradient this way will grow quadratically in N . The cost of estimating a Hessian using finite differencing grow even faster in N . Also, if the Hessian is estimated by taking finite differences of gradients, and those gradients themselves were finite-differences, the accumulated numerical error can be so large that the Hessian estimates are useless. 

