  We present a meta-algorithm for learning a posterior-inference algorithm for
restricted probabilistic programs. Our meta-algorithm takes a training set of
probabilistic programs that describe models with observations, and attempts to
learn an efficient method for inferring the posterior of a similar program. A
key feature of our approach is the use of what we call a white-box inference
algorithm that extracts information directly from model descriptions
themselves, given as programs. Concretely, our white-box inference algorithm is
equipped with multiple neural networks, one for each type of atomic command,
and computes an approximate posterior of a given probabilistic program by
analysing individual atomic commands in the program using these networks. The
parameters of the networks are learnt from a training set by our
meta-algorithm. We empirically demonstrate that the learnt inference algorithm
generalises well to programs that are new in terms of both parameters and model
structures, and report cases where our approach achieves greater test-time
efficiency than alternative approaches such as HMC. The overall results show
the promise as well as remaining challenges of our approach.

Machine Learning