Variational Search Distributions
Daniel M. Steinberg Rafael Oliveira Cheng Soon Ong Edwin V. Bonilla
We develop VSD, a method for conditioning a generative model of discrete, combinatorial designs on a rare desired class by efficiently evaluating a black-box (e.g. experiment, simulation) in a batch sequential manner. We call this task active generation; we formalize active generation's requirements and desiderata, and formulate a solution via variational inference. VSD uses off-the-shelf gradient based optimization routines, can learn powerful generative models for desirable designs, and can take advantage of scalable predictive models. We derive asymptotic convergence rates for learning the true conditional generative distribution of designs with certain configurations of our method. After illustrating the generative model on images, we empirically demonstrate that VSD can outperform existing baseline methods on a set of real sequence-design problems in various protein and DNA/RNA engineering tasks.
black box optimization, Bayesian optimization, variational inference, generative models, level set estimation, biological sequence design
A variational inference framework for learning generative models of (biological) sequences for black-box optimization and more.