Inference for determinantal point processes without spectral knowledge
Determinantal point processes (DPPs) are point process models that naturally encode diversity between the points of a given realization, through a positive definite kernel K. DPPs possess desirable properties, such as exact sampling or analyticity of the moments, but learning the parameters of kernel K through likelihood-based inference is not straightforward. First, the kernel that appears in the likelihood is not K, but another kernel L related to K through an often intractable spectral decomposition. This issue is typically bypassed in machine learning by directly parametrizing the kernel L, at the price of some interpretability of the model parameters. We follow this approach here. Second, the likelihood has an intractable normalizing constant, which takes the form of a large determinant in the case of a DPP over a finite set of objects, and the form of a Fredholm determinant in the case of a DPP over a continuous domain. Our main contribution is to derive bounds on the likelihood of a DPP, both for finite and continuous domains. Unlike previous work, our bounds are cheap to evaluate since they do not rely on approximating the spectrum of a large matrix or an operator. Through usual arguments, these bounds thus yield cheap variational inference and moderately expensive exact Markov chain Monte Carlo inference methods for DPPs.
Determinantal point processes (DPPs) have been receiving a significant amount of recent attention in the machine learning literature, due to their ability to model repulsion within realizations of a set of points.This repulsion is specified via a positive definite kernel function.One challenge, however, is learning the appropriate parameters of this kernel, given example point realizations.This estimation is difficult due to a difficult-to-compute normalization constant that, in effect, sums over all possible cardinalities and point configurations.A recent proposal [7] has provided a way to do this by observing that the spectrum can be estimated and used to provide incremental upper and lower bounds, resulting in a provably-correct scheme for "retrospective" Markov chain Monte Carlo (MCMC) of the kernel parameters.The present paper proposes a different type of bound that can also be used for MCMC inference, but that does not require estimation of the eigenvalues.This approach also lends itself to a variational approach to learning, and the paper draws connections to inducing-point methods to scalable inference in Gaussian process models.This is a technically strong and well-written paper.I was not aware of the inequality that provides the lower bound in Proposition 1 and it seems like an excellent fit for this problem.It is a clever way to avoid the estimation issues of [7].I love the connections to GP inducing points, as well as the ability to now perform variational inference.Overall this paper was a pleasure to read.I have two technical concerns that I would like to see addressed, however.First, although I like this approach very much, I do not find it all that compelling that it is a huge computational win to replace power iterations with nonlinear optimization problems of increasing dimension.Second, it is not obvious to me that simple increasing of the cardinality is guaranteed to result in convergence of the bounds, because of the coupled optimization problem.Although I am not certain, I believe this optimization is highly non-convex and so one needs to increase cardinality and also ensure that a global minimum is achieved.This is something that should be addressed directly, because it seems that a local minimum could prevent the procedure from being able to bound away the Bernoulli threshold and make it impossible to take an MCMC step.Presentation Issues: - P7 Figure 1: Please include axis labels for the figures. - The proof of the proposition is the main thing that makes this  paper possible and is the central insight.I would've liked to see  it in the main body of the paper and not the index.- P7 L361: "... the variational lower is ..."