Approximate Inference in Continuous Determinantal Point Processes
Determinantal point processes (DPPs) are random point processes well-suited for modeling repulsion. In machine learning, the focus of DPP-based models has been on diverse subset selection from a discrete and finite base set. This discrete setting admits an efficient sampling algorithm based on the eigendecomposition of the defining kernel matrix. Recently, there has been growing interest in using DPPs defined on continuous spaces. While the discrete-DPP sampler extends formally to the continuous case, computationally, the steps required are not tractable in general. In this paper, we present two efficient DPP sampling schemes that apply to a wide range of kernel functions: one based on low rank approximations via Nyström and random Fourier feature techniques and another based on Gibbs sampling. We demonstrate the utility of continuous DPPs in repulsive mixture modeling and synthesizing human poses spanning activity spaces.
This paper extends determinantal point process (DPP) 
      sampling schemes from discrete to continuous spaces by exploiting 
      low-rank kernels. Standard kernels are approximated using the Nystroem 
      or random Fourier features methods. The method is demonstrated on 
      repulsive mixture models and separated sampling. The paper 
      exploits a low rank kernel in order to derive an efficient "dual 
      sampler", as described in Alg 1. The authors show in which cases of 
      quality function and similarity kernel the necessary computations can 
      be carried out for the RFF and Nystroem approximations (Supp tab 1). 
      This is a non-trivial piece of work. In sec 5 the continuous DPP 
      is used as a prior on the means in a mixture model. However, note that 
      the approximation derived before is not actually used; as the authors 
      say "since we are sampling just one item from the DPP there is a 
      closed form and we do not rely on approximations." The results show 
      that DPPs do (as expected) have a repulsive effect, as evinced by 
      the mixture weight entropies. Aside: surely to qualify as a 
      repulsive mixture, the closeness of the means should really be 
      relative to their variances, but in supp mat p 4 the variances are 
      independent of the means ... In sec 6 a DPP is used to sample a 
      diverse set of poses from a given density. While the resulting poses 
      are credibly diverse, I suspect that visually similar results could 
      have been obtained e.g. using hard core processes (see p 1), or some 
      scheme like farthest point clustering (Gonzales, 1985). 
      Quality: As far as I can judge the material is technically 
      correct. Clarity: Generally clear. As to Fig 1, I wanted 
      more detail as to the defn of (15) -- for a given set X we can 
      evaluate this, but how is the sum defined and evaluated? 
      Originality: as the authors make clear, this work extends earlier 
      work on low-rank approximations in discrete spaces. And as they 
      mention the repulsive prior construction has been used before [20,32]. 
      Significance: it is not very clear about the applicability of the 
      main result. Sec 5 does not actually use it, and sec 6 is just a 
      visualization which does not require a precise sampler. Of course it 
      is possible that the result could be exploited in other interesting 
      ways. Other: * l 102. [11] are not the first to 
      show that a matrix with the form B^T B (D << N) has a fast 
      eigendecomposition, while the form of the citation implies it is. For 
      example this result was used in L. Sirovich and M. Kirby (1987). 
      "Low-dimensional procedure for the characterization of human faces". 
      Journal of the Optical Society of America A 4 (3): 519–524, but is 
      surely in any good advanced linear algebra text, e.g. Wilkinson? 
      * in eq 2 define the \bar{ \phi_n} notation (presumably as complex 
      conjugate). * l 241-242 why not sigma^2 I instead of 
      diag(sigma^2, \ldots, sigma^2), and the same with rho^2? * l 
      304 -- why use only 100 test observations? Reduce variance by using 
      many more!