Determinantal Point Processes (DPPs) provide an elegant and versatile way to sample sets of items that balance the point-wise quality with the set-wise diversity of selected items. For this reason, they have gained prominence in many machine learning applications that rely on subset selection. However, sampling from a DPP over a ground set of size N is a costly operation, requiring in general an O(N 3 ) preprocessing cost and an O(N k 3 ) sampling cost for subsets of size k. We approach this problem by introducing DPPNETs: generative deep models that produce DPP-like samples for arbitrary ground sets. We develop an inhibitive attention mechanism based on transformer networks that captures a notion of dissimilarity between feature vectors. We show theoretically that such an approximation is sensible as it maintains the guarantees of inhibition or dissimilarity that makes DPPs so powerful and unique. Empirically, we demonstrate that samples from our model receive high likelihood under the more expensive DPP alternative.
This paper proposes a scaleable algorithm for sampling from DppNets, a proposed model which approximates the distribution of a DPP. The approach builds upon a proposed inhibitive attention mechanism and transformer networks.
The proposed approach and focus on sampling is original as far as I can tell. The problem is also important to parts of the community as DPPs (or similar distributions) are used more and more frequently. However, the applicability of the proposed approach is limited as it is unclear how to deal with varying ground set sizes — the authors briefly discuss this issue in their conclusion referring to circumvent this problem by subsampling (this can however be problematic either requiring to sample from a DPP or incurring high probability of missing „important“ items).
Furthermore the used evaluation method is „biased“ in favor of DppNets as numerical results evaluate the likelihood of samples under the DPP which the DppNet is trained to approximate for. This makes it difficult to draw conclusions from the presented results. I understand that this evaluation is used as there is no standard way of measuring diversity of a subset of items, but it is also clear that „no“ baseline can be competitive. One possibility to overcome this bias would be to consider a downstream task and evaluate performance on that task. 
Furthermore, I suggest to make certain aspects of the paper more explicit and provide additional details. For instance, I would suggest to spell out a training algorithm, provide equations for the training criterion and the evaluation criterion. Please comment on the cost of training (constantly computing the marginal probabilities for training should be quite expensive) and the convergence of the training (maybe show a training curve; this would be interesting in the light of Theorem 1 and Corollary 1).
Certain parts of the paper are unclear or details are missing:
* Table 3: What is „DPP Gao“?
* How are results for k-medoids computed (including the standard error)? Are these results obtained by computing multiple k-medoids solutions with differing initial conditions?
* In the paper you say: „Furthermore, greedily sampling the mode from the DPPNET achieves a better NLL than DPP samples themselves.“ What are the implications of this? What is the NLL of an (approximate) mode of the original DPP? Is the statement that you want to make, that the greedy approximation works well?