Content-based recommendations with Poisson factorization
We develop collaborative topic Poisson factorization (CTPF), a generative model of articles and reader preferences. CTPF can be used to build recommender systems by learning from reader histories and content to recommend personalized articles of interest. In detail, CTPF models both reader behavior and article texts with Poisson distributions, connecting the latent topics that represent the texts with the latent preferences that represent the readers. This provides better recommendations than competing methods and gives an interpretable latent space for understanding patterns of readership. Further, we exploit stochastic variational inference to model massive real-world datasets. For example, we can fit CPTF to the full arXiv usage dataset, which contains over 43 million ratings and 42 million word counts, within a day. We demonstrate empirically that our model outperforms several baselines, including the previous state-of-the art approach.
The authors propose a new topic model collaborative topic Poisson factorization (CTPF) which combines ideas from Poisson factorization (PF) and collaborative topic regression (CTR). The idea behind the generative model is intuitive and appealing: learn a latent representation for users and articles that explains the observed ratings as well as the content of the articles. The authors use stochastic variational inference which enables them to apply the model to large datasets. The authors claim that the use of a shared latent representation allows them to attack the cold-start recommendation problem. I like the idea a lot and I think it can potentially be very useful (e.g. as a web service for discovering new scientific articles) assuming the authors' claims are true. However, the presentation needs to be improved before I can whole-heartedly accept the paper.- I felt that the paper is targeted at someone who knows the PF and CTR; the text in some sections reads like a "diff file" highlighting differences between the CTPF model from the older models :) It'd be better to present the CTPF model first and then perhaps describe how PF and CTR arise as special cases of CTPF. - Figure 1: It might be better to move it to the discussion section. \theta and \epsilon are not defined in section 1. I was confused by what "topic offsets" meant until I read the latter sections. (Line 72 describes the red bars as "preferences of readers who have the EM paper in their libraries", which is not helpful either.)- Evaluation: Do you assume that articles not rated are irrelevant? If so, does it not add noise to the computed precision and recall? Are these precision/recall values significant enough that users of such a system would notice an improvement in quality. Can you provide the performance of the "ideal" recommender which sorts by true relevance? Perhaps measures such as Normalized Discounted Cumulative Gain (NDCG) used in information retrieval would be more useful?- The evaluation compares models just based on precision and recall. I think it would be better to compare other model properties as well (e.g. distribution over number of ratings per article, distribution of number of ratings per article)- Line 88: Not clear to me why Poisson factorization handles sparse data better than Gaussian factorization. It's very common in recommender systems to let the objective function depend just on the observed ratings which requires us to update just the latent vectors for users/articles with non-zero observations.- Line 130: PF model seems to use a separate rate parameter (gamma distribution) for each document and user. Is a separate rate not needed here? Line 140: \epsilon is referred to as "document user weights" here and "topic offsets" elsewhere, which makes it hard to follow. It'd be better to use a consistent reference. - How was K set in the experiments? How sensitive is the algorithm to Gamma hyper-parameters? Some sensitivity analysis on toy data might be helpful.Line 374: "Content [1]" is misleading. The fLDA paper uses different priors, likelihood and a different inference scheme. I would encourage the authors to compare to fLDA if the source code is available. I guess "topic offset variables" refers to \epsilon?- Line 414: why are train/test/validation proportions different for the two datasets? 1% of documents as a test set seems too small. Do you average results on multiple train/test splits? - Figure 5: what do you mean by "readers in the field"? In A), B), C) it would be simpler if you specify what you mean in terms of \theta and \epsilon. There's some discrepancy between the figure and the caption ("ontologies and applications" vs "information retrieval").- Line 431: The paper seems to end quite abruptly. It'd be good to have a discussion section to summarize the contributions and discuss future directions.Typos, etc:- Line 107: use r instead of y- Line 285: \tau_0, \kappa not defined. Update (after author rebuttal):The authors have addressed most of my concerns. I have updated my rating, trusting the authors to make the necessary changes.I think it'd be helpful to add a brief discussion early on about implicit feedback and how it differs from explicit feedback, for example the differences explained in Hu et al. [13]. IMO, the PF model is more appealing in the implicit feedback setting; the authors' response about why PF is better than competitors in the implicit feedback setting might be worth adding to the main text. The fact that unobserved responses are treated as 0s in the evaluation is a bit unsatisfying and I am slightly worried how this assumption affects the authors' claims about 4x improvement over state-of-the-art. That said, I understand that evaluation on implicit feedback datasets is less straightforward. I strongly encourage the authors to make the data as well as the code publicly available so that others can study this problem in more detail.