Deconvolving Feedback Loops in Recommender Systems
Collaborative filtering is a popular technique to infer users' preferences on new content based on the collective information of all users preferences. Recommender systems then use this information to make personalized suggestions to users. When users accept these recommendations it creates a feedback loop in the recommender system, and these loops iteratively influence the collaborative filtering algorithm's predictions over time. We investigate whether it is possible to identify items affected by these feedback loops. We state sufficient assumptions to deconvolve the feedback loops while keeping the inverse solution tractable. We furthermore develop a metric to unravel the recommender system's influence on the entire user-item rating matrix. We use this metric on synthetic and real-world datasets to (1) identify the extent to which the recommender system affects the final rating matrix, (2) rank frequently recommended items, and (3) distinguish whether a user's rated item was recommended or an intrinsic preference. Our results indicate that it is possible to recover the ratings matrix of intrinsic user preferences using a single snapshot of the ratings matrix without any temporal information.
The paper presents a methodology and evaluation for finding out which ratings in a dataset are likely to have been generated due to the influence of a recommender system. A simple model is proposed, under which users deviate from their "true" ratings over a series of steps. In each step (a) item similarity is used to make recommendations, and (b) users pick a recommendation and "adopt" it, adding it to their set of ratings, with a certain probability. The observed rating matrix is the limit of this process. The authors show that, under certain assumptions (e.g., knowledge of the adoption probability), the "true" rating matrix can be recovered from the "observed" ratings, i.e., its inference is tractable. The difference of the two is used by the authors as indication that certain items were recommended, rather than inherent to users.  This intuition is applied and evaluated on a synthetic dataset, illustrating that this methodology can  succesfuly discern "true" from "recommended" ratings. The same methodology is turned next to real datasets, where its validity is reinforced through anecdotal evidence: e.g., no recommended ratings are discovered in a dataset collected in the absence of a recommender system, while in a dataset including both movies and TV shows, tv shows were less likely to be recommended.
I appreciated the model and methodology proposed by the authors, as well as the difficulty of the task at hand. The authors do a very good job motivating their assumptions, and I have no objection with the idea that certain somewhat strong assumptions are needed to perform this.   That said, I would have like to see a discussion on how this information (discerning "true" from  "recommended" ratings) could actually be put to use. Why is this an important problem to solve? User preferences may be affected by a variety of exogenous factors; does accounting for this one lead to better recommendations? If this is the case, is a recommender, who has full view of the system, constrained by the same absence of information as assumed here?   A technical question: the relationship between true and observed ratings ends up being determined by the singular value relationship (8) (or the cleaner (18) in the supplement). In short, it seems to be "boosting" items with high sigma^true_i. Is there some intuition to be gained from this? How is the algorithm identifying recommended items affected by the gap between sigma^true and sigma^obs, and can perhaps the inference occur on the singular value space, rather than by observing how ratings change?  Section 3.2. is a bit abrupt: when stating "We develop an algorithm" it is not clear that his is an algorithm to classify items rated as "true" or  "recommended".  "validating that the Jester..." mention the fact that Jester did not use an RS earlier