Estimating the class prior and posterior from noisy positives and unlabeled data
We develop a classification algorithm for estimating posterior distributions from positive-unlabeled data, that is robust to noise in the positive labels and effective for high-dimensional data. In recent years, several algorithms have been proposed to learn from positive-unlabeled data; however, many of these contributions remain theoretical, performing poorly on real high-dimensional data that is typically contaminated with noise. We build on this previous work to develop two practical classification algorithms that explicitly model the noise in the positive labels and utilize univariate transforms built on discriminative classifiers. We prove that these univariate transforms preserve the class prior, enabling estimation in the univariate space and avoiding kernel density estimation for high-dimensional data. The theoretical development and parametric and nonparametric algorithms proposed here constitute an important step towards wide-spread use of robust classification algorithms for positive-unlabeled data.
The paper proposes a method for estimating the class prior given on labelled sample, as well as noisy positive samples. This builds upon an existing approach to estimate the prior when the positive samples are clean. Experiments show good performance compared to some existing methods. 
UPDATE AFTER REBUTTAL:  Regarding the main result highlighted in the rebuttal: it's indeed nice to relate the observed P(y|x) to the true one, but again this was already done in [1] using similar ideas, just without noise in the positives. The result here thus didn't really surprise me. As mentioned in my review, it's certainly valuable to have all this written down, but to me falls below the novelty threshold.   Regarding the related work on asymmetric noise: my basic point is that from (Scott et al., 2013) Lemma 1 + Proposition 3 (part 2), one has a means of relating the output of a "mixture proportion estimator" (MPE) to the mixing weights for generic asymmetric positive and negative label noise. As I understand, that means that the estimator of Sanderson & Scott that I mentioned could be used in the asymmetric noise scenario. The authors are correct that Sanderson & Scott focussed on an application to (multiclass) positive + unlabelled learning, without noise in the positives; but the point remains that their estimator is conceivably applicable in the scenario of noisy positives as well. I believe this is similarly true of Ramaswamy et al.'s estimator.  ==============  Learning from positive and unlabelled data is a fairly well studied problem, and the paper makes a reasonable case for attention to be made in the case where the positives are subject to noise. Existing methods in the PU literature for estimating the class prior are not directly applicable here, and so it is of interest to design methods for the scenario. The paper thus is well motivated.  On the technical side, the paper has a fairly careful theoretical analysis of the identifiably issues in this problem, which is appreciated. This is used to determine a canonical form in which estimation is performed, leading to a practical algorithm for the estimation. The basic idea is to build upon the AlphaMax framework of (Jain et al., 2016), and cast the estimation problem as determining the mixing weight for a certain mixture model. This estimation is performed on the outputs of any probabilistically-consistent scorer on the inputs (i.e. it does not suffer from the curse of dimensionality.) Thus, the basic technical content seems sound.  I do however have two concerns about the novelty of the work.   (1) The analysis in section 3 appears to closely follow that of (Jain et al., 2016), with some natural extensions to account for the fact that one is in the noisy positive setting. Unless I missed something, there don't seem to be any fundamentally new insights or techniques used in the proofs of the section: the two basic messages of (Jain et al., 2016) are translated, namely that working with all possible distributions leads to non-identifiability, while disallowing distributions that are themselves mixtures alleviates the problem. One has two calls to AlphaMax for the estimation, but the basic idea of mixture modelling is as per the original paper.   The idea of then using univariate transformations in section 4 appears particularly similar to what is done in (Jain et al., 2016). The paper claims that the latter was restricted to the case where the transformation results in calibrated probability, but doesn't Thm 9 of the latter also allow for arbitrary transformations?   Certainly it is valuable for all this to be written down in the noisy PU setting, but this leads to my next concern.   (2) The precise distinction of the scenario considered here to the general asymmetric label noise setting, seems worthy of a little more discussion. This is first to make clear why one does not just conduct all analysis in the general case, rather than assuming that one of the observed  distributions is in fact the underlying marginal (i.e. why one does not work in the general mutually contaminated framework of Scott et al., 2013). Is there some reason analogues of Theorem 1 would not exist here? (As noted, at least some of the results have precedent in Scott et al., 2013, albeit in a different language.)   The second reason is there are methods for the latter which I think could be applied for the scenario considered here, in which case the paper should compare against them. In particular, I refer to the practically viable estimators of  Sanderson and Scott. Class proportion estimation with application to multiclass anomaly rejection. AISTATS 2014.  Liu and Tao. Classification with Noisy Labels by Importance Reweighting. PAMI 2016. (arXiV draft since 2014)  Ramaswamy et al. Mixture Proportion Estimation via Kernel Embedding of Distributions. ICML 2016. (arXiV draft since early 2016, so reasonable for authors to be unaware of)  All three of the above seem like they would be relevant at the end of the third paragraph in the introduction, as a contrast to the cited work of Scott et al., 2013 that relies on density estimation. I note that the first method in particular also does not require the use of calibrated probabilities, but rather a probabilistically consistent ranker (as the estimate is based on a derivative of the RHS of the ROC curve).  Overall, I think the paper's contribution is reasonable, being an extension of (Jain et al., 2016) to the noisy PU case, but I think the novelty is a weak point.  Other comments:  - section 3 is said to include "a few missing results needed for our approach" -- explicitly identifying which these are seems prudent.  - I found the use of a subscript and superscript in a^lambda_mu a bit confusing. Consider just a(lambda, mu)?  - the introduction of \Pi^res could perhaps be deferred till after Lemma 1.  - consider swapping statements 4 and 5 for lemma 1, since the proof seems to do the same.  - in the proof of lemma 1, consider not having all the equations inline to improve readability.  - typo: "Theorem 1 (identifiablity)"  - I felt that section 4 could be more explicit about the suggested algorithm, i.e. that one simply perform the mixture modelling on the classifier outputs on the positive and unlabelled samples.  - it might be worth explaining in more detail what exactly the probabilistic model summarised in Figure 1 corresponds to. That is, is it meant to represent an assumed generative view of the data, which must be satisfied for the data generation process? Or a means by which one processes the available data one has? Put another way, suppose one is able to draw from the unlabelled distribution, and then separately from the noisy positive distribution; what relevance does Fig 1 have?  - is (12) related to the result about the label probability function (Proposition 1) in  Ward et al. Presence-only data and the EM algorithm. Biometrics, 2009.