Color Constancy by Learning to Predict Chromaticity from Luminance
Color constancy is the recovery of true surface color from observed color, and requires estimating the chromaticity of scene illumination to correct for the bias it induces. In this paper, we show that the per-pixel color statistics of natural scenes-without any spatial or semantic context-can by themselves be a powerful cue for color constancy. Specifically, we describe an illuminant estimation method that is built around a "classifier" for identifying the true chromaticity of a pixel given its luminance (absolute brightness across color channels). During inference, each pixel's observed color restricts its true chromaticity to those values that can be explained by one of a candidate set of illuminants, and applying the classifier over these values yields a distribution over the corresponding illuminants. A global estimate for the scene illuminant is computed through a simple aggregation of these distributions across all pixels. We begin by simply defining the luminance-to-chromaticity classifier by computing empirical histograms over discretized chromaticity and luminance values from a training set of natural images. These histograms reflect a preference for hues corresponding to smooth reflectance functions, and for achromatic colors in brighter pixels. Despite its simplicity, the resulting estimation algorithm outperforms current state-of-the-art color constancy methods. Next, we propose a method to learn the luminanceto-chromaticity classifier "end-to-end". Using stochastic gradient descent, we set chromaticity-luminance likelihoods to minimize errors in the final scene illuminant estimates on a training set. This leads to further improvements in accuracy, most significantly in the tail of the error distribution.
The algorithm presented here is simple and interesting. Pixel luminance, chrominance, and illumination chrominance are all histogrammed, and then evaluation is simply each pixel's luminance voting on each pixel's true chrominance for each of the "memorized" illuminations. The model can be trained generative by simply counting pixels in the training set, or can be trained end-to-end for a slight performance boost. This algorithm's simplicity and speed are appealing, and additionally it seems like it may be a useful building block for a more sophisticated spatially-varying illumination model.I found the paper reasonable easy to read, though much of the math could use a revision to make things more clear and hopefully reduce the (unusually large) number of symbols which are defined for what seems to be a fairly simple algorithm. For example, why define x-hat and g()? is v(n) == v_n? Is { g(v(n), m-hat_i}_i really the easiest way to refer to that set --- and is that math even correct, given that i is used as a subscript twice? Perhaps some figures which correspond to the math would be useful, as the only variable which appears in a figure is L[].The evaluation shows that this technique produces state-of-the-art results on the standard color checker dataset. The evaluation contains one component which I take issue with. When comparing the authors results to the results published in [19], we see that the authors very slightly outperform [19]. However, in this paper the authors have presented a modified set of error metrics, which in the supplement they explain are derived from the metrics in [7,19] subject to a correction, with the claim that this correction undoes an unfair advantage that the authors of [7,19] have achieved by using incorrectly black-leveled images. It may be true that [7,19] are evaluated incorrectly, but such an allegation should be taken very seriously, and should not be dealt with in this way. This number (as I understand it) assumes that the models of [7,19] were trained on the incorrect dataset and then evaluated on the correct dataset, which will obviously increase error metrics. But this does not mean that these numbers are indicative of how [7,19] would perform if trained *and* tested on the correct dataset. Indeed, given that incorrect black-level correction can completely ruin a color constancy algorithm, I would not be surprised in [7,19] actually performs better if trained and tested correctly. So I think the correction the authors present here is not valid.Though it is unfortunate that [7,19] contains incorrect numbers which may be cited, we should not combat such innacuracies with other innacuracies. If published, the number presented here may be taken as truthful by future work, when in fact it is even more speculative and unreliable as the numbers presented in [7,19]. So I think the authors should not present their modified error numbers. The error metrics presented here should be the same as in [7,19], but perhaps indicated with an asterix or with a grayed-out font color that they are likely not trustworthy, with an explanation of why in the supplement. Then the onus will be on [7,19] to revise their results, and we wont have two contradictory (and equally incorrect) numbers for the performance of [7,19] circulating in publications. I don't suggest this to diminish the author's work, as even if [7,19] are taken on face value the work presented here outperforms it. This is simply a matter of scientific honesty and ettiquette that we should handle as carefully as possible.