DeepExposure: Learning to Expose Photos with Asynchronously Reinforced Adversarial Learning
The accurate exposure is the key of capturing high-quality photos in computational photography, especially for mobile phones that are limited by sizes of camera modules. Inspired by luminosity masks usually applied by professional photographers, in this paper, we develop a novel algorithm for learning local exposures with deep reinforcement adversarial learning. To be specific, we segment an image into sub-images that can reflect variations of dynamic range exposures according to raw low-level features. Based on these sub-images, a local exposure for each sub-image is automatically learned by virtue of policy network sequentially while the reward of learning is globally designed for striking a balance of overall exposures. The aesthetic evaluation function is approximated by discriminator in generative adversarial networks. The reinforcement learning and the adversarial learning are trained collaboratively by asynchronous deterministic policy gradient and generative loss approximation. To further simply the algorithmic architecture, we also prove the feasibility of leveraging the discriminator as the value function. Further more, we employ each local exposure to retouch the raw input image respectively, thus delivering multiple retouched images under different exposures which are fused with exposure blending. The extensive experiments verify that our algorithms are superior to state-of-the-art methods in terms of quantitative accuracy and visual illustration.
The paper proposes an image retouching method in a local processing and fusion manner. The method first segments the image into sub-images, then each sub-image is passed into a reinforment-learning-based method to output an optimal exposure for this region. The exposure-corrected images under all the exposures (estimated from all the sub-images) are fused through a blending approach.   Pros:  1. The authors propose an image retouching method by exploiting local adjustment instead of global adjustment. This could better handle the cases that bright and dark regions appear in the same picture. The three components are reasonably assembled. 2. The proposed method achieves good quantitative result in terms of PSNR.  Cons: 1. The proposed method consists of three components, segmentation, local retouching, and fusion. The authors use the method [8] for segmentation, and the method [24] for blending. And the retouching strategy is very similar to [13], which also uses reinforcement-learning to update the exposure and a discriminator to represent the loss/AE function (and approximating the loss using CNN). The major difference is that [13] applies global adjustment instead of local one. Considering those, the proposed method is more a systematic work by combining several existing components, thus marginalize its novelty. If there are other differences between [13] and the local retouching component of this method (which I did not see much), the authors should provide a comparison by replacing the component with the method in [13], to demonstrate the advantages of the proposed one. 2. The authors provide the quantitative results on MIT-Adobe 5K in terms of PSNR. As there are no ground-truth images for this task, are the expert retouched results treated as the reference? If so, the results using this metric is less informative, as the quantitative evaluation with respect to the expect retouched results could be biased.  Since the task is subjective, a user study to evaluate the results is suggested.  3. From the quantitative results, the improvement over [13] is little and less than expected (less than 0.06db considering [13] also use the approximation function). I would like to see more analysis on this.  4. In Figure 3, the results using [5] look better than the proposed method. Some more analysis on this and quantitative comparison with [5] are recommended.  5. In Figure 3, the results using the proposed method look over-exposed. What is the reason for that? Is it because the exposed images are favored by the discriminator? I would like to see some analysis on that.   In general, the novelty of the paper is limited and the experimental validation is a bit lacking.   ---------- The rebuttal addresses some of my mis-understanding before, especially on the non-differentiable form and the generative loss approximation. And therefore the usage of asynchronous reinforcement learning makes sense. In this sense, the proposed method is technically sound and novel in terms of the modification on the local adjustment.   For the comparison to [13] with local adjustment in the rebuttal, I don't quite understand the setup (the setup still feels like the proposed framework). And the provided quantitative is hard to explain, which is much worse (0.36db lower) than global retouching itself. A more reasonable way is directly applying the global retouching [13] on different segments to estimate their exposure settings (differentiable loss directly on the segment) and then merge all the retouched results of different exposure settings via the exposure fusion. Since there are a few modifications and there lacks of ablation study (and difficult to evaluate) for each small modification. I think this experiment at least helps understand whether the improvement is from simply local adjustment or others.