Shape and Illumination from Shading using the Generic Viewpoint Assumption
The Generic Viewpoint Assumption (GVA) states that the position of the viewer or the light in a scene is not special. Thus, any estimated parameters from an observation should be stable under small perturbations such as object, viewpoint or light positions. The GVA has been analyzed and quantified in previous works, but has not been put to practical use in actual vision tasks. In this paper, we show how to utilize the GVA to estimate shape and illumination from a single shading image, without the use of other priors. We propose a novel linearized Spherical Harmonics (SH) shading model which enables us to obtain a computationally efficient form of the GVA term. Together with a data term, we build a model whose unknowns are shape and SH illumination. The model parameters are estimated using the Alternating Direction Method of Multipliers embedded in a multi-scale estimation framework. In this prior-free framework, we obtain competitive shape and illumination estimation results under a variety of models and lighting conditions, requiring fewer assumptions than competing methods.
This paper proposes a method for estimation of shape and illumination from shading by using the generic view assumption proposed in [9]. The authors formulate a cost function that consists of a data term and a regularization term based on the generic viewpoint assumption that depends on both shape and illumination. The cost is minimized with respect to these unknowns using a message passing variant for alternating direction method of multipliers. The experimental results on the MIT/Berkeley intrinsic images database show that the method performs close to the prior art method by Barron and Malik, but it does not outperform it.The idea of using the generic viewpoint assumption (GVA) directly in the cost function for shape and illumination from shading is novel, to the best of my knowledge. Therefore, the idea presented in the paper is original. The addressed problem is also a very significant one in computer vision. However, the authors have not made their case to convince the reader why incorporating the GVA actually makes sense and why they expect it to outperform previous work. Particularly, they should explain why using GVA makes more sense than using natural shape statistics priors (e.g. shape smoothness). GVA states that the viewpoint is typically not in a special relation to the object, thus small variations in the viewpoint (or light source) will induce small variations in the object appearance. In that case, one can say that invariance to such small variations under GVA relies a lot on the natural statistics of the scene, such as smoothness. Hence, using GVA as a prior could indirectly assume shape smoothness as well. However, the smoothness then might be enforced only with respect to the viewpoint change considered in the GVA term in the cost function (in this paper only with respect to rotations). Could this interpretation explain the unusual streaks in the shape reconstruction results and the dependence of their orientation on the viewpoint? I think it is very important to discuss this issue in the paper, and justify why the GVA prior is better than the smoothness prior. Even though the authors state that their GVA is not a prior, looking at the cost function it seems to me that it is a prior.The clarity of the paper should be significantly improved. Besides my comment related to the GVA assumption above, the authors need to put more effort into explaining the mathematical formulation of the GVA term in the cost function, given in the Eq.14. It is not clear at all how the authors arrive to this formulation from Eq.9-13. Moreover, how they obtain Eq.11-13 is not clear either. The supplemental material only provides the derivation of the linearization formula. There is also a typo in the supplemental in two equations that are not enumerated (please enumerate them!) and are between Eq.5 and Eq.6 - it should be h here instead of f. Overall, the mathematical derivation of the cost function in Eq.15 is not clear at all, in addition to the unclear motivation of using this cost. The obtained results do not justify the method either. Numerical results show that the proposed method does not outperform state of the art, which makes one wonder why would someone use the proposed method. Visual comparisons are difficult to interpret since the proposed method gives shapes with strange artifacts. These are some streaks, whose orientation seems to be dependent on the viewpoint. These streaks are visible on the 3D shape renderings, but not in the rendered image from a specific viewpoint. This makes the reader wonder how would the rendered image look like from another viewpoint?In detail:Sec. 3. "We found it to be highly accurate for the models in our experiments" - what about the models not presented in the paper?Sec. 4.1 - Authors need to be more careful with the language to make sure the reader understands that this was done in [9]. E.g. line 180: "We get the …" - Actually it has been obtained in [9]In supple. mat. Eq.6 uses a variable 'x'. How is that obtained?Sec.6 What is the mask and how it affects the results?Sec.6. Why do you do a DC shift?