Our network decomposes an input image into shape, lighting, and albedo with four disentangled representations: identity, expression, pose, and lighting, which allows expression transfer between different face images.
Recovering 3D geometry shape, albedo and lighting from a single image has wide applications in many areas, which is also a typical ill-posed problem. In order to eliminate the ambiguity, face prior knowledge like linear 3D morphable models (3DMM) learned from limited scan data are often adopted to the reconstruction process. However, methods based on linear parametric models cannot generalize well for facial images in the wild with various ages, ethnicity, expressions, poses, and lightings. Recent methods aim to learn a nonlinear parametric model using convolutional neural networks (CNN) to regress the face shape and texture directly. However, the models were only trained on a dataset that is generated from a linear 3DMM. Moreover, the identity and expression representations are entangled in these models, which hurdles many facial editing applications. In this paper, we train our model with adversarial loss in a semi-supervised manner on hybrid batches of unlabeled and labeled face images to exploit the value of large amounts of unlabeled face images from unconstrained photo collections. A novel center loss is introduced to make sure that different facial images from the same person have the same identity shape and albedo. Besides, our proposed model disentangles identity, expression, pose, and lighting representations, which improves the overall reconstruction performance and facilitates facial editing applications, e.g., expression transfer. Comprehensive experiments demonstrate that our model produces high-quality reconstruction compared to state-of-the-art methods and is robust to various expression, pose, and lighting conditions.
Overview:
This paper introduces a model for image-based facial 3D reconstruction. The proposed model is an encoder-decoder architecture that is trained in semi-supervised way to map images to sets of vectors representing identity (which encodes albedo and geometry), pose, expression and lighting. The encoder is a standard image CNN, whereas the decoders for geometry and albedo rely on spectral graph CNNs (similar to e.g. COMA, Ranjan’18). 
The main contribution of the work with respect to the existing methods is the use of additional loss terms that enable semi-supervised training and learning somewhat more disentangled representations. Authors report quantitative results on MICC Florence, with marginal improvements over the baselines (the choice of the baselines is reasonable).
Decision:
The overall architecture is very similar to existing works such as COMA (Ranjan’18) and (Tran’19), including the specific architecture for geometry decoders, and thus the contributions are primarily in the newly added loss terms. 
I also find the promise of “disentangled” representation a bit over-stated, as the albedo and base geometry still seem to be encoded in the same “identity” vector (see related question below).
The numerical improvements seem fairly modest with respect to (Tran’19). In addition, there is no numerical ablation study that would demonstrate the actual utility of the main contributions (such as adversarial loss): there are qualitative results but they are not very convincing. 
Thus, the final rating “weak reject”.
Additional comments / typos:
* I am not fully following the argument about sharing identity for albedo and shape on p2: “albedo and face shape are decoded ...”. Would it not be more beneficial to have a fully decoupled representation between the albedo and the facial geometry? I do not see how albedo information would be useful for encoding face geometry and vise-versa. 
* Authors claim that one of the main drawbacks e.g. of (Train’19) is the fact that they train on data generated from linear 3DMM. This is indeed the case, but it does not seem like here the authors fully overcome this issue: they do have additional weakly-supervised data, but they still strongly rely on linear 3DMM supervision (p6, “pairwise shape loss”, “adversarial loss”), and do not seem to provide experimental evidence that the model will work without it.
* In particular, the “adversarial training” actually corresponds to learning the distribution of the linear 3DMM. Would it not mean that ultimately the model will be limited to learning only linear ? Could you please elaborate on this?
* p3: “allows ene-to-end … training”
* p3: “framework to exact … representations“.
* p8: “evaluation matric”
Update:
Authors did not provide any response, thus I keep my rating.