query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : As a computer vision researcher, I need a system that does 3D face reconstruction from images from a single camera. My current idea is to leverage the highly correlated spectral and spatial information in 3D visual face models, which are treated as tensors. I want to introduce a tensor completion model to generate a vast amount of possible face completions from all angles and poses. Given these generated images related to a single facial image, the system should use semantic segmentation to identify different facial parts. From these facial parts, the system should discover geometry and global structure of the face. In the end, the system should also determine the best subset of generated images that can be used to optimize the likelihood of the 3D facial reconstruction, conditioned on learned geometry and global structure of the face. The system should outperform the current state-of-the-art facial reconstruction models.

positive:
3D face reconstruction is a fundamental Computer Vision problem of
extraordinary difficulty. Current systems often assume the availability of
multiple facial images (sometimes from the same subject) as input, and must
address a number of methodological challenges such as establishing dense
correspondences across large facial poses, expressions, and non-uniform
illumination. In general these methods require complex and inefficient
pipelines for model building and fitting. In this work, we propose to address
many of these limitations by training a Convolutional Neural Network (CNN) on
an appropriate dataset consisting of 2D images and 3D facial models or scans.
Our CNN works with just a single 2D facial image, does not require accurate
alignment nor establishes dense correspondence between images, works for
arbitrary facial poses and expressions, and can be used to reconstruct the
whole 3D facial geometry (including the non-visible parts of the face)
bypassing the construction (during training) and fitting (during testing) of a
3D Morphable Model. We achieve this via a simple CNN architecture that performs
direct regression of a volumetric representation of the 3D facial geometry from
a single 2D image. We also demonstrate how the related task of facial landmark
localization can be incorporated into the proposed framework and help improve
reconstruction quality, especially for the cases of large poses and facial
expressions. Testing code will be made available online, along with pre-trained
models http://aaronsplace.co.uk/papers/jackson2017recon

positive:
Traditional methods for image-based 3D face reconstruction and facial motion
retargeting fit a 3D morphable model (3DMM) to the face, which has limited
modeling capacity and fail to generalize well to in-the-wild data. Use of
deformation transfer or multilinear tensor as a personalized 3DMM for
blendshape interpolation does not address the fact that facial expressions
result in different local and global skin deformations in different persons.
Moreover, existing methods learn a single albedo per user which is not enough
to capture the expression-specific skin reflectance variations. We propose an
end-to-end framework that jointly learns a personalized face model per user and
per-frame facial motion parameters from a large corpus of in-the-wild videos of
user expressions. Specifically, we learn user-specific expression blendshapes
and dynamic (expression-specific) albedo maps by predicting personalized
corrections on top of a 3DMM prior. We introduce novel constraints to ensure
that the corrected blendshapes retain their semantic meanings and the
reconstructed geometry is disentangled from the albedo. Experimental results
show that our personalization accurately captures fine-grained facial dynamics
in a wide range of conditions and efficiently decouples the learned face model
from facial motion, resulting in more accurate face reconstruction and facial
motion retargeting compared to state-of-the-art methods.

positive:
In this paper, we present a large-scale detailed 3D face dataset, FaceScape,
and the corresponding benchmark to evaluate single-view facial 3D
reconstruction. By training on FaceScape data, a novel algorithm is proposed to
predict elaborate riggable 3D face models from a single image input. FaceScape
dataset provides 18,760 textured 3D faces, captured from 938 subjects and each
with 20 specific expressions. The 3D models contain the pore-level facial
geometry that is also processed to be topologically uniformed. These fine 3D
facial models can be represented as a 3D morphable model for rough shapes and
displacement maps for detailed geometry. Taking advantage of the large-scale
and high-accuracy dataset, a novel algorithm is further proposed to learn the
expression-specific dynamic details using a deep neural network. The learned
relationship serves as the foundation of our 3D face prediction system from a
single image input. Different than the previous methods, our predicted 3D
models are riggable with highly detailed geometry under different expressions.
We also use FaceScape data to generate the in-the-wild and in-the-lab benchmark
to evaluate recent methods of single-view face reconstruction. The accuracy is
reported and analyzed on the dimensions of camera pose and focal length, which
provides a faithful and comprehensive evaluation and reveals new challenges.
The unprecedented dataset, benchmark, and code have been released to the public
for research purpose.

positive:
Monocular 3D facial shape reconstruction from a single 2D facial image has
been an active research area due to its wide applications. Inspired by the
success of deep neural networks (DNN), we propose a DNN-based approach for
End-to-End 3D FAce Reconstruction (UH-E2FAR) from a single 2D image. Different
from recent works that reconstruct and refine the 3D face in an iterative
manner using both an RGB image and an initial 3D facial shape rendering, our
DNN model is end-to-end, and thus the complicated 3D rendering process can be
avoided. Moreover, we integrate in the DNN architecture two components, namely
a multi-task loss function and a fusion convolutional neural network (CNN) to
improve facial expression reconstruction. With the multi-task loss function, 3D
face reconstruction is divided into neutral 3D facial shape reconstruction and
expressive 3D facial shape reconstruction. The neutral 3D facial shape is
class-specific. Therefore, higher layer features are useful. In comparison, the
expressive 3D facial shape favors lower or intermediate layer features. With
the fusion-CNN, features from different intermediate layers are fused and
transformed for predicting the 3D expressive facial shape. Through extensive
experiments, we demonstrate the superiority of our end-to-end framework in
improving the accuracy of 3D face reconstruction.

positive:
Deep learning methods have brought many breakthroughs to computer vision,
especially in 2D face recognition. However, the bottleneck of deep learning
based 3D face recognition is that it is difficult to collect millions of 3D
faces, whether for industry or academia. In view of this situation, there are
many methods to generate more 3D faces from existing 3D faces through 3D face
data augmentation, which are used to train deep 3D face recognition models.
However, to the best of our knowledge, there is no method to generate 3D faces
from 2D face images for training deep 3D face recognition models. This letter
focuses on the role of reconstructed 3D facial surfaces in 3D face
identification and proposes a framework of 2D-aided deep 3D face
identification. In particular, we propose to reconstruct millions of 3D face
scans from a large scale 2D face database (i.e.VGGFace2), using a deep learning
based 3D face reconstruction method (i.e.ExpNet). Then, we adopt a two-phase
training approach: In the first phase, we use millions of face images to
pre-train the deep convolutional neural network (DCNN), and in the second
phase, we use normal component images (NCI) of reconstructed 3D face scans to
train the DCNN. Extensive experimental results illustrate that the proposed
approach can greatly improve the rank-1 score of 3D face identification on the
FRGC v2.0, the Bosphorus, and the BU-3DFE 3D face databases, compared to the
model trained by 2D face images. Finally, our proposed approach achieves
state-of-the-art rank-1 scores on the FRGC v2.0 (97.6%), Bosphorus (98.4%), and
BU-3DFE (98.8%) databases. The experimental results show that the reconstructed
3D facial surfaces are useful and our 2D-aided deep 3D face identification
framework is meaningful, facing the scarcity of 3D faces.

positive:
We present a novel 3D face reconstruction technique that leverages sparse
photometric stereo (PS) and latest advances on face registration/modeling from
a single image. We observe that 3D morphable faces approach provides a
reasonable geometry proxy for light position calibration. Specifically, we
develop a robust optimization technique that can calibrate per-pixel lighting
direction and illumination at a very high precision without assuming uniform
surface albedos. Next, we apply semantic segmentation on input images and the
geometry proxy to refine hairy vs. bare skin regions using tailored filters.
Experiments on synthetic and real data show that by using a very small set of
images, our technique is able to reconstruct fine geometric details such as
wrinkles, eyebrows, whelks, pores, etc, comparable to and sometimes surpassing
movie quality productions.

positive:
Reconstructing 3D facial geometry from a single RGB image has recently
instigated wide research interest. However, it is still an ill-posed problem
and most methods rely on prior models hence undermining the accuracy of the
recovered 3D faces. In this paper, we exploit the Epipolar Plane Images (EPI)
obtained from light field cameras and learn CNN models that recover horizontal
and vertical 3D facial curves from the respective horizontal and vertical EPIs.
Our 3D face reconstruction network (FaceLFnet) comprises a densely connected
architecture to learn accurate 3D facial curves from low resolution EPIs. To
train the proposed FaceLFnets from scratch, we synthesize photo-realistic light
field images from 3D facial scans. The curve by curve 3D face estimation
approach allows the networks to learn from only 14K images of 80 identities,
which still comprises over 11 Million EPIs/curves. The estimated facial curves
are merged into a single pointcloud to which a surface is fitted to get the
final 3D face. Our method is model-free, requires only a few training samples
to learn FaceLFnet and can reconstruct 3D faces with high accuracy from single
light field images under varying poses, expressions and lighting conditions.
Comparison on the BU-3DFE and BU-4DFE datasets show that our method reduces
reconstruction errors by over 20% compared to recent state of the art.

positive:
Face alignment and 3D face reconstruction are traditionally accomplished as
separated tasks. By exploring the strong correlation between 2D landmarks and
3D shapes, in contrast, we propose a joint face alignment and 3D face
reconstruction method to simultaneously solve these two problems for 2D face
images of arbitrary poses and expressions. This method, based on a summation
model of 3D faces and cascaded regression in 2D and 3D shape spaces,
iteratively and alternately applies two cascaded regressors, one for updating
2D landmarks and the other for 3D shape. The 3D shape and the landmarks are
correlated via a 3D-to-2D mapping matrix, which is updated in each iteration to
refine the location and visibility of 2D landmarks. Unlike existing methods,
the proposed method can fully automatically generate both
pose-and-expression-normalized (PEN) and expressive 3D faces and localize both
visible and invisible 2D landmarks. Based on the PEN 3D faces, we devise a
method to enhance face recognition accuracy across poses and expressions. Both
linear and nonlinear implementations of the proposed method are presented and
evaluated in this paper. Extensive experiments show that the proposed method
can achieve the state-of-the-art accuracy in both face alignment and 3D face
reconstruction, and benefit face recognition owing to its reconstructed PEN 3D
face.

negative:
Recently, 3D face reconstruction from a single image has achieved great
success with the help of deep learning and shape prior knowledge, but they
often fail to produce accurate geometry details. On the other hand, photometric
stereo methods can recover reliable geometry details, but require dense inputs
and need to solve a complex optimization problem. In this paper, we present a
lightweight strategy that only requires sparse inputs or even a single image to
recover high-fidelity face shapes with images captured under near-field lights.
To this end, we construct a dataset containing 84 different subjects with 29
expressions under 3 different lights. Data augmentation is applied to enrich
the data in terms of diversity in identity, lighting, expression, etc. With
this constructed dataset, we propose a novel neural network specially designed
for photometric stereo based 3D face reconstruction. Extensive experiments and
comparisons demonstrate that our method can generate high-quality
reconstruction results with one to three facial images captured under
near-field lights. Our full framework is available at
https://github.com/Juyong/FacePSNet.

negative:
In this work, we present a new method for 3D face reconstruction from
sparse-view RGB images. Unlike previous methods which are built upon 3D
morphable models (3DMMs) with limited details, we leverage an implicit
representation to encode rich geometric features. Our overall pipeline consists
of two major components, including a geometry network, which learns a
deformable neural signed distance function (SDF) as the 3D face representation,
and a rendering network, which learns to render on-surface points of the neural
SDF to match the input images via self-supervised optimization. To handle
in-the-wild sparse-view input of the same target with different expressions at
test time, we propose residual latent code to effectively expand the shape
space of the learned implicit face representation as well as a novel
view-switch loss to enforce consistency among different views. Our experimental
results on several benchmark datasets demonstrate that our approach outperforms
alternative baselines and achieves superior face reconstruction results
compared to state-of-the-art methods.

negative:
We present a minimalistic but effective neural network that computes dense
facial correspondences in highly unconstrained RGB images. Our network learns a
per-pixel flow and a matchability mask between 2D input photographs of a person
and the projection of a textured 3D face model. To train such a network, we
generate a massive dataset of synthetic faces with dense labels using
renderings of a morphable face model with variations in pose, expressions,
lighting, and occlusions. We found that a training refinement using real
photographs is required to drastically improve the ability to handle real
images. When combined with a facial detection and 3D face fitting step, we show
that our approach outperforms the state-of-the-art face alignment methods in
terms of accuracy and speed. By directly estimating dense correspondences, we
do not rely on the full visibility of sparse facial landmarks and are not
limited to the model space of regression-based approaches. We also assess our
method on video frames and demonstrate successful per-frame processing under
extreme pose variations, occlusions, and lighting conditions. Compared to
existing 3D facial tracking techniques, our fitting does not rely on previous
frames or frontal facial initialization and is robust to imperfect face
detections.

negative:
Recovery of a 3D head model including the complete face and hair regions is
still a challenging problem in computer vision and graphics. In this paper, we
consider this problem using only a few multi-view portrait images as input.
Previous multi-view stereo methods that have been based, either on optimization
strategies or deep learning techniques, suffer from low-frequency geometric
structures such as unclear head structures and inaccurate reconstruction in
hair regions. To tackle this problem, we propose a prior-guided implicit neural
rendering network. Specifically, we model the head geometry with a learnable
signed distance field (SDF) and optimize it via an implicit differentiable
renderer with the guidance of some human head priors, including the facial
prior knowledge, head semantic segmentation information and 2D hair orientation
maps. The utilization of these priors can improve the reconstruction accuracy
and robustness, leading to a high-quality integrated 3D head model. Extensive
ablation studies and comparisons with state-of-the-art methods demonstrate that
our method can generate high-fidelity 3D head geometries with the guidance of
these priors.

negative:
Reconstructing the detailed geometric structure of a face from a given image
is a key to many computer vision and graphics applications, such as motion
capture and reenactment. The reconstruction task is challenging as human faces
vary extensively when considering expressions, poses, textures, and intrinsic
geometries. While many approaches tackle this complexity by using additional
data to reconstruct the face of a single subject, extracting facial surface
from a single image remains a difficult problem. As a result, single-image
based methods can usually provide only a rough estimate of the facial geometry.
In contrast, we propose to leverage the power of convolutional neural networks
to produce a highly detailed face reconstruction from a single image. For this
purpose, we introduce an end-to-end CNN framework which derives the shape in a
coarse-to-fine fashion. The proposed architecture is composed of two main
blocks, a network that recovers the coarse facial geometry (CoarseNet),
followed by a CNN that refines the facial features of that geometry (FineNet).
The proposed networks are connected by a novel layer which renders a depth
image given a mesh in 3D. Unlike object recognition and detection problems,
there are no suitable datasets for training CNNs to perform face geometry
reconstruction. Therefore, our training regime begins with a supervised phase,
based on synthetic images, followed by an unsupervised phase that uses only
unconstrained facial images. The accuracy and robustness of the proposed model
is demonstrated by both qualitative and quantitative evaluation tests.

negative:
We introduce the concept of unconstrained real-time 3D facial performance
capture through explicit semantic segmentation in the RGB input. To ensure
robustness, cutting edge supervised learning approaches rely on large training
datasets of face images captured in the wild. While impressive tracking quality
has been demonstrated for faces that are largely visible, any occlusion due to
hair, accessories, or hand-to-face gestures would result in significant visual
artifacts and loss of tracking accuracy. The modeling of occlusions has been
mostly avoided due to its immense space of appearance variability. To address
this curse of high dimensionality, we perform tracking in unconstrained images
assuming non-face regions can be fully masked out. Along with recent
breakthroughs in deep learning, we demonstrate that pixel-level facial
segmentation is possible in real-time by repurposing convolutional neural
networks designed originally for general semantic segmentation. We develop an
efficient architecture based on a two-stream deconvolution network with
complementary characteristics, and introduce carefully designed training
samples and data augmentation strategies for improved segmentation accuracy and
robustness. We adopt a state-of-the-art regression-based facial tracking
framework with segmented face images as training, and demonstrate accurate and
uninterrupted facial performance capture in the presence of extreme occlusion
and even side views. Furthermore, the resulting segmentation can be directly
used to composite partial 3D face models on the input images and enable
seamless facial manipulation tasks, such as virtual make-up or face
replacement.

negative:
State-of-the-art face super-resolution methods employ deep convolutional
neural networks to learn a mapping between low- and high- resolution facial
patterns by exploring local appearance knowledge. However, most of these
methods do not well exploit facial structures and identity information, and
struggle to deal with facial images that exhibit large pose variations. In this
paper, we propose a novel face super-resolution method that explicitly
incorporates 3D facial priors which grasp the sharp facial structures. Our work
is the first to explore 3D morphable knowledge based on the fusion of
parametric descriptions of face attributes (e.g., identity, facial expression,
texture, illumination, and face pose). Furthermore, the priors can easily be
incorporated into any network and are extremely efficient in improving the
performance and accelerating the convergence speed. Firstly, a 3D face
rendering branch is set up to obtain 3D priors of salient facial structures and
identity knowledge. Secondly, the Spatial Attention Module is used to better
exploit this hierarchical information (i.e., intensity similarity, 3D facial
structure, and identity content) for the super-resolution problem. Extensive
experiments demonstrate that the proposed 3D priors achieve superior face
super-resolution results over the state-of-the-arts.

negative:
Methods for generating synthetic data have become of increasing importance to
build large datasets required for Convolution Neural Networks (CNN) based deep
learning techniques for a wide range of computer vision applications. In this
work, we extend existing methodologies to show how 2D thermal facial data can
be mapped to provide 3D facial models. For the proposed research work we have
used tufts datasets for generating 3D varying face poses by using a single
frontal face pose. The system works by refining the existing image quality by
performing fusion based image preprocessing operations. The refined outputs
have better contrast adjustments, decreased noise level and higher exposedness
of the dark regions. It makes the facial landmarks and temperature patterns on
the human face more discernible and visible when compared to original raw data.
Different image quality metrics are used to compare the refined version of
images with original images. In the next phase of the proposed study, the
refined version of images is used to create 3D facial geometry structures by
using Convolution Neural Networks (CNN). The generated outputs are then
imported in blender software to finally extract the 3D thermal facial outputs
of both males and females. The same technique is also used on our thermal face
data acquired using prototype thermal camera (developed under Heliaus EU
project) in an indoor lab environment which is then used for generating
synthetic 3D face data along with varying yaw face angles and lastly facial
depth map is generated.

negative:
State-of-the-art methods for 3D reconstruction of faces from a single image
require 2D-3D pairs of ground-truth data for supervision. Such data is costly
to acquire, and most datasets available in the literature are restricted to
pairs for which the input 2D images depict faces in a near fronto-parallel
pose. Therefore, many data-driven methods for single-image 3D facial
reconstruction perform poorly on profile and near-profile faces. We propose a
method to improve the performance of single-image 3D facial reconstruction
networks by utilizing the network to synthesize its own training data for
fine-tuning, comprising: (i) single-image 3D reconstruction of faces in
near-frontal images without ground-truth 3D shape; (ii) application of a
rigid-body transformation to the reconstructed face model; (iii) rendering of
the face model from new viewpoints; and (iv) use of the rendered image and
corresponding 3D reconstruction as additional data for supervised fine-tuning.
The new 2D-3D pairs thus produced have the same high-quality observed for near
fronto-parallel reconstructions, thereby nudging the network towards more
uniform performance as a function of the viewing angle of input faces.
Application of the proposed technique to the fine-tuning of a state-of-the-art
single-image 3D-reconstruction network for faces demonstrates the usefulness of
the method, with particularly significant gains for profile or near-profile
views.

negative:
The reconstruction of dense 3D models of face geometry and appearance from a
single image is highly challenging and ill-posed. To constrain the problem,
many approaches rely on strong priors, such as parametric face models learned
from limited 3D scan data. However, prior models restrict generalization of the
true diversity in facial geometry, skin reflectance and illumination. To
alleviate this problem, we present the first approach that jointly learns 1) a
regressor for face shape, expression, reflectance and illumination on the basis
of 2) a concurrently learned parametric face model. Our multi-level face model
combines the advantage of 3D Morphable Models for regularization with the
out-of-space generalization of a learned corrective space. We train end-to-end
on in-the-wild images without dense annotations by fusing a convolutional
encoder with a differentiable expert-designed renderer and a self-supervised
training loss, both defined at multiple detail levels. Our approach compares
favorably to the state-of-the-art in terms of reconstruction quality, better
generalizes to real world faces, and runs at over 250 Hz.

negative:
Existing single view, 3D face reconstruction methods can produce beautifully
detailed 3D results, but typically only for near frontal, unobstructed
viewpoints. We describe a system designed to provide detailed 3D
reconstructions of faces viewed under extreme conditions, out of plane
rotations, and occlusions. Motivated by the concept of bump mapping, we propose
a layered approach which decouples estimation of a global shape from its
mid-level details (e.g., wrinkles). We estimate a coarse 3D face shape which
acts as a foundation and then separately layer this foundation with details
represented by a bump map. We show how a deep convolutional encoder-decoder can
be used to estimate such bump maps. We further show how this approach naturally
extends to generate plausible details for occluded facial regions. We test our
approach and its components extensively, quantitatively demonstrating the
invariance of our estimated facial details. We further provide numerous
qualitative examples showing that our method produces detailed 3D face shapes
in viewing conditions where existing state of the art often break down.

negative:
3D face reconstruction is an important task in the field of computer vision.
Although 3D face reconstruction has being developing rapidly in recent years,
it is still a challenge for face reconstruction under large pose. That is
because much of the information about a face in a large pose will be
unknowable. In order to address this issue, this paper proposes a novel 3D face
reconstruction algorithm (PIFR) based on 3D Morphable Model (3DMM). After input
a single face image, it generates a frontal image by normalizing the image.
Then we set weighted sum of the 3D parameters of the two images. Our method
solves the problem of face reconstruction of a single image of a traditional
method in a large pose, works on arbitrary Pose and Expressions, greatly
improves the accuracy of reconstruction. Experiments on the challenging AFW,
LFPW and AFLW database show that our algorithm significantly improves the
accuracy of 3D face reconstruction even under extreme poses .

negative:
With the powerfulness of convolution neural networks (CNN), CNN based face
reconstruction has recently shown promising performance in reconstructing
detailed face shape from 2D face images. The success of CNN-based methods
relies on a large number of labeled data. The state-of-the-art synthesizes such
data using a coarse morphable face model, which however has difficulty to
generate detailed photo-realistic images of faces (with wrinkles). This paper
presents a novel face data generation method. Specifically, we render a large
number of photo-realistic face images with different attributes based on
inverse rendering. Furthermore, we construct a fine-detailed face image dataset
by transferring different scales of details from one image to another. We also
construct a large number of video-type adjacent frame pairs by simulating the
distribution of real video data. With these nicely constructed datasets, we
propose a coarse-to-fine learning framework consisting of three convolutional
networks. The networks are trained for real-time detailed 3D face
reconstruction from monocular video as well as from a single image. Extensive
experimental results demonstrate that our framework can produce high-quality
reconstruction but with much less computation time compared to the
state-of-the-art. Moreover, our method is robust to pose, expression and
lighting due to the diversity of data.

negative:
It has been recently shown that neural networks can recover the geometric
structure of a face from a single given image. A common denominator of most
existing face geometry reconstruction methods is the restriction of the
solution space to some low-dimensional subspace. While such a model
significantly simplifies the reconstruction problem, it is inherently limited
in its expressiveness. As an alternative, we propose an Image-to-Image
translation network that jointly maps the input image to a depth image and a
facial correspondence map. This explicit pixel-based mapping can then be
utilized to provide high quality reconstructions of diverse faces under extreme
expressions, using a purely geometric refinement process. In the spirit of
recent approaches, the network is trained only with synthetic data, and is then
evaluated on in-the-wild facial images. Both qualitative and quantitative
analyses demonstrate the accuracy and the robustness of our approach.

negative:
Recently, 3D face reconstruction and face alignment tasks are gradually
combined into one task: 3D dense face alignment. Its goal is to reconstruct the
3D geometric structure of face with pose information. In this paper, we propose
a graph convolution network to regress 3D face coordinates. Our method directly
performs feature learning on the 3D face mesh, where the geometric structure
and details are well preserved. Extensive experiments show that our approach
gains superior performance over state-of-the-art methods on several challenging
datasets.

negative:
Existing face completion solutions are primarily driven by end-to-end models
that directly generate 2D completions of 2D masked faces. By having to
implicitly account for geometric and photometric variations in facial shape and
appearance, such approaches result in unrealistic completions, especially under
large variations in pose, shape, illumination and mask sizes. To alleviate
these limitations, we introduce 3DFaceFill, an analysis-by-synthesis approach
for face completion that explicitly considers the image formation process. It
comprises three components, (1) an encoder that disentangles the face into its
constituent 3D mesh, 3D pose, illumination and albedo factors, (2) an
autoencoder that inpaints the UV representation of facial albedo, and (3) a
renderer that resynthesizes the completed face. By operating on the UV
representation, 3DFaceFill affords the power of correspondence and allows us to
naturally enforce geometrical priors (e.g. facial symmetry) more effectively.
Quantitatively, 3DFaceFill improves the state-of-the-art by up to 4dB higher
PSNR and 25% better LPIPS for large masks. And, qualitatively, it leads to
demonstrably more photorealistic face completions over a range of masks and
occlusions while preserving consistency in global and component-wise shape,
pose, illumination and eye-gaze.

negative:
In the past few years, a lot of work has been done towards reconstructing the
3D facial structure from single images by capitalizing on the power of Deep
Convolutional Neural Networks (DCNNs). In the most recent works, differentiable
renderers were employed in order to learn the relationship between the facial
identity features and the parameters of a 3D morphable model for shape and
texture. The texture features either correspond to components of a linear
texture space or are learned by auto-encoders directly from in-the-wild images.
In all cases, the quality of the facial texture reconstruction of the
state-of-the-art methods is still not capable of modeling textures in high
fidelity. In this paper, we take a radically different approach and harness the
power of Generative Adversarial Networks (GANs) and DCNNs in order to
reconstruct the facial texture and shape from single images. That is, we
utilize GANs to train a very powerful generator of facial texture in UV space.
Then, we revisit the original 3D Morphable Models (3DMMs) fitting approaches
making use of non-linear optimization to find the optimal latent parameters
that best reconstruct the test image but under a new perspective. We optimize
the parameters with the supervision of pretrained deep identity features
through our end-to-end differentiable framework. We demonstrate excellent
results in photorealistic and identity preserving 3D face reconstructions and
achieve for the first time, to the best of our knowledge, facial texture
reconstruction with high-frequency details.

negative:
Monocular image-based 3D reconstruction of faces is a long-standing problem
in computer vision. Since image data is a 2D projection of a 3D face, the
resulting depth ambiguity makes the problem ill-posed. Most existing methods
rely on data-driven priors that are built from limited 3D face scans. In
contrast, we propose multi-frame video-based self-supervised training of a deep
network that (i) learns a face identity model both in shape and appearance
while (ii) jointly learning to reconstruct 3D faces. Our face model is learned
using only corpora of in-the-wild video clips collected from the Internet. This
virtually endless source of training data enables learning of a highly general
3D face model. In order to achieve this, we propose a novel multi-frame
consistency loss that ensures consistent shape and appearance across multiple
frames of a subject's face, thus minimizing depth ambiguity. At test time we
can use an arbitrary number of frames, so that we can perform both monocular as
well as multi-frame reconstruction.

negative:
Over the last years, many face analysis tasks have accomplished astounding
performance, with applications including face generation and 3D face
reconstruction from a single "in-the-wild" image. Nevertheless, to the best of
our knowledge, there is no method which can produce render-ready
high-resolution 3D faces from "in-the-wild" images and this can be attributed
to the: (a) scarcity of available data for training, and (b) lack of robust
methodologies that can successfully be applied on very high-resolution data. In
this work, we introduce the first method that is able to reconstruct
photorealistic render-ready 3D facial geometry and BRDF from a single
"in-the-wild" image. We capture a large dataset of facial shape and
reflectance, which we have made public. We define a fast facial photorealistic
differentiable rendering methodology with accurate facial skin diffuse and
specular reflection, self-occlusion and subsurface scattering approximation.
With this, we train a network that disentangles the facial diffuse and specular
BRDF components from a shape and texture with baked illumination, reconstructed
with a state-of-the-art 3DMM fitting method. Our method outperforms the
existing arts by a significant margin and reconstructs high-resolution 3D faces
from a single low-resolution image, that can be rendered in various
applications, and bridge the uncanny valley.

negative:
We present an approach to generate high fidelity 3D face avatar with a
high-resolution UV texture map from a single image. To estimate the face
geometry, we use a deep neural network to directly predict vertex coordinates
of the 3D face model from the given image. The 3D face geometry is further
refined by a non-rigid deformation process to more accurately capture facial
landmarks before texture projection. A key novelty of our approach is to train
the shape regression network on facial images synthetically generated using a
high-quality rendering engine. Moreover, our shape estimator fully leverages
the discriminative power of deep facial identity features learned from millions
of facial images. We have conducted extensive experiments to demonstrate the
superiority of our optimized 2D-to-3D rendering approach, especially its
excellent generalization property on real-world selfie images. Our proposed
system of rendering 3D avatars from 2D images has a wide range of applications
from virtual/augmented reality (VR/AR) and telepsychiatry to human-computer
interaction and social networks.

negative:
With the increasing demands of applications in virtual reality such as 3D
films, virtual Human-Machine Interactions and virtual agents, the analysis of
3D human face analysis is considered to be more and more important as a
fundamental step for those virtual reality tasks. Due to information provided
by an additional dimension, 3D facial reconstruction enables aforementioned
tasks to be achieved with higher accuracy than those based on 2D facial
analysis. The denser the 3D facial model is, the more information it could
provide. However, most existing dense 3D facial reconstruction methods require
complicated processing and high system cost. To this end, this paper presents a
novel method that simplifies the process of dense 3D facial reconstruction by
employing only one frame of depth data obtained with an off-the-shelf RGB-D
sensor. The experiments showed competitive results with real world data.

negative:
We present a novel method to jointly learn a 3D face parametric model and 3D
face reconstruction from diverse sources. Previous methods usually learn 3D
face modeling from one kind of source, such as scanned data or in-the-wild
images. Although 3D scanned data contain accurate geometric information of face
shapes, the capture system is expensive and such datasets usually contain a
small number of subjects. On the other hand, in-the-wild face images are easily
obtained and there are a large number of facial images. However, facial images
do not contain explicit geometric information. In this paper, we propose a
method to learn a unified face model from diverse sources. Besides scanned face
data and face images, we also utilize a large number of RGB-D images captured
with an iPhone X to bridge the gap between the two sources. Experimental
results demonstrate that with training data from more sources, we can learn a
more powerful face model.

negative:
Fast and robust three-dimensional reconstruction of facial geometric
structure from a single image is a challenging task with numerous applications.
Here, we introduce a learning-based approach for reconstructing a
three-dimensional face from a single image. Recent face recovery methods rely
on accurate localization of key characteristic points. In contrast, the
proposed approach is based on a Convolutional-Neural-Network (CNN) which
extracts the face geometry directly from its image. Although such deep
architectures outperform other models in complex computer vision problems,
training them properly requires a large dataset of annotated examples. In the
case of three-dimensional faces, currently, there are no large volume data
sets, while acquiring such big-data is a tedious task. As an alternative, we
propose to generate random, yet nearly photo-realistic, facial images for which
the geometric form is known. The suggested model successfully recovers facial
shapes from real images, even for faces with extreme expressions and under
various lighting conditions.

negative:
Recently, it was shown that excellent results can be achieved in both face
landmark localization and pose-invariant face recognition. These breakthroughs
are attributed to the efforts of the community to manually annotate facial
images in many different poses and to collect 3D faces data. In this paper, we
propose a novel method for joint face landmark localization and frontal face
reconstruction (pose correction) using a small set of frontal images only. By
observing that the frontal facial image is the one with the minimum rank from
all different poses we formulate an appropriate model which is able to jointly
recover the facial landmarks as well as the frontalized version of the face. To
this end, a suitable optimization problem, involving the minimization of the
nuclear norm and the matrix $\ell_1$ norm, is solved. The proposed method is
assessed in frontal face reconstruction (pose correction), face landmark
localization, and pose-invariant face recognition and verification by
conducting experiments on $6$ facial images databases. The experimental results
demonstrate the effectiveness of the proposed method.

negative:
Face recognition has been of great importance in many applications as a
biometric for its throughput, convenience, and non-invasiveness. Recent
advancements in deep Convolutional Neural Network (CNN) architectures have
boosted significantly the performance of face recognition based on
two-dimensional (2D) facial texture images and outperformed the previous state
of the art using conventional methods. However, the accuracy of 2D face
recognition is still challenged by the change of pose, illumination, make-up,
and expression. On the other hand, the geometric information contained in
three-dimensional (3D) face data has the potential to overcome the fundamental
limitations of 2D face data.
  We propose a multi-Channel deep 3D face network for face recognition based on
3D face data. We compute the geometric information of a 3D face based on its
piecewise-linear triangular mesh structure and then conformally flatten
geometric information along with the color from 3D to 2D plane to leverage the
state-of-the-art deep CNN architectures. We modify the input layer of the
network to take images with nine channels instead of three only such that more
geometric information can be explicitly fed to it. We pre-train the network
using images from the VGG-Face \cite{Parkhi2015} and then fine-tune it with the
generated multi-channel face images. The face recognition accuracy of the
multi-Channel deep 3D face network has achieved 98.6. The experimental results
also clearly show that the network performs much better when a 9-channel image
is flattened to plane based on the conformal map compared with the orthographic
projection.

negative:
Recently, deep learning based 3D face reconstruction methods have shown
promising results in both quality and efficiency.However, training deep neural
networks typically requires a large volume of data, whereas face images with
ground-truth 3D face shapes are scarce. In this paper, we propose a novel deep
3D face reconstruction approach that 1) leverages a robust, hybrid loss
function for weakly-supervised learning which takes into account both low-level
and perception-level information for supervision, and 2) performs multi-image
face reconstruction by exploiting complementary information from different
images for shape aggregation. Our method is fast, accurate, and robust to
occlusion and large pose. We provide comprehensive experiments on three
datasets, systematically comparing our method with fifteen recent methods and
demonstrating its state-of-the-art performance.

negative:
Facial motion retargeting is an important problem in both computer graphics
and vision, which involves capturing the performance of a human face and
transferring it to another 3D character. Learning 3D morphable model (3DMM)
parameters from 2D face images using convolutional neural networks is common in
2D face alignment, 3D face reconstruction etc. However, existing methods either
require an additional face detection step before retargeting or use a cascade
of separate networks to perform detection followed by retargeting in a
sequence. In this paper, we present a single end-to-end network to jointly
predict the bounding box locations and 3DMM parameters for multiple faces.
First, we design a novel multitask learning framework that learns a
disentangled representation of 3DMM parameters for a single face. Then, we
leverage the trained single face model to generate ground truth 3DMM parameters
for multiple faces to train another network that performs joint face detection
and motion retargeting for images with multiple faces. Experimental results
show that our joint detection and retargeting network has high face detection
accuracy and is robust to extreme expressions and poses while being faster than
state-of-the-art methods.

negative:
The "interpretation through synthesis" approach to analyze face images,
particularly Active Appearance Models (AAMs) method, has become one of the most
successful face modeling approaches over the last two decades. AAM models have
ability to represent face images through synthesis using a controllable
parameterized Principal Component Analysis (PCA) model. However, the accuracy
and robustness of the synthesized faces of AAM are highly depended on the
training sets and inherently on the generalizability of PCA subspaces. This
paper presents a novel Deep Appearance Models (DAMs) approach, an efficient
replacement for AAMs, to accurately capture both shape and texture of face
images under large variations. In this approach, three crucial components
represented in hierarchical layers are modeled using the Deep Boltzmann
Machines (DBM) to robustly capture the variations of facial shapes and
appearances. DAMs are therefore superior to AAMs in inferencing a
representation for new face images under various challenging conditions. The
proposed approach is evaluated in various applications to demonstrate its
robustness and capabilities, i.e. facial super-resolution reconstruction,
facial off-angle reconstruction or face frontalization, facial occlusion
removal and age estimation using challenging face databases, i.e. Labeled Face
Parts in the Wild (LFPW), Helen and FG-NET. Comparing to AAMs and other deep
learning based approaches, the proposed DAMs achieve competitive results in
those applications, thus this showed their advantages in handling occlusions,
facial representation, and reconstruction.

negative:
A lot of work has been done towards reconstructing the 3D facial structure
from single images by capitalizing on the power of Deep Convolutional Neural
Networks (DCNNs). In the recent works, the texture features either correspond
to components of a linear texture space or are learned by auto-encoders
directly from in-the-wild images. In all cases, the quality of the facial
texture reconstruction is still not capable of modeling facial texture with
high-frequency details. In this paper, we take a radically different approach
and harness the power of Generative Adversarial Networks (GANs) and DCNNs in
order to reconstruct the facial texture and shape from single images. That is,
we utilize GANs to train a very powerful facial texture prior \edit{from a
large-scale 3D texture dataset}. Then, we revisit the original 3D Morphable
Models (3DMMs) fitting making use of non-linear optimization to find the
optimal latent parameters that best reconstruct the test image but under a new
perspective. In order to be robust towards initialisation and expedite the
fitting process, we propose a novel self-supervised regression based approach.
We demonstrate excellent results in photorealistic and identity preserving 3D
face reconstructions and achieve for the first time, to the best of our
knowledge, facial texture reconstruction with high-frequency details.

negative:
This paper proposes an encoder-decoder network to disentangle shape features
during 3D face reconstruction from single 2D images, such that the tasks of
reconstructing accurate 3D face shapes and learning discriminative shape
features for face recognition can be accomplished simultaneously. Unlike
existing 3D face reconstruction methods, our proposed method directly regresses
dense 3D face shapes from single 2D images, and tackles identity and residual
(i.e., non-identity) components in 3D face shapes explicitly and separately
based on a composite 3D face shape model with latent representations. We devise
a training process for the proposed network with a joint loss measuring both
face identification error and 3D face shape reconstruction error. To construct
training data we develop a method for fitting 3D morphable model (3DMM) to
multiple 2D images of a subject. Comprehensive experiments have been done on
MICC, BU3DFE, LFW and YTF databases. The results show that our method expands
the capacity of 3DMM for capturing discriminative shape features and facial
detail, and thus outperforms existing methods both in 3D face reconstruction
accuracy and in face recognition accuracy.

negative:
As a classic statistical model of 3D facial shape and albedo, 3D Morphable
Model (3DMM) is widely used in facial analysis, e.g., model fitting, image
synthesis. Conventional 3DMM is learned from a set of 3D face scans with
associated well-controlled 2D face images, and represented by two sets of PCA
basis functions. Due to the type and amount of training data, as well as, the
linear bases, the representation power of 3DMM can be limited. To address these
problems, this paper proposes an innovative framework to learn a nonlinear 3DMM
model from a large set of in-the-wild face images, without collecting 3D face
scans. Specifically, given a face image as input, a network encoder estimates
the projection, lighting, shape and albedo parameters. Two decoders serve as
the nonlinear 3DMM to map from the shape and albedo parameters to the 3D shape
and albedo, respectively. With the projection parameter, lighting, 3D shape,
and albedo, a novel analytically-differentiable rendering layer is designed to
reconstruct the original input face. The entire network is end-to-end trainable
with only weak supervision. We demonstrate the superior representation power of
our nonlinear 3DMM over its linear counterpart, and its contribution to face
alignment, 3D reconstruction, and face editing.

negative:
This paper investigates the evaluation of dense 3D face reconstruction from a
single 2D image in the wild. To this end, we organise a competition that
provides a new benchmark dataset that contains 2000 2D facial images of 135
subjects as well as their 3D ground truth face scans. In contrast to previous
competitions or challenges, the aim of this new benchmark dataset is to
evaluate the accuracy of a 3D dense face reconstruction algorithm using real,
accurate and high-resolution 3D ground truth face scans. In addition to the
dataset, we provide a standard protocol as well as a Python script for the
evaluation. Last, we report the results obtained by three state-of-the-art 3D
face reconstruction systems on the new benchmark dataset. The competition is
organised along with the 2018 13th IEEE Conference on Automatic Face & Gesture
Recognition.

negative:
In this paper, we address the problem of makeup transfer, which aims at
transplanting the makeup from the reference face to the source face while
preserving the identity of the source. Existing makeup transfer methods have
made notable progress in generating realistic makeup faces, but do not perform
well in terms of color fidelity and spatial transformation. To tackle these
issues, we propose a novel Facial Attribute Transformer (FAT) and its variant
Spatial FAT for high-quality makeup transfer. Drawing inspirations from the
Transformer in NLP, FAT is able to model the semantic correspondences and
interactions between the source face and reference face, and then precisely
estimate and transfer the facial attributes. To further facilitate shape
deformation and transformation of facial parts, we also integrate thin plate
splines (TPS) into FAT, thus creating Spatial FAT, which is the first method
that can transfer geometric attributes in addition to color and texture.
Extensive qualitative and quantitative experiments demonstrate the
effectiveness and superiority of our proposed FATs in the following aspects:
(1) ensuring high-fidelity color transfer; (2) allowing for geometric
transformation of facial parts; (3) handling facial variations (such as poses
and shadows) and (4) supporting high-resolution face generation.

negative:
Meaningful facial parts can convey key cues for both facial action unit
detection and expression prediction. Textured 3D face scan can provide both
detailed 3D geometric shape and 2D texture appearance cues of the face which
are beneficial for Facial Expression Recognition (FER). However, accurate
facial parts extraction as well as their fusion are challenging tasks. In this
paper, a novel system for 3D FER is designed based on accurate facial parts
extraction and deep feature fusion of facial parts. In particular, each
textured 3D face scan is firstly represented as a 2D texture map and a depth
map with one-to-one dense correspondence. Then, the facial parts of both
texture map and depth map are extracted using a novel 4-stage process consists
of facial landmark localization, facial rotation correction, facial resizing,
facial parts bounding box extraction and post-processing procedures. Finally,
deep fusion Convolutional Neural Networks (CNNs) features of all facial parts
are learned from both texture maps and depth maps, respectively and nonlinear
SVMs are used for expression prediction. Experiments are conducted on the
BU-3DFE database, demonstrating the effectiveness of combing different facial
parts, texture and depth cues and reporting the state-of-the-art results in
comparison with all existing methods under the same setting.

negative:
Recently, a lot of attention has been focused on the incorporation of 3D data
into face analysis and its applications. Despite providing a more accurate
representation of the face, 3D facial images are more complex to acquire than
2D pictures. As a consequence, great effort has been invested in developing
systems that reconstruct 3D faces from an uncalibrated 2D image. However, the
3D-from-2D face reconstruction problem is ill-posed, thus prior knowledge is
needed to restrict the solutions space. In this work, we review 3D face
reconstruction methods proposed in the last decade, focusing on those that only
use 2D pictures captured under uncontrolled conditions. We present a
classification of the proposed methods based on the technique used to add prior
knowledge, considering three main strategies, namely, statistical model
fitting, photometry, and deep learning, and reviewing each of them separately.
In addition, given the relevance of statistical 3D facial models as prior
knowledge, we explain the construction procedure and provide a list of the most
popular publicly available 3D facial models. After the exhaustive study of
3D-from-2D face reconstruction approaches, we observe that the deep learning
strategy is rapidly growing since the last few years, becoming the standard
choice in replacement of the widespread statistical model fitting. Unlike the
other two strategies, photometry-based methods have decreased in number due to
the need for strong underlying assumptions that limit the quality of their
reconstructions compared to statistical model fitting and deep learning
methods. The review also identifies current challenges and suggests avenues for
future research.

negative:
3D face reconstruction from a single image is a challenging problem,
especially under partial occlusions and extreme poses. This is because the
uncertainty of the estimated 2D landmarks will affect the quality of face
reconstruction. In this paper, we propose a novel joint 2D and 3D optimization
method to adaptively reconstruct 3D face shapes from a single image, which
combines the depths of 3D landmarks to solve the uncertain detections of
invisible landmarks. The strategy of our method involves two aspects: a
coarse-to-fine pose estimation using both 2D and 3D landmarks, and an adaptive
2D and 3D re-weighting based on the refined pose parameter to recover accurate
3D faces. Experimental results on multiple datasets demonstrate that our method
can generate high-quality reconstruction from a single color image and is
robust for self-occlusion and large poses.

negative:
3D face reconstruction from a single image is a classical and challenging
problem, with wide applications in many areas. Inspired by recent works in face
animation from RGB-D or monocular video inputs, we develop a novel method for
reconstructing 3D faces from unconstrained 2D images, using a coarse-to-fine
optimization strategy. First, a smooth coarse 3D face is generated from an
example-based bilinear face model, by aligning the projection of 3D face
landmarks with 2D landmarks detected from the input image. Afterwards, using
local corrective deformation fields, the coarse 3D face is refined using
photometric consistency constraints, resulting in a medium face shape. Finally,
a shape-from-shading method is applied on the medium face to recover fine
geometric details. Our method outperforms state-of-the-art approaches in terms
of accuracy and detail recovery, which is demonstrated in extensive experiments
using real world models and publicly available datasets.

negative:
This paper presents a part-based face detection approach where the spatial
relationship between the face parts is represented by a hidden 3D model with
six parameters. The computational complexity of the search in the six
dimensional pose space is addressed by proposing meaningful 3D pose candidates
by image-based regression from detected face keypoint locations. The 3D pose
candidates are evaluated using a parameter sensitive classifier based on
difference features relative to the 3D pose. A compatible subset of candidates
is then obtained by non-maximal suppression. Experiments on two standard face
detection datasets show that the proposed 3D model based approach obtains
results comparable to or better than state of the art.

negative:
We present a single-image 3D face synthesis technique that can handle
challenging facial expressions while recovering fine geometric details. Our
technique employs expression analysis for proxy face geometry generation and
combines supervised and unsupervised learning for facial detail synthesis. On
proxy generation, we conduct emotion prediction to determine a new
expression-informed proxy. On detail synthesis, we present a Deep Facial Detail
Net (DFDN) based on Conditional Generative Adversarial Net (CGAN) that employs
both geometry and appearance loss functions. For geometry, we capture 366
high-quality 3D scans from 122 different subjects under 3 facial expressions.
For appearance, we use additional 20K in-the-wild face images and apply
image-based rendering to accommodate lighting variations. Comprehensive
experiments demonstrate that our framework can produce high-quality 3D faces
with realistic details under challenging facial expressions.

negative:
We present a versatile model, FaceAnime, for various video generation tasks
from still images. Video generation from a single face image is an interesting
problem and usually tackled by utilizing Generative Adversarial Networks (GANs)
to integrate information from the input face image and a sequence of sparse
facial landmarks. However, the generated face images usually suffer from
quality loss, image distortion, identity change, and expression mismatching due
to the weak representation capacity of the facial landmarks. In this paper, we
propose to "imagine" a face video from a single face image according to the
reconstructed 3D face dynamics, aiming to generate a realistic and
identity-preserving face video, with precisely predicted pose and facial
expression. The 3D dynamics reveal changes of the facial expression and motion,
and can serve as a strong prior knowledge for guiding highly realistic face
video generation. In particular, we explore face video prediction and exploit a
well-designed 3D dynamic prediction network to predict a 3D dynamic sequence
for a single face image. The 3D dynamics are then further rendered by the
sparse texture mapping algorithm to recover structural details and sparse
textures for generating face frames. Our model is versatile for various AR/VR
and entertainment applications, such as face video retargeting and face video
prediction. Superior experimental results have well demonstrated its
effectiveness in generating high-fidelity, identity-preserving, and visually
pleasant face video clips from a single source face image.

negative:
Facial feature detection from facial images has attracted great attention in
the field of computer vision. It is a nontrivial task since the appearance and
shape of the face tend to change under different conditions. In this paper, we
propose a hierarchical probabilistic model that could infer the true locations
of facial features given the image measurements even if the face is with
significant facial expression and pose. The hierarchical model implicitly
captures the lower level shape variations of facial components using the
mixture model. Furthermore, in the higher level, it also learns the joint
relationship among facial components, the facial expression, and the pose
information through automatic structure learning and parameter estimation of
the probabilistic model. Experimental results on benchmark databases
demonstrate the effectiveness of the proposed hierarchical probabilistic model.

negative:
With the rapid progress of deepfake techniques in recent years, facial video
forgery can generate highly deceptive video contents and bring severe security
threats. And detection of such forgery videos is much more urgent and
challenging. Most existing detection methods treat the problem as a vanilla
binary classification problem. In this paper, the problem is treated as a
special fine-grained classification problem since the differences between fake
and real faces are very subtle. It is observed that most existing face forgery
methods left some common artifacts in the spatial domain and time domain,
including generative defects in the spatial domain and inter-frame
inconsistencies in the time domain. And a spatial-temporal model is proposed
which has two components for capturing spatial and temporal forgery traces in
global perspective respectively. The two components are designed using a novel
long distance attention mechanism. The one component of the spatial domain is
used to capture artifacts in a single frame, and the other component of the
time domain is used to capture artifacts in consecutive frames. They generate
attention maps in the form of patches. The attention method has a broader
vision which contributes to better assembling global information and extracting
local statistic information. Finally, the attention maps are used to guide the
network to focus on pivotal parts of the face, just like other fine-grained
classification methods. The experimental results on different public datasets
demonstrate that the proposed method achieves the state-of-the-art performance,
and the proposed long distance attention method can effectively capture pivotal
parts for face forgery.

negative:
Facial image super-resolution (SR) is an important preprocessing for facial
image analysis, face recognition, and image-based 3D face reconstruction.
Recent convolutional neural network (CNN) based method has shown excellent
performance by learning mapping relation using pairs of low-resolution (LR) and
high-resolution (HR) facial images. However, since the HR facial image
reconstruction using CNN is conventionally aimed to increase the PSNR and SSIM
metrics, the reconstructed HR image might not be realistic even with high
scores. An adversarial framework is proposed in this study to reconstruct the
HR facial image by simultaneously generating an HR image with and without blur.
First, the spatial resolution of the LR facial image is increased by eight
times using a five-layer CNN. Then, the encoder extracts the features of the
up-scaled image. These features are finally sent to two branches (decoders) to
generate an HR facial image with and without blur. In addition, local and
global discriminators are combined to focus on the reconstruction of HR facial
structures. Experiment results show that the proposed algorithm generates a
realistic HR facial image. Furthermore, the proposed method can generate a
variety of different facial images.

negative:
3D face reconstruction from a single image is a task that has garnered
increased interest in the Computer Vision community, especially due to its
broad use in a number of applications such as realistic 3D avatar creation,
pose invariant face recognition and face hallucination. Since the introduction
of the 3D Morphable Model in the late 90's, we witnessed an explosion of
research aiming at particularly tackling this task. Nevertheless, despite the
increasing level of detail in the 3D face reconstructions from single images
mainly attributed to deep learning advances, finer and highly deformable
components of the face such as the tongue are still absent from all 3D face
models in the literature, although being very important for the realness of the
3D avatar representations. In this work we present the first, to the best of
our knowledge, end-to-end trainable pipeline that accurately reconstructs the
3D face together with the tongue. Moreover, we make this pipeline robust in
"in-the-wild" images by introducing a novel GAN method tailored for 3D tongue
surface generation. Finally, we make publicly available to the community the
first diverse tongue dataset, consisting of 1,800 raw scans of 700 individuals
varying in gender, age, and ethnicity backgrounds. As we demonstrate in an
extensive series of quantitative as well as qualitative experiments, our model
proves to be robust and realistically captures the 3D tongue structure, even in
adverse "in-the-wild" conditions.

negative:
We present a fully automatic approach to real-time 3D face reconstruction
from monocular in-the-wild videos. With the use of a cascaded-regressor based
face tracking and a 3D Morphable Face Model shape fitting, we obtain a
semi-dense 3D face shape. We further use the texture information from multiple
frames to build a holistic 3D face representation from the video frames. Our
system is able to capture facial expressions and does not require any
person-specific training. We demonstrate the robustness of our approach on the
challenging 300 Videos in the Wild (300-VW) dataset. Our real-time fitting
framework is available as an open source library at http://4dface.org.

negative:
Recently, data-driven single-view reconstruction methods have shown great
progress in modeling 3D dressed humans. However, such methods suffer heavily
from depth ambiguities and occlusions inherent to single view inputs. In this
paper, we tackle this problem by considering a small set of input views and
investigate the best strategy to suitably exploit information from these views.
We propose a data-driven end-to-end approach that reconstructs an implicit 3D
representation of dressed humans from sparse camera views. Specifically, we
introduce three key components: first a spatially consistent reconstruction
that allows for arbitrary placement of the person in the input views using a
perspective camera model; second an attention-based fusion layer that learns to
aggregate visual information from several viewpoints; and third a mechanism
that encodes local 3D patterns under the multi-view context. In the
experiments, we show the proposed approach outperforms the state of the art on
standard data both quantitatively and qualitatively. To demonstrate the
spatially consistent reconstruction, we apply our approach to dynamic scenes.
Additionally, we apply our method on real data acquired with a multi-camera
platform and demonstrate our approach can obtain results comparable to
multi-view stereo with dramatically less views.

negative:
Face occlusions, covering either the majority or discriminative parts of the
face, can break facial perception and produce a drastic loss of information.
Biometric systems such as recent deep face recognition models are not immune to
obstructions or other objects covering parts of the face. While most of the
current face recognition methods are not optimized to handle occlusions, there
have been a few attempts to improve robustness directly in the training stage.
Unlike those, we propose to study the effect of generative face completion on
the recognition. We offer a face completion encoder-decoder, based on a
convolutional operator with a gating mechanism, trained with an ample set of
face occlusions. To systematically evaluate the impact of realistic occlusions
on recognition, we propose to play the occlusion game: we render 3D objects
onto different face parts, providing precious knowledge of what the impact is
of effectively removing those occlusions. Extensive experiments on the Labeled
Faces in the Wild (LFW), and its more difficult variant LFW-BLUFR, testify that
face completion is able to partially restore face perception in machine vision
systems for improved recognition.

negative:
Recently, face super-resolution (FSR) methods either feed whole face image
into convolutional neural networks (CNNs) or utilize extra facial priors (e.g.,
facial parsing maps, facial landmarks) to focus on facial structure, thereby
maintaining the consistency of the facial structure while restoring facial
details. However, the limited receptive fields of CNNs and inaccurate facial
priors will reduce the naturalness and fidelity of the reconstructed face. In
this paper, we propose a novel paradigm based on the self-attention mechanism
(i.e., the core of Transformer) to fully explore the representation capacity of
the facial structure feature. Specifically, we design a Transformer-CNN
aggregation network (TANet) consisting of two paths, in which one path uses
CNNs responsible for restoring fine-grained facial details while the other
utilizes a resource-friendly Transformer to capture global information by
exploiting the long-distance visual relation modeling. By aggregating the
features from the above two paths, the consistency of global facial structure
and fidelity of local facial detail restoration are strengthened
simultaneously. Experimental results of face reconstruction and recognition
verify that the proposed method can significantly outperform the
state-of-the-art methods.

negative:
We introduce an approach that accurately reconstructs 3D human poses and
detailed 3D full-body geometric models from single images in realtime. The key
idea of our approach is a novel end-to-end multi-task deep learning framework
that uses single images to predict five outputs simultaneously: foreground
segmentation mask, 2D joints positions, semantic body partitions, 3D part
orientations and uv coordinates (uv map). The multi-task network architecture
not only generates more visual cues for reconstruction, but also makes each
individual prediction more accurate. The CNN regressor is further combined with
an optimization based algorithm for accurate kinematic pose reconstruction and
full-body shape modeling. We show that the realtime reconstruction reaches
accurate fitting that has not been seen before, especially for wild images. We
demonstrate the results of our realtime 3D pose and human body reconstruction
system on various challenging in-the-wild videos. We show the system advances
the frontier of 3D human body and pose reconstruction from single images by
quantitative evaluations and comparisons with state-of-the-art methods.

negative:
In this work, we propose a high fidelity face swapping method, called
HifiFace, which can well preserve the face shape of the source face and
generate photo-realistic results. Unlike other existing face swapping works
that only use face recognition model to keep the identity similarity, we
propose 3D shape-aware identity to control the face shape with the geometric
supervision from 3DMM and 3D face reconstruction method. Meanwhile, we
introduce the Semantic Facial Fusion module to optimize the combination of
encoder and decoder features and make adaptive blending, which makes the
results more photo-realistic. Extensive experiments on faces in the wild
demonstrate that our method can preserve better identity, especially on the
face shape, and can generate more photo-realistic results than previous
state-of-the-art methods.

negative:
3D face reconstruction is a challenging problem but also an important task in
the field of computer vision and graphics. Recently, many researchers put
attention to the problem and a large number of articles have been published.
Single image reconstruction is one of the branches of 3D face reconstruction,
which has a lot of applications in our life. This paper is a review of the
recent literature on 3D face reconstruction from a single image.

negative:
Most 3D face reconstruction methods rely on 3D morphable models, which
disentangle the space of facial deformations into identity geometry,
expressions and skin reflectance. These models are typically learned from a
limited number of 3D scans and thus do not generalize well across different
identities and expressions. We present the first approach to learn complete 3D
models of face identity geometry, albedo and expression just from images and
videos. The virtually endless collection of such data, in combination with our
self-supervised learning-based approach allows for learning face models that
generalize beyond the span of existing approaches. Our network design and loss
functions ensure a disentangled parameterization of not only identity and
albedo, but also, for the first time, an expression basis. Our method also
allows for in-the-wild monocular reconstruction at test time. We show that our
learned models better generalize and lead to higher quality image-based
reconstructions than existing approaches.

negative:
3D Morphable Model (3DMM) based methods have achieved great success in
recovering 3D face shapes from single-view images. However, the facial textures
recovered by such methods lack the fidelity as exhibited in the input images.
Recent work demonstrates high-quality facial texture recovering with generative
networks trained from a large-scale database of high-resolution UV maps of face
textures, which is hard to prepare and not publicly available. In this paper,
we introduce a method to reconstruct 3D facial shapes with high-fidelity
textures from single-view images in-the-wild, without the need to capture a
large-scale face texture database. The main idea is to refine the initial
texture generated by a 3DMM based method with facial details from the input
image. To this end, we propose to use graph convolutional networks to
reconstruct the detailed colors for the mesh vertices instead of reconstructing
the UV map. Experiments show that our method can generate high-quality results
and outperforms state-of-the-art methods in both qualitative and quantitative
comparisons.

negative:
Face completion is a challenging generation task because it requires
generating visually pleasing new pixels that are semantically consistent with
the unmasked face region. This paper proposes a geometry-aware Face Completion
and Editing NETwork (FCENet) by systematically studying facial geometry from
the unmasked region. Firstly, a facial geometry estimator is learned to
estimate facial landmark heatmaps and parsing maps from the unmasked face
image. Then, an encoder-decoder structure generator serves to complete a face
image and disentangle its mask areas conditioned on both the masked face image
and the estimated facial geometry images. Besides, since low-rank property
exists in manually labeled masks, a low-rank regularization term is imposed on
the disentangled masks, enforcing our completion network to manage occlusion
area with various shape and size. Furthermore, our network can generate diverse
results from the same masked input by modifying estimated facial geometry,
which provides a flexible mean to edit the completed face appearance. Extensive
experimental results qualitatively and quantitatively demonstrate that our
network is able to generate visually pleasing face completion results and edit
face attributes as well.

negative:
We propose a straightforward method that simultaneously reconstructs the 3D
facial structure and provides dense alignment. To achieve this, we design a 2D
representation called UV position map which records the 3D shape of a complete
face in UV space, then train a simple Convolutional Neural Network to regress
it from a single 2D image. We also integrate a weight mask into the loss
function during training to improve the performance of the network. Our method
does not rely on any prior face model, and can reconstruct full facial geometry
along with semantic meaning. Meanwhile, our network is very light-weighted and
spends only 9.8ms to process an image, which is extremely faster than previous
works. Experiments on multiple challenging datasets show that our method
surpasses other state-of-the-art methods on both reconstruction and alignment
tasks by a large margin.

negative:
In this paper, we propose an effective face completion algorithm using a deep
generative model. Different from well-studied background completion, the face
completion task is more challenging as it often requires to generate
semantically new pixels for the missing key components (e.g., eyes and mouths)
that contain large appearance variations. Unlike existing nonparametric
algorithms that search for patches to synthesize, our algorithm directly
generates contents for missing regions based on a neural network. The model is
trained with a combination of a reconstruction loss, two adversarial losses and
a semantic parsing loss, which ensures pixel faithfulness and local-global
contents consistency. With extensive experimental results, we demonstrate
qualitatively and quantitatively that our model is able to deal with a large
area of missing pixels in arbitrary shapes and generate realistic face
completion results.

negative:
Video facial expression recognition is useful for many applications and
received much interest lately. Although some solutions give really good results
in a controlled environment (no occlusion), recognition in the presence of
partial facial occlusion remains a challenging task. To handle occlusions,
solutions based on the reconstruction of the occluded part of the face have
been proposed. These solutions are mainly based on the texture or the geometry
of the face. However, the similarity of the face movement between different
persons doing the same expression seems to be a real asset for the
reconstruction. In this paper we exploit this asset and propose a new solution
based on an auto-encoder with skip connections to reconstruct the occluded part
of the face in the optical flow domain. To the best of our knowledge, this is
the first proposition to directly reconstruct the movement for facial
expression recognition. We validated our approach in the controlled dataset CK+
on which different occlusions were generated. Our experiments show that the
proposed method reduce significantly the gap, in terms of recognition accuracy,
between occluded and non-occluded situations. We also compare our approach with
existing state-of-the-art solutions. In order to lay the basis of a
reproducible and fair comparison in the future, we also propose a new
experimental protocol that includes occlusion generation and reconstruction
evaluation.

negative:
Occlusions are very common in face images in the wild, leading to the
degraded performance of face-related tasks. Although much effort has been
devoted to removing occlusions from face images, the varying shapes and
textures of occlusions still challenge the robustness of current methods. As a
result, current methods either rely on manual occlusion masks or only apply to
specific occlusions. This paper proposes a novel face de-occlusion model based
on face segmentation and 3D face reconstruction, which automatically removes
all kinds of face occlusions with even blurred boundaries,e.g., hairs. The
proposed model consists of a 3D face reconstruction module, a face segmentation
module, and an image generation module. With the face prior and the occlusion
mask predicted by the first two, respectively, the image generation module can
faithfully recover the missing facial textures. To supervise the training, we
further build a large occlusion dataset, with both manually labeled and
synthetic occlusions. Qualitative and quantitative results demonstrate the
effectiveness and robustness of the proposed method.

negative:
The last few years have witnessed the great success of non-linear generative
models in synthesizing high-quality photorealistic face images. Many recent 3D
facial texture reconstruction and pose manipulation from a single image
approaches still rely on large and clean face datasets to train image-to-image
Generative Adversarial Networks (GANs). Yet the collection of such a large
scale high-resolution 3D texture dataset is still very costly and difficult to
maintain age/ethnicity balance. Moreover, regression-based approaches suffer
from generalization to the in-the-wild conditions and are unable to fine-tune
to a target-image. In this work, we propose an unsupervised approach for
one-shot 3D facial texture completion that does not require large-scale texture
datasets, but rather harnesses the knowledge stored in 2D face generators. The
proposed approach rotates an input image in 3D and fill-in the unseen regions
by reconstructing the rotated image in a 2D face generator, based on the
visible parts. Finally, we stitch the most visible textures at different angles
in the UV image-plane. Further, we frontalize the target image by projecting
the completed texture into the generator. The qualitative and quantitative
experiments demonstrate that the completed UV textures and frontalized images
are of high quality, resembles the original identity, can be used to train a
texture GAN model for 3DMM fitting and improve pose-invariant face recognition.

negative:
Learned 3D representations of human faces are useful for computer vision
problems such as 3D face tracking and reconstruction from images, as well as
graphics applications such as character generation and animation. Traditional
models learn a latent representation of a face using linear subspaces or
higher-order tensor generalizations. Due to this linearity, they can not
capture extreme deformations and non-linear expressions. To address this, we
introduce a versatile model that learns a non-linear representation of a face
using spectral convolutions on a mesh surface. We introduce mesh sampling
operations that enable a hierarchical mesh representation that captures
non-linear variations in shape and expression at multiple scales within the
model. In a variational setting, our model samples diverse realistic 3D faces
from a multivariate Gaussian distribution. Our training data consists of 20,466
meshes of extreme expressions captured over 12 different subjects. Despite
limited training data, our trained model outperforms state-of-the-art face
models with 50% lower reconstruction error, while using 75% fewer parameters.
We also show that, replacing the expression space of an existing
state-of-the-art face model with our autoencoder, achieves a lower
reconstruction error. Our data, model and code are available at
http://github.com/anuragranj/coma

negative:
Impressive progress has been made in audio-driven 3D facial animation
recently, but synthesizing 3D talking-head with rich emotion is still unsolved.
This is due to the lack of 3D generative models and available 3D emotional
dataset with synchronized audios. To address this, we introduce 3D-TalkEmo, a
deep neural network that generates 3D talking head animation with various
emotions. We also create a large 3D dataset with synchronized audios and
videos, rich corpus, as well as various emotion states of different persons
with the sophisticated 3D face reconstruction methods. In the emotion
generation network, we propose a novel 3D face representation structure -
geometry map by classical multi-dimensional scaling analysis. It maps the
coordinates of vertices on a 3D face to a canonical image plane, while
preserving the vertex-to-vertex geodesic distance metric in a least-square
sense. This maintains the adjacency relationship of each vertex and holds the
effective convolutional structure for the 3D facial surface. Taking a neutral
3D mesh and a speech signal as inputs, the 3D-TalkEmo is able to generate vivid
facial animations. Moreover, it provides access to change the emotion state of
the animated speaker.
  We present extensive quantitative and qualitative evaluation of our method,
in addition to user studies, demonstrating the generated talking-heads of
significantly higher quality compared to previous state-of-the-art methods.

negative:
Satellite multi-view stereo (MVS) imagery is particularly suited for
large-scale Earth surface reconstruction. Differing from the perspective camera
model (pin-hole model) that is commonly used for close-range and aerial
cameras, the cubic rational polynomial camera (RPC) model is the mainstream
model for push-broom linear-array satellite cameras. However, the homography
warping used in the prevailing learning based MVS methods is only applicable to
pin-hole cameras. In order to apply the SOTA learning based MVS technology to
the satellite MVS task for large-scale Earth surface reconstruction, RPC
warping should be considered. In this work, we propose, for the first time, a
rigorous RPC warping module. The rational polynomial coefficients are recorded
as a tensor, and the RPC warping is formulated as a series of tensor
transformations. Based on the RPC warping, we propose the deep learning based
satellite MVS (SatMVS) framework for large-scale and wide depth range Earth
surface reconstruction. We also introduce a large-scale satellite image dataset
consisting of 519 5120${\times}$5120 images, which we call the TLC SatMVS
dataset. The satellite images were acquired from a three-line camera (TLC) that
catches triple-view images simultaneously, forming a valuable supplement to the
existing open-source WorldView-3 datasets with single-scanline images.
Experiments show that the proposed RPC warping module and the SatMVS framework
can achieve a superior reconstruction accuracy compared to the pin-hole fitting
method and conventional MVS methods. Code and data are available at
https://github.com/WHU-GPCV/SatMVS.

negative:
Deep convolutional networks (CNNs) have achieved great success in face
completion to generate plausible facial structures. These methods, however, are
limited in maintaining global consistency among face components and recovering
fine facial details. On the other hand, reflectional symmetry is a prominent
property of face image and benefits face recognition and consistency modeling,
yet remaining uninvestigated in deep face completion. In this work, we leverage
two kinds of symmetry-enforcing subnets to form a symmetry-consistent CNN model
(i.e., SymmFCNet) for effective face completion. For missing pixels on only one
of the half-faces, an illumination-reweighted warping subnet is developed to
guide the warping and illumination reweighting of the other half-face. As for
missing pixels on both of half-faces, we present a generative reconstruction
subnet together with a perceptual symmetry loss to enforce symmetry consistency
of recovered structures. The SymmFCNet is constructed by stacking generative
reconstruction subnet upon illumination-reweighted warping subnet, and can be
end-to-end learned from training set of unaligned face images. Experiments show
that SymmFCNet can generate high quality results on images with synthetic and
real occlusion, and performs favorably against state-of-the-arts.

negative:
Hyperspectral imaging, providing abundant spatial and spectral information
simultaneously, has attracted a lot of interest in recent years. Unfortunately,
due to the hardware limitations, the hyperspectral image (HSI) is vulnerable to
various degradations, such noises (random noise, HSI denoising), blurs
(Gaussian and uniform blur, HSI deblurring), and down-sampled (both spectral
and spatial downsample, HSI super-resolution). Previous HSI restoration methods
are designed for one specific task only. Besides, most of them start from the
1-D vector or 2-D matrix models and cannot fully exploit the structurally
spectral-spatial correlation in 3-D HSI. To overcome these limitations, in this
work, we propose a unified low-rank tensor recovery model for comprehensive HSI
restoration tasks, in which non-local similarity between spectral-spatial cubic
and spectral correlation are simultaneously captured by 3-order tensors.
Further, to improve the capability and flexibility, we formulate it as a
weighted low-rank tensor recovery (WLRTR) model by treating the singular values
differently, and study its analytical solution. We also consider the exclusive
stripe noise in HSI as the gross error by extending WLRTR to robust principal
component analysis (WLRTR-RPCA). Extensive experiments demonstrate the proposed
WLRTR models consistently outperform state-of-the-arts in typical low level
vision HSI tasks, including denoising, destriping, deblurring and
super-resolution.

negative:
Face is one of the most important things for communication with the world
around us. It also forms our identity and expressions. Estimating the face
structure is a fundamental task in computer vision with applications in
different areas such as face recognition and medical surgeries. Recently, deep
learning techniques achieved significant results for 3D face reconstruction
from flat images. The main challenge of such techniques is a vital need for
large 3D face datasets. Usually, this challenge is handled by synthetic face
generation. However, synthetic datasets suffer from the existence of
non-possible faces. Here, we propose a face manifold learning method for
synthetic diverse face dataset generation. First, the face structure is divided
into the shape and expression groups. Then, a fully convolutional autoencoder
network is exploited to deal with the non-possible faces, and, simultaneously,
preserving the dataset diversity. Simulation results show that the proposed
method is capable of denoising highly corrupted faces. The diversity of the
generated dataset is evaluated qualitatively and quantitatively and compared to
the existing methods. Experiments show that our manifold learning method
outperforms the state of the art methods significantly.

negative:
In this paper, we aim to address the problem of heterogeneous or
cross-spectral face recognition using machine learning to synthesize visual
spectrum face from infrared images. The synthesis of visual-band face images
allows for more optimal extraction of facial features to be used for face
identification and/or verification. We explore the ability to use Generative
Adversarial Networks (GANs) for face image synthesis, and examine the
performance of these images using pre-trained Convolutional Neural Networks
(CNNs). The features extracted using CNNs are applied in face identification
and verification. We explore the performance in terms of acceptance rate when
using various similarity measures for face verification.

negative:
Attribute guided face image synthesis aims to manipulate attributes on a face
image. Most existing methods for image-to-image translation can either perform
a fixed translation between any two image domains using a single attribute or
require training data with the attributes of interest for each subject.
Therefore, these methods could only train one specific model for each pair of
image domains, which limits their ability in dealing with more than two
domains. Another disadvantage of these methods is that they often suffer from
the common problem of mode collapse that degrades the quality of the generated
images. To overcome these shortcomings, we propose attribute guided face image
generation method using a single model, which is capable to synthesize multiple
photo-realistic face images conditioned on the attributes of interest. In
addition, we adopt the proposed model to increase the realism of the simulated
face images while preserving the face characteristics. Compared to existing
models, synthetic face images generated by our method present a good
photorealistic quality on several face datasets. Finally, we demonstrate that
generated facial images can be used for synthetic data augmentation, and
improve the performance of the classifier used for facial expression
recognition.

negative:
An important goal in human-robot-interaction (HRI) is for machines to achieve
a close to human level of face perception. One of the important differences
between machine learning and human intelligence is the lack of
compositionality. This paper introduces a new scheme to enable generative
adversarial networks to learn the distribution of face images composed of
smaller parts. This results in a more flexible machine face perception and
easier generalization to outside training examples. We demonstrate that this
model is able to produce realistic high-quality face images by generating and
piecing together the parts. Additionally, we demonstrate that this model learns
the relations between the facial parts and their distributions. Therefore, the
specific facial parts are interchangeable between generated face images.

negative:
3D face reconstruction from a single 2D image is a challenging problem with
broad applications. Recent methods typically aim to learn a CNN-based 3D face
model that regresses coefficients of 3D Morphable Model (3DMM) from 2D images
to render 3D face reconstruction or dense face alignment. However, the shortage
of training data with 3D annotations considerably limits performance of those
methods. To alleviate this issue, we propose a novel 2D-assisted
self-supervised learning (2DASL) method that can effectively use "in-the-wild"
2D face images with noisy landmark information to substantially improve 3D face
model learning. Specifically, taking the sparse 2D facial landmarks as
additional information, 2DSAL introduces four novel self-supervision schemes
that view the 2D landmark and 3D landmark prediction as a self-mapping process,
including the 2D and 3D landmark self-prediction consistency, cycle-consistency
over the 2D landmark prediction and self-critic over the predicted 3DMM
coefficients based on landmark predictions. Using these four self-supervision
schemes, the 2DASL method significantly relieves demands on the the
conventional paired 2D-to-3D annotations and gives much higher-quality 3D face
models without requiring any additional 3D annotations. Experiments on multiple
challenging datasets show that our method outperforms state-of-the-arts for
both 3D face reconstruction and dense face alignment by a large margin.

negative:
In this paper, we propose a novel tensor learning and coding model for
third-order data completion. Our model is to learn a data-adaptive dictionary
from the given observations, and determine the coding coefficients of
third-order tensor tubes. In the completion process, we minimize the
low-rankness of each tensor slice containing the coding coefficients. By
comparison with the traditional pre-defined transform basis, the advantages of
the proposed model are that (i) the dictionary can be learned based on the
given data observations so that the basis can be more adaptively and accurately
constructed, and (ii) the low-rankness of the coding coefficients can allow the
linear combination of dictionary features more effectively. Also we develop a
multi-block proximal alternating minimization algorithm for solving such tensor
learning and coding model, and show that the sequence generated by the
algorithm can globally converge to a critical point. Extensive experimental
results for real data sets such as videos, hyperspectral images, and traffic
data are reported to demonstrate these advantages and show the performance of
the proposed tensor learning and coding method is significantly better than the
other tensor completion methods in terms of several evaluation metrics.

negative:
Faces manifest large variations in many aspects, such as identity,
expression, pose, and face styling. Therefore, it is a great challenge to
disentangle and extract these characteristics from facial images, especially in
an unsupervised manner. In this work, we introduce cycle-consistency in facial
characteristics as free supervisory signal to learn facial representations from
unlabeled facial images. The learning is realized by superimposing the facial
motion cycle-consistency and identity cycle-consistency constraints. The main
idea of the facial motion cycle-consistency is that, given a face with
expression, we can perform de-expression to a neutral face via the removal of
facial motion and further perform re-expression to reconstruct back to the
original face. The main idea of the identity cycle-consistency is to exploit
both de-identity into mean face by depriving the given neutral face of its
identity via feature re-normalization and re-identity into neutral face by
adding the personal attributes to the mean face. At training time, our model
learns to disentangle two distinct facial representations to be useful for
performing cycle-consistent face reconstruction. At test time, we use the
linear protocol scheme for evaluating facial representations on various tasks,
including facial expression recognition and head pose regression. We also can
directly apply the learnt facial representations to person recognition,
frontalization and image-to-image translation. Our experiments show that the
results of our approach is competitive with those of existing methods,
demonstrating the rich and unique information embedded in the disentangled
representations. Code is available at https://github.com/JiaRenChang/FaceCycle .

negative:
3D face reconstruction and face alignment are two fundamental and highly
related topics in computer vision. Recently, some works start to use deep
learning models to estimate the 3DMM coefficients to reconstruct 3D face
geometry. However, the performance is restricted due to the limitation of the
pre-defined face templates. To address this problem, some end-to-end methods,
which can completely bypass the calculation of 3DMM coefficients, are proposed
and attract much attention. In this report, we introduce and analyse three
state-of-the-art methods in 3D face reconstruction and face alignment. Some
potential improvement on PRN are proposed to further enhance its accuracy and
speed.

negative:
In this paper, we address the problem of 3D object mesh reconstruction from
RGB videos. Our approach combines the best of multi-view geometric and
data-driven methods for 3D reconstruction by optimizing object meshes for
multi-view photometric consistency while constraining mesh deformations with a
shape prior. We pose this as a piecewise image alignment problem for each mesh
face projection. Our approach allows us to update shape parameters from the
photometric error without any depth or mask information. Moreover, we show how
to avoid a degeneracy of zero photometric gradients via rasterizing from a
virtual viewpoint. We demonstrate 3D object mesh reconstruction results from
both synthetic and real-world videos with our photometric mesh optimization,
which is unachievable with either na\"ive mesh generation networks or
traditional pipelines of surface reconstruction without heavy manual
post-processing.

negative:
Face images in the wild undergo large intra-personal variations, such as
poses, illuminations, occlusions, and low resolutions, which cause great
challenges to face-related applications. This paper addresses this challenge by
proposing a new deep learning framework that can recover the canonical view of
face images. It dramatically reduces the intra-person variances, while
maintaining the inter-person discriminativeness. Unlike the existing face
reconstruction methods that were either evaluated in controlled 2D environment
or employed 3D information, our approach directly learns the transformation
from the face images with a complex set of variations to their canonical views.
At the training stage, to avoid the costly process of labeling canonical-view
images from the training set by hand, we have devised a new measurement to
automatically select or synthesize a canonical-view image for each identity. As
an application, this face recovery approach is used for face verification.
Facial features are learned from the recovered canonical-view face images by
using a facial component-based convolutional neural network. Our approach
achieves the state-of-the-art performance on the LFW dataset.

negative:
Face recognition performance has improved remarkably in the last decade. Much
of this success can be attributed to the development of deep learning
techniques such as convolutional neural networks (CNNs). While CNNs have pushed
the state-of-the-art forward, their training process requires a large amount of
clean and correctly labelled training data. If a CNN is intended to tolerate
facial pose, then we face an important question: should this training data be
diverse in its pose distribution, or should face images be normalized to a
single pose in a pre-processing step? To address this question, we evaluate a
number of popular facial landmarking and pose correction algorithms to
understand their effect on facial recognition performance. Additionally, we
introduce a new, automatic, single-image frontalization scheme that exceeds the
performance of current algorithms. CNNs trained using sets of different
pre-processing methods are used to extract features from the Point and Shoot
Challenge (PaSC) and CMU Multi-PIE datasets. We assert that the subsequent
verification and recognition performance serves to quantify the effectiveness
of each pose correction scheme.

negative:
Facial analysis has attracted much attention in the technology for
human-machine interface. Different methods of classification based on sparse
representation and Gabor kernels have been widely applied in the fields of
facial analysis. However, most of these methods treat face from a whole view
standpoint. In terms of the importance of different facial views, in this
paper, we present multi-view face analysis based on sparse representation and
Gabor wavelet coefficients. To evaluate the performance, we conduct face
analysis experiments including face recognition (FR) and face expression
recognition (FER) on JAFFE database. Experiments are conducted from two parts:
(1) Face images are divided into three facial parts which are forehead, eye and
mouth. (2) Face images are divided into 8 parts by the orientation of Gabor
kernels. Experimental results demonstrate that the proposed methods can
significantly boost the performance and perform better than the other methods.

negative:
Recovering color images and videos from highly undersampled data is a
fundamental and challenging task in face recognition and computer vision. By
the multi-dimensional nature of color images and videos, in this paper, we
propose a novel tensor completion approach, which is able to efficiently
explore the sparsity of tensor data under the discrete cosine transform (DCT).
Specifically, we introduce two ``sparse + low-rank'' tensor completion models
as well as two implementable algorithms for finding their solutions. The first
one is a DCT-based sparse plus weighted nuclear norm induced low-rank
minimization model. The second one is a DCT-based sparse plus $p$-shrinking
mapping induced low-rank optimization model. Moreover, we accordingly propose
two implementable augmented Lagrangian-based algorithms for solving the
underlying optimization models. A series of numerical experiments including
color image inpainting and video data recovery demonstrate that our proposed
approach performs better than many existing state-of-the-art tensor completion
methods, especially for the case when the ratio of missing data is high.

negative:
Face hallucination, which is the task of generating a high-resolution face
image from a low-resolution input image, is a well-studied problem that is
useful in widespread application areas. Face hallucination is particularly
challenging when the input face resolution is very low (e.g., 10 x 12 pixels)
and/or the image is captured in an uncontrolled setting with large pose and
illumination variations. In this paper, we revisit the algorithm introduced in
[1] and present a deep interpretation of this framework that achieves
state-of-the-art under such challenging scenarios. In our deep network
architecture the global and local constraints that define a face can be
efficiently modeled and learned end-to-end using training data. Conceptually
our network design can be partitioned into two sub-networks: the first one
implements the holistic face reconstruction according to global constraints,
and the second one enhances face-specific details and enforces local patch
statistics. We optimize the deep network using a new loss function for
super-resolution that combines reconstruction error with a learned face quality
measure in adversarial setting, producing improved visual results. We conduct
extensive experiments in both controlled and uncontrolled setups and show that
our algorithm improves the state of the art both numerically and visually.

negative:
Face parsing is a basic task in face image analysis. It amounts to labeling
each pixel with appropriate facial parts such as eyes and nose. In the paper,
we present a interlinked convolutional neural network (iCNN) for solving this
problem in an end-to-end fashion. It consists of multiple convolutional neural
networks (CNNs) taking input in different scales. A special interlinking layer
is designed to allow the CNNs to exchange information, enabling them to
integrate local and contextual information efficiently. The hallmark of iCNN is
the extensive use of downsampling and upsampling in the interlinking layers,
while traditional CNNs usually uses downsampling only. A two-stage pipeline is
proposed for face parsing and both stages use iCNN. The first stage localizes
facial parts in the size-reduced image and the second stage labels the pixels
in the identified facial parts in the original image. On a benchmark dataset we
have obtained better results than the state-of-the-art methods.

negative:
In this paper, we present a novel single shot face-related task analysis
method, called Face-SSD, for detecting faces and for performing various
face-related (classification/regression) tasks including smile recognition,
face attribute prediction and valence-arousal estimation in the wild. Face-SSD
uses a Fully Convolutional Neural Network (FCNN) to detect multiple faces of
different sizes and recognise/regress one or more face-related classes.
Face-SSD has two parallel branches that share the same low-level filters, one
branch dealing with face detection and the other one with face analysis tasks.
The outputs of both branches are spatially aligned heatmaps that are produced
in parallel - therefore Face-SSD does not require that face detection, facial
region extraction, size normalisation, and facial region processing are
performed in subsequent steps. Our contributions are threefold: 1) Face-SSD is
the first network to perform face analysis without relying on pre-processing
such as face detection and registration in advance - Face-SSD is a simple and a
single FCNN architecture simultaneously performing face detection and
face-related task analysis - those are conventionally treated as separate
consecutive tasks; 2) Face-SSD is a generalised architecture that is applicable
for various face analysis tasks without modifying the network structure - this
is in contrast to designing task-specific architectures; and 3) Face-SSD
achieves real-time performance (21 FPS) even when detecting multiple faces and
recognising multiple classes in a given image. Experimental results show that
Face-SSD achieves state-of-the-art performance in various face analysis tasks
by reaching a recognition accuracy of 95.76% for smile detection, 90.29% for
attribute prediction, and Root Mean Square (RMS) error of 0.44 and 0.39 for
valence and arousal estimation.

negative:
The COVID-19 pandemic has drastically changed accepted norms globally. Within
the past year, masks have been used as a public health response to limit the
spread of the virus. This sudden change has rendered many face recognition
based access control, authentication and surveillance systems ineffective.
Official documents such as passports, driving license and national identity
cards are enrolled with fully uncovered face images. However, in the current
global situation, face matching systems should be able to match these reference
images with masked face images. As an example, in an airport or security
checkpoint it is safer to match the unmasked image of the identifying document
to the masked person rather than asking them to remove the mask. We find that
current facial recognition techniques are not robust to this form of occlusion.
  To address this unique requirement presented due to the current circumstance,
we propose a set of re-purposed datasets and a benchmark for researchers to
use. We also propose a contrastive visual representation learning based
pre-training workflow which is specialized to masked vs unmasked face matching.
We ensure that our method learns robust features to differentiate people across
varying data collection scenarios. We achieve this by training over many
different datasets and validating our result by testing on various holdout
datasets. The specialized weights trained by our method outperform standard
face recognition features for masked to unmasked face matching. We believe the
provided synthetic mask generating code, our novel training approach and the
trained weights from the masked face models will help in adopting existing face
recognition systems to operate in the current global environment. We
open-source all contributions for broader use by the research community.

negative:
Human face synthesis involves transferring knowledge about the identity and
identity-dependent face shape (IDFS) of a human face to target face images
where the context (e.g., facial expressions, head poses, and other background
factors) may change dramatically. Human faces are non-rigid, so facial
expression leads to deformation of face shape, and head pose also affects the
face observed in 2D images. A key challenge in face transfer is to match the
face with unobserved new contexts, adapting the face appearance to different
poses and expressions accordingly. In this work, we find a way to provide prior
knowledge for generative models to reason about the appropriate appearance of a
human face in response to various expressions and poses. We propose a novel
context-aware face transfer method, called CarTrans, that incorporates causal
effects of contextual factors into face representation, and thus is able to be
aware of the uncertainty of new contexts. We estimate the effect of facial
expression and head pose in terms of counterfactual inference by designing a
controlled intervention trial, thus avoiding the requirement of a large number
of observations to cover the pose-expression space well. Moreover, we propose a
kernel regression-based encoder that eliminates the identity specificity of
target faces when encoding contextual information from target images. The
resulting method shows impressive performance, allowing fine-grained control
over face shape and appearance under various contextual conditions.

negative:
CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful
technique for tensor completion through explicitly capturing the multilinear
latent factors. The existing CP algorithms require the tensor rank to be
manually specified, however, the determination of tensor rank remains a
challenging problem especially for CP rank. In addition, existing approaches do
not take into account uncertainty information of latent factors, as well as
missing entries. To address these issues, we formulate CP factorization using a
hierarchical probabilistic model and employ a fully Bayesian treatment by
incorporating a sparsity-inducing prior over multiple latent factors and the
appropriate hyperpriors over all hyperparameters, resulting in automatic rank
determination. To learn the model, we develop an efficient deterministic
Bayesian inference algorithm, which scales linearly with data size. Our method
is characterized as a tuning parameter-free approach, which can effectively
infer underlying multilinear factors with a low-rank constraint, while also
providing predictive distributions over missing entries. Extensive simulations
on synthetic data illustrate the intrinsic capability of our method to recover
the ground-truth of CP rank and prevent the overfitting problem, even when a
large amount of entries are missing. Moreover, the results from real-world
applications, including image inpainting and facial image synthesis,
demonstrate that our method outperforms state-of-the-art approaches for both
tensor factorization and tensor completion in terms of predictive performance.

negative:
Predicting attributes from face images in the wild is a challenging computer
vision problem. To automatically describe face attributes from face containing
images, traditionally one needs to cascade three technical blocks --- face
localization, facial descriptor construction, and attribute classification ---
in a pipeline. As a typical classification problem, face attribute prediction
has been addressed using deep learning. Current state-of-the-art performance
was achieved by using two cascaded Convolutional Neural Networks (CNNs), which
were specifically trained to learn face localization and attribute description.
In this paper, we experiment with an alternative way of employing the power of
deep representations from CNNs. Combining with conventional face localization
techniques, we use off-the-shelf architectures trained for face recognition to
build facial descriptors. Recognizing that the describable face attributes are
diverse, our face descriptors are constructed from different levels of the CNNs
for different attributes to best facilitate face attribute prediction.
Experiments on two large datasets, LFWA and CelebA, show that our approach is
entirely comparable to the state-of-the-art. Our findings not only demonstrate
an efficient face attribute prediction approach, but also raise an important
question: how to leverage the power of off-the-shelf CNN representations for
novel tasks.

negative:
We introduce a deep convolutional neural networks (CNN) architecture to
classify facial attributes and recognize face images simultaneously via a
shared learning paradigm to improve the accuracy for facial attribute
prediction and face recognition performance. In this method, we use facial
attributes as an auxiliary source of information to assist CNN features
extracted from the face images to improve the face recognition performance.
Specifically, we use a shared CNN architecture that jointly predicts facial
attributes and recognize face images simultaneously via a shared learning
parameters, and then we use facial attribute features an an auxiliary source of
information concatenated by face features to increase the discrimination of the
CNN for face recognition. This process assists the CNN classifier to better
recognize face images. The experimental results show that our model increases
both the face recognition and facial attribute prediction performance,
especially for the identity attributes such as gender and race. We evaluated
our method on several standard datasets labeled by identities and face
attributes and the results show that the proposed method outperforms
state-of-the-art face recognition models.

negative:
Face image quality is an important factor to enable high performance face
recognition systems. Face quality assessment aims at estimating the suitability
of a face image for recognition. Previous work proposed supervised solutions
that require artificially or human labelled quality values. However, both
labelling mechanisms are error-prone as they do not rely on a clear definition
of quality and may not know the best characteristics for the utilized face
recognition system. Avoiding the use of inaccurate quality labels, we proposed
a novel concept to measure face quality based on an arbitrary face recognition
model. By determining the embedding variations generated from random
subnetworks of a face model, the robustness of a sample representation and
thus, its quality is estimated. The experiments are conducted in a
cross-database evaluation setting on three publicly available databases. We
compare our proposed solution on two face embeddings against six
state-of-the-art approaches from academia and industry. The results show that
our unsupervised solution outperforms all other approaches in the majority of
the investigated scenarios. In contrast to previous works, the proposed
solution shows a stable performance over all scenarios. Utilizing the deployed
face recognition model for our face quality assessment methodology avoids the
training phase completely and further outperforms all baseline approaches by a
large margin. Our solution can be easily integrated into current face
recognition systems and can be modified to other tasks beyond face recognition.

negative:
State-of-the-art face recognition systems are based on deep (convolutional)
neural networks. Therefore, it is imperative to determine to what extent face
templates derived from deep networks can be inverted to obtain the original
face image. In this paper, we study the vulnerabilities of a state-of-the-art
face recognition system based on template reconstruction attack. We propose a
neighborly de-convolutional neural network (\textit{NbNet}) to reconstruct face
images from their deep templates. In our experiments, we assumed that no
knowledge about the target subject and the deep network are available. To train
the \textit{NbNet} reconstruction models, we augmented two benchmark face
datasets (VGG-Face and Multi-PIE) with a large collection of images synthesized
using a face generator. The proposed reconstruction was evaluated using type-I
(comparing the reconstructed images against the original face images used to
generate the deep template) and type-II (comparing the reconstructed images
against a different face image of the same subject) attacks. Given the images
reconstructed from \textit{NbNets}, we show that for verification, we achieve
TAR of 95.20\% (58.05\%) on LFW under type-I (type-II) attacks @ FAR of 0.1\%.
Besides, 96.58\% (92.84\%) of the images reconstruction from templates of
partition \textit{fa} (\textit{fb}) can be identified from partition
\textit{fa} in color FERET. Our study demonstrates the need to secure deep
templates in face recognition systems.

negative:
This work presents a flexible system to reconstruct 3D models of objects
captured with an RGB-D sensor. A major advantage of the method is that our
reconstruction pipeline allows the user to acquire a full 3D model of the
object. This is achieved by acquiring several partial 3D models in different
sessions that are automatically merged together to reconstruct a full model. In
addition, the 3D models acquired by our system can be directly used by
state-of-the-art object instance recognition and object tracking modules,
providing object-perception capabilities for different applications, such as
human-object interaction analysis or robot grasping. The system does not impose
constraints in the appearance of objects (textured, untextured) nor in the
modelling setup (moving camera with static object or a turn-table setup). The
proposed reconstruction system has been used to model a large number of objects
resulting in metrically accurate and visually appealing 3D models.

negative:
A head-mounted display (HMD) could be an important component of augmented
reality system. However, as the upper face region is seriously occluded by the
device, the user experience could be affected in applications such as
telecommunication and multi-player video games. In this paper, we first present
a novel experimental setup that consists of two near-infrared (NIR) cameras to
point to the eye regions and one visible-light RGB camera to capture the
visible face region. The main purpose of this paper is to synthesize realistic
face images without occlusions based on the images captured by these cameras.
To this end, we propose a novel synthesis framework that contains four modules:
3D head reconstruction, face alignment and tracking, face synthesis, and eye
synthesis. In face synthesis, we propose a novel algorithm that can robustly
align and track a personalized 3D head model given a face that is severely
occluded by the HMD. In eye synthesis, in order to generate accurate eye
movements and dynamic wrinkle variations around eye regions, we propose another
novel algorithm to colorize the NIR eye images and further remove the "red eye"
effects caused by the colorization. Results show that both hardware setup and
system framework are robust to synthesize realistic face images in video
sequences.

negative:
3D reconstruction from 2D images is a central problem in computer vision.
Recent works have been focusing on reconstruction directly from a single image.
It is well known however that only one image cannot provide enough information
for such a reconstruction. A prior knowledge that has been entertained are 3D
CAD models due to its online ubiquity. A fundamental question is how to
compactly represent millions of CAD models while allowing generalization to new
unseen objects with fine-scaled geometry. We introduce an approach to compactly
represent a 3D mesh. Our method first selects a 3D model from a graph structure
by using a novel free-form deformation FFD 3D-2D registration, and then the
selected 3D model is refined to best fit the image silhouette. We perform a
comprehensive quantitative and qualitative analysis that demonstrates
impressive dense and realistic 3D reconstruction from single images.

negative:
Deep learning-based hyperspectral image super-resolution (SR) methods have
achieved great success recently. However, most existing models can not
effectively explore spatial information and spectral information between bands
simultaneously, obtaining relatively low performance. To address this issue, in
this paper, we propose a novel spectral-spatial residual network for
hyperspectral image super-resolution (SSRNet). Our method can effectively
explore spatial-spectral information by using 3D convolution instead of 2D
convolution, which enables the network to better extract potential information.
Furthermore, we design a spectral-spatial residual module (SSRM) to adaptively
learn more effective features from all the hierarchical features in units
through local feature fusion, significantly improving the performance of the
algorithm. In each unit, we employ spatial and temporal separable 3D
convolution to extract spatial and spectral information, which not only reduces
unaffordable memory usage and high computational cost, but also makes the
network easier to train. Extensive evaluations and comparisons on three
benchmark datasets demonstrate that the proposed approach achieves superior
performance in comparison to existing state-of-the-art methods.

negative:
With the recent success of deep neural networks, remarkable progress has been
achieved on face recognition. However, collecting large-scale real-world
training data for face recognition has turned out to be challenging, especially
due to the label noise and privacy issues. Meanwhile, existing face recognition
datasets are usually collected from web images, lacking detailed annotations on
attributes (e.g., pose and expression), so the influences of different
attributes on face recognition have been poorly investigated. In this paper, we
address the above-mentioned issues in face recognition using synthetic face
images, i.e., SynFace. Specifically, we first explore the performance gap
between recent state-of-the-art face recognition models trained with synthetic
and real face images. We then analyze the underlying causes behind the
performance gap, e.g., the poor intra-class variations and the domain gap
between synthetic and real face images. Inspired by this, we devise the SynFace
with identity mixup (IM) and domain mixup (DM) to mitigate the above
performance gap, demonstrating the great potentials of synthetic data for face
recognition. Furthermore, with the controllable face synthesis model, we can
easily manage different factors of synthetic face generation, including pose,
expression, illumination, the number of identities, and samples per identity.
Therefore, we also perform a systematically empirical analysis on synthetic
face images to provide some insights on how to effectively utilize synthetic
data for face recognition.

negative:
Face recognition remains a hot topic in computer vision, and it is
challenging to tackle the problem that both the training and testing images are
corrupted. In this paper, we propose a novel semi-supervised method based on
the theory of the low-rank matrix recovery for face recognition, which can
simultaneously learn discriminative low-rank and sparse representations for
both training and testing images. To this end, a correlation penalty term is
introduced into the formulation of our proposed method to learn an incoherent
dictionary. Experimental results on several face image databases demonstrate
the effectiveness of our method, i.e., the proposed method is robust to the
illumination, expression and pose variations, as well as images with noises
such as block occlusion or uniform noises.

negative:
Prior knowledge of face shape and structure plays an important role in face
inpainting. However, traditional face inpainting methods mainly focus on the
generated image resolution of the missing portion without consideration of the
special particularities of the human face explicitly and generally produce
discordant facial parts. To solve this problem, we present a domain embedded
multi-model generative adversarial model for inpainting of face images with
large cropped regions. We firstly represent only face regions using the latent
variable as the domain knowledge and combine it with the non-face parts
textures to generate high-quality face images with plausible contents. Two
adversarial discriminators are finally used to judge whether the generated
distribution is close to the real distribution or not. It can not only
synthesize novel image structures but also explicitly utilize the embedded face
domain knowledge to generate better predictions with consistency on structures
and appearance. Experiments on both CelebA and CelebA-HQ face datasets
demonstrate that our proposed approach achieved state-of-the-art performance
and generates higher quality inpainting results than existing ones.

negative:
Face detection and tracking in low resolution image is not a trivial task due
to the limitation in the appearance features for face characterization.
Moreover, facial expression gives additional distortion on this small and noisy
face. In this paper, we propose deformable symmetric Gabor wavelet network face
model for face detection in low resolution image. Our model optimizes the
rotation, translation, dilation, perspective and partial deformation amount of
the face model with symmetry constraints. Symmetry constraints help our model
to be more robust to noise and distortion. Experimental results on our low
resolution face image dataset and videos show promising face detection and
tracking results under various challenging conditions.

negative:
Face liveness detection is an essential prerequisite for face recognition
applications. Previous face liveness detection methods usually train a binary
classifier to differentiate between a fake face and a real face before face
recognition. The client identity information is not utilized in previous face
liveness detection methods. However, in practical face recognition
applications, face spoofing attacks are always aimed at a specific client, and
the client identity information can provide useful clues for face liveness
detection. In this paper, we propose a face liveness detection method based on
the client identity using Siamese network. We detect face liveness after face
recognition instead of before face recognition, that is, we detect face
liveness with the client identity information. We train a Siamese network with
image pairs. Each image pair consists of two real face images or one real and
one fake face images. The face images in each pair come from a same client.
Given a test face image, the face image is firstly recognized by face
recognition system, then the real face image of the identified client is
retrieved to help the face liveness detection. Experiment results demonstrate
the effectiveness of our method.

negative:
Robust face detection is one of the most important pre-processing steps to
support facial expression analysis, facial landmarking, face recognition, pose
estimation, building of 3D facial models, etc. Although this topic has been
intensely studied for decades, it is still challenging due to numerous variants
of face images in real-world scenarios. In this paper, we present a novel
approach named Multiple Scale Faster Region-based Convolutional Neural Network
(MS-FRCNN) to robustly detect human facial regions from images collected under
various challenging conditions, e.g. large occlusions, extremely low
resolutions, facial expressions, strong illumination variations, etc. The
proposed approach is benchmarked on two challenging face detection databases,
i.e. the Wider Face database and the Face Detection Dataset and Benchmark
(FDDB), and compared against recent other face detection methods, e.g.
Two-stage CNN, Multi-scale Cascade CNN, Faceness, Aggregate Chanel Features,
HeadHunter, Multi-view Face Detection, Cascade CNN, etc. The experimental
results show that our proposed approach consistently achieves highly
competitive results with the state-of-the-art performance against other recent
face detection methods.

negative:
Due to the COVID-19 situation, face masks have become a main part of our
daily life. Wearing mouth-and-nose protection has been made a mandate in many
public places, to prevent the spread of the COVID-19 virus. However, face masks
affect the performance of face recognition, since a large area of the face is
covered. The effect of wearing a face mask on the different components of the
face recognition system in a collaborative environment is a problem that is
still to be fully studied. This work studies, for the first time, the effect of
wearing a face mask on face image quality by utilising state-of-the-art face
image quality assessment methods of different natures. This aims at providing
better understanding on the effect of face masks on the operation of face
recognition as a whole system. In addition, we further studied the effect of
simulated masks on face image utility in comparison to real face masks. We
discuss the correlation between the mask effect on face image quality and that
on the face verification performance by automatic systems and human experts,
indicating a consistent trend between both factors. The evaluation is conducted
on the database containing (1) no-masked faces, (2) real face masks, and (3)
simulated face masks, by synthetically generating digital facial masks on
no-masked faces. Finally, a visual interpretation of the face areas
contributing to the quality score of a selected set of quality assessment
methods is provided to give a deeper insight into the difference of network
decisions in masked and non-masked faces, among other variations.

negative:
With the availability of more powerful computers, iterative reconstruction
algorithms are the subject of an ongoing work in the design of more efficient
reconstruction algorithms for X-ray computed tomography. In this work, we show
how two analytical reconstruction algorithms can be improved by correcting the
corresponding reconstructions using a randomized iterative reconstruction
algorithm. The combined analytical reconstruction followed by randomized
iterative reconstruction can also be viewed as a reconstruction algorithm
which, in the experiments we have conducted, uses up to $35\%$ less projection
angles as compared to the analytical reconstruction algorithms and produces the
same results in terms of quality of reconstruction, without increasing the
execution time significantly.

negative:
Face recognition is a long standing challenge in the field of Artificial
Intelligence (AI). The goal is to create systems that accurately detect,
recognize, verify, and understand human faces. There are significant technical
hurdles in making these systems accurate, particularly in unconstrained
settings due to confounding factors related to pose, resolution, illumination,
occlusion, and viewpoint. However, with recent advances in neural networks,
face recognition has achieved unprecedented accuracy, largely built on
data-driven deep learning methods. While this is encouraging, a critical aspect
that is limiting facial recognition accuracy and fairness is inherent facial
diversity. Every face is different. Every face reflects something unique about
us. Aspects of our heritage - including race, ethnicity, culture, geography -
and our individual identify - age, gender, and other visible manifestations of
self-expression, are reflected in our faces. We expect face recognition to work
equally accurately for every face. Face recognition needs to be fair. As we
rely on data-driven methods to create face recognition technology, we need to
ensure necessary balance and coverage in training data. However, there are
still scientific questions about how to represent and extract pertinent facial
features and quantitatively measure facial diversity. Towards this goal,
Diversity in Faces (DiF) provides a data set of one million annotated human
face images for advancing the study of facial diversity. The annotations are
generated using ten well-established facial coding schemes from the scientific
literature. The facial coding schemes provide human-interpretable quantitative
measures of facial features. We believe that by making the extracted coding
schemes available on a large set of faces, we can accelerate research and
development towards creating more fair and accurate facial recognition systems.

negative:
Face image quality can be defined as a measure of the utility of a face image
to automatic face recognition. In this work, we propose (and compare) two
methods for automatic face image quality based on target face quality values
from (i) human assessments of face image quality (matcher-independent), and
(ii) quality values computed from similarity scores (matcher-dependent). A
support vector regression model trained on face features extracted using a deep
convolutional neural network (ConvNet) is used to predict the quality of a face
image. The proposed methods are evaluated on two unconstrained face image
databases, LFW and IJB-A, which both contain facial variations with multiple
quality factors. Evaluation of the proposed automatic face image quality
measures shows we are able to reduce the FNMR at 1% FMR by at least 13% for two
face matchers (a COTS matcher and a ConvNet matcher) by using the proposed face
quality to select subsets of face images and video frames for matching
templates (i.e., multiple faces per subject) in the IJB-A protocol. To our
knowledge, this is the first work to utilize human assessments of face image
quality in designing a predictor of unconstrained face quality that is shown to
be effective in cross-database evaluation.

negative:
The widespread deployment of surveillance cameras for facial recognition
gives rise to many privacy concerns. This study proposes a privacy-friendly
alternative to large scale facial recognition. While there are multiple
techniques to preserve privacy, our work is based on the minimization principle
which implies minimizing the amount of collected personal data. Instead of
running facial recognition software on all video data, we propose to
automatically extract a high quality snapshot of each detected person without
revealing his or her identity. This snapshot is then encrypted and access is
only granted after legal authorization. We introduce a novel unsupervised face
image quality assessment method which is used to select the high quality
snapshots. For this, we train a variational autoencoder on high quality face
images from a publicly available dataset and use the reconstruction probability
as a metric to estimate the quality of each face crop. We experimentally
confirm that the reconstruction probability can be used as biometric quality
predictor. Unlike most previous studies, we do not rely on a manually defined
face quality metric as everything is learned from data. Our face quality
assessment method outperforms supervised, unsupervised and general image
quality assessment methods on the task of improving face verification
performance by rejecting low quality images. The effectiveness of the whole
system is validated qualitatively on still images and videos.