query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : In archaeology, a lot of old photos are produced by polarization cameras, and I am trying to reconstruct an accurate 3D model of an object using images from these polarization cameras. However, I have difficulties determining the surface of the object because of the features of the polarization camera. I need a model that can analyze the angle of polarization of reflected light in captured images. I think both geometric and photometric cues extracted from input color polarization images should be utilized to complete the analysis. I also need a model that can optimize photometric rendering errors and handle different materials given an initial standard geometric reconstruction. I will test the models on synthetic and real data and use them as an inspiration for my model.

positive:
Reconstructing an object's high-quality 3D shape with inherent spectral
reflectance property, beyond typical device-dependent RGB albedos, opens the
door to applications requiring a high-fidelity 3D model in terms of both
geometry and photometry. In this paper, we propose a novel Multi-View Inverse
Rendering (MVIR) method called Spectral MVIR for jointly reconstructing the 3D
shape and the spectral reflectance for each point of object surfaces from
multi-view images captured using a standard RGB camera and low-cost lighting
equipment such as an LED bulb or an LED projector. Our main contributions are
twofold: (i) We present a rendering model that considers both geometric and
photometric principles in the image formation by explicitly considering camera
spectral sensitivity, light's spectral power distribution, and light source
positions. (ii) Based on the derived model, we build a cost-optimization MVIR
framework for the joint reconstruction of the 3D shape and the per-vertex
spectral reflectance while estimating the light source positions and the
shadows. Different from most existing spectral-3D acquisition methods, our
method does not require expensive special equipment and cumbersome geometric
calibration. Experimental results using both synthetic and real-world data
demonstrate that our Spectral MVIR can acquire a high-quality 3D model with
accurate spectral reflectance property.

positive:
This paper makes a first attempt to bring the Shape from Polarization (SfP)
problem to the realm of deep learning. The previous state-of-the-art methods
for SfP have been purely physics-based. We see value in these principled
models, and blend these physical models as priors into a neural network
architecture. This proposed approach achieves results that exceed the previous
state-of-the-art on a challenging dataset we introduce. This dataset consists
of polarization images taken over a range of object textures, paints, and
lighting conditions. We report that our proposed method achieves the lowest
test error on each tested condition in our dataset, showing the value of
blending data-driven and physics-driven approaches.

positive:
We present a novel method for efficient acquisition of shape and spatially
varying reflectance of 3D objects using polarization cues. Unlike previous
works that have exploited polarization to estimate material or object
appearance under certain constraints (known shape or multiview acquisition), we
lift such restrictions by coupling polarization imaging with deep learning to
achieve high quality estimate of 3D object shape (surface normals and depth)
and SVBRDF using single-view polarization imaging under frontal flash
illumination. In addition to acquired polarization images, we provide our deep
network with strong novel cues related to shape and reflectance, in the form of
a normalized Stokes map and an estimate of diffuse color. We additionally
describe modifications to network architecture and training loss which provide
further qualitative improvements. We demonstrate our approach to achieve
superior results compared to recent works employing deep learning in
conjunction with flash illumination.

positive:
Semantic Segmentation (SS) is promising for outdoor scene perception in
safety-critical applications like autonomous vehicles, assisted navigation and
so on. However, traditional SS is primarily based on RGB images, which limits
the reliability of SS in complex outdoor scenes, where RGB images lack
necessary information dimensions to fully perceive unconstrained environments.
As preliminary investigation, we examine SS in an unexpected obstacle detection
scenario, which demonstrates the necessity of multimodal fusion. Thereby, in
this work, we present EAFNet, an Efficient Attention-bridged Fusion Network to
exploit complementary information coming from different optical sensors.
Specifically, we incorporate polarization sensing to obtain supplementary
information, considering its optical characteristics for robust representation
of diverse materials. By using a single-shot polarization sensor, we build the
first RGB-P dataset which consists of 394 annotated pixel-aligned
RGB-Polarization images. A comprehensive variety of experiments shows the
effectiveness of EAFNet to fuse polarization and RGB information, as well as
the flexibility to be adapted to other sensor combination scenarios.

positive:
Autonomous robotics is critically affected by the robustness of its scene
understanding algorithms. We propose a two-axis pipeline based on polarization
indices to analyze dynamic urban scenes. As robots evolve in unknown
environments, they are prone to encountering specular obstacles. Usually,
specular phenomena are rarely taken into account by algorithms which causes
misinterpretations and erroneous estimates. By exploiting all the light
properties, systems can greatly increase their robustness to events. In
addition to the conventional photometric characteristics, we propose to include
polarization sensing.
  We demonstrate in this paper that the contribution of polarization
measurement increases both the performances of segmentation and the quality of
depth estimation. Our polarimetry-based approaches are compared here with other
state-of-the-art RGB-centric methods showing interest of using polarization
imaging.

positive:
This paper describes a multimodal vision sensor that integrates three types
of cameras, including a stereo camera, a polarization camera and a panoramic
camera. Each sensor provides a specific dimension of information: the stereo
camera measures depth per pixel, the polarization obtains the degree of
polarization, and the panoramic camera captures a 360-degree landscape. Data
fusion and advanced environment perception could be built upon the combination
of sensors. Designed especially for autonomous driving, this vision sensor is
shipped with a robust semantic segmentation network. In addition, we
demonstrate how cross-modal enhancement could be achieved by registering the
color image and the polarization image. An example of water hazard detection is
given. To prove the multimodal vision sensor's compatibility with different
devices, a brief runtime performance analysis is carried out.

positive:
The stomatopod (mantis shrimp) visual system has recently provided a
blueprint for the design of paradigm-shifting polarization and multispectral
imaging sensors, enabling solutions to challenging medical and remote sensing
problems. However, these bioinspired sensors lack the high dynamic range (HDR)
and asynchronous polarization vision capabilities of the stomatopod visual
system, limiting temporal resolution to \~12 ms and dynamic range to \~ 72 dB.
Here we present a novel stomatopod-inspired polarization camera which mimics
the sustained and transient biological visual pathways to save power and sample
data beyond the maximum Nyquist frame rate. This bio-inspired sensor
simultaneously captures both synchronous intensity frames and asynchronous
polarization brightness change information with sub-millisecond latencies over
a million-fold range of illumination. Our PDAVIS camera is comprised of 346x260
pixels, organized in 2-by-2 macropixels, which filter the incoming light with
four linear polarization filters offset by 45 degrees. Polarization information
is reconstructed using both low cost and latency event-based algorithms and
more accurate but slower deep neural networks. Our sensor is used to image HDR
polarization scenes which vary at high speeds and to observe dynamical
properties of single collagen fibers in bovine tendon under rapid cyclical
loads

positive:
This paper tackles the problem of estimating 3D body shape of clothed humans
from single polarized 2D images, i.e. polarization images. Polarization images
are known to be able to capture polarized reflected lights that preserve rich
geometric cues of an object, which has motivated its recent applications in
reconstructing surface normal of the objects of interest. Inspired by the
recent advances in human shape estimation from single color images, in this
paper, we attempt at estimating human body shapes by leveraging the geometric
cues from single polarization images. A dedicated two-stage deep learning
approach, SfP, is proposed: given a polarization image, stage one aims at
inferring the fined-detailed body surface normal; stage two gears to
reconstruct the 3D body shape of clothing details. Empirical evaluations on a
synthetic dataset (SURREAL) as well as a real-world dataset (PHSPD) demonstrate
the qualitative and quantitative performance of our approach in estimating
human poses and shapes. This indicates polarization camera is a promising
alternative to the more conventional color or depth imaging for human shape
estimation. Further, normal maps inferred from polarization imaging play a
significant role in accurately recovering the body shapes of clothed people.

positive:
Photometric stereo provides an important method for high-fidelity 3D
reconstruction based on multiple intensity images captured under different
illumination directions. In this paper, we present a complete framework,
including a multilight source illumination and acquisition hardware system and
a two-stage convolutional neural network (CNN) architecture, to construct
inter- and intraframe representations for accurate normal estimation of
non-Lambertian objects. We experimentally investigate numerous network design
alternatives for identifying the optimal scheme to deploy inter- and intraframe
feature extraction modules for the photometric stereo problem. Moreover, we
propose utilizing the easily obtained object mask to eliminate adverse
interference from invalid background regions in intraframe spatial
convolutions, thus effectively improving the accuracy of normal estimation for
surfaces made of dark materials or with cast shadows. Experimental results
demonstrate that the proposed masked two-stage photometric stereo CNN model
(MT-PS-CNN) performs favourably against state-of-the-art photometric stereo
techniques in terms of both accuracy and efficiency. In addition, the proposed
method is capable of predicting accurate and rich surface normal details for
non-Lambertian objects of complex geometry and performs stably given inputs
captured in both sparse and dense lighting distributions.

positive:
This paper addresses the problem of simultaneous 3D reconstruction and
material recognition and segmentation. Enabling robots to recognise different
materials (concrete, metal etc.) in a scene is important for many tasks, e.g.
robotic interventions in nuclear decommissioning. Previous work on 3D semantic
reconstruction has predominantly focused on recognition of everyday domestic
objects (tables, chairs etc.), whereas previous work on material recognition
has largely been confined to single 2D images without any 3D reconstruction.
Meanwhile, most 3D semantic reconstruction methods rely on computationally
expensive post-processing, using Fully-Connected Conditional Random Fields
(CRFs), to achieve consistent segmentations. In contrast, we propose a deep
learning method which performs 3D reconstruction while simultaneously
recognising different types of materials and labelling them at the pixel level.
Unlike previous methods, we propose a fully end-to-end approach, which does not
require hand-crafted features or CRF post-processing. Instead, we use only
learned features, and the CRF segmentation constraints are incorporated inside
the fully end-to-end learned system. We present the results of experiments, in
which we trained our system to perform real-time 3D semantic reconstruction for
23 different materials in a real-world application. The run-time performance of
the system can be boosted to around 10Hz, using a conventional GPU, which is
enough to achieve real-time semantic reconstruction using a 30fps RGB-D camera.
To the best of our knowledge, this work is the first real-time end-to-end
system for simultaneous 3D reconstruction and material recognition.

positive:
Photometric Stereo methods seek to reconstruct the 3d shape of an object from
motionless images obtained with varying illumination. Most existing methods
solve a restricted problem where the physical reflectance model, such as
Lambertian reflectance, is known in advance. In contrast, we do not restrict
ourselves to a specific reflectance model. Instead, we offer a method that
works on a wide variety of reflectances. Our approach uses a simple yet
uncommonly used property of the problem - the sought after normals are points
on a unit hemisphere. We present a novel embedding method that maps pixels to
normals on the unit hemisphere. Our experiments demonstrate that this approach
outperforms existing manifold learning methods for the task of hemisphere
embedding. We further show successful reconstructions of objects from a wide
variety of reflectances including smooth, rough, diffuse and specular surfaces,
even in the presence of significant attached shadows. Finally, we empirically
prove that under these challenging settings we obtain more accurate shape
reconstructions than existing methods.

positive:
In this paper we present ADOP, a novel point-based, differentiable neural
rendering pipeline. Like other neural renderers, our system takes as input
calibrated camera images and a proxy geometry of the scene, in our case a point
cloud. To generate a novel view, the point cloud is rasterized with learned
feature vectors as colors and a deep neural network fills the remaining holes
and shades each output pixel. The rasterizer renders points as one-pixel
splats, which makes it very fast and allows us to compute gradients with
respect to all relevant input parameters efficiently. Furthermore, our pipeline
contains a fully differentiable physically-based photometric camera model,
including exposure, white balance, and a camera response function. Following
the idea of inverse rendering, we use our renderer to refine its input in order
to reduce inconsistencies and optimize the quality of its output. In
particular, we can optimize structural parameters like the camera pose, lens
distortions, point positions and features, and a neural environment map, but
also photometric parameters like camera response function, vignetting, and
per-image exposure and white balance. Because our pipeline includes photometric
parameters, e.g.~exposure and camera response function, our system can smoothly
handle input images with varying exposure and white balance, and generates
high-dynamic range output. We show that due to the improved input, we can
achieve high render quality, also for difficult input, e.g. with imperfect
camera calibrations, inaccurate proxy geometry, or varying exposure. As a
result, a simpler and thus faster deep neural network is sufficient for
reconstruction. In combination with the fast point rasterization, ADOP achieves
real-time rendering rates even for models with well over 100M points.
https://github.com/darglein/ADOP

positive:
We present a new data-driven approach with physics-based priors to
scene-level normal estimation from a single polarization image. Existing shape
from polarization (SfP) works mainly focus on estimating the normal of a single
object rather than complex scenes in the wild. A key barrier to high-quality
scene-level SfP is the lack of real-world SfP data in complex scenes. Hence, we
contribute the first real-world scene-level SfP dataset with paired input
polarization images and ground-truth normal maps. Then we propose a
learning-based framework with a multi-head self-attention module and viewing
encoding, which is designed to handle increasing polarization ambiguities
caused by complex materials and non-orthographic projection in scene-level SfP.
Our trained model can be generalized to far-field outdoor scenes as the
relationship between polarized light and surface normals is not affected by
distance. Experimental results demonstrate that our approach significantly
outperforms existing SfP models on two datasets. Our dataset and source code
will be publicly available at https://github.com/ChenyangLEI/sfp-wild

negative:
This paper presents an uncalibrated deep neural network framework for the
photometric stereo problem. For training models to solve the problem, existing
neural network-based methods either require exact light directions or
ground-truth surface normals of the object or both. However, in practice, it is
challenging to procure both of this information precisely, which restricts the
broader adoption of photometric stereo algorithms for vision application. To
bypass this difficulty, we propose an uncalibrated neural inverse rendering
approach to this problem. Our method first estimates the light directions from
the input images and then optimizes an image reconstruction loss to calculate
the surface normals, bidirectional reflectance distribution function value, and
depth. Additionally, our formulation explicitly models the concave and convex
parts of a complex surface to consider the effects of interreflections in the
image formation process. Extensive evaluation of the proposed method on the
challenging subjects generally shows comparable or better results than the
supervised and classical approaches.

negative:
In current practice, scene survey is carried out by workers using total
stations. The method has high accuracy, but it incurs high costs if continuous
monitoring is needed. Techniques based on photogrammetry, with the relatively
cheaper digital cameras, have gained wide applications in many fields. Besides
point measurement, photogrammetry can also create a three-dimensional (3D)
model of the scene. Accurate 3D model reconstruction depends on high quality
images. Degraded images will result in large errors in the reconstructed 3D
model. In this paper, we propose a method that can be used to improve the
visibility of the images, and eventually reduce the errors of the 3D scene
model. The idea is inspired by image dehazing. Each original image is first
transformed into multiple exposure images by means of gamma-correction
operations and adaptive histogram equalization. The transformed images are
analyzed by the computation of the local binary patterns. The image is then
enhanced, with each pixel generated from the set of transformed image pixels
weighted by a function of the local pattern feature and image saturation.
Performance evaluation has been performed on benchmark image dehazing datasets.
Experimentations have been carried out on outdoor and indoor surveys. Our
analysis finds that the method works on different types of degradation that
exist in both outdoor and indoor images. When fed into the photogrammetry
software, the enhanced images can reconstruct 3D scene models with
sub-millimeter mean errors.

negative:
Shape from Polarization (SfP) estimates surface normals using photos captured
at different polarizer rotations. Fundamentally, the SfP model assumes that
light is reflected either diffusely or specularly. However, this model is not
valid for many real-world surfaces exhibiting a mixture of diffuse and specular
properties. To address this challenge, previous methods have used a sequential
solution: first, use an existing algorithm to separate the scene into diffuse
and specular components, then apply the appropriate SfP model. In this paper,
we propose a new method that jointly uses viewpoint and polarization data to
holistically separate diffuse and specular components, recover refractive
index, and ultimately recover 3D shape. By involving the physics of
polarization in the separation process, we demonstrate competitive results with
a benchmark method, while recovering additional information (e.g. refractive
index).

negative:
We present a novel framework to learn to convert the perpixel photometric
information at each view into spatially distinctive and view-invariant
low-level features, which can be plugged into existing multi-view stereo
pipeline for enhanced 3D reconstruction. Both the illumination conditions
during acquisition and the subsequent per-pixel feature transform can be
jointly optimized in a differentiable fashion. Our framework automatically
adapts to and makes efficient use of the geometric information available in
different forms of input data. High-quality 3D reconstructions of a variety of
challenging objects are demonstrated on the data captured with an illumination
multiplexing device, as well as a point light. Our results compare favorably
with state-of-the-art techniques.

negative:
The emergence of the single-chip polarized color sensor now allows for
simultaneously capturing chromatic and polarimetric information of the scene on
a monochromatic image plane. However, unlike the usual camera with an embedded
demosaicing method, the latest polarized color camera is not delivered with an
in-built demosaicing tool. For demosaicing, the users have to down-sample the
captured images or to use traditional interpolation techniques. Neither of them
can perform well since the polarization and color are interdependent.
Therefore, joint chromatic and polarimetric demosaicing is the key to obtaining
high-quality polarized color images. In this paper, we propose a joint
chromatic and polarimetric demosaicing model to address this challenging
problem. Instead of mechanically demosaicing for the multi-channel polarized
color image, we further present a sparse representation-based optimization
strategy that utilizes chromatic information and polarimetric information to
jointly optimize the model. To avoid the interaction between color and
polarization during demosaicing, we separately construct the corresponding
dictionaries. We also build an optical data acquisition system to collect a
dataset, which contains various sources of polarization, such as illumination,
reflectance and birefringence. Results of both qualitative and quantitative
experiments have shown that our method is capable of faithfully recovering full
RGB information of four polarization angles for each pixel from a single mosaic
input image. Moreover, the proposed method can perform well not only on the
synthetic data but the real captured data.

negative:
Neural implicit surfaces have become an important technique for multi-view 3D
reconstruction but their accuracy remains limited. In this paper, we argue that
this comes from the difficulty to learn and render high frequency textures with
neural networks. We thus propose to add to the standard neural rendering
optimization a direct photo-consistency term across the different views.
Intuitively, we optimize the implicit geometry so that it warps views on each
other in a consistent way. We demonstrate that two elements are key to the
success of such an approach: (i) warping entire patches, using the predicted
occupancy and normals of the 3D points along each ray, and measuring their
similarity with a robust structural similarity (SSIM); (ii) handling visibility
and occlusion in such a way that incorrect warps are not given too much
importance while encouraging a reconstruction as complete as possible. We
evaluate our approach, dubbed NeuralWarp, on the standard DTU and EPFL
benchmarks and show it outperforms state of the art unsupervised implicit
surfaces reconstructions by over 20% on both datasets.

negative:
Since specular reflection often exists in the real captured images and causes
deviation between the recorded color and intrinsic color, specular reflection
separation can bring advantages to multiple applications that require
consistent object surface appearance. However, due to the color of an object is
significantly influenced by the color of the illumination, the existing
researches still suffer from the near-duplicate challenge, that is, the
separation becomes unstable when the illumination color is close to the surface
color. In this paper, we derive a polarization guided model to incorporate the
polarization information into a designed iteration optimization separation
strategy to separate the specular reflection. Based on the analysis of
polarization, we propose a polarization guided model to generate a polarization
chromaticity image, which is able to reveal the geometrical profile of the
input image in complex scenarios, such as diversity of illumination. The
polarization chromaticity image can accurately cluster the pixels with similar
diffuse color. We further use the specular separation of all these clusters as
an implicit prior to ensure that the diffuse components will not be mistakenly
separated as the specular components. With the polarization guided model, we
reformulate the specular reflection separation into a unified optimization
function which can be solved by the ADMM strategy. The specular reflection will
be detected and separated jointly by RGB and polarimetric information. Both
qualitative and quantitative experimental results have shown that our method
can faithfully separate the specular reflection, especially in some challenging
scenarios.

negative:
Stereo cameras are a popular choice for obstacle avoidance for outdoor
lighweight, low-cost robotics applications. However, they are unable to sense
thin and reflective objects well. Currently, many algorithms are tuned to
perform well on indoor scenes like the Middlebury dataset. When navigating
outdoors, reflective objects, like windows and glass, and thin obstacles, like
wires, are not well handled by most stereo disparity algorithms. Reflections,
repeating patterns and objects parallel to the cameras' baseline causes
mismatches between image pairs which leads to bad disparity estimates. Thin
obstacles are difficult for many sliding window based disparity methods to
detect because they do not take up large portions of the pixels in the sliding
window. We use a trinocular camera setup and micropolarizer camera capable of
detecting reflective objects to overcome these issues. We present a
hierarchical disparity algorithm that reduces noise, separately identify wires
using semantic object triangulation in three images, and use information about
the polarization of light to estimate the disparity of reflective objects. We
evaluate our approach on outdoor data that we collected. Our method contained
an average of 9.27% of bad pixels compared to a typical stereo algorithm's
18.4% of bad pixels in scenes containing reflective objects. Our trinocular and
semantic wire disparity methods detected 53% of wire pixels, whereas a typical
two camera stereo algorithm detected 5%.

negative:
Several approaches to image stitching use different constraints to estimate
the motion model between image pairs. These constraints can be roughly divided
into two categories: geometric constraints and photometric constraints. In this
paper, geometric and photometric constraints are combined to improve the
alignment quality, which is based on the observation that these two kinds of
constraints are complementary. On the one hand, geometric constraints (e.g.,
point and line correspondences) are usually spatially biased and are
insufficient in some extreme scenes, while photometric constraints are always
evenly and densely distributed. On the other hand, photometric constraints are
sensitive to displacements and are not suitable for images with large
parallaxes, while geometric constraints are usually imposed by feature matching
and are more robust to handle parallaxes. The proposed method therefore
combines them together in an efficient mesh-based image warping framework. It
achieves better alignment quality than methods only with geometric constraints,
and can handle larger parallax than photometric-constraint-based method.
Experimental results on various images illustrate that the proposed method
outperforms representative state-of-the-art image stitching methods reported in
the literature.

negative:
This paper focuses on a new problem of estimating human pose and shape from
single polarization images. Polarization camera is known to be able to capture
the polarization of reflected lights that preserves rich geometric cues of an
object surface. Inspired by the recent applications in surface normal
reconstruction from polarization images, in this paper, we attempt to estimate
human pose and shape from single polarization images by leveraging the
polarization-induced geometric cues. A dedicated two-stage pipeline is
proposed: given a single polarization image, stage one (Polar2Normal) focuses
on the fine detailed human body surface normal estimation; stage two
(Polar2Shape) then reconstructs clothed human shape from the polarization image
and the estimated surface normal. To empirically validate our approach, a
dedicated dataset (PHSPD) is constructed, consisting of over 500K frames with
accurate pose and parametric shape annotations. Empirical evaluations on this
real-world dataset as well as a synthetic dataset, SURREAL, demonstrate the
effectiveness of our approach. It suggests polarization camera as a promising
alternative to the more conventional RGB camera for human pose and shape
estimation.

negative:
Inspired by the recent advance of image-based object reconstruction using
deep learning, we present an active reconstruction model using a guided view
planner. We aim to reconstruct a 3D model using images observed from a planned
sequence of informative and discriminative views. But where are such
informative and discriminative views around an object? To address this we
propose a unified model for view planning and object reconstruction, which is
utilized to learn a guided information acquisition model and to aggregate
information from a sequence of images for reconstruction. Experiments show that
our model (1) increases our reconstruction accuracy with an increasing number
of views (2) and generally predicts a more informative sequence of views for
object reconstruction compared to other alternative methods.

negative:
The problem of obtaining dense reconstruction of an object in a natural
sequence of images has been long studied in computer vision. Classically this
problem has been solved through the application of bundle adjustment (BA). More
recently, excellent results have been attained through the application of
photometric bundle adjustment (PBA) methods -- which directly minimize the
photometric error across frames. A fundamental drawback to BA & PBA, however,
is: (i) their reliance on having to view all points on the object, and (ii) for
the object surface to be well textured. To circumvent these limitations we
propose semantic PBA which incorporates a 3D object prior, obtained through
deep learning, within the photometric bundle adjustment problem. We demonstrate
state of the art performance in comparison to leading methods for object
reconstruction across numerous natural sequences.

negative:
Polarization images are known to be able to capture polarized reflected
lights that preserve rich geometric cues of an object, which has motivated its
recent applications in reconstructing detailed surface normal of the objects of
interest. Meanwhile, inspired by the recent breakthroughs in human shape
estimation from a single color image, we attempt to investigate the new
question of whether the geometric cues from polarization camera could be
leveraged in estimating detailed human body shapes. This has led to the
curation of Polarization Human Shape and Pose Dataset (PHSPD), our home-grown
polarization image dataset of various human shapes and poses.

negative:
Current self-driving car systems operate well in sunny weather but struggle
in adverse conditions. One of the most commonly encountered adverse conditions
involves water on the road caused by rain, sleet, melting snow or flooding.
While some advances have been made in using conventional RGB camera and LIDAR
technology for detecting water hazards, other sources of information such as
polarization offer a promising and potentially superior approach to this
problem in terms of performance and cost. In this paper, we present a novel
stereo-polarization system for detecting and tracking water hazards based on
polarization and color variation of reflected light, with consideration of the
effect of polarized light from sky as function of reflection and azimuth
angles. To evaluate this system, we present a new large `water on road'
datasets spanning approximately 2 km of driving in various on-road and off-road
conditions and demonstrate for the first time reliable water detection and
tracking over a wide range of realistic car driving water conditions using
polarized vision as the primary sensing modality. Our system successfully
detects water hazards up to more than 100m. Finally, we discuss several
interesting challenges and propose future research directions for further
improving robust autonomous car perception in hazardous wet conditions using
polarization sensors.

negative:
Wide field-of-view (FOV) cameras, which capture a larger scene area than
narrow FOV cameras, are used in many applications including 3D reconstruction,
autonomous driving, and video surveillance. However, wide-angle images contain
distortions that violate the assumptions underlying pinhole camera models,
resulting in object distortion, difficulties in estimating scene distance,
area, and direction, and preventing the use of off-the-shelf deep models
trained on undistorted images for downstream computer vision tasks. Image
rectification, which aims to correct these distortions, can solve these
problems. In this paper, we comprehensively survey progress in wide-angle image
rectification from transformation models to rectification methods.
Specifically, we first present a detailed description and discussion of the
camera models used in different approaches. Then, we summarize several
distortion models including radial distortion and projection distortion. Next,
we review both traditional geometry-based image rectification methods and deep
learning-based methods, where the former formulate distortion parameter
estimation as an optimization problem and the latter treat it as a regression
problem by leveraging the power of deep neural networks. We evaluate the
performance of state-of-the-art methods on public datasets and show that
although both kinds of methods can achieve good results, these methods only
work well for specific camera models and distortion types. We also provide a
strong baseline model and carry out an empirical study of different distortion
models on synthetic datasets and real-world wide-angle images. Finally, we
discuss several potential research directions that are expected to further
advance this area in the future.

negative:
In this paper, we present a novel deep method to reconstruct a point cloud of
an object from a single still image. Prior arts in the field struggle to
reconstruct an accurate and scalable 3D model due to either the inefficient and
expensive 3D representations, the dependency between the output and number of
model parameters or the lack of a suitable computing operation. We propose to
overcome these by deforming a random point cloud to the object shape through
two steps: feature blending and deformation. In the first step, the global and
point-specific shape features extracted from a 2D object image are blended with
the encoded feature of a randomly generated point cloud, and then this mixture
is sent to the deformation step to produce the final representative point set
of the object. In the deformation process, we introduce a new layer termed as
GraphX that considers the inter-relationship between points like common graph
convolutions but operates on unordered sets. Moreover, with a simple trick, the
proposed model can generate an arbitrary-sized point cloud, which is the first
deep method to do so. Extensive experiments verify that we outperform existing
models and halve the state-of-the-art distance score in single image 3D
reconstruction.

negative:
Simultaneous reconstruction of geometry and reflectance properties in
uncontrolled environments remains a challenging problem. In this paper, we
propose an efficient method to reconstruct the scene's 3D geometry and
reflectance from multi-view photography using conventional hand-held cameras.
Our method automatically builds a virtual scene in a differentiable rendering
system that roughly matches the real world's scene parameters, optimized by
minimizing photometric objectives alternatingly and stochastically. With the
optimal scene parameters evaluated, photo-realistic novel views for various
viewing angles and distances can then be generated by our approach. We present
the results of captured scenes with complex geometry and various reflection
types. Our method also shows superior performance compared to state-of-the-art
alternatives in novel view synthesis visually and quantitatively.

negative:
This paper presents a method of passive non-line-of-sight (NLOS) imaging
using polarization cues. A key observation is that the oblique light has a
different polarimetric signal. It turns out this effect is due to the
polarization axis rotation, a phenomena which can be used to better condition
the light transport matrix for non-line-of-sight imaging. Our analysis and
results show that the use of a polarizer in front of the camera is not only a
separate technique, but it can be seen as an enhancement technique for more
advanced forms of passive NLOS imaging. For example, this paper shows that
polarization can enhance passive NLOS imaging both with and without occluders.
In all tested cases, despite the light attenuation from polarization optics,
recovery of the occluded images is improved.

negative:
We present a novel 3D face reconstruction technique that leverages sparse
photometric stereo (PS) and latest advances on face registration/modeling from
a single image. We observe that 3D morphable faces approach provides a
reasonable geometry proxy for light position calibration. Specifically, we
develop a robust optimization technique that can calibrate per-pixel lighting
direction and illumination at a very high precision without assuming uniform
surface albedos. Next, we apply semantic segmentation on input images and the
geometry proxy to refine hairy vs. bare skin regions using tailored filters.
Experiments on synthetic and real data show that by using a very small set of
images, our technique is able to reconstruct fine geometric details such as
wrinkles, eyebrows, whelks, pores, etc, comparable to and sometimes surpassing
movie quality productions.

negative:
Photometric stereo is a method that seeks to reconstruct the normal vectors
of an object from a set of images of the object illuminated under different
light sources. While effective in some situations, classical photometric stereo
relies on a diffuse surface model that cannot handle objects with complex
reflectance patterns, and it is sensitive to non-idealities in the images. In
this work, we propose a novel approach to photometric stereo that relies on
dictionary learning to produce robust normal vector reconstructions.
Specifically, we develop two formulations for applying dictionary learning to
photometric stereo. We propose a model that applies dictionary learning to
regularize and reconstruct the normal vectors from the images under the classic
Lambertian reflectance model. We then generalize this model to explicitly model
non-Lambertian objects. We investigate both approaches through extensive
experimentation on synthetic and real benchmark datasets and observe
state-of-the-art performance compared to existing robust photometric stereo
methods.

negative:
We present a method to capture both 3D shape and spatially varying
reflectance with a multi-view photometric stereo (MVPS) technique that works
for general isotropic materials. Our algorithm is suitable for perspective
cameras and nearby point light sources. Our data capture setup is simple, which
consists of only a digital camera, some LED lights, and an optional automatic
turntable. From a single viewpoint, we use a set of photometric stereo images
to identify surface points with the same distance to the camera. We collect
this information from multiple viewpoints and combine it with
structure-from-motion to obtain a precise reconstruction of the complete 3D
shape. The spatially varying isotropic bidirectional reflectance distribution
function (BRDF) is captured by simultaneously inferring a set of basis BRDFs
and their mixing weights at each surface point. In experiments, we demonstrate
our algorithm with two different setups: a studio setup for highest precision
and a desktop setup for best usability. According to our experiments, under the
studio setting, the captured shapes are accurate to 0.5 millimeters and the
captured reflectance has a relative root-mean-square error (RMSE) of 9%. We
also quantitatively evaluate state-of-the-art MVPS on a newly collected
benchmark dataset, which is publicly available for inspiring future research.

negative:
This paper is concerned with polarimetric dense map reconstruction based on a
polarization camera with the help of relative depth information as a prior. In
general, polarization imaging is able to reveal information about surface
normal such as azimuth and zenith angles, which can support the development of
solutions to the problem of dense reconstruction, especially in texture-poor
regions. However, polarimetric shape cues are ambiguous due to two types of
polarized reflection (specular/diffuse). Although methods have been proposed to
address this issue, they either are offline and therefore not practical in
robotics applications, or use incomplete polarimetric cues, leading to
sub-optimal performance. In this paper, we propose an online reconstruction
method that uses full polarimetric cues available from the polarization camera.
With our online method, we can propagate sparse depth values both along and
perpendicular to iso-depth contours. Through comprehensive experiments on
challenging image sequences, we demonstrate that our method is able to
significantly improve the accuracy of the depthmap as well as increase its
density, specially in regions of poor texture.

negative:
The recent development of the on-chip micro-polarizer technology has made it
possible to acquire four spatially aligned and temporally synchronized
polarization images with the same ease of operation as a conventional camera.
In this paper, we investigate the use of this sensor technology in
high-dynamic-range (HDR) imaging. Specifically, observing that natural light
can be attenuated differently by varying the orientation of the polarization
filter, we treat the multiple images captured by the polarization camera as a
set captured under different exposure times. In our approach, we first study
the relationship among polarizer orientation, degree and angle of polarization
of light to the exposure time of a pixel in the polarization image.
Subsequently, we propose a deep snapshot HDR reconstruction framework to
recover an HDR image using the polarization images. A polarized HDR dataset is
created to train and evaluate our approach. We demonstrate that our approach
performs favorably against state-of-the-art HDR reconstruction algorithms.

negative:
In this paper, we address the problem of 3D object mesh reconstruction from
RGB videos. Our approach combines the best of multi-view geometric and
data-driven methods for 3D reconstruction by optimizing object meshes for
multi-view photometric consistency while constraining mesh deformations with a
shape prior. We pose this as a piecewise image alignment problem for each mesh
face projection. Our approach allows us to update shape parameters from the
photometric error without any depth or mask information. Moreover, we show how
to avoid a degeneracy of zero photometric gradients via rasterizing from a
virtual viewpoint. We demonstrate 3D object mesh reconstruction results from
both synthetic and real-world videos with our photometric mesh optimization,
which is unachievable with either na\"ive mesh generation networks or
traditional pipelines of surface reconstruction without heavy manual
post-processing.

negative:
Pixel-wise operations between polarimetric images are important for
processing polarization information. For the lack of such operations, the
polarization information cannot be fully utilized in convolutional neural
network(CNN). In this paper, a novel end-to-end CNN framework for polarization
vision tasks is proposed, which enables the networks to take full advantage of
polarimetric images. The framework consists of two sub-networks: a
polarization-parameter-constructing network (PPCN) and a task network. PPCN
implements pixel-wise operations between images in the CNN form with 1x1
convolution kernels. It takes raw polarimetric images as input, and outputs
polarization-parametric images to task network so as to complete a vison task.
By training together, the PPCN can learn to provide the most suitable
polarization-parametric images for the task network and the dataset. Taking
faster R-CNN as task network, the experimental results show that compared with
existing methods, the proposed framework achieves much higher
mean-average-precision (mAP) in object detection task

negative:
This paper addresses the problem of photometric stereo for non-Lambertian
surfaces. Existing approaches often adopt simplified reflectance models to make
the problem more tractable, but this greatly hinders their applications on
real-world objects. In this paper, we propose a deep fully convolutional
network, called PS-FCN, that takes an arbitrary number of images of a static
object captured under different light directions with a fixed camera as input,
and predicts a normal map of the object in a fast feed-forward pass. Unlike the
recently proposed learning based method, PS-FCN does not require a pre-defined
set of light directions during training and testing, and can handle multiple
images and light directions in an order-agnostic manner. Although we train
PS-FCN on synthetic data, it can generalize well on real datasets. We further
show that PS-FCN can be easily extended to handle the problem of uncalibrated
photometric stereo.Extensive experiments on public real datasets show that
PS-FCN outperforms existing approaches in calibrated photometric stereo, and
promising results are achieved in uncalibrated scenario, clearly demonstrating
its effectiveness.

negative:
We introduce a novel learning-based method to reconstruct the high-quality
geometry and complex, spatially-varying BRDF of an arbitrary object from a
sparse set of only six images captured by wide-baseline cameras under
collocated point lighting. We first estimate per-view depth maps using a deep
multi-view stereo network; these depth maps are used to coarsely align the
different views. We propose a novel multi-view reflectance estimation network
architecture that is trained to pool features from these coarsely aligned
images and predict per-view spatially-varying diffuse albedo, surface normals,
specular roughness and specular albedo. We do this by jointly optimizing the
latent space of our multi-view reflectance network to minimize the photometric
error between images rendered with our predictions and the input images. While
previous state-of-the-art methods fail on such sparse acquisition setups, we
demonstrate, via extensive experiments on synthetic and real data, that our
method produces high-quality reconstructions that can be used to render
photorealistic images.

negative:
Reconstruction of geometric structures from images using supervised learning
suffers from limited available amount of accurate data. One type of such data
is accurate real-world RGB-D images. A major challenge in acquiring such ground
truth data is the accurate alignment between RGB images and the point cloud
measured by a depth scanner. To overcome this difficulty, we consider a
differential optimization method that aligns a colored point cloud with a given
color image through iterative geometric and color matching. In the proposed
framework, the optimization minimizes the photometric difference between the
colors of the point cloud and the corresponding colors of the image pixels.
Unlike other methods that try to reduce this photometric error, we analyze the
computation of the gradient on the image plane and propose a different direct
scheme. We assume that the colors produced by the geometric scanner camera and
the color camera sensor are different and therefore characterized by different
chromatic acquisition properties. Under these multimodal conditions, we find
the transformation between the camera image and the point cloud colors. We
alternately optimize for aligning the position of the point cloud and matching
the different color spaces. The alignments produced by the proposed method are
demonstrated on both synthetic data with quantitative evaluation and real
scenes with qualitative results.

negative:
Recently, 3D face reconstruction from a single image has achieved great
success with the help of deep learning and shape prior knowledge, but they
often fail to produce accurate geometry details. On the other hand, photometric
stereo methods can recover reliable geometry details, but require dense inputs
and need to solve a complex optimization problem. In this paper, we present a
lightweight strategy that only requires sparse inputs or even a single image to
recover high-fidelity face shapes with images captured under near-field lights.
To this end, we construct a dataset containing 84 different subjects with 29
expressions under 3 different lights. Data augmentation is applied to enrich
the data in terms of diversity in identity, lighting, expression, etc. With
this constructed dataset, we propose a novel neural network specially designed
for photometric stereo based 3D face reconstruction. Extensive experiments and
comparisons demonstrate that our method can generate high-quality
reconstruction results with one to three facial images captured under
near-field lights. Our full framework is available at
https://github.com/Juyong/FacePSNet.

negative:
This paper proposes an artificial neural network to determine orientation
using polarized skylight. This neural network has specific dilated convolution,
which can extract light intensity information of different polarization
directions. Then, the degree of polarization (DOP) and angle of polarization
(AOP) are directly extracted in the network. In addition, the exponential
function encoding of orientation is designed as the network output, which can
better reflect the insect's encoding of polarization information, and improve
the accuracy of orientation determination. Finally, training and testing were
conducted on a public polarized skylight navigation dataset, and the
experimental results proved the stability and effectiveness of the network.

negative:
We present a method for estimating neural scenes representations of objects
given only a single image. The core of our method is the estimation of a
geometric scaffold for the object and its use as a guide for the reconstruction
of the underlying radiance field. Our formulation is based on a generative
process that first maps a latent code to a voxelized shape, and then renders it
to an image, with the object appearance being controlled by a second latent
code. During inference, we optimize both the latent codes and the networks to
fit a test image of a new object. The explicit disentanglement of shape and
appearance allows our model to be fine-tuned given a single image. We can then
render new views in a geometrically consistent manner and they represent
faithfully the input object. Additionally, our method is able to generalize to
images outside of the training domain (more realistic renderings and even real
photographs). Finally, the inferred geometric scaffold is itself an accurate
estimate of the object's 3D shape. We demonstrate in several experiments the
effectiveness of our approach in both synthetic and real images.

negative:
We propose a method to refine geometry of 3D meshes from a consumer level
depth camera, e.g. Kinect, by exploiting shading cues captured from an infrared
(IR) camera. A major benefit to using an IR camera instead of an RGB camera is
that the IR images captured are narrow band images that filter out most
undesired ambient light, which makes our system robust against natural indoor
illumination. Moreover, for many natural objects with colorful textures in the
visible spectrum, the subjects appear to have a uniform albedo in the IR
spectrum. Based on our analyses on the IR projector light of the Kinect, we
define a near light source IR shading model that describes the captured
intensity as a function of surface normals, albedo, lighting direction, and
distance between light source and surface points. To resolve the ambiguity in
our model between the normals and distances, we utilize an initial 3D mesh from
the Kinect fusion and multi-view information to reliably estimate surface
details that were not captured and reconstructed by the Kinect fusion. Our
approach directly operates on the mesh model for geometry refinement. We ran
experiments on our algorithm for geometries captured by both the Kinect I and
Kinect II, as the depth acquisition in Kinect I is based on a structured-light
technique and that of the Kinect II is based on a time-of-flight (ToF)
technology. The effectiveness of our approach is demonstrated through several
challenging real-world examples. We have also performed a user study to
evaluate the quality of the mesh models before and after our refinements.

negative:
While the problem of estimating shapes and diffuse reflectances of human
faces from images has been extensively studied, there is relatively less work
done on recovering the specular albedo. This paper presents a lightweight
solution for inferring photorealistic facial reflectance and geometry. Our
system processes video streams from two views of a subject, and outputs two
reflectance maps for diffuse and specular albedos, as well as a vector map of
surface normals. A model-based optimization approach is used, consisting of the
three stages of multi-view face model fitting, facial reflectance inference and
facial geometry refinement. Our approach is based on a novel formulation built
upon the 3D morphable model (3DMM) for representing 3D textured faces in
conjunction with the Blinn-Phong reflection model. It has the advantage of
requiring only a simple setup with two video streams, and is able to exploit
the interaction between the diffuse and specular reflections across multiple
views as well as time frames. As a result, the method is able to reliably
recover high-fidelity facial reflectance and geometry, which facilitates
various applications such as generating photorealistic facial images under new
viewpoints or illumination conditions.

negative:
Event cameras have a lot of advantages over traditional cameras, such as low
latency, high temporal resolution, and high dynamic range. However, since the
outputs of event cameras are the sequences of asynchronous events overtime
rather than actual intensity images, existing algorithms could not be directly
applied. Therefore, it is demanding to generate intensity images from events
for other tasks. In this paper, we unlock the potential of event camera-based
conditional generative adversarial networks to create images/videos from an
adjustable portion of the event data stream. The stacks of space-time
coordinates of events are used as inputs and the network is trained to
reproduce images based on the spatio-temporal intensity changes. The usefulness
of event cameras to generate high dynamic range(HDR) images even in extreme
illumination conditions and also non blurred images under rapid motion is also
shown.In addition, the possibility of generating very high frame rate videos is
demonstrated, theoretically up to 1 million frames per second (FPS) since the
temporal resolution of event cameras are about 1{\mu}s. Proposed methods are
evaluated by comparing the results with the intensity images captured on the
same pixel grid-line of events using online available real datasets and
synthetic datasets produced by the event camera simulator.

negative:
Direct methods have shown promise on visual odometry and SLAM, leading to
greater accuracy and robustness over feature-based methods. However, offline
3-d reconstruction from internet images has not yet benefited from a joint,
photometric optimization over dense geometry and camera parameters. Issues such
as the lack of brightness constancy, and the sheer volume of data, make this a
more challenging task. This work presents a framework for jointly optimizing
millions of scene points and hundreds of camera poses and intrinsics, using a
photometric cost that is invariant to local lighting changes. The improvement
in metric reconstruction accuracy that it confers over feature-based bundle
adjustment is demonstrated on the large-scale Tanks & Temples benchmark. We
further demonstrate qualitative reconstruction improvements on an internet
photo collection, with challenging diversity in lighting and camera intrinsics.

negative:
Our previous work has demonstrated that Rayleigh model, which is widely used
in polarized skylight navigation to describe skylight polarization patterns,
does not contain three-dimensional (3D) attitude information [1]. However, it
is still necessary to further explore whether the skylight polarization
patterns contain 3D attitude information. So, in this paper, a social spider
optimization (SSO) method is proposed to estimate three Euler angles, which
considers the difference of each pixel among polarization images based on
template matching (TM) to make full use of the captured polarization
information. In addition, to explore this problem, we not only use angle of
polarization (AOP) and degree of polarization (DOP) information, but also the
light intensity (LI) information. So, a sky model is established, which
combines Berry model and Hosek model to fully describe AOP, DOP, and LI
information in the sky, and considers the influence of four neutral points,
ground albedo, atmospheric turbidity, and wavelength. The results of simulation
show that the SSO algorithm can estimate 3D attitude and the established sky
model contains 3D attitude information. However, when there are measurement
noise or model error, the accuracy of 3D attitude estimation drops
significantly. Especially in field experiment, it is very difficult to estimate
3D attitude. Finally, the results are discussed in detail.

negative:
Polarized light microscopy provides high contrast to birefringent specimen
and is widely used as a diagnostic tool in pathology. However, polarization
microscopy systems typically operate by analyzing images collected from two or
more light paths in different states of polarization, which lead to relatively
complex optical designs, high system costs or experienced technicians being
required. Here, we present a deep learning-based holographic polarization
microscope that is capable of obtaining quantitative birefringence retardance
and orientation information of specimen from a phase recovered hologram, while
only requiring the addition of one polarizer/analyzer pair to an existing
holographic imaging system. Using a deep neural network, the reconstructed
holographic images from a single state of polarization can be transformed into
images equivalent to those captured using a single-shot computational polarized
light microscope (SCPLM). Our analysis shows that a trained deep neural network
can extract the birefringence information using both the sample specific
morphological features as well as the holographic amplitude and phase
distribution. To demonstrate the efficacy of this method, we tested it by
imaging various birefringent samples including e.g., monosodium urate (MSU) and
triamcinolone acetonide (TCA) crystals. Our method achieves similar results to
SCPLM both qualitatively and quantitatively, and due to its simpler optical
design and significantly larger field-of-view, this method has the potential to
expand the access to polarization microscopy and its use for medical diagnosis
in resource limited settings.

negative:
A division-of-focal-plane or microgrid image polarimeter enables us to
acquire a set of polarization images in one shot. Since the polarimeter
consists of an image sensor equipped with a monochrome or color polarization
filter array (MPFA or CPFA), the demosaicking process to interpolate missing
pixel values plays a crucial role in obtaining high-quality polarization
images. In this paper, we propose a novel MPFA demosaicking method based on
edge-aware residual interpolation (EARI) and also extend it to CPFA
demosaicking. The key of EARI is a new edge detector for generating an
effective guide image used to interpolate the missing pixel values. We also
present a newly constructed full color-polarization image dataset captured
using a 3-CCD camera and a rotating polarizer. Using the dataset, we
experimentally demonstrate that our EARI-based method outperforms existing
methods in MPFA and CPFA demosaicking.

negative:
In 3D reconstruction, the recovery of the calibration parameters of the
cameras is paramount since it provides metric information about the observed
scene, e.g., measures of angles and ratios of distances. Autocalibration
enables the estimation of the camera parameters without using a calibration
device, but by enforcing simple constraints on the camera parameters. In the
absence of information about the internal camera parameters such as the focal
length and the principal point, the knowledge of the camera pixel shape is
usually the only available constraint. Given a projective reconstruction of a
rigid scene, we address the problem of the autocalibration of a minimal set of
cameras with known pixel shape and otherwise arbitrarily varying intrinsic and
extrinsic parameters. We propose an algorithm that only requires 5 cameras (the
theoretical minimum), thus halving the number of cameras required by previous
algorithms based on the same constraint. To this purpose, we introduce as our
basic geometric tool the six-line conic variety (SLCV), consisting in the set
of planes intersecting six given lines of 3D space in points of a conic. We
show that the set of solutions of the Euclidean upgrading problem for three
cameras with known pixel shape can be parameterized in a computationally
efficient way. This parameterization is then used to solve autocalibration from
five or more cameras, reducing the three-dimensional search space to a
two-dimensional one. We provide experiments with real images showing the good
performance of the technique.

negative:
The reconstruction of dense 3D models of face geometry and appearance from a
single image is highly challenging and ill-posed. To constrain the problem,
many approaches rely on strong priors, such as parametric face models learned
from limited 3D scan data. However, prior models restrict generalization of the
true diversity in facial geometry, skin reflectance and illumination. To
alleviate this problem, we present the first approach that jointly learns 1) a
regressor for face shape, expression, reflectance and illumination on the basis
of 2) a concurrently learned parametric face model. Our multi-level face model
combines the advantage of 3D Morphable Models for regularization with the
out-of-space generalization of a learned corrective space. We train end-to-end
on in-the-wild images without dense annotations by fusing a convolutional
encoder with a differentiable expert-designed renderer and a self-supervised
training loss, both defined at multiple detail levels. Our approach compares
favorably to the state-of-the-art in terms of reconstruction quality, better
generalizes to real world faces, and runs at over 250 Hz.

negative:
We present a new color photometric stereo (CPS) method that recovers high
quality, detailed 3D face geometry in a single shot. Our system uses three
uncalibrated near point lights of different colors and a single camera. For
robust self-calibration of the light sources, we use 3D morphable model (3DMM)
and semantic segmentation of facial parts. We address the spectral ambiguity
problem by incorporating albedo consensus, albedo similarity, and proxy prior
into a unified framework. We avoid the need for spatial constancy of albedo;
instead, we use a new measure for albedo similarity that is based on the albedo
norm profile. Experiments show that our new approach produces state-of-the-art
results from single image with high-fidelity geometry that includes details
such as wrinkles.

negative:
Removing perspective distortion from hand held camera captured document
images is one of the primitive tasks in document analysis, but unfortunately,
no such method exists that can reliably remove the perspective distortion from
document images automatically. In this paper, we propose a convolutional neural
network based method for recovering homography from hand-held camera captured
documents.
  Our proposed method works independent of document's underlying content and is
trained end-to-end in a fully automatic way. Specifically, this paper makes
following three contributions: Firstly, we introduce a large scale synthetic
dataset for recovering homography from documents images captured under
different geometric and photometric transformations; secondly, we show that a
generic convolutional neural network based architecture can be successfully
used for regressing the corners positions of documents captured under wild
settings; thirdly, we show that L1 loss can be reliably used for corners
regression. Our proposed method gives state-of-the-art performance on the
tested datasets, and has potential to become an integral part of document
analysis pipeline.

negative:
The reflections caused by common semi-reflectors, such as glass windows, can
impact the performance of computer vision algorithms. State-of-the-art methods
can remove reflections on synthetic data and in controlled scenarios. However,
they are based on strong assumptions and do not generalize well to real-world
images. Contrary to a common misconception, real-world images are challenging
even when polarization information is used. We present a deep learning approach
to separate the reflected and the transmitted components of the recorded
irradiance, which explicitly uses the polarization properties of light. To
train it, we introduce an accurate synthetic data generation pipeline, which
simulates realistic reflections, including those generated by curved and
non-ideal surfaces, non-static scenes, and high-dynamic-range scenes.

negative:
Scientific imaging techniques such as optical and electron microscopy and
computed tomography (CT) scanning are used to study the 3D structure of an
object through 2D observations. These observations are related to the original
3D object through orthogonal integral projections. For common 3D reconstruction
algorithms, computational efficiency requires the modeling of the 3D structures
to take place in Fourier space by applying the Fourier slice theorem. At
present, it is unclear how to differentiate through the projection operator,
and hence current learning algorithms can not rely on gradient based methods to
optimize 3D structure models. In this paper we show how back-propagation
through the projection operator in Fourier space can be achieved. We
demonstrate the validity of the approach with experiments on 3D reconstruction
of proteins. We further extend our approach to learning probabilistic models of
3D objects. This allows us to predict regions of low sampling rates or estimate
noise. A higher sample efficiency can be reached by utilizing the learned
uncertainties of the 3D structure as an unsupervised estimate of the model fit.
Finally, we demonstrate how the reconstruction algorithm can be extended with
an amortized inference scheme on unknown attributes such as object pose.
Through empirical studies we show that joint inference of the 3D structure and
the object pose becomes more difficult when the ground truth object contains
more symmetries. Due to the presence of for instance (approximate) rotational
symmetries, the pose estimation can easily get stuck in local optima,
inhibiting a fine-grained high-quality estimate of the 3D structure.

negative:
Visual perception is regularly used by humans and robots for navigation. By
either implicitly or explicitly mapping the environment, ego-motion can be
determined and a path of actions can be planned. The process of mapping and
navigation are delicately intertwined; therefore, improving one can often lead
to an improvement of the other. Both processes are sensitive to the interior
orientation parameters of the camera system and mathematically modelling these
systematic errors can often improve the precision and accuracy of the overall
solution. This paper presents an automatic camera calibration method suitable
for any lens, without having prior knowledge about the sensor. Statistical
inference is performed to map the environment and localize the camera
simultaneously. K-nearest neighbour regression is used to model the geometric
distortions of the images. A normal-angle lens Nikon camera and wide-angle lens
GoPro camera were calibrated using the proposed method, as well as the
conventional bundle adjustment with self-calibration method (for comparison).
Results showed that the mapping error was reduced from an average of 14.9 mm to
1.2 mm (i.e. a 92% improvement) and 66.6 mm to 1.5 mm (i.e. a 98% improvement)
using the proposed method for the Nikon and GoPro cameras, respectively. In
contrast, the conventional approach achieved an average 3D error of 0.9 mm
(i.e. 94% improvement) and 3.3 mm (i.e. 95% improvement) for the Nikon and
GoPro cameras, respectively. Thus, the proposed method performs well
irrespective of the lens/sensor used: it yields results that are comparable to
the conventional approach for normal-angle lens cameras, and it has the
additional benefit of improving calibration results for wide-angle lens
cameras.

negative:
This paper shows that accurate underwater 3D shape reconstruction is possible
using a single camera, observing a target through a refractive interface. We
provide unified reconstruction techniques for a variety of scenarios such as
single static camera and moving refractive interface, single moving camera and
static refractive interface, and single moving camera and moving refractive
interface. In our basic setup, we assume that the refractive interface is
planar, and simultaneously estimate the unknown transformations of the planar
interface and the camera, and the unknown target shape using bundle adjustment.
We also extend it to relax the planarity assumption, which enables us to use
waves of the refractive interface for the reconstruction task. Experiments with
real data show the superiority of our method to existing methods.

negative:
Recently, a lot of attention has been focused on the incorporation of 3D data
into face analysis and its applications. Despite providing a more accurate
representation of the face, 3D facial images are more complex to acquire than
2D pictures. As a consequence, great effort has been invested in developing
systems that reconstruct 3D faces from an uncalibrated 2D image. However, the
3D-from-2D face reconstruction problem is ill-posed, thus prior knowledge is
needed to restrict the solutions space. In this work, we review 3D face
reconstruction methods proposed in the last decade, focusing on those that only
use 2D pictures captured under uncontrolled conditions. We present a
classification of the proposed methods based on the technique used to add prior
knowledge, considering three main strategies, namely, statistical model
fitting, photometry, and deep learning, and reviewing each of them separately.
In addition, given the relevance of statistical 3D facial models as prior
knowledge, we explain the construction procedure and provide a list of the most
popular publicly available 3D facial models. After the exhaustive study of
3D-from-2D face reconstruction approaches, we observe that the deep learning
strategy is rapidly growing since the last few years, becoming the standard
choice in replacement of the widespread statistical model fitting. Unlike the
other two strategies, photometry-based methods have decreased in number due to
the need for strong underlying assumptions that limit the quality of their
reconstructions compared to statistical model fitting and deep learning
methods. The review also identifies current challenges and suggests avenues for
future research.

negative:
Three-dimensional reconstruction of objects from shading information is a
challenging task in computer vision. As most of the approaches facing the
Photometric Stereo problem use simplified far-field assumptions, real-world
scenarios have essentially more complex physical effects that need to be
handled for accurately reconstructing the 3D shape. An increasing number of
methods have been proposed to address the problem when point light sources are
assumed to be nearby the target object. The proximity of the light sources
complicates the modeling of the image formation as the light behaviour requires
non-linear parameterisation to describe its propagation and attenuation.
  To understand the capability of the approaches dealing with this near-field
scenario, the literature till now has used synthetically rendered photometric
images or minimal and very customised real-world data. In order to fill the gap
in evaluating near-field photometric stereo methods, we introduce LUCES the
first real-world 'dataset for near-fieLd point light soUrCe photomEtric Stereo'
of 14 objects of a varying of materials. A device counting 52 LEDs has been
designed to lit each object positioned 10 to 30 centimeters away from the
camera. Together with the raw images, in order to evaluate the 3D
reconstructions, the dataset includes both normal and depth maps for comparing
different features of the retrieved 3D geometry. Furthermore, we evaluate the
performance of the latest near-field Photometric Stereo algorithms on the
proposed dataset to assess the SOTA method with respect to actual close range
effects and object materials.

negative:
The ubiquity of smartphone cameras has led to more and more documents being
captured by cameras rather than scanned. Unlike flatbed scanners, photographed
documents are often folded and crumpled, resulting in large local variance in
text structure. The problem of document rectification is fundamental to the
Optical Character Recognition (OCR) process on documents, and its ability to
overcome geometric distortions significantly affects recognition accuracy.
Despite the great progress in recent OCR systems, most still rely on a
pre-process that ensures the text lines are straight and axis aligned. Recent
works have tackled the problem of rectifying document images taken in-the-wild
using various supervision signals and alignment means. However, they focused on
global features that can be extracted from the document's boundaries, ignoring
various signals that could be obtained from the document's content.
  We present CREASE: Content Aware Rectification using Angle Supervision, the
first learned method for document rectification that relies on the document's
content, the location of the words and specifically their orientation, as hints
to assist in the rectification process. We utilize a novel pixel-wise angle
regression approach and a curvature estimation side-task for optimizing our
rectification model. Our method surpasses previous approaches in terms of OCR
accuracy, geometric error and visual similarity.

negative:
Vehicles, search and rescue personnel, and endoscopes use flash lights to
locate, identify, and view objects in their surroundings. Here we show the
first steps of how all these tasks can be done around corners with consumer
cameras. Recent techniques for NLOS imaging using consumer cameras have not
been able to both localize and identify the hidden object. We introduce a
method that couples traditional geometric understanding and data-driven
techniques. To avoid the limitation of large dataset gathering, we train the
data-driven models on rendered samples to computationally recover the hidden
scene on real data. The method has three independent operating modes: 1) a
regression output to localize a hidden object in 2D, 2) an identification
output to identify the object type or pose, and 3) a generative network to
reconstruct the hidden scene from a new viewpoint. The method is able to
localize 12cm wide hidden objects in 2D with 1.7cm accuracy. The method also
identifies the hidden object class with 87.7% accuracy (compared to 33.3%
random accuracy). This paper also provides an analysis on the distribution of
information that encodes the occluded object in the accessible scene. We show
that, unlike previously thought, the area that extends beyond the corner is
essential for accurate object localization and identification.

negative:
Nowadays, as cameras are rapidly adopted in our daily routine, images of
documents are becoming both abundant and prevalent. Unlike natural images that
capture physical objects, document-images contain a significant amount of text
with critical semantics and complicated layouts. In this work, we devise a
generic unsupervised technique to learn multimodal affinities between textual
entities in a document-image, considering their visual style, the content of
their underlying text and their geometric context within the image. We then use
these learned affinities to automatically cluster the textual entities in the
image into different semantic groups. The core of our approach is a deep
optimization scheme dedicated for an image provided by the user that detects
and leverages reliable pairwise connections in the multimodal representation of
the textual elements in order to properly learn the affinities. We show that
our technique can operate on highly varying images spanning a wide range of
documents and demonstrate its applicability for various editing operations
manipulating the content, appearance and geometry of the image.

negative:
Airborne LiDAR (Light Detection and Ranging) data is widely applied in
building reconstruction, with studies reporting success in typical buildings.
However, the reconstruction of curved buildings remains an open research
problem. To this end, we propose a new framework for curved building
reconstruction via assembling and deforming geometric primitives. The input
LiDAR point cloud are first converted into contours where individual buildings
are identified. After recognizing geometric units (primitives) from building
contours, we get initial models by matching basic geometric primitives to these
primitives. To polish assembly models, we employ a warping field for model
refinements. Specifically, an embedded deformation (ED) graph is constructed
via downsampling the initial model. Then, the point-to-model displacements are
minimized by adjusting node parameters in the ED graph based on our objective
function. The presented framework is validated on several highly curved
buildings collected by various LiDAR in different cities. The experimental
results, as well as accuracy comparison, demonstrate the advantage and
effectiveness of our method. {The new insight attributes to an efficient
reconstruction manner.} Moreover, we prove that the primitive-based framework
significantly reduces the data storage to 10-20 percent of classical mesh
models.

negative:
We present a novel formulation to removing reflection from polarized images
in the wild. We first identify the misalignment issues of existing reflection
removal datasets where the collected reflection-free images are not perfectly
aligned with input mixed images due to glass refraction. Then we build a new
dataset with more than 100 types of glass in which obtained transmission images
are perfectly aligned with input mixed images. Second, capitalizing on the
special relationship between reflection and polarized light, we propose a
polarized reflection removal model with a two-stage architecture. In addition,
we design a novel perceptual NCC loss that can improve the performance of
reflection removal and general image decomposition tasks. We conduct extensive
experiments, and results suggest that our model outperforms state-of-the-art
methods on reflection removal.

negative:
Event cameras, i.e., the Dynamic and Active-pixel Vision Sensor (DAVIS) ones,
capture the intensity changes in the scene and generates a stream of events in
an asynchronous fashion. The output rate of such cameras can reach up to 10
million events per second in high dynamic environments. DAVIS cameras use novel
vision sensors that mimic human eyes. Their attractive attributes, such as high
output rate, High Dynamic Range (HDR), and high pixel bandwidth, make them an
ideal solution for applications that require high-frequency tracking. Moreover,
applications that operate in challenging lighting scenarios can exploit the
high HDR of event cameras, i.e., 140 dB compared to 60 dB of traditional
cameras. In this paper, a novel asynchronous corner tracking method is proposed
that uses both events and intensity images captured by a DAVIS camera. The
Harris algorithm is used to extract features, i.e., frame-corners from
keyframes, i.e., intensity images. Afterward, a matching algorithm is used to
extract event-corners from the stream of events. Events are solely used to
perform asynchronous tracking until the next keyframe is captured. Neighboring
events, within a window size of 5x5 pixels around the event-corner, are used to
calculate the velocity and direction of extracted event-corners by fitting the
2D planar using a randomized Hough transform algorithm. Experimental evaluation
showed that our approach is able to update the location of the extracted
corners up to 100 times during the blind time of traditional cameras, i.e.,
between two consecutive intensity images.

negative:
Understanding 3D scenes from a single image is fundamental to a wide variety
of tasks, such as for robotics, motion planning, or augmented reality. Existing
works in 3D perception from a single RGB image tend to focus on geometric
reconstruction only, or geometric reconstruction with semantic segmentation or
instance segmentation. Inspired by 2D panoptic segmentation, we propose to
unify the tasks of geometric reconstruction, 3D semantic segmentation, and 3D
instance segmentation into the task of panoptic 3D scene reconstruction - from
a single RGB image, predicting the complete geometric reconstruction of the
scene in the camera frustum of the image, along with semantic and instance
segmentations. We thus propose a new approach for holistic 3D scene
understanding from a single RGB image which learns to lift and propagate 2D
features from an input image to a 3D volumetric scene representation. We
demonstrate that this holistic view of joint scene reconstruction, semantic,
and instance segmentation is beneficial over treating the tasks independently,
thus outperforming alternative approaches.

negative:
This paper primarily focuses on figuring out the best array of cameras, or
visual sensors, so that such a placement enables the maximum utilization of
these visual sensors. Maximizing the utilization of these cameras can convert
to another problem that is simpler for the formulation, that is, maximizing the
total coverage with these cameras. To solve the problem, the coverage problem
is first defined subject to the capabilities and limits of cameras. Then, poses
of cameras are analyzed for the best arrangement.

negative:
We propose a simple and efficient method for exploiting synthetic images when
training a Deep Network to predict a 3D pose from an image. The ability of
using synthetic images for training a Deep Network is extremely valuable as it
is easy to create a virtually infinite training set made of such images, while
capturing and annotating real images can be very cumbersome. However, synthetic
images do not resemble real images exactly, and using them for training can
result in suboptimal performance. It was recently shown that for exemplar-based
approaches, it is possible to learn a mapping from the exemplar representations
of real images to the exemplar representations of synthetic images. In this
paper, we show that this approach is more general, and that a network can also
be applied after the mapping to infer a 3D pose: At run time, given a real
image of the target object, we first compute the features for the image, map
them to the feature space of synthetic images, and finally use the resulting
features as input to another network which predicts the 3D pose. Since this
network can be trained very effectively by using synthetic images, it performs
very well in practice, and inference is faster and more accurate than with an
exemplar-based approach. We demonstrate our approach on the LINEMOD dataset for
3D object pose estimation from color images, and the NYU dataset for 3D hand
pose estimation from depth maps. We show that it allows us to outperform the
state-of-the-art on both datasets.

negative:
We present an unsupervised learning framework for decomposing images into
layers of automatically discovered object models. Contrary to recent approaches
that model image layers with autoencoder networks, we represent them as
explicit transformations of a small set of prototypical images. Our model has
three main components: (i) a set of object prototypes in the form of learnable
images with a transparency channel, which we refer to as sprites; (ii)
differentiable parametric functions predicting occlusions and transformation
parameters necessary to instantiate the sprites in a given image; (iii) a
layered image formation model with occlusion for compositing these instances
into complete images including background. By jointly learning the sprites and
occlusion/transformation predictors to reconstruct images, our approach not
only yields accurate layered image decompositions, but also identifies object
categories and instance parameters. We first validate our approach by providing
results on par with the state of the art on standard multi-object synthetic
benchmarks (Tetrominoes, Multi-dSprites, CLEVR6). We then demonstrate the
applicability of our model to real images in tasks that include clustering
(SVHN, GTSRB), cosegmentation (Weizmann Horse) and object discovery from
unfiltered social network images. To the best of our knowledge, our approach is
the first layered image decomposition algorithm that learns an explicit and
shared concept of object type, and is robust enough to be applied to real
images.

negative:
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world.

negative:
Moving object detection is important in computer vision. Event-based cameras
are bio-inspired cameras that work by mimicking the working of the human eye.
These cameras have multiple advantages over conventional frame-based cameras,
like reduced latency, HDR, reduced motion blur during high motion, low power
consumption, etc. In spite of these advantages, event-based cameras are
noise-sensitive and have low resolution. Moreover, the task of moving object
detection in these cameras is difficult, as event-based sensors lack useful
visual features like texture and color. In this paper, we investigate the
application of the k-means clustering technique in detecting moving objects in
event-based data.

negative:
This work presents a flexible system to reconstruct 3D models of objects
captured with an RGB-D sensor. A major advantage of the method is that our
reconstruction pipeline allows the user to acquire a full 3D model of the
object. This is achieved by acquiring several partial 3D models in different
sessions that are automatically merged together to reconstruct a full model. In
addition, the 3D models acquired by our system can be directly used by
state-of-the-art object instance recognition and object tracking modules,
providing object-perception capabilities for different applications, such as
human-object interaction analysis or robot grasping. The system does not impose
constraints in the appearance of objects (textured, untextured) nor in the
modelling setup (moving camera with static object or a turn-table setup). The
proposed reconstruction system has been used to model a large number of objects
resulting in metrically accurate and visually appealing 3D models.

negative:
While a great variety of 3D cameras have been introduced in recent years,
most publicly available datasets for object recognition and pose estimation
focus on one single camera. In this work, we present a dataset of 32 scenes
that have been captured by 7 different 3D cameras, totaling 49,294 frames. This
allows evaluating the sensitivity of pose estimation algorithms to the
specifics of the used camera and the development of more robust algorithms that
are more independent of the camera model. Vice versa, our dataset enables
researchers to perform a quantitative comparison of the data from several
different cameras and depth sensing technologies and evaluate their algorithms
before selecting a camera for their specific task. The scenes in our dataset
contain 20 different objects from the common benchmark YCB object and model set
[1], [2]. We provide full ground truth 6DoF poses for each object, per-pixel
segmentation, 2D and 3D bounding boxes and a measure of the amount of occlusion
of each object. We have also performed an initial evaluation of the cameras
using our dataset on a state-of-the-art object recognition and pose estimation
system [3].

negative:
Event cameras or neuromorphic cameras mimic the human perception system as
they measure the per-pixel intensity change rather than the actual intensity
level. In contrast to traditional cameras, such cameras capture new information
about the scene at MHz frequency in the form of sparse events. The high
temporal resolution comes at the cost of losing the familiar per-pixel
intensity information. In this work we propose a variational model that
accurately models the behaviour of event cameras, enabling reconstruction of
intensity images with arbitrary frame rate in real-time. Our method is
formulated on a per-event-basis, where we explicitly incorporate information
about the asynchronous nature of events via an event manifold induced by the
relative timestamps of events. In our experiments we verify that solving the
variational model on the manifold produces high-quality images without
explicitly estimating optical flow.

negative:
In this paper, we propose a novel technique to reconstruct 3D surface of an
underwater object using stereo images. Reconstructing the 3D surface of an
underwater object is really a challenging task due to degraded quality of
underwater images. There are various reason of quality degradation of
underwater images i.e., non-uniform illumination of light on the surface of
objects, scattering and absorption effects. Floating particles present in
underwater produces Gaussian noise on the captured underwater images which
degrades the quality of images. The degraded underwater images are preprocessed
by applying homomorphic, wavelet denoising and anisotropic filtering
sequentially. The uncalibrated rectification technique is applied to
preprocessed images to rectify the left and right images. The rectified left
and right image lies on a common plane. To find the correspondence points in a
left and right images, we have applied dense stereo matching technique i.e.,
graph cut method. Finally, we estimate the depth of images using triangulation
technique. The experimental result shows that the proposed method reconstruct
3D surface of underwater objects accurately using captured underwater stereo
images.

negative:
Non-Rigid Structure from Motion (NRSfM) refers to the problem of
reconstructing cameras and the 3D point cloud of a non-rigid object from an
ensemble of images with 2D correspondences. Current NRSfM algorithms are
limited from two perspectives: (i) the number of images, and (ii) the type of
shape variability they can handle. These difficulties stem from the inherent
conflict between the condition of the system and the degrees of freedom needing
to be modeled -- which has hampered its practical utility for many applications
within vision. In this paper we propose a novel hierarchical sparse coding
model for NRSFM which can overcome (i) and (ii) to such an extent, that NRSFM
can be applied to problems in vision previously thought too ill posed. Our
approach is realized in practice as the training of an unsupervised deep neural
network (DNN) auto-encoder with a unique architecture that is able to
disentangle pose from 3D structure. Using modern deep learning computational
platforms allows us to solve NRSfM problems at an unprecedented scale and shape
complexity. Our approach has no 3D supervision, relying solely on 2D point
correspondences. Further, our approach is also able to handle missing/occluded
2D points without the need for matrix completion. Extensive experiments
demonstrate the impressive performance of our approach where we exhibit
superior precision and robustness against all available state-of-the-art works
in some instances by an order of magnitude. We further propose a new quality
measure (based on the network weights) which circumvents the need for 3D
ground-truth to ascertain the confidence we have in the reconstructability. We
believe our work to be a significant advance over state-of-the-art in NRSFM.

negative:
With the availability of more powerful computers, iterative reconstruction
algorithms are the subject of an ongoing work in the design of more efficient
reconstruction algorithms for X-ray computed tomography. In this work, we show
how two analytical reconstruction algorithms can be improved by correcting the
corresponding reconstructions using a randomized iterative reconstruction
algorithm. The combined analytical reconstruction followed by randomized
iterative reconstruction can also be viewed as a reconstruction algorithm
which, in the experiments we have conducted, uses up to $35\%$ less projection
angles as compared to the analytical reconstruction algorithms and produces the
same results in terms of quality of reconstruction, without increasing the
execution time significantly.

negative:
Monocular 3D facial shape reconstruction from a single 2D facial image has
been an active research area due to its wide applications. Inspired by the
success of deep neural networks (DNN), we propose a DNN-based approach for
End-to-End 3D FAce Reconstruction (UH-E2FAR) from a single 2D image. Different
from recent works that reconstruct and refine the 3D face in an iterative
manner using both an RGB image and an initial 3D facial shape rendering, our
DNN model is end-to-end, and thus the complicated 3D rendering process can be
avoided. Moreover, we integrate in the DNN architecture two components, namely
a multi-task loss function and a fusion convolutional neural network (CNN) to
improve facial expression reconstruction. With the multi-task loss function, 3D
face reconstruction is divided into neutral 3D facial shape reconstruction and
expressive 3D facial shape reconstruction. The neutral 3D facial shape is
class-specific. Therefore, higher layer features are useful. In comparison, the
expressive 3D facial shape favors lower or intermediate layer features. With
the fusion-CNN, features from different intermediate layers are fused and
transformed for predicting the 3D expressive facial shape. Through extensive
experiments, we demonstrate the superiority of our end-to-end framework in
improving the accuracy of 3D face reconstruction.

negative:
We present a practical approach to capturing ear-to-ear face models
comprising both 3D meshes and intrinsic textures (i.e. diffuse and specular
albedo). Our approach is a hybrid of geometric and photometric methods and
requires no geometric calibration. Photometric measurements made in a
lightstage are used to estimate view dependent high resolution normal maps. We
overcome the problem of having a single photometric viewpoint by capturing in
multiple poses. We use uncalibrated multiview stereo to estimate a coarse base
mesh to which the photometric views are registered. We propose a novel approach
to robustly stitching surface normal and intrinsic texture data into a
seamless, complete and highly detailed face model. The resulting relightable
models provide photorealistic renderings in any view.

negative:
A basic problem in computer vision is to understand the structure of a
real-world scene given several images of it. Here we study several theoretical
aspects of the intra multi-view geometry of calibrated cameras when all that
they can reliably recognize is each other. With the proliferation of wearable
cameras, autonomous vehicles and drones, the geometry of these multiple cameras
is a timely and relevant problem to study.

negative:
We study the problem of object detection over scanned images of scientific
documents. We consider images that contain objects of varying aspect ratios and
sizes and range from coarse elements such as tables and figures to fine
elements such as equations and section headers. We find that current object
detectors fail to produce properly localized region proposals over such page
objects. We revisit the original R-CNN model and present a method for
generating fine-grained proposals over document elements. We also present a
region embedding model that uses the convolutional maps of a proposal's
neighbors as context to produce an embedding for each proposal. This region
embedding is able to capture the semantic relationships between a target region
and its surrounding context. Our end-to-end model produces an embedding for
each proposal, then classifies each proposal by using a multi-head attention
model that attends to the most important neighbors of a proposal. To evaluate
our model, we collect and annotate a dataset of publications from heterogeneous
journals. We show that our model, referred to as Attentive-RCNN, yields a 17%
mAP improvement compared to standard object detection models.

negative:
Event cameras sense intensity changes and have many advantages over
conventional cameras. To take advantage of event cameras, some methods have
been proposed to reconstruct intensity images from event streams. However, the
outputs are still in low resolution (LR), noisy, and unrealistic. The
low-quality outputs stem broader applications of event cameras, where high
spatial resolution (HR) is needed as well as high temporal resolution, dynamic
range, and no motion blur. We consider the problem of reconstructing and
super-resolving intensity images from LR events, when no ground truth (GT) HR
images and down-sampling kernels are available. To tackle the challenges, we
propose a novel end-to-end pipeline that reconstructs LR images from event
streams, enhances the image qualities and upsamples the enhanced images, called
EventSR. For the absence of real GT images, our method is primarily
unsupervised, deploying adversarial learning. To train EventSR, we create an
open dataset including both real-world and simulated scenes. The use of both
datasets boosts up the network performance, and the network architectures and
various loss functions in each phase help improve the image qualities. The
whole pipeline is trained in three phases. While each phase is mainly for one
of the three tasks, the networks in earlier phases are fine-tuned by respective
loss functions in an end-to-end manner. Experimental results show that EventSR
reconstructs high-quality SR images from events for both simulated and
real-world data.

negative:
Division-of-focal-plane (DoFP) polarization imaging technical recently has
been applied in many fields. However, the images captured by such sensors
cannot be used directly because they suffer from instantaneous field-of-view
errors and low resolution problem. This paper builds a fast DoFP demosaicing
system with proposed progressive polarization demosaicing convolutional neural
network (PPDN), which is specifically designed for edge-side GPU devices like
Navidia Jetson TX2. The proposed network consists of two parts: reconstruction
stage and refining stage. The former recovers four polarization channels from a
single DoFP image. The latter fine-tune the four channels to obtain more
accurate polarization information. PPDN can be implemented in another version:
PPDN-L (large), for the platforms of high computing resources. Experiments show
that PPDN can compete with the best existing methods with fewer parameters and
faster inference speed and meet the real-time demands of imaging system.

negative:
An innovative 3-D radar imaging technique is developed for fast and efficient
identification and characterization of radar backscattering components of
complex objects, when the collected scattered field is made of
polarization-diverse measurements. In this context, all the polarimetric
information seems irretrievably mixed. A direct model, derived from a simple
but original extension of the widespread "multiple scattering model" leads to a
high dimensional linear inverse problem. It is solved by a fast dedicated
imaging algorithm that performs to determine at a time three huge 3-D scatterer
maps which correspond to HH, VV and HV polarizations at emission and reception.
It is applied successfully to various mock-ups and data sets collected from an
accurate and dedicated 3D spherical experimental layout that provides
concentric polarization-diverse RCS measurements.

negative:
Despite significant progress on current state-of-the-art image generation
models, synthesis of document images containing multiple and complex object
layouts is a challenging task. This paper presents a novel approach, called
DocSynth, to automatically synthesize document images based on a given layout.
In this work, given a spatial layout (bounding boxes with object categories) as
a reference by the user, our proposed DocSynth model learns to generate a set
of realistic document images consistent with the defined layout. Also, this
framework has been adapted to this work as a superior baseline model for
creating synthetic document image datasets for augmenting real data during
training for document layout analysis tasks. Different sets of learning
objectives have been also used to improve the model performance.
Quantitatively, we also compare the generated results of our model with real
data using standard evaluation metrics. The results highlight that our model
can successfully generate realistic and diverse document images with multiple
objects. We also present a comprehensive qualitative analysis summary of the
different scopes of synthetic image generation tasks. Lastly, to our knowledge
this is the first work of its kind.

negative:
3D face reconstruction is a challenging problem but also an important task in
the field of computer vision and graphics. Recently, many researchers put
attention to the problem and a large number of articles have been published.
Single image reconstruction is one of the branches of 3D face reconstruction,
which has a lot of applications in our life. This paper is a review of the
recent literature on 3D face reconstruction from a single image.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
We introduce a novel learning method for 3D pose estimation from color
images. While acquiring annotations for color images is a difficult task, our
approach circumvents this problem by learning a mapping from paired color and
depth images captured with an RGB-D camera. We jointly learn the pose from
synthetic depth images that are easy to generate, and learn to align these
synthetic depth images with the real depth images. We show our approach for the
task of 3D hand pose estimation and 3D object pose estimation, both from color
images only. Our method achieves performances comparable to state-of-the-art
methods on popular benchmark datasets, without requiring any annotations for
the color images.

negative:
We propose a novel approach to synthesizing images that are effective for
training object detectors. Starting from a small set of real images, our
algorithm estimates the rendering parameters required to synthesize similar
images given a coarse 3D model of the target object. These parameters can then
be reused to generate an unlimited number of training images of the object of
interest in arbitrary 3D poses, which can then be used to increase
classification performances.
  A key insight of our approach is that the synthetically generated images
should be similar to real images, not in terms of image quality, but rather in
terms of features used during the detector training. We show in the context of
drone, plane, and car detection that using such synthetically generated images
yields significantly better performances than simply perturbing real images or
even synthesizing images in such way that they look very realistic, as is often
done when only limited amounts of training data are available.

negative:
Cross view feature fusion is the key to address the occlusion problem in
human pose estimation. The current fusion methods need to train a separate
model for every pair of cameras making them difficult to scale. In this work,
we introduce MetaFuse, a pre-trained fusion model learned from a large number
of cameras in the Panoptic dataset. The model can be efficiently adapted or
finetuned for a new pair of cameras using a small number of labeled images. The
strong adaptation power of MetaFuse is due in large part to the proposed
factorization of the original fusion model into two parts (1) a generic fusion
model shared by all cameras, and (2) lightweight camera-dependent
transformations. Furthermore, the generic model is learned from many cameras by
a meta-learning style algorithm to maximize its adaptation capability to
various camera poses. We observe in experiments that MetaFuse finetuned on the
public datasets outperforms the state-of-the-arts by a large margin which
validates its value in practice.

negative:
The distributed representation of correlated multi-view images is an
important problem that arise in vision sensor networks. This paper concentrates
on the joint reconstruction problem where the distributively compressed
correlated images are jointly decoded in order to improve the reconstruction
quality of all the compressed images. We consider a scenario where the images
captured at different viewpoints are encoded independently using common coding
solutions (e.g., JPEG, H.264 intra) with a balanced rate distribution among
different cameras. A central decoder first estimates the underlying correlation
model from the independently compressed images which will be used for the joint
signal recovery. The joint reconstruction is then cast as a constrained convex
optimization problem that reconstructs total-variation (TV) smooth images that
comply with the estimated correlation model. At the same time, we add
constraints that force the reconstructed images to be consistent with their
compressed versions. We show by experiments that the proposed joint
reconstruction scheme outperforms independent reconstruction in terms of image
quality, for a given target bit rate. In addition, the decoding performance of
our proposed algorithm compares advantageously to state-of-the-art distributed
coding schemes based on disparity learning and on the DISCOVER.

negative:
Imaging polarimetry allows more information to be extracted from a scene than
conventional intensity or colour imaging. However, a major challenge of imaging
polarimetry is image degradation due to noise. This paper investigates the
mitigation of noise through denoising algorithms and compares existing
denoising algorithms with a new method, based on BM3D. This algorithm, PBM3D,
gives visual quality superior to the state of the art across all images and
noise standard deviations tested. We show that denoising polarization images
using PBM3D allows the degree of polarization to be more accurately calculated
by comparing it to spectroscopy methods.

negative:
This paper presents an explore-and-classify framework for structured
architectural reconstruction from an aerial image. Starting from a potentially
imperfect building reconstruction by an existing algorithm, our approach 1)
explores the space of building models by modifying the reconstruction via
heuristic actions; 2) learns to classify the correctness of building models
while generating classification labels based on the ground-truth, and 3)
repeat. At test time, we iterate exploration and classification, seeking for a
result with the best classification score. We evaluate the approach using
initial reconstructions by two baselines and two state-of-the-art
reconstruction algorithms. Qualitative and quantitative evaluations demonstrate
that our approach consistently improves the reconstruction quality from every
initial reconstruction.

negative:
Event cameras are novel sensors that report brightness changes in the form of
a stream of asynchronous "events" instead of intensity frames. They offer
significant advantages with respect to conventional cameras: high temporal
resolution, high dynamic range, and no motion blur. While the stream of events
encodes in principle the complete visual signal, the reconstruction of an
intensity image from a stream of events is an ill-posed problem in practice.
Existing reconstruction approaches are based on hand-crafted priors and strong
assumptions about the imaging process as well as the statistics of natural
images. In this work we propose to learn to reconstruct intensity images from
event streams directly from data instead of relying on any hand-crafted priors.
We propose a novel recurrent network to reconstruct videos from a stream of
events, and train it on a large amount of simulated event data. During training
we propose to use a perceptual loss to encourage reconstructions to follow
natural image statistics. We further extend our approach to synthesize color
images from color event streams. Our network surpasses state-of-the-art
reconstruction methods by a large margin in terms of image quality (> 20%),
while comfortably running in real-time. We show that the network is able to
synthesize high framerate videos (> 5,000 frames per second) of high-speed
phenomena (e.g. a bullet hitting an object) and is able to provide high dynamic
range reconstructions in challenging lighting conditions. We also demonstrate
the effectiveness of our reconstructions as an intermediate representation for
event data. We show that off-the-shelf computer vision algorithms can be
applied to our reconstructions for tasks such as object classification and
visual-inertial odometry and that this strategy consistently outperforms
algorithms that were specifically designed for event data.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.

negative:
Novel vision sensors such as thermal, hyperspectral, polarization, and event
cameras provide information that is not available from conventional intensity
cameras. An obstacle to using these sensors with current powerful deep neural
networks is the lack of large labeled training datasets. This paper proposes a
Network Grafting Algorithm (NGA), where a new front end network driven by
unconventional visual inputs replaces the front end network of a pretrained
deep network that processes intensity frames. The self-supervised training uses
only synchronously-recorded intensity frames and novel sensor data to maximize
feature similarity between the pretrained network and the grafted network. We
show that the enhanced grafted network reaches competitive average precision
(AP50) scores to the pretrained network on an object detection task using
thermal and event camera datasets, with no increase in inference costs.
Particularly, the grafted network driven by thermal frames showed a relative
improvement of 49.11% over the use of intensity frames. The grafted front end
has only 5--8% of the total parameters and can be trained in a few hours on a
single GPU equivalent to 5% of the time that would be needed to train the
entire object detector from labeled data. NGA allows new vision sensors to
capitalize on previously pretrained powerful deep models, saving on training
cost and widening a range of applications for novel sensors.

negative:
Cosmic polarization rotation, which may result from parity-violating new
physics or the presence of primordial magnetic fields, converts $E$-mode
polarization of the cosmic microwave background (CMB) into $B$-mode
polarization. Anisotropic cosmic polarization rotation leads to statistical
anisotropy in CMB polarization and can be reconstructed with quadratic
estimator techniques similar to those designed for gravitational lensing of the
CMB. At the sensitivity of upcoming CMB surveys, lensing-induced $B$-mode
polarization will act as a limiting factor in the search for anisotropic cosmic
polarization rotation, meaning that an analysis which incorporates some form of
delensing will be required to improve constraints on the effect with future
surveys. In this paper we extend the ResUNet-CMB convolutional neural network
to reconstruct anisotropic cosmic polarization rotation in the presence of
gravitational lensing and patchy reionization, and we show that the network
simultaneously reconstructs all three effects with variance that is lower than
that from the standard quadratic estimator nearly matching the performance of
an iterative reconstruction method.

negative:
Polarization of the mammalian embryo at the right developmental time is
critical for its development to term and would be valuable in assessing the
potential of human embryos. However, tracking polarization requires invasive
fluorescence staining, impermissible in the in vitro fertilization clinic.
Here, we report the use of artificial intelligence to detect polarization from
unstained time-lapse movies of mouse embryos. We assembled a dataset of
bright-field movie frames from 8-cell-stage embryos, side-by-side with
corresponding images of fluorescent markers of cell polarization. We then used
an ensemble learning model to detect whether any bright-field frame showed an
embryo before or after onset of polarization. Our resulting model has an
accuracy of 85% for detecting polarization, significantly outperforming human
volunteers trained on the same data (61% accuracy). We discovered that our
self-learning model focuses upon the angle between cells as one known cue for
compaction, which precedes polarization, but it outperforms the use of this cue
alone. By compressing three-dimensional time-lapsed image data into
two-dimensions, we are able to reduce data to an easily manageable size for
deep learning processing. In conclusion, we describe a method for detecting a
key developmental feature of embryo development that avoids clinically
impermissible fluorescence staining.

negative:
For applications such as airport border control, biometric technologies that
can process many capture subjects quickly, efficiently, with weak supervision,
and with minimal discomfort are desirable. Facial recognition is particularly
appealing because it is minimally invasive yet offers relatively good
recognition performance. Unfortunately, the combination of weak supervision and
minimal invasiveness makes even highly accurate facial recognition systems
susceptible to spoofing via presentation attacks. Thus, there is great demand
for an effective and low cost system capable of rejecting such attacks.To this
end we introduce PARAPH -- a novel hardware extension that exploits different
measurements of light polarization to yield an image space in which
presentation media are readily discernible from Bona Fide facial
characteristics. The PARAPH system is inexpensive with an added cost of less
than 10 US dollars. The system makes two polarization measurements in rapid
succession, allowing them to be approximately pixel-aligned, with a frame rate
limited by the camera, not the system. There are no moving parts above the
molecular level, due to the efficient use of twisted nematic liquid crystals.
We present evaluation images using three presentation attack media next to an
actual face -- high quality photos on glossy and matte paper and a video of the
face on an LCD. In each case, the actual face in the image generated by PARAPH
is structurally discernible from the presentations, which appear either as
noise (print attacks) or saturated images (replay attacks).