query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am currently working on a project that involves transforming black-and-white videos into colored versions, with a particular focus on color propagation in videos. I have found that traditional methods of color propagation tend to be computationally demanding. Therefore, I am seeking a more efficient method, especially when additional video information is available. I am interested in a model that can utilize a set of colored key frames from the video and apply color propagation algorithms to colorize other frames based on the provided colored ones. Ideally, this method should incorporate both local and global strategies. The local strategies should ensure color consistency in neighboring regions and frames, thereby maintaining stable color propagation. Simultaneously, from a global perspective, the model should be capable of identifying the keyframes that require colorization, ensuring overall process efficiency. While I value the efficiency of the color propagation methods, I am willing to compromise on quality, provided the performance surpasses the baselines established within the past five years.

positive:
Videos contain highly redundant information between frames. Such redundancy
has been extensively studied in video compression and encoding, but is less
explored for more advanced video processing. In this paper, we propose a
learnable unified framework for propagating a variety of visual properties of
video images, including but not limited to color, high dynamic range (HDR), and
segmentation information, where the properties are available for only a few
key-frames. Our approach is based on a temporal propagation network (TPN),
which models the transition-related affinity between a pair of frames in a
purely data-driven manner. We theoretically prove two essential factors for
TPN: (a) by regularizing the global transformation matrix as orthogonal, the
"style energy" of the property can be well preserved during propagation; (b)
such regularization can be achieved by the proposed switchable TPN with
bi-directional training on pairs of frames. We apply the switchable TPN to
three tasks: colorizing a gray-scale video based on a few color key-frames,
generating an HDR video from a low dynamic range (LDR) video and a few HDR
frames, and propagating a segmentation mask from the first frame in videos.
Experimental results show that our approach is significantly more accurate and
efficient than the state-of-the-art methods.

positive:
Video colorization is a challenging and highly ill-posed problem. Although
recent years have witnessed remarkable progress in single image colorization,
there is relatively less research effort on video colorization and existing
methods always suffer from severe flickering artifacts (temporal inconsistency)
or unsatisfying colorization performance. We address this problem from a new
perspective, by jointly considering colorization and temporal consistency in a
unified framework. Specifically, we propose a novel temporally consistent video
colorization framework (TCVC). TCVC effectively propagates frame-level deep
features in a bidirectional way to enhance the temporal consistency of
colorization. Furthermore, TCVC introduces a self-regularization learning (SRL)
scheme to minimize the prediction difference obtained with different time
steps. SRL does not require any ground-truth color videos for training and can
further improve temporal consistency. Experiments demonstrate that our method
can not only obtain visually pleasing colorized video, but also achieve clearly
better temporal consistency than state-of-the-art methods.

positive:
We propose a hybrid recurrent Video Colorization with Hybrid Generative
Adversarial Network (VCGAN), an improved approach to video colorization using
end-to-end learning. The VCGAN addresses two prevalent issues in the video
colorization domain: Temporal consistency and unification of colorization
network and refinement network into a single architecture. To enhance
colorization quality and spatiotemporal consistency, the mainstream of
generator in VCGAN is assisted by two additional networks, i.e., global feature
extractor and placeholder feature extractor, respectively. The global feature
extractor encodes the global semantics of grayscale input to enhance
colorization quality, whereas the placeholder feature extractor acts as a
feedback connection to encode the semantics of the previous colorized frame in
order to maintain spatiotemporal consistency. If changing the input for
placeholder feature extractor as grayscale input, the hybrid VCGAN also has the
potential to perform image colorization. To improve the consistency of far
frames, we propose a dense long-term loss that smooths the temporal disparity
of every two remote frames. Trained with colorization and temporal losses
jointly, VCGAN strikes a good balance between color vividness and video
continuity. Experimental results demonstrate that VCGAN produces higher-quality
and temporally more consistent colorful videos than existing approaches.

positive:
Coloring line art images based on the colors of reference images is an
important stage in animation production, which is time-consuming and tedious.
In this paper, we propose a deep architecture to automatically color line art
videos with the same color style as the given reference images. Our framework
consists of a color transform network and a temporal constraint network. The
color transform network takes the target line art images as well as the line
art and color images of one or more reference images as input, and generates
corresponding target color images. To cope with larger differences between the
target line art image and reference color images, our architecture utilizes
non-local similarity matching to determine the region correspondences between
the target image and the reference images, which are used to transform the
local color information from the references to the target. To ensure global
color style consistency, we further incorporate Adaptive Instance Normalization
(AdaIN) with the transformation parameters obtained from a style embedding
vector that describes the global color style of the references, extracted by an
embedder. The temporal constraint network takes the reference images and the
target image together in chronological order, and learns the spatiotemporal
features through 3D convolution to ensure the temporal consistency of the
target image and the reference image. Our model can achieve even better
coloring results by fine-tuning the parameters with only a small amount of
samples when dealing with an animation of a new style. To evaluate our method,
we build a line art coloring dataset. Experiments show that our method achieves
the best performance on line art video coloring compared to the
state-of-the-art methods and other baselines.

positive:
Endoscopic videos from multicentres often have different imaging conditions,
e.g., color and illumination, which make the models trained on one domain
usually fail to generalize well to another. Domain adaptation is one of the
potential solutions to address the problem. However, few of existing works
focused on the translation of video-based data. In this work, we propose a
novel generative adversarial network (GAN), namely VideoGAN, to transfer the
video-based data across different domains. As the frames of a video may have
similar content and imaging conditions, the proposed VideoGAN has an X-shape
generator to preserve the intra-video consistency during translation.
Furthermore, a loss function, namely color histogram loss, is proposed to tune
the color distribution of each translated frame. Two colonoscopic datasets from
different centres, i.e., CVC-Clinic and ETIS-Larib, are adopted to evaluate the
performance of domain adaptation of our VideoGAN. Experimental results
demonstrate that the adapted colonoscopic video generated by our VideoGAN can
significantly boost the segmentation accuracy, i.e., an improvement of 5%, of
colorectal polyps on multicentre datasets. As our VideoGAN is a general network
architecture, we also evaluate its performance with the CamVid driving video
dataset on the cloudy-to-sunny translation task. Comprehensive experiments show
that the domain gap could be substantially narrowed down by our VideoGAN.

positive:
Applying image processing algorithms independently to each frame of a video
often leads to undesired inconsistent results over time. Developing temporally
consistent video-based extensions, however, requires domain knowledge for
individual tasks and is unable to generalize to other applications. In this
paper, we present an efficient end-to-end approach based on deep recurrent
network for enforcing temporal consistency in a video. Our method takes the
original unprocessed and per-frame processed videos as inputs to produce a
temporally consistent video. Consequently, our approach is agnostic to specific
image processing algorithms applied on the original video. We train the
proposed network by minimizing both short-term and long-term temporal losses as
well as the perceptual loss to strike a balance between temporal stability and
perceptual similarity with the processed frames. At test time, our model does
not require computing optical flow and thus achieves real-time speed even for
high-resolution videos. We show that our single model can handle multiple and
unseen tasks, including but not limited to artistic style transfer,
enhancement, colorization, image-to-image translation and intrinsic image
decomposition. Extensive objective evaluation and subject study demonstrate
that the proposed approach performs favorably against the state-of-the-art
methods on various types of videos.

positive:
Models optimized for accuracy on single images are often prohibitively slow
to run on each frame in a video. Recent work exploits the use of optical flow
to warp image features forward from select keyframes, as a means to conserve
computation on video. This approach, however, achieves only limited speedup,
even when optimized, due to the accuracy degradation introduced by repeated
forward warping, and the inference cost of optical flow estimation. To address
these problems, we propose a new scheme that propagates features using the
block motion vectors (BMV) present in compressed video (e.g. H.264 codecs),
instead of optical flow, and bi-directionally warps and fuses features from
enclosing keyframes to capture scene context on each video frame. Our
technique, interpolation-BMV, enables us to accurately estimate the features of
intermediate frames, while keeping inference costs low. We evaluate our system
on the CamVid and Cityscapes datasets, comparing to both a strong single-frame
baseline and related work. We find that we are able to substantially accelerate
segmentation on video, achieving near real-time frame rates (20+ frames per
second) on large images (e.g. 960 x 720 pixels), while maintaining competitive
accuracy. This represents an improvement of almost 6x over the single-frame
baseline and 2.5x over the fastest prior work.

positive:
In this paper, we tackle the problem of colorization of grayscale videos to
reduce bandwidth usage. For this task, we use some colored keyframes as
reference images from the colored version of the grayscale video. We propose a
model that extracts keyframes from a colored video and trains a Convolutional
Neural Network from scratch on these colored frames. Through the extracted
keyframes we get a good knowledge of the colors that have been used in the
video which helps us in colorizing the grayscale version of the video
efficiently. An application of the technique that we propose in this paper, is
in saving bandwidth while sending raw colored videos that haven't gone through
any compression. A raw colored video takes up around three times more memory
size than its grayscale version. We can exploit this fact and send a grayscale
video along with out trained model instead of a colored video. Later on, in
this paper we show how this technique can help to save bandwidth usage to upto
three times while transmitting raw colored videos.

positive:
This paper presents the first end-to-end network for exemplar-based video
colorization. The main challenge is to achieve temporal consistency while
remaining faithful to the reference style. To address this issue, we introduce
a recurrent framework that unifies the semantic correspondence and color
propagation steps. Both steps allow a provided reference image to guide the
colorization of every frame, thus reducing accumulated propagation errors.
Video frames are colorized in sequence based on the colorization history, and
its coherency is further enforced by the temporal consistency loss. All of
these components, learned end-to-end, help produce realistic videos with good
temporal stability. Experiments show our result is superior to the
state-of-the-art methods both quantitatively and qualitatively.

positive:
Video object segmentation is a fundamental step in many advanced vision
applications. Most existing algorithms are based on handcrafted features such
as HOG, super-pixel segmentation or texture-based techniques, while recently
deep features have been found to be more efficient. Existing algorithms observe
performance degradation in the presence of challenges such as illumination
variations, shadows, and color camouflage. To handle these challenges we
propose a fusion based moving object segmentation algorithm which exploits
color as well as depth information using GAN to achieve more accuracy. Our goal
is to segment moving objects in the presence of challenging background scenes,
in real environments. To address this problem, GAN is trained in an
unsupervised manner on color and depth information independently with
challenging video sequences. During testing, the trained GAN generates
backgrounds similar to that in the test sample. The generated background
samples are then compared with the test sample to segment moving objects. The
final result is computed by fusion of object boundaries in both modalities, RGB
and the depth. The comparison of our proposed algorithm with five
state-of-the-art methods on publicly available dataset has shown the strength
of our algorithm for moving object segmentation in videos in the presence of
challenging real scenarios.

positive:
A background model describes a scene without any foreground objects and has a
number of applications, ranging from video surveillance to computational
photography. Recent studies have introduced the method of Dynamic Mode
Decomposition (DMD) for robustly separating video frames into a background
model and foreground components. While the method introduced operates by
converting color images to grayscale, we in this study propose a technique to
obtain the background model in the color domain. The effectiveness of our
technique is demonstrated using a publicly available Scene Background
Initialisation (SBI) dataset. Our results both qualitatively and quantitatively
show that DMD can successfully obtain a colored background model.

positive:
We propose a technique that propagates information forward through video
data. The method is conceptually simple and can be applied to tasks that
require the propagation of structured information, such as semantic labels,
based on video content. We propose a 'Video Propagation Network' that processes
video frames in an adaptive manner. The model is applied online: it propagates
information forward without the need to access future frames. In particular we
combine two components, a temporal bilateral network for dense and video
adaptive filtering, followed by a spatial network to refine features and
increased flexibility. We present experiments on video object segmentation and
semantic video segmentation and show increased performance comparing to the
best previous task-specific methods, while having favorable runtime.
Additionally we demonstrate our approach on an example regression task of color
propagation in a grayscale video.

positive:
We address the problem of soft color segmentation, defined as decomposing a
given image into several RGBA layers, each containing only homogeneous color
regions. The resulting layers from decomposition pave the way for applications
that benefit from layer-based editing, such as recoloring and compositing of
images and videos. The current state-of-the-art approach for this problem is
hindered by slow processing time due to its iterative nature, and consequently
does not scale to certain real-world scenarios. To address this issue, we
propose a neural network based method for this task that decomposes a given
image into multiple layers in a single forward pass. Furthermore, our method
separately decomposes the color layers and the alpha channel layers. By
leveraging a novel training objective, our method achieves proper assignment of
colors amongst layers. As a consequence, our method achieve promising quality
without existing issue of inference speed for iterative approaches. Our
thorough experimental analysis shows that our method produces qualitative and
quantitative results comparable to previous methods while achieving a 300,000x
speed improvement. Finally, we utilize our proposed method on several
applications, and demonstrate its speed advantage, especially in video editing.

negative:
The remastering of vintage film comprises of a diversity of sub-tasks
including super-resolution, noise removal, and contrast enhancement which aim
to restore the deteriorated film medium to its original state. Additionally,
due to the technical limitations of the time, most vintage film is either
recorded in black and white, or has low quality colors, for which colorization
becomes necessary. In this work, we propose a single framework to tackle the
entire remastering task semi-interactively. Our work is based on temporal
convolutional neural networks with attention mechanisms trained on videos with
data-driven deterioration simulation. Our proposed source-reference attention
allows the model to handle an arbitrary number of reference color images to
colorize long videos without the need for segmentation while maintaining
temporal consistency. Quantitative analysis shows that our framework
outperforms existing approaches, and that, in contrast to existing approaches,
the performance of our framework increases with longer videos and more
reference color images.

negative:
Convolutional networks optimized for accuracy on challenging, dense
prediction tasks are prohibitively slow to run on each frame in a video. The
spatial similarity of nearby video frames, however, suggests opportunity to
reuse computation. Existing work has explored basic feature reuse and feature
warping based on optical flow, but has encountered limits to the speedup
attainable with these techniques. In this paper, we present a new, two part
approach to accelerating inference on video. First, we propose a fast feature
propagation technique that utilizes the block motion vectors present in
compressed video (e.g. H.264 codecs) to cheaply propagate features from frame
to frame. Second, we develop a novel feature estimation scheme, termed feature
interpolation, that fuses features propagated from enclosing keyframes to
render accurate feature estimates, even at sparse keyframe frequencies. We
evaluate our system on the Cityscapes and CamVid datasets, comparing to both a
frame-by-frame baseline and related work. We find that we are able to
substantially accelerate segmentation on video, achieving near real-time frame
rates (20.1 frames per second) on large images (960 x 720 pixels), while
maintaining competitive accuracy. This represents an improvement of almost 6x
over the single-frame baseline and 2.5x over the fastest prior work.

negative:
Video inpainting, which aims at filling in missing regions of a video,
remains challenging due to the difficulty of preserving the precise spatial and
temporal coherence of video contents. In this work we propose a novel
flow-guided video inpainting approach. Rather than filling in the RGB pixels of
each frame directly, we consider video inpainting as a pixel propagation
problem. We first synthesize a spatially and temporally coherent optical flow
field across video frames using a newly designed Deep Flow Completion network.
Then the synthesized flow field is used to guide the propagation of pixels to
fill up the missing regions in the video. Specifically, the Deep Flow
Completion network follows a coarse-to-fine refinement to complete the flow
fields, while their quality is further improved by hard flow example mining.
Following the guide of the completed flow, the missing video regions can be
filled up precisely. Our method is evaluated on DAVIS and YouTube-VOS datasets
qualitatively and quantitatively, achieving the state-of-the-art performance in
terms of inpainting quality and speed.

negative:
Segmentation of an object from a video is a challenging task in multimedia
applications. Depending on the application, automatic or interactive methods
are desired; however, regardless of the application type, efficient computation
of video object segmentation is crucial for time-critical applications;
specifically, mobile and interactive applications require near real-time
efficiencies. In this paper, we address the problem of video segmentation from
the perspective of efficiency. We initially redefine the problem of video
object segmentation as the propagation of MRF energies along the temporal
domain. For this purpose, a novel and efficient method is proposed to propagate
MRF energies throughout the frames via bilateral filters without using any
global texture, color or shape model. Recently presented bi-exponential filter
is utilized for efficiency, whereas a novel technique is also developed to
dynamically solve graph-cuts for varying, non-lattice graphs in general linear
filtering scenario. These improvements are experimented for both automatic and
interactive video segmentation scenarios. Moreover, in addition to the
efficiency, segmentation quality is also tested both quantitatively and
qualitatively. Indeed, for some challenging examples, significant time
efficiency is observed without loss of segmentation quality.

negative:
One of the challenges of computer vision is that it needs to adapt to color
deviations in changeable environments. Therefore, minimizing the adverse
effects of color deviation on the prediction is one of the main goals of vision
task. Current solutions focus on using generative models to augment training
data to enhance the invariance of input variation. However, such methods often
introduce new noise, which limits the gain from generated data. To this end,
this paper proposes a strategy eliminate deviation with deviation, which is
named Random Color Dropout (RCD). Our hypothesis is that if there are color
deviation between the query image and the gallery image, the retrieval results
of some examples will be better after ignoring the color information.
Specifically, this strategy balances the weights between color features and
color-independent features in the neural network by dropouting partial color
information in the training data, so as to overcome the effect of color
devitaion. The proposed RCD can be combined with various existing ReID models
without changing the learning strategy, and can be applied to other computer
vision fields, such as object detection. Experiments on several ReID baselines
and three common large-scale datasets such as Market1501, DukeMTMC, and MSMT17
have verified the effectiveness of this method. Experiments on Cross-domain
tests have shown that this strategy is significant eliminating the domain gap.
Furthermore, in order to understand the working mechanism of RCD, we analyzed
the effectiveness of this strategy from the perspective of classification,
which reveals that it may be better to utilize many instead of all of color
information in visual tasks with strong domain variations.

negative:
Automatic animation line art colorization is a challenging computer vision
problem, since the information of the line art is highly sparse and abstracted
and there exists a strict requirement for the color and style consistency
between frames. Recently, a lot of Generative Adversarial Network (GAN) based
image-to-image translation methods for single line art colorization have
emerged. They can generate perceptually appealing results conditioned on line
art images. However, these methods can not be adopted for the purpose of
animation colorization because there is a lack of consideration of the
in-between frame consistency. Existing methods simply input the previous
colored frame as a reference to color the next line art, which will mislead the
colorization due to the spatial misalignment of the previous colored frame and
the next line art especially at positions where apparent changes happen. To
address these challenges, we design a kind of correlation matching feature
transfer model (called CMFT) to align the colored reference feature in a
learnable way and integrate the model into an U-Net based generator in a
coarse-to-fine manner. This enables the generator to transfer the layer-wise
synchronized features from the deep semantic code to the content progressively.
Extension evaluation shows that CMFT model can effectively improve the
in-between consistency and the quality of colored frames especially when the
motion is intense and diverse.

negative:
For many years, channels of a color image have been processed individually,
or the image has been converted to grayscale one with respect to color image
processing. Pure quaternion representation of color images solves this issue as
it allows images to be processed in a holistic space. Nevertheless, it brings
additional costs due to the extra fourth dimension. In this paper, we propose
an approach for representing color images with full quaternion numbers that
enables us to process color images holistically without additional cost in
time, space and computation. With taking auto- and cross-correlation of color
channels into account, an autoencoder neural network is used to generate a
global model for transforming a color image into a full quaternion matrix. To
evaluate the model, we use UCID dataset, and the results indicate that the
model has an acceptable performance on color images. Moreover, we propose a
compression method based on the generated model and QSVD as a case study. The
method is compared with the same compression method using pure quaternion
representation and is assessed with UCID dataset. The results demonstrate that
the compression method using the proposed full quaternion representation fares
better than the other in terms of time, quality, and size of compressed files.

negative:
We present a novel approach of color transfer between images by exploring
their high-level semantic information. First, we set up a database which
consists of the collection of downloaded images from the internet, which are
segmented automatically by using matting techniques. We then, extract image
foregrounds from both source and multiple target images. Then by using image
matting algorithms, the system extracts the semantic information such as faces,
lips, teeth, eyes, eyebrows, etc., from the extracted foregrounds of the source
image. And, then the color is transferred between corresponding parts with the
same semantic information. Next we get the color transferred result by
seamlessly compositing different parts together using alpha blending. In the
final step, we present an efficient method of color consistency to optimize the
color of a collection of images showing the common scene. The main advantage of
our method over existing techniques is that it does not need face matching, as
one could use more than one target images. It is not restricted to head shot
images as we can also change the color style in the wild. Moreover, our
algorithm does not require to choose the same color style, same pose and image
size between source and target images. Our algorithm is not restricted to
one-to-one image color transfer and can make use of more than one target images
to transfer the color in different parts in the source image. Comparing with
other approaches, our algorithm is much better in color blending in the input
data.

negative:
In this work, we present a method for automatic colorization of grayscale
videos. The core of the method is a Generative Adversarial Network that is
trained and tested on sequences of frames in a sliding window manner. Network
convolutional and deconvolutional layers are three-dimensional, with frame
height, width and time as the dimensions taken into account. Multiple
chrominance estimates per frame are aggregated and combined with available
luminance information to recreate a colored sequence. Colorization trials are
run succesfully on a dataset of old black-and-white films. The usefulness of
our method is also validated with numerical results, computed with a newly
proposed metric that measures colorization consistency over a frame sequence.

negative:
We present a fully automatic approach to video colorization with
self-regularization and diversity. Our model contains a colorization network
for video frame colorization and a refinement network for spatiotemporal color
refinement. Without any labeled data, both networks can be trained with
self-regularized losses defined in bilateral and temporal space. The bilateral
loss enforces color consistency between neighboring pixels in a bilateral space
and the temporal loss imposes constraints between corresponding pixels in two
nearby frames. While video colorization is a multi-modal problem, our method
uses a perceptual loss with diversity to differentiate various modes in the
solution space. Perceptual experiments demonstrate that our approach
outperforms state-of-the-art approaches on fully automatic video colorization.
The results are shown in the supplementary video at
https://youtu.be/Y15uv2jnK-4

negative:
Document image enhancement and binarization methods are often used to improve
the accuracy and efficiency of document image analysis tasks such as text
recognition. Traditional non-machine-learning methods are constructed on
low-level features in an unsupervised manner but have difficulty with
binarization on documents with severely degraded backgrounds. Convolutional
neural network-based methods focus only on grayscale images and on local
textual features. In this paper, we propose a two-stage color document image
enhancement and binarization method using generative adversarial neural
networks. In the first stage, four color-independent adversarial networks are
trained to extract color foreground information from an input image for
document image enhancement. In the second stage, two independent adversarial
networks with global and local features are trained for image binarization of
documents of variable size. For the adversarial neural networks, we formulate
loss functions between a discriminator and generators having an encoder-decoder
structure. Experimental results show that the proposed method achieves better
performance than many classical and state-of-the-art algorithms over the
Document Image Binarization Contest (DIBCO) datasets, the LRDE Document
Binarization Dataset (LRDE DBD), and our shipping label image dataset. We plan
to release the shipping label dataset as well as our implementation code at
github.com/opensuh/DocumentBinarization/.

negative:
In industry, there exist plenty of scenarios where old gray photos need to be
automatically colored, such as video sites and archives. In this paper, we
present the HistoryNet focusing on historical person's diverse high fidelity
clothing colorization based on fine grained semantic understanding and prior.
Colorization of historical persons is realistic and practical, however,
existing methods do not perform well in the regards. In this paper, a
HistoryNet including three parts, namely, classification, fine grained semantic
parsing and colorization, is proposed. Classification sub-module supplies
classifying of images according to the eras, nationalities and garment types;
Parsing sub-network supplies the semantic for person contours, clothing and
background in the image to achieve more accurate colorization of clothes and
persons and prevent color overflow. In the training process, we integrate
classification and semantic parsing features into the coloring generation
network to improve colorization. Through the design of classification and
parsing subnetwork, the accuracy of image colorization can be improved and the
boundary of each part of image can be more clearly. Moreover, we also propose a
novel Modern Historical Movies Dataset (MHMD) containing 1,353,166 images and
42 labels of eras, nationalities, and garment types for automatic colorization
from 147 historical movies or TV series made in modern time. Various
quantitative and qualitative comparisons demonstrate that our method
outperforms the state-of-the-art colorization methods, especially on military
uniforms, which has correct colors according to the historical literatures.

negative:
For semantic segmentation, most existing real-time deep models trained with
each frame independently may produce inconsistent results for a video sequence.
Advanced methods take into considerations the correlations in the video
sequence, e.g., by propagating the results to the neighboring frames using
optical flow, or extracting the frame representations with other frames, which
may lead to inaccurate results or unbalanced latency. In this work, we process
efficient semantic video segmentation in a per-frame fashion during the
inference process. Different from previous per-frame models, we explicitly
consider the temporal consistency among frames as extra constraints during the
training process and embed the temporal consistency into the segmentation
network. Therefore, in the inference process, we can process each frame
independently with no latency, and improve the temporal consistency with no
extra computational cost and post-processing. We employ compact models for
real-time execution. To narrow the performance gap between compact models and
large models, new knowledge distillation methods are designed. Our results
outperform previous keyframe based methods with a better trade-off between the
accuracy and the inference speed on popular benchmarks, including the
Cityscapes and Camvid. The temporal consistency is also improved compared with
corresponding baselines which are trained with each frame independently. Code
is available at: https://tinyurl.com/segment-video

negative:
We propose the first deep learning approach for exemplar-based local
colorization. Given a reference color image, our convolutional neural network
directly maps a grayscale image to an output colorized image. Rather than using
hand-crafted rules as in traditional exemplar-based methods, our end-to-end
colorization network learns how to select, propagate, and predict colors from
the large-scale data. The approach performs robustly and generalizes well even
when using reference images that are unrelated to the input grayscale image.
More importantly, as opposed to other learning-based colorization methods, our
network allows the user to achieve customizable results by simply feeding
different references. In order to further reduce manual effort in selecting the
references, the system automatically recommends references with our proposed
image retrieval algorithm, which considers both semantic and luminance
information. The colorization can be performed fully automatically by simply
picking the top reference suggestion. Our approach is validated through a user
study and favorable quantitative comparisons to the-state-of-the-art methods.
Furthermore, our approach can be naturally extended to video colorization. Our
code and models will be freely available for public use.

negative:
Compared to color images captured by conventional RGB cameras, monochrome
images usually have better signal-to-noise ratio (SNR) and richer textures due
to its higher quantum efficiency. It is thus natural to apply a mono-color
dual-camera system to restore color images with higher visual quality. In this
paper, we propose a mono-color image enhancement algorithm that colorizes the
monochrome image with the color one. Based on the assumption that adjacent
structures with similar luminance values are likely to have similar colors, we
first perform dense scribbling to assign colors to the monochrome pixels
through block matching. Two types of outliers, including occlusion and color
ambiguity, are detected and removed from the initial scribbles. We also
introduce a sampling strategy to accelerate the scribbling process. Then, the
dense scribbles are propagated to the entire image. To alleviate incorrect
color propagation in the regions that have no color hints at all, we generate
extra color seeds based on the existed scribbles to guide the propagation
process. Experimental results show that, our algorithm can efficiently restore
color images with higher SNR and richer details from the mono-color image
pairs, and achieves good performance in solving the color bleeding problem.

negative:
A novel color image enhancement method is proposed based on Retinex to
enhance color images under non-uniform illumination or poor visibility
conditions. Different from the conventional Retinex algorithms, the Weighted
Guided Image Filter is used as a surround function instead of the Gaussian
filter to estimate the background illumination, which can overcome the
drawbacks of local blur and halo artifact that may appear by Gaussian filter.
To avoid color distortion, the image is converted to the HSI color model, and
only the intensity channel is enhanced. Then a linear color restoration
algorithm is adopted to convert the enhanced intensity image back to the RGB
color model, which ensures the hue is constant and undistorted. Experimental
results show that the proposed method is effective to enhance both color and
gray images with low exposure and non-uniform illumination, resulting in better
visual quality than traditional method. At the same time, the objective
evaluation indicators are also superior to the conventional methods. In
addition, the efficiency of the proposed method is also improved thanks to the
linear color restoration algorithm.

negative:
Greyscale image colorization for applications in image restoration has seen
significant improvements in recent years. Many of these techniques that use
learning-based methods struggle to effectively colorize sparse inputs. With the
consistent growth of the anime industry, the ability to colorize sparse input
such as line art can reduce significant cost and redundant work for production
studios by eliminating the in-between frame colorization process. Simply using
existing methods yields inconsistent colors between related frames resulting in
a flicker effect in the final video. In order to successfully automate key
areas of large-scale anime production, the colorization of line arts must be
temporally consistent between frames. This paper proposes a method to colorize
line art frames in an adversarial setting, to create temporally coherent video
of large anime by improving existing image to image translation methods. We
show that by adding an extra condition to the generator and discriminator, we
can effectively create temporally consistent video sequences from anime line
arts. Code and models available at: https://github.com/Harry-Thasarathan/TCVC

negative:
Although recent inpainting approaches have demonstrated significant
improvements with deep neural networks, they still suffer from artifacts such
as blunt structures and abrupt colors when filling in the missing regions. To
address these issues, we propose an external-internal inpainting scheme with a
monochromic bottleneck that helps image inpainting models remove these
artifacts. In the external learning stage, we reconstruct missing structures
and details in the monochromic space to reduce the learning dimension. In the
internal learning stage, we propose a novel internal color propagation method
with progressive learning strategies for consistent color restoration.
Extensive experiments demonstrate that our proposed scheme helps image
inpainting models produce more structure-preserved and visually compelling
results.

negative:
Color transfer is an image editing process that adjusts the colors of a
picture to match a target picture's color theme. A natural color transfer not
only matches the color styles but also prevents after-transfer artifacts due to
image compression, noise, and gradient smoothness change. The recently
discovered color homography theorem proves that colors across a change in
photometric viewing condition are related by a homography. In this paper, we
propose a color-homography-based color transfer decomposition which encodes
color transfer as a combination of chromaticity shift and shading adjustment. A
powerful form of shading adjustment is shown to be a global shading curve by
which the same shading homography can be applied elsewhere. Our experiments
show that the proposed color transfer decomposition provides a very close
approximation to many popular color transfer methods. The advantage of our
approach is that the learned color transfer can be applied to many other images
(e.g. other frames in a video), instead of a frame-to-frame basis. We
demonstrate two applications for color transfer enhancement and video color
grading re-application. This simple model of color transfer is also important
for future color transfer algorithm design.

negative:
We consider the reconstruction problem of video snapshot compressive imaging
(SCI), which captures high-speed videos using a low-speed 2D sensor (detector).
The underlying principle of SCI is to modulate sequential high-speed frames
with different masks and then these encoded frames are integrated into a
snapshot on the sensor and thus the sensor can be of low-speed. On one hand,
video SCI enjoys the advantages of low-bandwidth, low-power and low-cost. On
the other hand, applying SCI to large-scale problems (HD or UHD videos) in our
daily life is still challenging and one of the bottlenecks lies in the
reconstruction algorithm. Exiting algorithms are either too slow (iterative
optimization algorithms) or not flexible to the encoding process (deep learning
based end-to-end networks). In this paper, we develop fast and flexible
algorithms for SCI based on the plug-and-play (PnP) framework. In addition to
the PnP-ADMM method, we further propose the PnP-GAP (generalized alternating
projection) algorithm with a lower computational workload. We first employ the
image deep denoising priors to show that PnP can recover a UHD color video with
30 frames from a snapshot measurement. Since videos have strong temporal
correlation, by employing the video deep denoising priors, we achieve a
significant improvement in the results. Furthermore, we extend the proposed PnP
algorithms to the color SCI system using mosaic sensors, where each pixel only
captures the red, green or blue channels. A joint reconstruction and
demosaicing paradigm is developed for flexible and high quality reconstruction
of color video SCI systems. Extensive results on both simulation and real
datasets verify the superiority of our proposed algorithm.

negative:
We propose a deep learning approach for user-guided image colorization. The
system directly maps a grayscale image, along with sparse, local user "hints"
to an output colorization with a Convolutional Neural Network (CNN). Rather
than using hand-defined rules, the network propagates user edits by fusing
low-level cues along with high-level semantic information, learned from
large-scale data. We train on a million images, with simulated user inputs. To
guide the user towards efficient input selection, the system recommends likely
colors based on the input image and current user inputs. The colorization is
performed in a single feed-forward pass, enabling real-time use. Even with
randomly simulated user inputs, we show that the proposed system helps novice
users quickly create realistic colorizations, and offers large improvements in
colorization quality with just a minute of use. In addition, we demonstrate
that the framework can incorporate other user "hints" to the desired
colorization, showing an application to color histogram transfer. Our code and
models are available at https://richzhang.github.io/ideepcolor.

negative:
Video semantic segmentation is active in recent years benefited from the
great progress of image semantic segmentation. For such a task, the per-frame
image segmentation is generally unacceptable in practice due to high
computation cost. To tackle this issue, many works use the flow-based feature
propagation to reuse the features of previous frames. However, the optical flow
estimation inevitably suffers inaccuracy and then causes the propagated
features distorted. In this paper, we propose distortion-aware feature
correction to alleviate the issue, which improves video segmentation
performance by correcting distorted propagated features. To be specific, we
firstly propose to transfer distortion patterns from feature into image space
and conduct effective distortion map prediction. Benefited from the guidance of
distortion maps, we proposed Feature Correction Module (FCM) to rectify
propagated features in the distorted areas. Our proposed method can
significantly boost the accuracy of video semantic segmentation at a low price.
The extensive experimental results on Cityscapes and CamVid show that our
method outperforms the recent state-of-the-art methods.

negative:
Video object segmentation, aiming to segment the foreground objects given the
annotation of the first frame, has been attracting increasing attentions. Many
state-of-the-art approaches have achieved great performance by relying on
online model updating or mask-propagation techniques. However, most online
models require high computational cost due to model fine-tuning during
inference. Most mask-propagation based models are faster but with relatively
low performance due to failure to adapt to object appearance variation. In this
paper, we are aiming to design a new model to make a good balance between speed
and performance. We propose a model, called NPMCA-net, which directly localizes
foreground objects based on mask-propagation and non-local technique by
matching pixels in reference and target frames. Since we bring in information
of both first and previous frames, our network is robust to large object
appearance variation, and can better adapt to occlusions. Extensive experiments
show that our approach can achieve a new state-of-the-art performance with a
fast speed at the same time (86.5% IoU on DAVIS-2016 and 72.2% IoU on
DAVIS-2017, with speed of 0.11s per frame) under the same level comparison.
Source code is available at https://github.com/siyueyu/NPMCA-net.

negative:
In this work we focus on the problem of colorization for image compression.
Since color information occupies a large proportion of the total storage size
of an image, a method that can predict accurate color from its grayscale
version can produce dramatic reduction in image file size. But colorization for
compression poses several challenges. First, while colorization for artistic
purposes simply involves predicting plausible chroma, colorization for
compression requires generating output colors that are as close as possible to
the ground truth. Second, many objects in the real world exhibit multiple
possible colors. Thus, to disambiguate the colorization problem some additional
information must be stored to reproduce the true colors with good accuracy. To
account for the multimodal color distribution of objects we propose a deep
tree-structured network that generates multiple color hypotheses for every
pixel from a grayscale picture (as opposed to a single color produced by most
prior colorization approaches). We show how to leverage the multimodal output
of our model to reproduce with high fidelity the true colors of an image by
storing very little additional information. In the experiments we show that our
proposed method outperforms traditional JPEG color coding by a large margin,
producing colors that are nearly indistinguishable from the ground truth at the
storage cost of just a few hundred bytes for high-resolution pictures!

negative:
Error propagation is a general but crucial problem in online semi-supervised
video object segmentation. We aim to suppress error propagation through a
correction mechanism with high reliability. The key insight is to disentangle
the correction from the conventional mask propagation process with reliable
cues. We introduce two modulators, propagation and correction modulators, to
separately perform channel-wise re-calibration on the target frame embeddings
according to local temporal correlations and reliable references respectively.
Specifically, we assemble the modulators with a cascaded propagation-correction
scheme. This avoids overriding the effects of the reliable correction modulator
by the propagation modulator. Although the reference frame with the ground
truth label provides reliable cues, it could be very different from the target
frame and introduce uncertain or incomplete correlations. We augment the
reference cues by supplementing reliable feature patches to a maintained pool,
thus offering more comprehensive and expressive object representations to the
modulators. In addition, a reliability filter is designed to retrieve reliable
patches and pass them in subsequent frames. Our model achieves state-of-the-art
performance on YouTube-VOS18/19 and DAVIS17-Val/Test benchmarks. Extensive
experiments demonstrate that the correction mechanism provides considerable
performance gain by fully utilizing reliable guidance. Code is available at:
https://github.com/JerryX1110/RPCMVOS.

negative:
Color and structure are the two pillars that construct an image. Usually, the
structure is well expressed through a rich spectrum of colors, allowing objects
in an image to be recognized by neural networks. However, under extreme
limitations of color space, the structure tends to vanish, and thus a neural
network might fail to understand the image. Interested in exploring this
interplay between color and structure, we study the scientific problem of
identifying and preserving the most informative image structures while
constraining the color space to just a few bits, such that the resulting image
can be recognized with possibly high accuracy. To this end, we propose a color
quantization network, ColorCNN, which learns to structure the images from the
classification loss in an end-to-end manner. Given a color space size, ColorCNN
quantizes colors in the original image by generating a color index map and an
RGB color palette. Then, this color-quantized image is fed to a pre-trained
task network to evaluate its performance. In our experiment, with only a 1-bit
color space (i.e., two colors), the proposed network achieves 82.1% top-1
accuracy on the CIFAR10 dataset, outperforming traditional color quantization
methods by a large margin. For applications, when encoded with PNG, the
proposed color quantization shows superiority over other image compression
methods in the extremely low bit-rate regime. The code is available at:
https://github.com/hou-yz/color_distillation.

negative:
This thesis presents methods and approaches to image color correction, color
enhancement, and color editing. To begin, we study the color correction problem
from the standpoint of the camera's image signal processor (ISP). A camera's
ISP is hardware that applies a series of in-camera image processing and color
manipulation steps, many of which are nonlinear in nature, to render the
initial sensor image to its final photo-finished representation saved in the
8-bit standard RGB (sRGB) color space. As white balance (WB) is one of the
major procedures applied by the ISP for color correction, this thesis presents
two different methods for ISP white balancing. Afterward, we discuss another
scenario of correcting and editing image colors, where we present a set of
methods to correct and edit WB settings for images that have been improperly
white-balanced by the ISP. Then, we explore another factor that has a
significant impact on the quality of camera-rendered colors, in which we
outline two different methods to correct exposure errors in camera-rendered
images. Lastly, we discuss post-capture auto color editing and manipulation. In
particular, we propose auto image recoloring methods to generate different
realistic versions of the same camera-rendered image with new colors. Through
extensive evaluations, we demonstrate that our methods provide superior
solutions compared to existing alternatives targeting color correction, color
enhancement, and color editing.

negative:
Event cameras are novel, bio-inspired visual sensors, whose pixels output
asynchronous and independent timestamped spikes at local intensity changes,
called 'events'. Event cameras offer advantages over conventional frame-based
cameras in terms of latency, high dynamic range (HDR) and temporal resolution.
Until recently, event cameras have been limited to outputting events in the
intensity channel, however, recent advances have resulted in the development of
color event cameras, such as the Color-DAVIS346. In this work, we present and
release the first Color Event Camera Dataset (CED), containing 50 minutes of
footage with both color frames and events. CED features a wide variety of
indoor and outdoor scenes, which we hope will help drive forward event-based
vision research. We also present an extension of the event camera simulator
ESIM that enables simulation of color events. Finally, we present an evaluation
of three state-of-the-art image reconstruction methods that can be used to
convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to
visualise the event stream, and for use in downstream vision applications.

negative:
Lossy compression brings artifacts into the compressed image and degrades the
visual quality. In recent years, many compression artifacts removal methods
based on convolutional neural network (CNN) have been developed with great
success. However, these methods usually train a model based on one specific
value or a small range of quality factors. Obviously, if the test image's
quality factor does not match to the assumed value range, then degraded
performance will be resulted. With this motivation and further consideration of
practical usage, a highly robust compression artifacts removal network is
proposed in this paper. Our proposed network is a single model approach that
can be trained for handling a wide range of quality factors while consistently
delivering superior or comparable image artifacts removal performance. To
demonstrate, we focus on the JPEG compression with quality factors, ranging
from 1 to 60. Note that a turnkey success of our proposed network lies in the
novel utilization of the quantization tables as part of the training data.
Furthermore, it has two branches in parallel---i.e., the restoration branch and
the global branch. The former effectively removes the local artifacts, such as
ringing artifacts removal. On the other hand, the latter extracts the global
features of the entire image that provides highly instrumental image quality
improvement, especially effective on dealing with the global artifacts, such as
blocking, color shifting. Extensive experimental results performed on color and
grayscale images have clearly demonstrated the effectiveness and efficacy of
our proposed single-model approach on the removal of compression artifacts from
the decoded image.

negative:
This paper addresses fast semantic segmentation on video.Video segmentation
often calls for real-time, or even fasterthan real-time, processing. One common
recipe for conserving computation arising from feature extraction is to
propagate features of few selected keyframes. However, recent advances in fast
image segmentation make these solutions less attractive. To leverage fast image
segmentation for furthering video segmentation, we propose a simple yet
efficient propagation framework. Specifically, we perform lightweight flow
estimation in 1/8-downscaled image space for temporal warping in segmentation
outpace space. Moreover, we introduce a guided spatially-varying convolution
for fusing segmentations derived from the previous and current frames, to
mitigate propagation error and enable lightweight feature extraction on
non-keyframes. Experimental results on Cityscapes and CamVid show that our
scheme achieves the state-of-the-art accuracy-throughput trade-off on video
segmentation.

negative:
Near-infrared imaging can capture haze-free near-infrared gray images and
visible color images, according to physical scattering models, e.g., Rayleigh
or Mie models. However, there exist serious discrepancies in brightness and
image structures between the near-infrared gray images and the visible color
images. The direct use of the near-infrared gray images brings about another
color distortion problem in the dehazed images. Therefore, the color distortion
should also be considered for near-infrared dehazing. To reflect this point,
this paper presents an approach of adding a new color regularization to
conventional dehazing framework. The proposed color regularization can model
the color prior for unknown haze-free images from two captured images. Thus,
natural-looking colors and fine details can be induced on the dehazed images.
The experimental results show that the proposed color regularization model can
help remove the color distortion and the haze at the same time. Also, the
effectiveness of the proposed color regularization is verified by comparing
with other conventional regularizations. It is also shown that the proposed
color regularization can remove the edge artifacts which arise from the use of
the conventional dark prior model.

negative:
Over the last decade, the process of automatic image colorization has been of
significant interest for several application areas including restoration of
aged or degraded images. This problem is highly ill-posed due to the large
degrees of freedom during the assignment of color information. Many of the
recent developments in automatic colorization involve images that contain a
common theme or require highly processed data such as semantic maps as input.
In our approach, we attempt to fully generalize the colorization procedure
using a conditional Deep Convolutional Generative Adversarial Network (DCGAN),
extend current methods to high-resolution images and suggest training
strategies that speed up the process and greatly stabilize it. The network is
trained over datasets that are publicly available such as CIFAR-10 and
Places365. The results of the generative model and traditional deep neural
networks are compared.

negative:
Photorealistic image stylization concerns transferring style of a reference
photo to a content photo with the constraint that the stylized photo should
remain photorealistic. While several photorealistic image stylization methods
exist, they tend to generate spatially inconsistent stylizations with
noticeable artifacts. In this paper, we propose a method to address these
issues. The proposed method consists of a stylization step and a smoothing
step. While the stylization step transfers the style of the reference photo to
the content photo, the smoothing step ensures spatially consistent
stylizations. Each of the steps has a closed-form solution and can be computed
efficiently. We conduct extensive experimental validations. The results show
that the proposed method generates photorealistic stylization outputs that are
more preferred by human subjects as compared to those by the competing methods
while running much faster. Source code and additional results are available at
https://github.com/NVIDIA/FastPhotoStyle .

negative:
Recently, some correlation filter based trackers with detection proposals
have achieved state-of-the-art tracking results. However, a large number of
redundant proposals given by the proposal generator may degrade the performance
and speed of these trackers. In this paper, we propose an adaptive proposal
selection algorithm which can generate a small number of high-quality proposals
to handle the problem of scale variations for visual object tracking.
Specifically, we firstly utilize the color histograms in the HSV color space to
represent the instances (i.e., the initial target in the first frame and the
predicted target in the previous frame) and proposals. Then, an adaptive
strategy based on the color similarity is formulated to select high-quality
proposals. We further integrate the proposed adaptive proposal selection
algorithm with coarse-to-fine deep features to validate the generalization and
efficiency of the proposed tracker. Experiments on two benchmark datasets
demonstrate that the proposed algorithm performs favorably against several
state-of-the-art trackers.

negative:
The ability to characterize the color content of natural imagery is an
important application of image processing. The pixel by pixel coloring of
images may be viewed naturally as points in color space, and the inherent
structure and distribution of these points affords a quantization, through
clustering, of the color information in the image. In this paper, we present a
novel topologically driven clustering algorithm that permits segmentation of
the color features in a digital image. The algorithm blends Locally Linear
Embedding (LLE) and vector quantization by mapping color information to a lower
dimensional space, identifying distinct color regions, and classifying pixels
together based on both a proximity measure and color content. It is observed
that these techniques permit a significant reduction in color resolution while
maintaining the visually important features of images.

negative:
While many image colorization algorithms have recently shown the capability
of producing plausible color versions from gray-scale photographs, they still
suffer from limited semantic understanding. To address this shortcoming, we
propose to exploit pixelated object semantics to guide image colorization. The
rationale is that human beings perceive and distinguish colors based on the
semantic categories of objects. Starting from an autoregressive model, we
generate image color distributions, from which diverse colored results are
sampled. We propose two ways to incorporate object semantics into the
colorization model: through a pixelated semantic embedding and a pixelated
semantic generator. Specifically, the proposed convolutional neural network
includes two branches. One branch learns what the object is, while the other
branch learns the object colors. The network jointly optimizes a color
embedding loss, a semantic segmentation loss and a color generation loss, in an
end-to-end fashion. Experiments on PASCAL VOC2012 and COCO-stuff reveal that
our network, when trained with semantic segmentation labels, produces more
realistic and finer results compared to the colorization state-of-the-art.

negative:
Image denoising and demosaicking are the most important early stages in
digital camera pipelines. They constitute a severely ill-posed problem that
aims at reconstructing a full color image from a noisy color filter array (CFA)
image. In most of the literature, denoising and demosaicking are treated as two
independent problems, without considering their interaction, or asking which
should be applied first. Several recent works have started addressing them
jointly in works that involve heavy weight CNNs, thus incompatible with low
power portable imaging devices. Hence, the question of how to combine denoising
and demosaicking to reconstruct full color images remains very relevant: Is
denoising to be applied first, or should that be demosaicking first? In this
paper, we review the main variants of these strategies and carry-out an
extensive evaluation to find the best way to reconstruct full color images from
a noisy mosaic. We conclude that demosaicking should applied first, followed by
denoising. Yet we prove that this requires an adaptation of classic denoising
algorithms to demosaicked noise, which we justify and specify.

negative:
Nowadays modern displays are capable to render video content with high
dynamic range (HDR) and wide color gamut (WCG). However, most available
resources are still in standard dynamic range (SDR). Therefore, there is an
urgent demand to transform existing SDR-TV contents into their HDR-TV versions.
In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the
formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step
solution pipeline including adaptive global color mapping, local enhancement
and highlight generation. Moreover, the above analysis inspires us to present a
lightweight network that utilizes global statistics as guidance to conduct
image-adaptive color mapping. In addition, we construct a dataset using HDR
videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate
the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results
achieve state-of-the-art performance in quantitative comparisons and visual
quality. The code and dataset are available at
https://github.com/chxy95/HDRTVNet.

negative:
Recent years have seen remarkable progress in semantic segmentation. Yet, it
remains a challenging task to apply segmentation techniques to video-based
applications. Specifically, the high throughput of video streams, the sheer
cost of running fully convolutional networks, together with the low-latency
requirements in many real-world applications, e.g. autonomous driving, present
a significant challenge to the design of the video segmentation framework. To
tackle this combined challenge, we develop a framework for video semantic
segmentation, which incorporates two novel components: (1) a feature
propagation module that adaptively fuses features over time via spatially
variant convolution, thus reducing the cost of per-frame computation; and (2)
an adaptive scheduler that dynamically allocate computation based on accuracy
prediction. Both components work together to ensure low latency while
maintaining high segmentation quality. On both Cityscapes and CamVid, the
proposed framework obtained competitive performance compared to the state of
the art, while substantially reducing the latency, from 360 ms to 119 ms.

negative:
Automatic document content processing is affected by artifacts caused by the
shape of the paper, non-uniform and diverse color of lighting conditions.
Fully-supervised methods on real data are impossible due to the large amount of
data needed. Hence, the current state of the art deep learning models are
trained on fully or partially synthetic images. However, document shadow or
shading removal results still suffer because: (a) prior methods rely on
uniformity of local color statistics, which limit their application on
real-scenarios with complex document shapes and textures and; (b) synthetic or
hybrid datasets with non-realistic, simulated lighting conditions are used to
train the models. In this paper we tackle these problems with our two main
contributions. First, a physically constrained learning-based method that
directly estimates document reflectance based on intrinsic image formation
which generalizes to challenging illumination conditions. Second, a new dataset
that clearly improves previous synthetic ones, by adding a large range of
realistic shading and diverse multi-illuminant conditions, uniquely customized
to deal with documents in-the-wild. The proposed architecture works in a
self-supervised manner where only the synthetic texture is used as a weak
training signal (obviating the need for very costly ground truth with
disentangled versions of shading and reflectance). The proposed approach leads
to a significant generalization of document reflectance estimation in real
scenes with challenging illumination. We extensively evaluate on the real
benchmark datasets available for intrinsic image decomposition and document
shadow removal tasks. Our reflectance estimation scheme, when used as a
pre-processing step of an OCR pipeline, shows a 26% improvement of character
error rate (CER), thus, proving the practical applicability.

negative:
Although adversarial samples of deep neural networks (DNNs) have been
intensively studied on static images, their extensions in videos are never
explored. Compared with images, attacking a video needs to consider not only
spatial cues but also temporal cues. Moreover, to improve the imperceptibility
as well as reduce the computation cost, perturbations should be added on as
fewer frames as possible, i.e., adversarial perturbations are temporally
sparse. This further motivates the propagation of perturbations, which denotes
that perturbations added on the current frame can transfer to the next frames
via their temporal interactions. Thus, no (or few) extra perturbations are
needed for these frames to misclassify them. To this end, we propose an
l2,1-norm based optimization algorithm to compute the sparse adversarial
perturbations for videos. We choose the action recognition as the targeted
task, and networks with a CNN+RNN architecture as threat models to verify our
method. Thanks to the propagation, we can compute perturbations on a shortened
version video, and then adapt them to the long version video to fool DNNs.
Experimental results on the UCF101 dataset demonstrate that even only one frame
in a video is perturbed, the fooling rate can still reach 59.7%.

negative:
We propose a new algorithm for color transfer between images that have
perceptually similar semantic structures. We aim to achieve a more accurate
color transfer that leverages semantically-meaningful dense correspondence
between images. To accomplish this, our algorithm uses neural representations
for matching. Additionally, the color transfer should be spatially variant and
globally coherent. Therefore, our algorithm optimizes a local linear model for
color transfer satisfying both local and global constraints. Our proposed
approach jointly optimizes matching and color transfer, adopting a
coarse-to-fine strategy. The proposed method can be successfully extended from
one-to-one to one-to-many color transfer. The latter further addresses the
problem of mismatching elements of the input image. We validate our proposed
method by testing it on a large variety of image content.

negative:
One of the important requirements in image retrieval, indexing,
classification, clustering and etc. is extracting efficient features from
images. The color feature is one of the most widely used visual features. Use
of color histogram is the most common way for representing color feature. One
of disadvantage of the color histogram is that it does not take the color
spatial distribution into consideration. In this paper dynamic color
distribution entropy of neighborhoods method based on color distribution
entropy is presented, which effectively describes the spatial information of
colors. The image retrieval results in compare to improved color distribution
entropy show the acceptable efficiency of this approach.

negative:
We propose a novel framework to produce cartoon videos by fetching the color
information from two input keyframes while following the animated motion guided
by a user sketch. The key idea of the proposed approach is to estimate the
dense cross-domain correspondence between the sketch and cartoon video frames,
and employ a blending module with occlusion estimation to synthesize the middle
frame guided by the sketch. After that, the input frames and the synthetic
frame equipped with established correspondence are fed into an arbitrary-time
frame interpolation pipeline to generate and refine additional inbetween
frames. Finally, a module to preserve temporal consistency is employed.
Compared to common frame interpolation methods, our approach can address frames
with relatively large motion and also has the flexibility to enable users to
control the generated video sequences by editing the sketch guidance. By
explicitly considering the correspondence between frames and the sketch, we can
achieve higher quality results than other image synthesis methods. Our results
show that our system generalizes well to different movie frames, achieving
better results than existing solutions.

negative:
Evaluation of automatic video summaries is a challenging problem. In the past
years, some evaluation methods are presented that utilize only a single feature
like color feature to detect similarity between automatic video summaries and
ground-truth user summaries. One of the drawbacks of using a single feature is
that sometimes it gives a false similarity detection which makes the assessment
of the quality of the generated video summary less perceptual and not accurate.
In this paper, a novel method for evaluating automatic video summaries is
presented. This method is based on comparing automatic video summaries
generated by video summarization techniques with ground-truth user summaries.
The objective of this evaluation method is to quantify the quality of video
summaries, and allow comparing different video summarization techniques
utilizing both color and texture features of the video frames and using the
Bhattacharya distance as a dissimilarity measure due to its advantages. Our
Experiments show that the proposed evaluation method overcomes the drawbacks of
other methods and gives a more perceptual evaluation of the quality of the
automatic video summaries.

negative:
Matching people across multiple camera views known as person
re-identification, is a challenging problem due to the change in visual
appearance caused by varying lighting conditions. The perceived color of the
subject appears to be different with respect to illumination. Previous works
use color as it is or address these challenges by designing color spaces
focusing on a specific cue. In this paper, we propose a data driven approach
for learning color patterns from pixels sampled from images across two camera
views. The intuition behind this work is that, even though pixel values of same
color would be different across views, they should be encoded with the same
values. We model color feature generation as a learning problem by jointly
learning a linear transformation and a dictionary to encode pixel values. We
also analyze different photometric invariant color spaces. Using color as the
only cue, we compare our approach with all the photometric invariant color
spaces and show superior performance over all of them. Combining with other
learned low-level and high-level features, we obtain promising results in
ViPER, Person Re-ID 2011 and CAVIAR4REID datasets.

negative:
Object detection in videos has drawn increasing attention since it is more
practical in real scenarios. Most of the deep learning methods use CNNs to
process each decoded frame in a video stream individually. However, the free of
charge yet valuable motion information already embedded in the video
compression format is usually overlooked. In this paper, we propose a fast
object detection method by taking advantage of this with a novel Motion aided
Memory Network (MMNet). The MMNet has two major advantages: 1) It significantly
accelerates the procedure of feature extraction for compressed videos. It only
need to run a complete recognition network for I-frames, i.e. a few reference
frames in a video, and it produces the features for the following P frames
(predictive frames) with a light weight memory network, which runs fast; 2)
Unlike existing methods that establish an additional network to model motion of
frames, we take full advantage of both motion vectors and residual errors that
are freely available in video streams. To our best knowledge, the MMNet is the
first work that investigates a deep convolutional detector on compressed
videos. Our method is evaluated on the large-scale ImageNet VID dataset, and
the results show that it is 3x times faster than single image detector R-FCN
and 10x times faster than high-performance detector MANet at a minor accuracy
loss.

negative:
Semantic video segmentation is challenging due to the sheer amount of data
that needs to be processed and labeled in order to construct accurate models.
In this paper we present a deep, end-to-end trainable methodology to video
segmentation that is capable of leveraging information present in unlabeled
data in order to improve semantic estimates. Our model combines a convolutional
architecture and a spatio-temporal transformer recurrent layer that are able to
temporally propagate labeling information by means of optical flow, adaptively
gated based on its locally estimated uncertainty. The flow, the recognition and
the gated temporal propagation modules can be trained jointly, end-to-end. The
temporal, gated recurrent flow propagation component of our model can be
plugged into any static semantic segmentation architecture and turn it into a
weakly supervised video processing one. Our extensive experiments in the
challenging CityScapes and Camvid datasets, and based on multiple deep
architectures, indicate that the resulting model can leverage unlabeled
temporal frames, next to a labeled one, in order to improve both the video
segmentation accuracy and the consistency of its temporal labeling, at no
additional annotation cost and with little extra computation.

negative:
Several researches in leaf identification did not include color information
as features. The main reason is caused by a fact that they used green colored
leaves as samples. However, for foliage plants, plants with colorful leaves,
fancy patterns in their leaves, and interesting plants with unique shape, color
and also texture could not be neglected. For example, Epipremnum pinnatum
'Aureum' and Epipremnum pinnatum 'Marble Queen' have similar patterns, same
shape, but different colors. Combination of shape, color, texture features, and
other attribute contained on the leaf is very useful in leaf identification. In
this research, Polar Fourier Transform and three kinds of geometric features
were used to represent shape features, color moments that consist of mean,
standard deviation, skewness were used to represent color features, texture
features are extracted from GLCMs, and vein features were added to improve
performance of the identification system. The identification system uses
Probabilistic Neural Network (PNN) as a classifier. The result shows that the
system gives average accuracy of 93.0833% for 60 kinds of foliage plants.

negative:
By assuming that the most frequently occuring color in a video or a region of
a video I propose a new algorithm for detecting foreground objects in a video.
The process of detecting the foreground objects is complicated because of the
fact that there may be swaying trees, objects of the background being moved
around or lighting changes in the video. To deal with such complexities many
have come up with solutions which heavily rely on expensive floating point
operations. In this paper I used a data structure called Octree which is
implemented only using binary operations. Traditionally octrees were used for
color quantization but here in this paper I used it as a data structure to
store the most frequently occuring colors in a video as well. For each of the
starting few video frames, I constructed a Octree using all the colors of that
frame. Next I pruned all the trees by removing nodes below a certain height and
gave the leaf nodes a color which is dependant on the topological path from
that node to its parent. Hence any two leaf nodes in two different octrees with
the same topological path from themselves to the root will represent the same
color. Next I merged all these individual trees into a single tree retaining
only those nodes whose topological path to itself from the root is most common
among all the trees. The colors represented by the leaf nodes in the resultant
tree will be the most frequently occuring colors in the starting video frames
of the video. Hence any color of an incomming frame that is not close to any of
the colors represented by the leaf node of the merged tree can be regarded as
belonging to a foreground object.
  As an Octree is constructed using only binary operations, it is very fast
compared to other leading algorithms.

negative:
Semi-supervised video object segmentation aims to separate a target object
from a video sequence, given the mask in the first frame. Most of current
prevailing methods utilize information from additional modules trained in other
domains like optical flow and instance segmentation, and as a result they do
not compete with other methods on common ground. To address this issue, we
propose a simple yet strong transductive method, in which additional modules,
datasets, and dedicated architectural designs are not needed. Our method takes
a label propagation approach where pixel labels are passed forward based on
feature similarity in an embedding space. Different from other propagation
methods, ours diffuses temporal information in a holistic manner which take
accounts of long-term object appearance. In addition, our method requires few
additional computational overhead, and runs at a fast $\sim$37 fps speed. Our
single model with a vanilla ResNet50 backbone achieves an overall score of 72.3
on the DAVIS 2017 validation set and 63.1 on the test set. This simple yet high
performing and efficient method can serve as a solid baseline that facilitates
future research. Code and models are available at
\url{https://github.com/microsoft/transductive-vos.pytorch}.

negative:
This paper proposes an automatic image correction method for portrait
photographs, which promotes consistency of facial skin color by suppressing
skin color changes due to background colors. In portrait photographs, skin
color is often distorted due to the lighting environment (e.g., light reflected
from a colored background wall and over-exposure by a camera strobe), and if
the photo is artificially combined with another background color, this color
change is emphasized, resulting in an unnatural synthesized result. In our
framework, after roughly extracting the face region and rectifying the skin
color distribution in a color space, we perform color and brightness correction
around the face in the original image to achieve a proper color balance of the
facial image, which is not affected by luminance and background colors. Unlike
conventional algorithms for color correction, our final result is attained by a
color correction process with a guide image. In particular, our guided image
filtering for the color correction does not require a perfectly-aligned guide
image required in the original guide image filtering method proposed by He et
al. Experimental results show that our method generates more natural results
than conventional methods on not only headshot photographs but also natural
scene photographs. We also show automatic yearbook style photo generation as an
another application.

negative:
We present a novel algorithm for transferring artistic styles of semantically
meaningful local regions of an image onto local regions of a target video while
preserving its photorealism. Local regions may be selected either fully
automatically from an image, through using video segmentation algorithms, or
from casual user guidance such as scribbles. Our method, based on a deep neural
network architecture inspired by recent work in photorealistic style transfer,
is real-time and works on arbitrary inputs without runtime optimization once
trained on a diverse dataset of artistic styles. By augmenting our video
dataset with noisy semantic labels and jointly optimizing over style, content,
mask, and temporal losses, our method can cope with a variety of imperfections
in the input and produce temporally coherent videos without visual artifacts.
We demonstrate our method on a variety of style images and target videos,
including the ability to transfer different styles onto multiple objects
simultaneously, and smoothly transition between styles in time.

negative:
In this work we discuss the known algorithms for linear colour segmentation
based on a physical approach and propose a new modification of segmentation
algorithm. This algorithm is based on a region adjacency graph framework
without a pre-segmentation stage. Proposed edge weight functions are defined
from linear image model with normal noise. The colour space projective
transform is introduced as a novel pre-processing technique for better handling
of shadow and highlight areas. The resulting algorithm is tested on a benchmark
dataset consisting of the images of 19 natural scenes selected from the
Barnard's DXC-930 SFU dataset and 12 natural scene images newly published for
common use. The dataset is provided with pixel-by-pixel ground truth colour
segmentation for every image. Using this dataset, we show that the proposed
algorithm modifications lead to qualitative advantages over other model-based
segmentation algorithms, and also show the positive effect of each proposed
modification. The source code and datasets for this work are available for free
access at http://github.com/visillect/segmentation.

negative:
In this paper, we proposed three methods to solve color recognition of
Rubik's cube, which includes one offline method and two online methods. Scatter
balance \& extreme learning machine (SB-ELM), a offline method, is proposed to
illustrate the efficiency of training based method. We also point out the
conception of color drifting which indicates offline methods are always
ineffectiveness and can not work well in continuous change circumstance. By
contrast, dynamic weight label propagation is proposed for labeling blocks
color by known center blocks color of Rubik's cube. Furthermore, weak label
hierarchic propagation, another online method, is also proposed for unknown all
color information but only utilizes weak label of center block in color
recognition. We finally design a Rubik's cube robot and construct a dataset to
illustrate the efficiency and effectiveness of our online methods and to
indicate the ineffectiveness of offline method by color drifting in our
dataset.

negative:
Decolorization is the process to convert a color image or video to its
grayscale version, and it has received great attention in recent years. An
ideal decolorization algorithm should preserve the original color contrast as
much as possible. Meanwhile, it should provide the final decolorized result as
fast as possible. However, most of the current methods are suffering from
either unsatisfied color information preservation or high computational cost,
limiting their application value. In this paper, a simple but effective
technique is proposed for real-time decolorization. Based on the typical
rgb2gray() color conversion model, which produces a grayscale image by linearly
combining R, G, and B channels, we propose a dominant color hypothesis and a
corresponding distance measurement metric to evaluate the quality of grayscale
conversion. The local optimum scheme provides several "good" candidates in a
confidence interval, from which the "best" result can be extracted.
Experimental results demonstrate that remarkable simplicity of the proposed
method facilitates the process of high resolution images and videos in
real-time using a common CPU.

negative:
Information about the illuminant color is well contained in both achromatic
regions and the specular components of highlight regions. In this paper, we
propose a novel way to achieve color constancy by exploiting such clues. The
key to our approach lies in the use of suitably extracted derivative colors,
which are able to compute the illuminant color robustly with kernel density
estimation. While extracting derivative colors from achromatic regions to
approximate the illuminant color well is basically straightforward, the success
of our extraction in highlight regions is attributed to the different rates of
variation of the diffuse and specular magnitudes in the dichromatic reflection
model. The proposed approach requires no training phase and is simple to
implement. More significantly, it performs quite satisfactorily under
inter-database parameter settings. Our experiments on three standard databases
demonstrate its effectiveness and fine performance in comparison to
state-of-the-art methods.

negative:
In this paper, we investigate the effect of color space selection on
detectability and discriminability of colored objects under various conditions.
20 color spaces from the literature are evaluated on a large dataset of
simulated and real images. We measure the suitability of color spaces from two
different perspectives: detectability and discriminability of various color
groups. Through experimental evaluation, we found that there is no single
optimal color space suitable for all color groups. The color spaces have
different levels of sensitivity to different color groups and they are useful
depending on the color of the sought object. Overall, the best results were
achieved in both simulated and real images using color spaces C1C2C3, UVW and
XYZ. In addition, using a simulated environment, we show a practical
application of color space selection in the context of top-down control in
active visual search. The results indicate that on average color space C1C2C3
followed by HSI and XYZ achieve the best time in searching for objects of
various colors. Here, the right choice of color space can improve time of
search on average by 20%. As part of our contribution, we also introduce a
large dataset of simulated 3D objects

negative:
Recent state-of-the-art video generation systems employ Generative
Adversarial Networks (GANs) or Variational Autoencoders (VAEs) to produce novel
videos. However, VAE models typically produce blurry outputs when faced with
sub-optimal conditioning of the input, and GANs are known to be unstable for
large output sizes. In addition, the output videos of these models are
difficult to evaluate, partly because the GAN loss function is not an accurate
measure of convergence. In this work, we propose using a state-of-the-art
neural flow generator called Glow to generate videos conditioned on a textual
label, one frame at a time. Neural flow models are more stable than standard
GANs, as they only optimize a single cross entropy loss function, which is
monotonic and avoids the circular convergence issues of the GAN minimax
objective. In addition, we also show how to condition Glow on external context,
while still preserving the invertible nature of each "flow" layer. Finally, we
evaluate the proposed Glow model by calculating cross entropy on a held-out
validation set of videos, in order to compare multiple versions of the proposed
model via an ablation study. We show generated videos and discuss future
improvements.

negative:
Color transfer between images uses the statistics information of image
effectively. We present a novel approach of local color transfer between images
based on the simple statistics and locally linear embedding. A sketching
interface is proposed for quickly and easily specifying the color
correspondences between target and source image. The user can specify the
correspondences of local region using scribes, which more accurately transfers
the target color to the source image while smoothly preserving the boundaries,
and exhibits more natural output results. Our algorithm is not restricted to
one-to-one image color transfer and can make use of more than one target images
to transfer the color in different regions in the source image. Moreover, our
algorithm does not require to choose the same color style and image size
between source and target images. We propose the sub-sampling to reduce the
computational load. Comparing with other approaches, our algorithm is much
better in color blending in the input data. Our approach preserves the other
color details in the source image. Various experimental results show that our
approach specifies the correspondences of local color region in source and
target images. And it expresses the intention of users and generates more
actual and natural results of visual effect.

negative:
Research on the detection of AI-generated videos has focused almost
exclusively on face videos, usually referred to as deepfakes. Manipulations
like face swapping, face reenactment and expression manipulation have been the
subject of an intense research with the development of a number of efficient
tools to distinguish artificial videos from genuine ones. Much less attention
has been paid to the detection of artificial non-facial videos. Yet, new tools
for the generation of such kind of videos are being developed at a fast pace
and will soon reach the quality level of deepfake videos. The goal of this
paper is to investigate the detectability of a new kind of AI-generated videos
framing driving street sequences (here referred to as DeepStreets videos),
which, by their nature, can not be analysed with the same tools used for facial
deepfakes. Specifically, we present a simple frame-based detector, achieving
very good performance on state-of-the-art DeepStreets videos generated by the
Vid2vid architecture. Noticeably, the detector retains very good performance on
compressed videos, even when the compression level used during training does
not match that used for the test videos.

negative:
We consider the cross-modal task of producing color representations for text
phrases. Motivated by the fact that a significant fraction of user queries on
an image search engine follow an (attribute, object) structure, we propose a
generative adversarial network that generates color profiles for such bigrams.
We design our pipeline to learn composition - the ability to combine seen
attributes and objects to unseen pairs. We propose a novel dataset curation
pipeline from existing public sources. We describe how a set of phrases of
interest can be compiled using a graph propagation technique, and then mapped
to images. While this dataset is specialized for our investigations on color,
the method can be extended to other visual dimensions where composition is of
interest. We provide detailed ablation studies that test the behavior of our
GAN architecture with loss functions from the contrastive learning literature.
We show that the generative model achieves lower Frechet Inception Distance
than discriminative ones, and therefore predicts color profiles that better
match those from real images. Finally, we demonstrate improved performance in
image retrieval and classification, indicating the crucial role that color
plays in these downstream tasks.

negative:
Mappings between color spaces are ubiquitous in image processing problems
such as gamut mapping, decolorization, and image optimization for color-blind
people. Simple color transformations often result in information loss and
ambiguities (for example, when mapping from RGB to grayscale), and one wishes
to find an image-specific transformation that would preserve as much as
possible the structure of the original image in the target color space. In this
paper, we propose Laplacian colormaps, a generic framework for
structure-preserving color transformations between images. We use the image
Laplacian to capture the structural information, and show that if the color
transformation between two images preserves the structure, the respective
Laplacians have similar eigenvectors, or in other words, are approximately
jointly diagonalizable. Employing the relation between joint diagonalizability
and commutativity of matrices, we use Laplacians commutativity as a criterion
of color mapping quality and minimize it w.r.t. the parameters of a color
transformation to achieve optimal structure preservation. We show numerous
applications of our approach, including color-to-gray conversion, gamut
mapping, multispectral image fusion, and image optimization for color deficient
viewers.

negative:
Most of the existing works in video synthesis focus on generating videos
using adversarial learning. Despite their success, these methods often require
input reference frame or fail to generate diverse videos from the given data
distribution, with little to no uniformity in the quality of videos that can be
generated. Different from these methods, we focus on the problem of generating
videos from latent noise vectors, without any reference input frames. To this
end, we develop a novel approach that jointly optimizes the input latent space,
the weights of a recurrent neural network and a generator through
non-adversarial learning. Optimizing for the input latent space along with the
network weights allows us to generate videos in a controlled environment, i.e.,
we can faithfully generate all videos the model has seen during the learning
process as well as new unseen videos. Extensive experiments on three
challenging and diverse datasets well demonstrate that our approach generates
superior quality videos compared to the existing state-of-the-art methods.

negative:
With the dominance of keyframe-based SLAM in the field of robotics, the
relative frame poses between keyframes have typically been sacrificed for a
faster algorithm to achieve online applications. However, those approaches can
become insufficient for applications that may require refined poses of all
frames, not just keyframes which are relatively sparse compared to all input
frames. This paper proposes a novel algorithm to correct the relative frames
between keyframes after the keyframes have been updated by a back-end
optimization process. The correction model is derived using conservation of the
measurement constraint between landmarks and the robot pose. The proposed
algorithm is designed to be easily integrable to existing keyframe-based SLAM
systems while exhibiting robust and accurate performance superior to existing
interpolation methods. The algorithm also requires low computational resources
and hence has a minimal burden on the whole SLAM pipeline. We provide the
evaluation of the proposed pose correction algorithm in comparison to existing
interpolation methods in various vector spaces, and our method has demonstrated
excellent accuracy in both KITTI and EuRoC datasets.

negative:
Portrait photo retouching is a photo retouching task that emphasizes
human-region priority and group-level consistency. The lookup table-based
method achieves promising retouching performance by learning image-adaptive
weights to combine 3-dimensional lookup tables (3D LUTs) and conducting
pixel-to-pixel color transformation. However, this paradigm ignores the local
context cues and applies the same transformation to portrait pixels and
background pixels when they exhibit the same raw RGB values. In contrast, an
expert usually conducts different operations to adjust the color temperatures
and tones of portrait regions and background regions. This inspires us to model
local context cues to improve the retouching quality explicitly. Firstly, we
consider an image patch and predict pixel-adaptive lookup table weights to
precisely retouch the center pixel. Secondly, as neighboring pixels exhibit
different affinities to the center pixel, we estimate a local attention mask to
modulate the influence of neighboring pixels. Thirdly, the quality of the local
attention mask can be further improved by applying supervision, which is based
on the affinity map calculated by the groundtruth portrait mask. As for
group-level consistency, we propose to directly constrain the variance of mean
color components in the Lab space. Extensive experiments on PPR10K dataset
verify the effectiveness of our method, e.g. on high-resolution photos, the
PSNR metric receives over 0.5 gains while the group-level consistency metric
obtains at least 2.1 decreases.

negative:
In this paper the use of Random Sprays Retinex (RSR) algorithm for global
illumination estimation is proposed and its feasibility tested. Like other
algorithms based on the Retinex model, RSR also provides local illumination
estimation and brightness adjustment for each pixel and it is faster than other
path-wise Retinex algorithms. As the assumption of the uniform illumination
holds in many cases, it should be possible to use the mean of local
illumination estimations of RSR as a global illumination estimation for images
with (assumed) uniform illumination allowing also the accuracy to be easily
measured. Therefore we propose a method for estimating global illumination
estimation based on local RSR results. To our best knowledge this is the first
time that RSR algorithm is used to obtain global illumination estimation. For
our tests we use a publicly available color constancy image database for
testing. The results are presented and discussed and it turns out that the
proposed method outperforms many existing unsupervised color constancy
algorithms. The source code is available at
http://www.fer.unizg.hr/ipg/resources/color_constancy/.

negative:
A novel method to convert color/multi-spectral images to gray-level images is
introduced to increase the performance of document binarization methods. The
method uses the distribution of the pixel data of the input document image in a
color space to find a transformation, called the dual transform, which balances
the amount of information on all color channels. Furthermore, in order to
reduce the intensity variations on the gray output, a color reduction
preprocessing step is applied. Then, a channel is selected as the gray value
representation of the document image based on the homogeneity criterion on the
text regions. In this way, the proposed method can provide a
luminance-independent contrast enhancement. The performance of the method is
evaluated against various images from two databases, the ICDAR'03 Robust
Reading, the KAIST and the DIBCO'09 datasets, subjectively and objectively with
promising results. The ground truth images for the images from the ICDAR'03
Robust Reading dataset have been created manually by the authors.

negative:
Thumbnail is the face of online videos. The explosive growth of videos both
in number and variety underpins the importance of a good thumbnail because it
saves potential viewers time to choose videos and even entice them to click on
them. A good thumbnail should be a frame that best represents the content of a
video while at the same time capturing viewers' attention. However, the
techniques and models in the past only focus on frames within a video, and we
believe such narrowed focus leave out much useful information that are part of
a video. In this paper, we expand the definition of content to include title,
description, and audio of a video and utilize information provided by these
modalities in our selection model. Specifically, our model will first sample
frames uniformly in time and return the top 1,000 frames in this subset with
the highest aesthetic scores by a Double-column Convolutional Neural Network,
to avoid the computational burden of processing all frames in downstream task.
Then, the model incorporates frame features extracted from VGG16, text features
from ELECTRA, and audio features from TRILL. These models were selected because
of their results on popular datasets as well as their competitive performances.
After feature extraction, the time-series features, frames and audio, will be
fed into Transformer encoder layers to return a vector representing their
corresponding modality. Each of the four features (frames, title, description,
audios) will pass through a context gating layer before concatenation. Finally,
our model will generate a vector in the latent space and select the frame that
is most similar to this vector in the latent space. To the best of our
knowledge, we are the first to propose a multi-modal deep learning model to
select video thumbnail, which beats the result from the previous
State-of-The-Art models.

negative:
Image colorization is the process of estimating RGB colors for grayscale
images or video frames to improve their aesthetic and perceptual quality. Deep
learning techniques for image colorization have progressed notably over the
last decade, calling the need for a systematic survey and benchmarking of these
techniques. This article presents a comprehensive survey of recent
state-of-the-art deep learning-based image colorization techniques, describing
their fundamental block architectures, inputs, optimizers, loss functions,
training protocols, and training data \textit{etc.} It categorizes the existing
colorization techniques into seven classes and discusses important factors
governing their performance, such as benchmark datasets and evaluation metrics.
We highlight the limitations of existing datasets and introduce a new dataset
specific to colorization. Using the existing datasets and our new one, we
perform an extensive experimental evaluation of existing image colorization
methods. Finally, we discuss the limitations of existing methods and recommend
possible solutions as well as future research directions for this rapidly
evolving topic of deep image colorization. Dataset and codes for evaluation are
publicly available at https://github.com/saeed-anwar/ColorSurvey

negative:
Color modelling and extraction is an important topic in fashion, art, and
design. Recommender systems, color-based retrieval, decorating, and fashion
design can benefit from color extraction tools. Research has shown that
modeling color so that it can be automatically analyzed and / or extracted is a
difficult task. Unlike machines, color perception, although very subjective, is
much simpler for humans. That being said, the first step in color modeling is
to estimate the number of colors in the item / object. This is because color
models can take advantage of the number of colors as the seed for better
modelling, e.g., to make color extraction further deterministic. We aim in this
work to develop and test models that can count the number of colors of clothing
and other items. We propose a novel color counting method based on cumulative
color histogram, which stands out among other methods. We compare the method we
propose with other methods that utilize exhaustive color search that uses
Gaussian Mixture Models (GMMs) and K-Means as bases for scoring the optimal
number of colors, in addition to another method that relies on deep learning
models. Unfortunately, the GMM, K-Means, and Deep Learning models all fail to
accurately capture the number of colors. Our proposed method can provide the
color baseline that can be used in AI-based fashion applications, and can also
find applications in other areas, for example, interior design. To the best of
our knowledge, this work is the first of its kind that addresses the problem of
color-counting machine.

negative:
Color texture representation is an important step in the task of texture
classification. Shortest paths was used to extract color texture features from
RGB and HSV color spaces. In this paper, we propose to use shortest paths in
the HSI space to build a texture representation for classification. In
particular, two undirected graphs are used to model the H channel and the S and
I channels respectively in order to represent a color texture image. Moreover,
the shortest paths is constructed by using four pairs of pixels according to
different scales and directions of the texture image. Experimental results on
colored Brodatz and USPTex databases reveal that our proposed method is
effective, and the highest classification accuracy rate is 96.93% in the
Brodatz database.

negative:
Although many methods have been proposed to deal with nature image
super-resolution (SR) and get impressive performance, the text images SR is not
good due to their ignorance of document images. In this paper, we propose a
matting-based dual generative adversarial network (mdGAN) for document image
SR. Firstly, the input image is decomposed into document text, foreground and
background layers using deep image matting. Then two parallel branches are
constructed to recover text boundary information and color information
respectively. Furthermore, in order to improve the restoration accuracy of
characters in output image, we use the input image's corresponding ground truth
text label as extra supervise information to refine the two-branch networks
during training. Experiments on real text images demonstrate that our method
outperforms several state-of-the-art methods quantitatively and qualitatively.

negative:
We propose the first approach for the decomposition of a monocular color
video into direct and indirect illumination components in real time. We
retrieve, in separate layers, the contribution made to the scene appearance by
the scene reflectance, the light sources and the reflections from various
coherent scene regions to one another. Existing techniques that invert global
light transport require image capture under multiplexed controlled lighting, or
only enable the decomposition of a single image at slow off-line frame rates.
In contrast, our approach works for regular videos and produces temporally
coherent decomposition layers at real-time frame rates. At the core of our
approach are several sparsity priors that enable the estimation of the
per-pixel direct and indirect illumination layers based on a small set of
jointly estimated base reflectance colors. The resulting variational
decomposition problem uses a new formulation based on sparse and dense sets of
non-linear equations that we solve efficiently using a novel alternating
data-parallel optimization strategy. We evaluate our approach qualitatively and
quantitatively, and show improvements over the state of the art in this field,
in both quality and runtime. In addition, we demonstrate various real-time
appearance editing applications for videos with consistent illumination.

negative:
In real-world applications, e.g. law enforcement and video retrieval, one
often needs to search a certain person in long videos with just one portrait.
This is much more challenging than the conventional settings for person
re-identification, as the search may need to be carried out in the environments
different from where the portrait was taken. In this paper, we aim to tackle
this challenge and propose a novel framework, which takes into account the
identity invariance along a tracklet, thus allowing person identities to be
propagated via both the visual and the temporal links. We also develop a novel
scheme called Progressive Propagation via Competitive Consensus, which
significantly improves the reliability of the propagation process. To promote
the study of person search, we construct a large-scale benchmark, which
contains 127K manually annotated tracklets from 192 movies. Experiments show
that our approach remarkably outperforms mainstream person re-id methods,
raising the mAP from 42.16% to 62.27%.

negative:
We present color sails, a discrete-continuous color gamut representation that
extends the color gradient analogy to three dimensions and allows interactive
control of the color blending behavior. Our representation models a wide
variety of color distributions in a compact manner, and lends itself to
applications such as color exploration for graphic design, illustration and
similar fields. We propose a Neural Network that can fit a color sail to any
image. Then, the user can adjust color sail parameters to change the base
colors, their blending behavior and the number of colors, exploring a wide
range of options for the original design. In addition, we propose a Deep
Learning model that learns to automatically segment an image into
color-compatible alpha masks, each equipped with its own color sail. This
allows targeted color exploration by either editing their corresponding color
sails or using standard software packages. Our model is trained on a custom
diverse dataset of art and design. We provide both quantitative evaluations,
and a user study, demonstrating the effectiveness of color sail interaction.
Interactive demos are available at www.colorsails.com.

negative:
We use methods from Riemann geometry to investigate transformations between
the color spaces of color-normal and color weak observers. The two main
applications are the simulation of the perception of a color weak observer for
a color normal observer and the compensation of color images in a way that a
color weak observer has approximately the same perception as a color normal
observer. The metrics in the color spaces of interest are characterized with
the help of ellipsoids defined by the just-noticable-differences between color
which are measured with the help of color-matching experiments. The constructed
mappings are isometries of Riemann spaces that preserve the perceived
color-differences for both observers. Among the two approaches to build such an
isometry, we introduce normal coordinates in Riemann spaces as a tool to
construct a global color-weak compensation map. Compared to previously used
methods this method is free from approximation errors due to local
linearizations and it avoids the problem of shifting locations of the origin of
the local coordinate system. We analyse the variations of the Riemann metrics
for different observers obtained from new color matching experiments and
describe three variations of the basic method. The performance of the methods
is evaluated with the help of semantic differential (SD) tests.

negative:
Color in an image is resolved into 3 or 4 color components and 2-Dimages of
these components are stored in separate channels. Most of the color image
enhancement algorithms are applied channel-by-channel on each image. But such a
system of color image processing is not processing the original color. When a
color image is represented as a quaternion image, processing is done in
original colors. This paper proposes an implementation of the quaternion
approach of enhancement algorithm for enhancing color images and is referred as
the modified alpha-rooting by the two-dimensional quaternion discrete Fourier
transform (2-D QDFT). Enhancement results of this proposed method are compared
with the channel-by-channel image enhancement by the 2-D DFT. Enhancements in
color images are quantitatively measured by the color enhancement measure
estimation (CEME), which allows for selecting optimum parameters for processing
by the genetic algorithm. Enhancement of color images by the quaternion based
method allows for obtaining images which are closer to the genuine
representation of the real original color.

negative:
Now that everyone can easily record videos, the quantity of which is
continuously increasing, research on methods for improved video retrieval is
important in the contemporary world. In cases where target videos are to be
identified within a large collection gathered by individuals, the appropriate
information must be obtained to retrieve the correct video within a large
number of similar items in the target database. The purpose of this research is
to retrieve target videos in such cases by introducing an interaction, or a
dialog, between the system and the user. We propose a system to retrieve videos
by asking questions about the content of the videos and leveraging the user's
responses to the questions. Additionally, we confirmed the usefulness of the
proposed system through experiments using the dataset called AVSD which
includes videos and dialogs about the videos.

negative:
Video prediction, forecasting the future frames from a sequence of input
frames, is a challenging task since the view changes are influenced by various
factors, such as the global context surrounding the scene and local motion
dynamics. In this paper, we propose a new framework to integrate these
complementary attributes to predict complex pixel dynamics through deep
networks. We present global context propagation networks that iteratively
aggregate the non-local neighboring representations to preserve the contextual
information over the past frames. To capture the local motion pattern of
objects, we also devise local filter memory networks that generate adaptive
filter kernels by storing the prototypical motion of moving objects in the
memory. The proposed framework, utilizing the outputs from both networks, can
address blurry predictions and color distortion. We conduct experiments on
Caltech pedestrian and UCF101 datasets, and demonstrate state-of-the-art
results. Especially for multi-step prediction, we obtain an outstanding
performance in quantitative and qualitative evaluation.

negative:
Plant aliveness is proven through laboratory experiments and special
scientific instruments. In this paper, we aim to detect the degree of animation
of plants based on the magnification of the small color changes in the plant's
green leaves using the Eulerian video magnification. Capturing the video under
a controlled environment, e.g., using a tripod and direct current (DC) light
sources, reduces camera movements and minimizes light fluctuations; we aim to
reduce the external factors as much as possible. The acquired video is then
stabilized and a proposed algorithm used to reduce the illumination variations.
Lastly, the Euler magnification is utilized to magnify the color changes on the
light invariant video. The proposed system does not require any special purpose
instruments as it uses a digital camera with a regular frame rate. The results
of magnified color changes on both natural and plastic leaves show that the
live green leaves have color changes in contrast to the plastic leaves. Hence,
we can argue that the color changes of the leaves are due to biological
operations, such as photosynthesis. To date, this is possibly the first work
that focuses on interpreting visually, some biological operations of plants
without any special purpose instruments.

negative:
Creating and labelling datasets of videos for use in training Human Activity
Recognition models is an arduous task. In this paper, we approach this by using
3D rendering tools to generate a synthetic dataset of videos, and show that a
classifier trained on these videos can generalise to real videos. We use five
different augmentation techniques to generate the videos, leading to a wide
variety of accurately labelled unique videos. We fine tune a pre-trained I3D
model on our videos, and find that the model is able to achieve a high accuracy
of 73% on the HMDB51 dataset over three classes. We also find that augmenting
the HMDB training set with our dataset provides a 2% improvement in the
performance of the classifier. Finally, we discuss possible extensions to the
dataset, including virtual try on and modeling motion of the people.

negative:
The majority of existing color naming methods focuses on the eleven basic
color terms of the English language. However, in many applications, different
sets of color names are used for the accurate description of objects. Labeling
data to learn these domain-specific color names is an expensive and laborious
task. Therefore, in this article we aim to learn color names from weakly
labeled data. For this purpose, we add an attention branch to the color naming
network. The attention branch is used to modulate the pixel-wise color naming
predictions of the network. In experiments, we illustrate that the attention
branch correctly identifies the relevant regions. Furthermore, we show that our
method obtains state-of-the-art results for pixel-wise and image-wise
classification on the EBAY dataset and is able to learn color names for various
domains.

negative:
Universal style transfer aims to transfer arbitrary visual styles to content
images. Existing feed-forward based methods, while enjoying the inference
efficiency, are mainly limited by inability of generalizing to unseen styles or
compromised visual quality. In this paper, we present a simple yet effective
method that tackles these limitations without training on any pre-defined
styles. The key ingredient of our method is a pair of feature transforms,
whitening and coloring, that are embedded to an image reconstruction network.
The whitening and coloring transforms reflect a direct matching of feature
covariance of the content image to a given style image, which shares similar
spirits with the optimization of Gram matrix based cost in neural style
transfer. We demonstrate the effectiveness of our algorithm by generating
high-quality stylized images with comparisons to a number of recent methods. We
also analyze our method by visualizing the whitened features and synthesizing
textures via simple feature coloring.

negative:
In this paper, we present a color transfer algorithm to colorize a broad
range of gray images without any user intervention. The algorithm uses a
machine learning-based approach to automatically colorize grayscale images. The
algorithm uses the superpixel representation of the reference color images to
learn the relationship between different image features and their corresponding
color values. We use this learned information to predict the color value of
each grayscale image superpixel. As compared to processing individual image
pixels, our use of superpixels helps us to achieve a much higher degree of
spatial consistency as well as speeds up the colorization process. The
predicted color values of the gray-scale image superpixels are used to provide
a 'micro-scribble' at the centroid of the superpixels. These color scribbles
are refined by using a voting based approach. To generate the final
colorization result, we use an optimization-based approach to smoothly spread
the color scribble across all pixels within a superpixel. Experimental results
on a broad range of images and the comparison with existing state-of-the-art
colorization methods demonstrate the greater effectiveness of the proposed
algorithm.

negative:
Colors play a particularly important role in both designing and accessing Web
pages. A well-designed color scheme improves Web pages' visual aesthetic and
facilitates user interactions. As far as we know, existing color assessment
studies focus on images; studies on color assessment and editing for Web pages
are rare. This paper investigates color assessment for Web pages based on
existing online color theme-rating data sets and applies this assessment to Web
color edit. This study consists of three parts. First, we study the extraction
of a Web page's color theme. Second, we construct color assessment models that
score the color compatibility of a Web page by leveraging machine learning
techniques. Third, we incorporate the learned color assessment model into a new
application, namely, color transfer for Web pages. Our study combines
techniques from computer graphics, Web mining, computer vision, and machine
learning. Experimental results suggest that our constructed color assessment
models are effective, and useful in the color transfer for Web pages, which has
received little attention in both Web mining and computer graphics communities.

negative:
We suggest a new color distance based on two observations. First, perceptual
color differences were designed to be used to compare very similar colors. They
do not capture human perception for medium and large color differences well.
Thresholding was proposed to solve the problem for large color differences,
i.e. two totally different colors are always the same distance apart. We show
that thresholding alone cannot improve medium color differences. We suggest to
alleviate this problem using basic color terms. Second, when a color distance
is used for edge detection, many small distances around the just noticeable
difference may account for false edges. We suggest to reduce the effect of
small distances.

negative:
Manually re-drawing an image in a certain artistic style takes a professional
artist a long time. Doing this for a video sequence single-handedly is beyond
imagination. We present two computational approaches that transfer the style
from one image (for example, a painting) to a whole video sequence. In our
first approach, we adapt to videos the original image style transfer technique
by Gatys et al. based on energy minimization. We introduce new ways of
initialization and new loss functions to generate consistent and stable
stylized video sequences even in cases with large motion and strong occlusion.
Our second approach formulates video stylization as a learning problem. We
propose a deep network architecture and training procedures that allow us to
stylize arbitrary-length videos in a consistent and stable way, and nearly in
real time. We show that the proposed methods clearly outperform simpler
baselines both qualitatively and quantitatively. Finally, we propose a way to
adapt these approaches also to 360 degree images and videos as they emerge with
recent virtual reality hardware.

negative:
In recent years, recognition of text from natural scene image and video frame
has got increased attention among the researchers due to its various
complexities and challenges. Because of low resolution, blurring effect,
complex background, different fonts, color and variant alignment of text within
images and video frames, etc., text recognition in such scenario is difficult.
Most of the current approaches usually apply a binarization algorithm to
convert them into binary images and next OCR is applied to get the recognition
result. In this paper, we present a novel approach based on color channel
selection for text recognition from scene images and video frames. In the
approach, at first, a color channel is automatically selected and then selected
color channel is considered for text recognition. Our text recognition
framework is based on Hidden Markov Model (HMM) which uses Pyramidal Histogram
of Oriented Gradient features extracted from selected color channel. From each
sliding window of a color channel our color-channel selection approach analyzes
the image properties from the sliding window and then a multi-label Support
Vector Machine (SVM) classifier is applied to select the color channel that
will provide the best recognition results in the sliding window. This color
channel selection for each sliding window has been found to be more fruitful
than considering a single color channel for the whole word image. Five
different features have been analyzed for multi-label SVM based color channel
selection where wavelet transform based feature outperforms others. Our
framework has been tested on different publicly available scene/video text
image datasets. For Devanagari script, we collected our own data dataset. The
performances obtained from experimental results are encouraging and show the
advantage of the proposed method.

negative:
The color sensation evoked by an object depends on both the spectral power
distribution of the illumination and the reflectance properties of the object
being illuminated. The color sensation can be characterized by three
color-space values, such as XYZ, RGB, HSV, L*a*b*, etc. It is straightforward
to compute the three values given the illuminant and reflectance curves. The
converse process of computing a reflectance curve given the color-space values
and the illuminant is complicated by the fact that an infinite number of
different reflectance curves can give rise to a single set of color-space
values (metamerism). This paper presents five algorithms for generating a
reflectance curve from a specified sRGB triplet, written for a general
audience. The algorithms are designed to generate reflectance curves that are
similar to those found with naturally occurring colored objects. The computed
reflectance curves are compared to a database of thousands of reflectance
curves measured from paints and pigments available both commercially and in
nature, and the similarity is quantified. One particularly useful application
of these algorithms is in the field of computer graphics, where modeling color
transformations sometimes requires wavelength-specific information, such as
when modeling subtractive color mixture.

negative:
In this paper, a color transfer framework to evoke different emotions for
images based on color combinations is proposed. The purpose of this color
transfer is to change the "look and feel" of images, i.e., evoking different
emotions. Colors are confirmed as the most attractive factor in images. In
addition, various studies in both art and science areas have concluded that
other than single color, color combinations are necessary to evoke specific
emotions. Therefore, we propose a novel framework to transfer color of images
based on color combinations, using a predefined color emotion model. The
contribution of this new framework is three-fold. First, users do not need to
provide reference images as used in traditional color transfer algorithms. In
most situations, users may not have enough aesthetic knowledge or path to
choose desired reference images. Second, because of the usage of color
combinations instead of single color for emotions, a new color transfer
algorithm that does not require an image library is proposed. Third, again
because of the usage of color combinations, artifacts that are normally seen in
traditional frameworks using single color are avoided. We present encouraging
results generated from this new framework and its potential in several possible
applications including color transfer of photos and paintings.

negative:
Pretrained language models have been shown to encode relational information,
such as the relations between entities or concepts in knowledge-bases --
(Paris, Capital, France). However, simple relations of this type can often be
recovered heuristically and the extent to which models implicitly reflect
topological structure that is grounded in world, such as perceptual structure,
is unknown. To explore this question, we conduct a thorough case study on
color. Namely, we employ a dataset of monolexemic color terms and color chips
represented in CIELAB, a color space with a perceptually meaningful distance
metric.
  Using two methods of evaluating the structural alignment of colors in this
space with text-derived color term representations, we find significant
correspondence. Analyzing the differences in alignment across the color
spectrum, we find that warmer colors are, on average, better aligned to the
perceptual color space than cooler ones, suggesting an intriguing connection to
findings from recent work on efficient communication in color naming. Further
analysis suggests that differences in alignment are, in part, mediated by
collocationality and differences in syntactic usage, posing questions as to the
relationship between color perception and usage and context.

negative:
Color image segmentation is a very emerging topic for image processing
research. Since it has the ability to present the result in a way that is much
more close to the human yes perceive, so todays more research is going on this
area. Choosing a proper color space is a very important issue for color image
segmentation process. Generally LAB and HSV are the two frequently chosen color
spaces. In this paper a comparative analysis is performed between these two
color spaces with respect to color image segmentation. For measuring their
performance, we consider the parameters: mse and psnr . It is found that HSV
color space is performing better than LAB.