query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am currently engaged in a project that involves compressing High Dynamic Range (HDR) images to be compatible with older JPEG formats. I have found that the existing JPEG XT standard, which is designed to support high dynamic range imaging, does not fully accommodate HDR images. Consequently, I am in need of a model that can convert HDR images into a JPEG XT version with minimal loss of information. I am interested in learning how to incorporate image compression methods into this task. My objective is to develop a universal model that can effectively handle this task without the need to find the optimal setting for each individual image. I am seeking guidance on how to design and train such a model, including the selection of appropriate training data and suitable performance metrics. I anticipate that a neural network-based compression algorithm could potentially outperform traditional algorithms. Therefore, I would appreciate any insights into the compression methods based on CNN, autoencoders, or GANs.

positive:
Detection of contrast adjustments in the presence of JPEG postprocessing is
known to be a challenging task. JPEG post processing is often applied
innocently, as JPEG is the most common image format, or it may correspond to a
laundering attack, when it is purposely applied to erase the traces of
manipulation. In this paper, we propose a CNN-based detector for generic
contrast adjustment, which is robust to JPEG compression. The proposed system
relies on a patch-based Convolutional Neural Network (CNN), trained to
distinguish pristine images from contrast adjusted images, for some selected
adjustment operators of different nature. Robustness to JPEG compression is
achieved by training the CNN with JPEG examples, compressed over a range of
Quality Factors (QFs). Experimental results show that the detector works very
well and scales well with respect to the adjustment type, yielding very good
performance under a large variety of unseen tonal adjustments.

positive:
Convolutional neural networks (CNNs) have achieved astonishing advances over
the past decade, defining state-of-the-art in several computer vision tasks.
CNNs are capable of learning robust representations of the data directly from
the RGB pixels. However, most image data are usually available in compressed
format, from which the JPEG is the most widely used due to transmission and
storage purposes demanding a preliminary decoding process that have a high
computational load and memory usage. For this reason, deep learning methods
capable of leaning directly from the compressed domain have been gaining
attention in recent years. These methods adapt typical CNNs to work on the
compressed domain, but the common architectural modifications lead to an
increase in computational complexity and the number of parameters. In this
paper, we investigate the usage of CNNs that are designed to work directly with
the DCT coefficients available in JPEG compressed images, proposing a
handcrafted and data-driven techniques for reducing the computational
complexity and the number of parameters for these models in order to keep their
computational cost similar to their RGB baselines. We make initial ablation
studies on a subset of ImageNet in order to analyse the impact of different
frequency ranges, image resolution, JPEG quality and classification task
difficulty on the performance of the models. Then, we evaluate the models on
the complete ImageNet dataset. Our results indicate that DCT models are capable
of obtaining good performance, and that it is possible to reduce the
computational complexity and the number of parameters from these models while
retaining a similar classification accuracy through the use of our proposed
techniques.

positive:
With the growth of computer vision based applications and services, an
explosive amount of images have been uploaded to cloud servers which host such
computer vision algorithms, usually in the form of deep learning models. JPEG
has been used as the {\em de facto} compression and encapsulation method before
one uploads the images, due to its wide adaptation. However, standard JPEG
configuration does not always perform well for compressing images that are to
be processed by a deep learning model, e.g., the standard quality level of JPEG
leads to 50\% of size overhead (compared with the best quality level selection)
on ImageNet under the same inference accuracy in popular computer vision models
including InceptionNet, ResNet, etc. Knowing this, designing a better JPEG
configuration for online computer vision services is still extremely
challenging: 1) Cloud-based computer vision models are usually a black box to
end-users; thus it is difficult to design JPEG configuration without knowing
their model structures. 2) JPEG configuration has to change when different
users use it. In this paper, we propose a reinforcement learning based JPEG
configuration framework. In particular, we design an agent that adaptively
chooses the compression level according to the input image's features and
backend deep learning models. Then we train the agent in a reinforcement
learning way to adapt it for different deep learning cloud services that act as
the {\em interactive training environment} and feeding a reward with
comprehensive consideration of accuracy and data size. In our real-world
evaluation on Amazon Rekognition, Face++ and Baidu Vision, our approach can
reduce the size of images by 1/2 -- 1/3 while the overall classification
accuracy only decreases slightly.

positive:
We propose an efficient two-layer near-lossless coding method using an
extended histogram packing technique with backward compatibility to the legacy
JPEG standard. The JPEG XT, which is the international standard to compress HDR
images, adopts a two-layer coding method for backward compatibility to the
legacy JPEG standard. However, there are two problems with this two-layer
coding method. One is that it does not exhibit better near-lossless performance
than other methods for HDR image compression with single-layer structure. The
other problem is that the determining the appropriate values of the coding
parameters may be required for each input image to achieve good compression
performance of near-lossless compression with the two-layer coding method of
the JPEG XT. To solve these problems, we focus on a histogram-packing technique
that takes into account the histogram sparseness of HDR images. We used
zero-skip quantization, which is an extension of the histogram-packing
technique proposed for lossless coding, for implementing the proposed
near-lossless coding method. The experimental results indicate that the
proposed method exhibits not only a better near-lossless compression
performance than that of the two-layer coding method of the JPEG XT, but also
there are no issue regarding the combination of parameter values without losing
backward compatibility to the JPEG standard.

positive:
An efficient two-layer coding method using the histogram packing technique
with the backward compatibility to the legacy JPEG is proposed in this paper.
The JPEG XT, which is the international standard to compress HDR images, adopts
two-layer coding scheme for backward compatibility to the legacy JPEG. However,
this two-layer coding structure does not give better lossless performance than
the other existing methods for HDR image compression with single-layer
structure. Moreover, the lossless compression of the JPEG XT has a problem on
determination of the coding parameters; The lossless performance is affected by
the input images and/or the parameter values. That is, finding appropriate
combination of the values is necessary to achieve good lossless performance. It
is firstly pointed out that the histogram packing technique considering the
histogram sparseness of HDR images is able to improve the performance of
lossless compression. Then, a novel two-layer coding with the histogram packing
technique and an additional lossless encoder is proposed. The experimental
results demonstrate that not only the proposed method has a better lossless
compression performance than that of the JPEG XT, but also there is no need to
determine image-dependent parameter values for good compression performance
without losing the backward compatibility to the well known legacy JPEG
standard.

positive:
Deep learning for computer vision depends on lossy image compression: it
reduces the storage required for training and test data and lowers transfer
costs in deployment. Mainstream datasets and imaging pipelines all rely on
standard JPEG compression. In JPEG, the degree of quantization of frequency
coefficients controls the lossiness: an 8 by 8 quantization table (Q-table)
decides both the quality of the encoded image and the compression ratio. While
a long history of work has sought better Q-tables, existing work either seeks
to minimize image distortion or to optimize for models of the human visual
system. This work asks whether JPEG Q-tables exist that are "better" for
specific vision networks and can offer better quality--size trade-offs than
ones designed for human perception or minimal distortion. We reconstruct an
ImageNet test set with higher resolution to explore the effect of JPEG
compression under novel Q-tables. We attempt several approaches to tune a
Q-table for a vision task. We find that a simple sorted random sampling method
can exceed the performance of the standard JPEG Q-table. We also use
hyper-parameter tuning techniques including bounded random search, Bayesian
optimization, and composite heuristic optimization methods. The new Q-tables we
obtained can improve the compression rate by 10% to 200% when the accuracy is
fixed, or improve accuracy up to $2\%$ at the same compression rate.

positive:
Lossy compression brings artifacts into the compressed image and degrades the
visual quality. In recent years, many compression artifacts removal methods
based on convolutional neural network (CNN) have been developed with great
success. However, these methods usually train a model based on one specific
value or a small range of quality factors. Obviously, if the test image's
quality factor does not match to the assumed value range, then degraded
performance will be resulted. With this motivation and further consideration of
practical usage, a highly robust compression artifacts removal network is
proposed in this paper. Our proposed network is a single model approach that
can be trained for handling a wide range of quality factors while consistently
delivering superior or comparable image artifacts removal performance. To
demonstrate, we focus on the JPEG compression with quality factors, ranging
from 1 to 60. Note that a turnkey success of our proposed network lies in the
novel utilization of the quantization tables as part of the training data.
Furthermore, it has two branches in parallel---i.e., the restoration branch and
the global branch. The former effectively removes the local artifacts, such as
ringing artifacts removal. On the other hand, the latter extracts the global
features of the entire image that provides highly instrumental image quality
improvement, especially effective on dealing with the global artifacts, such as
blocking, color shifting. Extensive experimental results performed on color and
grayscale images have clearly demonstrated the effectiveness and efficacy of
our proposed single-model approach on the removal of compression artifacts from
the decoded image.

positive:
The popularity of Convolutional Neural Network (CNN) in the field of Image
Processing and Computer Vision has motivated researchers and industrialist
experts across the globe to solve different challenges with high accuracy. The
simplest way to train a CNN classifier is to directly feed the original RGB
pixels images into the network. However, if we intend to classify images
directly with its compressed data, the same approach may not work better, like
in case of JPEG compressed images. This research paper investigates the issues
of modifying the input representation of the JPEG compressed data, and then
feeding into the CNN. The architecture is termed as DCT-CompCNN. This novel
approach has shown that CNNs can also be trained with JPEG compressed DCT
coefficients, and subsequently can produce a better performance in comparison
with the conventional CNN approach. The efficiency of the modified input
representation is tested with the existing ResNet-50 architecture and the
proposed DCT-CompCNN architecture on a public image classification datasets
like Dog Vs Cat and CIFAR-10 datasets, reporting a better performance

positive:
We propose a novel JPEG XT image compression with hue compensation for
two-layer HDR coding. LDR images produced from JPEG XT bitstreams have some
distortion in hue due to tone mapping operations. In order to suppress the
color distortion, we apply a novel hue compensation method based on the
maximally saturated colors. Moreover, the bitstreams generated by using the
proposed method are fully compatible with the JPEG XT standard. In an
experiment, the proposed method is demonstrated not only to produce images with
small hue degradation but also to maintain well-mapped luminance, in terms of
three kinds of criterion: TMQI, hue value in CIEDE2000, and the maximally
saturated color on the constant-hue plane.

positive:
We propose a new approach to the problem of optimizing autoencoders for lossy
image compression. New media formats, changing hardware technology, as well as
diverse requirements and content types create a need for compression algorithms
which are more flexible than existing codecs. Autoencoders have the potential
to address this need, but are difficult to optimize directly due to the
inherent non-differentiabilty of the compression loss. We here show that
minimal changes to the loss are sufficient to train deep autoencoders
competitive with JPEG 2000 and outperforming recently proposed approaches based
on RNNs. Our network is furthermore computationally efficient thanks to a
sub-pixel architecture, which makes it suitable for high-resolution images.
This is in contrast to previous work on autoencoders for compression using
coarser approximations, shallower architectures, computationally expensive
methods, or focusing on small images.

positive:
Most image data available are often stored in a compressed format, from which
JPEG is the most widespread. To feed this data on a convolutional neural
network (CNN), a preliminary decoding process is required to obtain RGB pixels,
demanding a high computational load and memory usage. For this reason, the
design of CNNs for processing JPEG compressed data has gained attention in
recent years. In most existing works, typical CNN architectures are adapted to
facilitate the learning with the DCT coefficients rather than RGB pixels.
Although they are effective, their architectural changes either raise the
computational costs or neglect relevant information from DCT inputs. In this
paper, we examine different ways of speeding up CNNs designed for DCT inputs,
exploiting learning strategies to reduce the computational complexity by taking
full advantage of DCT inputs. Our experiments were conducted on the ImageNet
dataset. Results show that learning how to combine all DCT inputs in a
data-driven fashion is better than discarding them by hand, and its combination
with a reduction of layers has proven to be effective for reducing the
computational costs while retaining accuracy.

positive:
Display technology must offer high dynamic range (HDR) contrast-based depth
induction and 3D personalization simultaneously. Efficient algorithms to
compress HDR stereo data is critical. Direct capturing of HDR content is
complicated due to the high expense and scarcity of HDR cameras. The HDR 3D
images could be generated in low-cost by fusing low-dynamic-range (LDR) images
acquired using a stereo camera with various exposure settings. In this paper,
an efficient scheme for coding multi-exposure stereo images is proposed based
on a tensor low-rank approximation scheme. The multi-exposure fusion can be
realized to generate HDR stereo output at the decoder for increased realism and
exaggerated binocular 3D depth cues.
  For exploiting spatial redundancy in LDR stereo images, the stack of
multi-exposure stereo images is decomposed into a set of projection matrices
and a core tensor following an alternating least squares Tucker decomposition
model. The compact, low-rank representation of the scene, thus, generated is
further processed by 3D extension of High Efficiency Video Coding standard. The
encoding with 3D-HEVC enhance the proposed scheme efficiency by exploiting
intra-frame, inter-view and the inter-component redundancies in low-rank
approximated representation. We consider constant luminance property of IPT and
Y'CbCr color space to precisely approximate intensity prediction and
perceptually minimize the encoding distortion. Besides, the proposed scheme
gives flexibility to adjust the bitrate of tensor latent components by changing
the rank of core tensor and its quantization. Extensive experiments on natural
scenes demonstrate that the proposed scheme outperforms state-of-the-art
JPEG-XT and 3D-HEVC range coding standards.

positive:
Deep convolutional neural networks (DCNNs) have aided high dynamic range
(HDR) imaging recently and have received a lot of attention. The quality of
DCNN-generated HDR images has overperformed the traditional counterparts.
However, DCNNs are prone to be computationally intensive and power-hungry, and
hence cannot be implemented on various embedded computing platforms with
limited power and hardware resources. Embedded systems have a huge market, and
utilizing DCNNs' powerful functionality into them will further reduce human
intervention. To address the challenge, we propose LightFuse, a lightweight
CNN-based algorithm for extreme dual-exposure image fusion, which achieves
better functionality than a conventional DCNN and can be deployed in embedded
systems. Two sub-networks are utilized: a GlobalNet (G) and a DetailNet (D).
The goal of G is to learn the global illumination information on the spatial
dimension, whereas D aims to enhance local details on the channel dimension.
Both G and D are based solely on depthwise convolution (D_Conv) and pointwise
convolution (P_Conv) to reduce required parameters and computations.
Experimental results show that this proposed technique could generate HDR
images in extremely exposed regions with sufficient details to be legible. Our
model outperforms other state-of-the-art approaches in peak signal-to-noise
ratio (PSNR) score by 0.9 to 8.7 while achieving 16.7 to 306.2 times parameter
reduction.

positive:
Deep Neural Networks trained as image auto-encoders have recently emerged as
a promising direction for advancing the state-of-the-art in image compression.
The key challenge in learning such networks is twofold: To deal with
quantization, and to control the trade-off between reconstruction error
(distortion) and entropy (rate) of the latent image representation. In this
paper, we focus on the latter challenge and propose a new technique to navigate
the rate-distortion trade-off for an image compression auto-encoder. The main
idea is to directly model the entropy of the latent representation by using a
context model: A 3D-CNN which learns a conditional probability model of the
latent distribution of the auto-encoder. During training, the auto-encoder
makes use of the context model to estimate the entropy of its representation,
and the context model is concurrently updated to learn the dependencies between
the symbols in the latent representation. Our experiments show that this
approach, when measured in MS-SSIM, yields a state-of-the-art image compression
system based on a simple convolutional auto-encoder.

positive:
In this paper we propose a deep residual autoencoder exploiting
Residual-in-Residual Dense Blocks (RRDB) to remove artifacts in JPEG compressed
images that is independent from the Quality Factor (QF) used. The proposed
approach leverages both the learning capacity of deep residual networks and
prior knowledge of the JPEG compression pipeline. The proposed model operates
in the YCbCr color space and performs JPEG artifact restoration in two phases
using two different autoencoders: the first one restores the luma channel
exploiting 2D convolutions; the second one, using the restored luma channel as
a guide, restores the chroma channels explotining 3D convolutions. Extensive
experimental results on three widely used benchmark datasets (i.e. LIVE1,
BDS500, and CLASSIC-5) show that our model is able to outperform the state of
the art with respect to all the evaluation metrics considered (i.e. PSNR,
PSNR-B, and SSIM). This results is remarkable since the approaches in the state
of the art use a different set of weights for each compression quality, while
the proposed model uses the same weights for all of them, making it applicable
to images in the wild where the QF used for compression is unkwnown.
Furthermore, the proposed model shows a greater robustness than
state-of-the-art methods when applied to compression qualities not seen during
training.

positive:
In the advent of a digital health revolution, vast amounts of clinical data
are being generated, stored and processed on a daily basis. This has made the
storage and retrieval of large volumes of health-care data, especially,
high-resolution medical images, particularly challenging. Effective image
compression for medical images thus plays a vital role in today's healthcare
information system, particularly in teleradiology. In this work, an X-ray image
compression method based on a Convolutional Recurrent Neural Networks RNN-Conv
is presented. The proposed architecture can provide variable compression rates
during deployment while it requires each network to be trained only once for a
specific dimension of X-ray images. The model uses a multi-level pooling scheme
that learns contextualized features for effective compression. We perform our
image compression experiments on the National Institute of Health (NIH)
ChestX-ray8 dataset and compare the performance of the proposed architecture
with a state-of-the-art RNN based technique and JPEG 2000. The experimental
results depict improved compression performance achieved by the proposed method
in terms of Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio
(PSNR) metrics. To the best of our knowledge, this is the first reported
evaluation on using a deep convolutional RNN for medical image compression.

positive:
This paper proposes a new end-to-end trainable model for lossy image
compression, which includes several novel components. The method incorporates
1) an adequate perceptual similarity metric; 2) saliency in the images; 3) a
hierarchical auto-regressive model. This paper demonstrates that the popularly
used evaluations metrics such as MS-SSIM and PSNR are inadequate for judging
the performance of image compression techniques as they do not align with the
human perception of similarity. Alternatively, a new metric is proposed, which
is learned on perceptual similarity data specific to image compression. The
proposed compression model incorporates the salient regions and optimizes on
the proposed perceptual similarity metric. The model not only generates images
which are visually better but also gives superior performance for subsequent
computer vision tasks such as object detection and segmentation when compared
to existing engineered or learned compression techniques.

positive:
We describe an image compression method, consisting of a nonlinear analysis
transformation, a uniform quantizer, and a nonlinear synthesis transformation.
The transforms are constructed in three successive stages of convolutional
linear filters and nonlinear activation functions. Unlike most convolutional
neural networks, the joint nonlinearity is chosen to implement a form of local
gain control, inspired by those used to model biological neurons. Using a
variant of stochastic gradient descent, we jointly optimize the entire model
for rate-distortion performance over a database of training images, introducing
a continuous proxy for the discontinuous loss function arising from the
quantizer. Under certain conditions, the relaxed loss function may be
interpreted as the log likelihood of a generative model, as implemented by a
variational autoencoder. Unlike these models, however, the compression model
must operate at any given point along the rate-distortion curve, as specified
by a trade-off parameter. Across an independent set of test images, we find
that the optimized method generally exhibits better rate-distortion performance
than the standard JPEG and JPEG 2000 compression methods. More importantly, we
observe a dramatic improvement in visual quality for all images at all bit
rates, which is supported by objective quality estimates using MS-SSIM.

positive:
A large fraction of Internet traffic is now driven by requests from mobile
devices with relatively small screens and often stringent bandwidth
requirements. Due to these factors, it has become the norm for modern
graphics-heavy websites to transmit low-resolution, low-bytecount image
previews (thumbnails) as part of the initial page load process to improve
apparent page responsiveness. Increasing thumbnail compression beyond the
capabilities of existing codecs is therefore a current research focus, as any
byte savings will significantly enhance the experience of mobile device users.
Toward this end, we propose a general framework for variable-rate image
compression and a novel architecture based on convolutional and deconvolutional
LSTM recurrent networks. Our models address the main issues that have prevented
autoencoder neural networks from competing with existing image compression
algorithms: (1) our networks only need to be trained once (not per-image),
regardless of input image dimensions and the desired compression rate; (2) our
networks are progressive, meaning that the more bits are sent, the more
accurate the image reconstruction; and (3) the proposed architecture is at
least as efficient as a standard purpose-trained autoencoder for a given number
of bits. On a large-scale benchmark of 32$\times$32 thumbnails, our LSTM-based
approaches provide better visual quality than (headerless) JPEG, JPEG2000 and
WebP, with a storage size that is reduced by 10% or more.

positive:
The field of neural image compression has witnessed exciting progress as
recently proposed architectures already surpass the established transform
coding based approaches. While, so far, research has mainly focused on
architecture and model improvements, in this work we explore content adaptive
optimization. To this end, we introduce an iterative procedure which adapts the
latent representation to the specific content we wish to compress while keeping
the parameters of the network and the predictive model fixed. Our experiments
show that this allows for an overall increase in rate-distortion performance,
independently of the specific architecture used. Furthermore, we also evaluate
this strategy in the context of adapting a pretrained network to other content
that is different in visual appearance or resolution. Here, our experiments
show that our adaptation strategy can largely close the gap as compared to
models specifically trained for the given content while having the benefit that
no additional data in the form of model parameter updates has to be
transmitted.

negative:
In recent years we have witnessed an increasing interest in applying Deep
Neural Networks (DNNs) to improve the rate-distortion performance in image
compression. However, the existing approaches either train a post-processing
DNN on the decoder side, or propose learning for image compression in an
end-to-end manner. This way, the trained DNNs are required in the decoder,
leading to the incompatibility to the standard image decoders (e.g., JPEG) in
personal computers and mobiles. Therefore, we propose learning to improve the
encoding performance with the standard decoder. In this paper, We work on JPEG
as an example. Specifically, a frequency-domain pre-editing method is proposed
to optimize the distribution of DCT coefficients, aiming at facilitating the
JPEG compression. Moreover, we propose learning the JPEG quantization table
jointly with the pre-editing network. Most importantly, we do not modify the
JPEG decoder and therefore our approach is applicable when viewing images with
the widely used standard JPEG decoder. The experiments validate that our
approach successfully improves the rate-distortion performance of JPEG in terms
of various quality metrics, such as PSNR, MS-SSIM and LPIPS. Visually, this
translates to better overall color retention especially when strong compression
is applied. The codes are available at
https://github.com/YannickStruempler/LearnedJPEG.

negative:
High Dynamic Range (HDR) images are the ones that contain a greater range of
luminosity as compared to the standard images. HDR images have a higher detail
and clarity of structure, objects, and color, which the standard images lack.
HDR images are useful in capturing scenes that pose high brightness, darker
areas, and shadows, etc. An HDR image comprises multiple narrow-range-exposure
images combined into one high-quality image. As these HDR images cannot be
displayed on standard display devices, the real challenge comes while
converting these HDR images to Low dynamic range (LDR) images. The conversion
of HDR image to LDR image is performed using Tone-mapped operators (TMOs). This
conversion results in the loss of much valuable information in structure,
color, naturalness, and exposures. The loss of information in the LDR image may
not directly be visible to the human eye. To calculate how good an LDR image is
after conversion, various metrics have been proposed previously. Some are not
noise resilient, some work on separate color channels (Red, Green, and Blue one
by one), and some lack capacity to identify the structure. To deal with this
problem, we propose a metric in this paper called the Tone Mapping Quality
Index (TMQI-3), which evaluates the quality of the LDR image based on its
objective score. TMQI-3 is noise resilient, takes account of structure and
naturalness, and works on all three color channels combined into one luminosity
component. This eliminates the need to use multiple metrics at the same time.
We compute results for several HDR and LDR images from the literature and show
that our quality index metric performs better than the baseline models.

negative:
Recent studies have shown that neural network (NN) based image classifiers
are highly vulnerable to adversarial examples, which poses a threat to
security-sensitive image recognition task. Prior work has shown that JPEG
compression can combat the drop in classification accuracy on adversarial
examples to some extent. But, as the compression ratio increases, traditional
JPEG compression is insufficient to defend those attacks but can cause an
abrupt accuracy decline to the benign images. In this paper, with the aim of
fully filtering the adversarial perturbations, we firstly make modifications to
traditional JPEG compression algorithm which becomes more favorable for NN.
Specifically, based on an analysis of the frequency coefficient, we design a
NN-favored quantization table for compression. Considering compression as a
data augmentation strategy, we then combine our model-agnostic preprocess with
noisy training. We fine-tune the pre-trained model by training with images
encoded at different compression levels, thus generating multiple classifiers.
Finally, since lower (higher) compression ratio can remove both perturbations
and original features slightly (aggressively), we use these trained multiple
models for model ensemble. The majority vote of the ensemble of models is
adopted as final predictions. Experiments results show our method can improve
defense efficiency while maintaining original accuracy.

negative:
As one of most fascinating machine learning techniques, deep neural network
(DNN) has demonstrated excellent performance in various intelligent tasks such
as image classification. DNN achieves such performance, to a large extent, by
performing expensive training over huge volumes of training data. To reduce the
data storage and transfer overhead in smart resource-limited Internet-of-Thing
(IoT) systems, effective data compression is a "must-have" feature before
transferring real-time produced dataset for training or classification. While
there have been many well-known image compression approaches (such as JPEG), we
for the first time find that a human-visual based image compression approach
such as JPEG compression is not an optimized solution for DNN systems,
especially with high compression ratios. To this end, we develop an image
compression framework tailored for DNN applications, named "DeepN-JPEG", to
embrace the nature of deep cascaded information process mechanism of DNN
architecture. Extensive experiments, based on "ImageNet" dataset with various
state-of-the-art DNNs, show that "DeepN-JPEG" can achieve ~3.5x higher
compression rate over the popular JPEG solution while maintaining the same
accuracy level for image recognition, demonstrating its great potential of
storage and power efficiency in DNN-based smart IoT system design.

negative:
A two-layer lossless image coding method compatible with JPEG XS is proposed.
JPEG XS is a new international standard for still image coding that has the
characteristics of very low latency and very low complexity. However, it does
not support lossless coding, although it can achieve visual lossless coding.
The proposed method has a two-layer structure similar to JPEG XT, which
consists of JPEG XS coding and a lossless coding method. As a result, it
enables us to losslessly restore original images, while maintaining
compatibility with JPEG XS.

negative:
The prime goal of digital imaging techniques is to reproduce the realistic
appearance of a scene. Low Dynamic Range (LDR) cameras are incapable of
representing the wide dynamic range of the real-world scene. The captured
images turn out to be either too dark (underexposed) or too bright
(overexposed). Specifically, saturation in overexposed regions makes the task
of reconstructing a High Dynamic Range (HDR) image from single LDR image
challenging. In this paper, we propose a deep learning based approach to
recover details in the saturated areas while reconstructing the HDR image. We
formulate this problem as an image-to-image (I2I) translation task. To this
end, we present a novel conditional GAN (cGAN) based framework trained in an
end-to-end fashion over the HDR-REAL and HDR-SYNTH datasets. Our framework uses
an overexposed mask obtained from a pre-trained segmentation model to
facilitate the hallucination task of adding details in the saturated regions.
We demonstrate the effectiveness of the proposed method by performing an
extensive quantitative and qualitative comparison with several state-of-the-art
single-image HDR reconstruction techniques.

negative:
Recent advances in generalized image understanding have seen a surge in the
use of deep convolutional neural networks (CNN) across a broad range of
image-based detection, classification and prediction tasks. Whilst the reported
performance of these approaches is impressive, this study investigates the
hitherto unapproached question of the impact of commonplace image and video
compression techniques on the performance of such deep learning architectures.
Focusing on the JPEG and H.264 (MPEG-4 AVC) as a representative proxy for
contemporary lossy image/video compression techniques that are in common use
within network-connected image/video devices and infrastructure, we examine the
impact on performance across five discrete tasks: human pose estimation,
semantic segmentation, object detection, action recognition, and monocular
depth estimation. As such, within this study we include a variety of network
architectures and domains spanning end-to-end convolution, encoder-decoder,
region-based CNN (R-CNN), dual-stream, and generative adversarial networks
(GAN). Our results show a non-linear and non-uniform relationship between
network performance and the level of lossy compression applied. Notably,
performance decreases significantly below a JPEG quality (quantization) level
of 15% and a H.264 Constant Rate Factor (CRF) of 40. However, retraining said
architectures on pre-compressed imagery conversely recovers network performance
by up to 78.4% in some cases. Furthermore, there is a correlation between
architectures employing an encoder-decoder pipeline and those that demonstrate
resilience to lossy image compression. The characteristics of the relationship
between input compression to output task performance can be used to inform
design decisions within future image/video devices and infrastructure.

negative:
This paper presents a set of full-resolution lossy image compression methods
based on neural networks. Each of the architectures we describe can provide
variable compression rates during deployment without requiring retraining of
the network: each network need only be trained once. All of our architectures
consist of a recurrent neural network (RNN)-based encoder and decoder, a
binarizer, and a neural network for entropy coding. We compare RNN types (LSTM,
associative LSTM) and introduce a new hybrid of GRU and ResNet. We also study
"one-shot" versus additive reconstruction architectures and introduce a new
scaled-additive framework. We compare to previous work, showing improvements of
4.3%-8.8% AUC (area under the rate-distortion curve), depending on the
perceptual metric used. As far as we know, this is the first neural network
architecture that is able to outperform JPEG at image compression across most
bitrates on the rate-distortion curve on the Kodak dataset images, with and
without the aid of entropy coding.

negative:
Mapping a single exposure low dynamic range (LDR) image into a high dynamic
range (HDR) is considered among the most strenuous image to image translation
tasks due to exposure-related missing information. This study tackles the
challenges of single-shot LDR to HDR mapping by proposing a novel two-stage
deep network. Notably, our proposed method aims to reconstruct an HDR image
without knowing hardware information, including camera response function (CRF)
and exposure settings. Therefore, we aim to perform image enhancement task like
denoising, exposure correction, etc., in the first stage. Additionally, the
second stage of our deep network learns tone mapping and bit-expansion from a
convex set of data samples. The qualitative and quantitative comparisons
demonstrate that the proposed method can outperform the existing LDR to HDR
works with a marginal difference. Apart from that, we collected an LDR image
dataset incorporating different camera systems. The evaluation with our
collected real-world LDR images illustrates that the proposed method can
reconstruct plausible HDR images without presenting any visual artefacts. Code
available: https://github. com/sharif-apu/twostageHDR_NTIRE21.

negative:
High dynamic range (HDR) imaging has recently drawn much attention in
multimedia community. In this paper, we proposed a HDR image forensics method
based on convolutional neural network (CNN).To our best knowledge, this is the
first time to apply deep learning method on HDR image forensics. The proposed
algorithm uses CNN to distinguish HDR images generated by multiple low dynamic
range (LDR) images from that expanded by single LDR image using inverse tone
mapping (iTM). To do this, we learn the change of statistical characteristics
extracted by the proposed CNN architectures and classify two kinds of HDR
images. Comparision results with some traditional statistical characteristics
shows efficiency of the proposed method in HDR image source identification.

negative:
Increasing popularity of high-dynamic-range (HDR) image and video content
brings the need for metrics that could predict the severity of image
impairments as seen on displays of different brightness levels and dynamic
range. Such metrics should be trained and validated on a sufficiently large
subjective image quality dataset to ensure robust performance. As the existing
HDR quality datasets are limited in size, we created a Unified Photometric
Image Quality dataset (UPIQ) with over 4,000 images by realigning and merging
existing HDR and standard-dynamic-range (SDR) datasets. The realigned quality
scores share the same unified quality scale across all datasets. Such
realignment was achieved by collecting additional cross-dataset quality
comparisons and re-scaling data with a psychometric scaling method. Images in
the proposed dataset are represented in absolute photometric and colorimetric
units, corresponding to light emitted from a display. We use the new dataset to
retrain existing HDR metrics and show that the dataset is sufficiently large
for training deep architectures. We show the utility of the dataset on
brightness aware image compression.

negative:
High dynamic range (HDR) imaging provides the capability of handling real
world lighting as opposed to the traditional low dynamic range (LDR) which
struggles to accurately represent images with higher dynamic range. However,
most imaging content is still available only in LDR. This paper presents a
method for generating HDR content from LDR content based on deep Convolutional
Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input
and generates images with an expanded range in an end-to-end fashion. The model
attempts to reconstruct missing information that was lost from the original
signal due to quantization, clipping, tone mapping or gamma correction. The
added information is reconstructed from learned features, as the network is
trained in a supervised fashion using a dataset of HDR images. The approach is
fully automatic and data driven; it does not require any heuristics or human
expertise. ExpandNet uses a multiscale architecture which avoids the use of
upsampling layers to improve image quality. The method performs well compared
to expansion/inverse tone mapping operators quantitatively on multiple metrics,
even for badly exposed inputs.

negative:
Bottleneck autoencoders have been actively researched as a solution to image
compression tasks. However, we observed that bottleneck autoencoders produce
subjectively low quality reconstructed images. In this work, we explore the
ability of sparse coding to improve reconstructed image quality for the same
degree of compression. We observe that sparse image compression produces
visually superior reconstructed images and yields higher values of pixel-wise
measures of reconstruction quality (PSNR and SSIM) compared to bottleneck
autoencoders. % In addition, we find that using alternative metrics that
correlate better with human perception, such as feature perceptual loss and the
classification accuracy, sparse image compression scores up to 18.06\% and
2.7\% higher, respectively, compared to bottleneck autoencoders. Although
computationally much more intensive, we find that sparse coding is otherwise
superior to bottleneck autoencoders for the same degree of compression.

negative:
In this paper, we propose a novel deep neural network model that reconstructs
a high dynamic range (HDR) image from a single low dynamic range (LDR) image.
The proposed model is based on a convolutional neural network composed of
dilated convolutional layers, and infers LDR images with various exposures and
illumination from a single LDR image of the same scene. Then, the final HDR
image can be formed by merging these inference results. It is relatively easy
for the proposed method to find the mapping between the LDR and an HDR with a
different bit depth because of the chaining structure inferring the
relationship between the LDR images with brighter (or darker) exposures from a
given LDR image. The method not only extends the range, but also has the
advantage of restoring the light information of the actual physical world. For
the HDR images obtained by the proposed method, the HDR-VDP2 Q score, which is
the most popular evaluation metric for HDR images, was 56.36 for a display with
a 1920$\times$1200 resolution, which is an improvement of 6 compared with the
scores of conventional algorithms. In addition, when comparing the peak
signal-to-noise ratio values for tone mapped HDR images generated by the
proposed and conventional algorithms, the average value obtained by the
proposed algorithm is 30.86 dB, which is 10 dB higher than those obtained by
the conventional algorithms.

negative:
Inpainting based image compression approaches, especially linear and
non-linear diffusion models, are an active research topic for lossy image
compression. The major challenge in these compression models is to find a small
set of descriptive supporting points, which allow for an accurate
reconstruction of the original image. It turns out in practice that this is a
challenging problem even for the simplest Laplacian interpolation model. In
this paper, we revisit the Laplacian interpolation compression model and
introduce two fast algorithms, namely successive preconditioning primal dual
algorithm and the recently proposed iPiano algorithm, to solve this problem
efficiently. Furthermore, we extend the Laplacian interpolation based
compression model to a more general form, which is based on principles from
bi-level optimization. We investigate two different variants of the Laplacian
model, namely biharmonic interpolation and smoothed Total Variation
regularization. Our numerical results show that significant improvements can be
obtained from the biharmonic interpolation model, and it can recover an image
with very high quality from only 5% pixels.

negative:
Camera sensors can only capture a limited range of luminance simultaneously,
and in order to create high dynamic range (HDR) images a set of different
exposures are typically combined. In this paper we address the problem of
predicting information that have been lost in saturated image areas, in order
to enable HDR reconstruction from a single exposure. We show that this problem
is well-suited for deep learning algorithms, and propose a deep convolutional
neural network (CNN) that is specifically designed taking into account the
challenges in predicting HDR values. To train the CNN we gather a large dataset
of HDR images, which we augment by simulating sensor saturation for a range of
cameras. To further boost robustness, we pre-train the CNN on a simulated HDR
dataset created from a subset of the MIT Places database. We demonstrate that
our approach can reconstruct high-resolution visually convincing HDR results in
a wide range of situations, and that it generalizes well to reconstruction of
images captured with arbitrary and low-end cameras that use unknown camera
response functions and post-processing. Furthermore, we compare to existing
methods for HDR expansion, and show high quality results also for image based
lighting. Finally, we evaluate the results in a subjective experiment performed
on an HDR display. This shows that the reconstructed HDR images are visually
convincing, with large improvements as compared to existing methods.

negative:
Since its invention, Generative adversarial networks (GANs) have shown
outstanding results in many applications. Generative Adversarial Networks are
powerful yet, resource-hungry deep-learning models. Their main difference from
ordinary deep learning models is the nature of their output. For example, GAN
output can be a whole image versus other models detecting objects or
classifying images. Thus, the architecture and numeric precision of the network
affect the quality and speed of the solution. Hence, accelerating GANs is
pivotal. Accelerating GANs can be classified into three main tracks: (1) Memory
compression, (2) Computation optimization, and (3) Data-flow optimization.
Because data transfer is the main source of energy usage, memory compression
leads to the most savings. Thus, in this paper, we survey memory compression
techniques for CNN-Based GANs. Additionally, the paper summarizes opportunities
and challenges in GANs acceleration and suggests open research problems to be
further investigated.

negative:
Deep learning based image compression has recently witnessed exciting
progress and in some cases even managed to surpass transform coding based
approaches that have been established and refined over many decades. However,
state-of-the-art solutions for deep image compression typically employ
autoencoders which map the input to a lower dimensional latent space and thus
irreversibly discard information already before quantization. Due to that, they
inherently limit the range of quality levels that can be covered. In contrast,
traditional approaches in image compression allow for a larger range of quality
levels. Interestingly, they employ an invertible transformation before
performing the quantization step which explicitly discards information.
Inspired by this, we propose a deep image compression method that is able to go
from low bit-rates to near lossless quality by leveraging normalizing flows to
learn a bijective mapping from the image space to a latent representation. In
addition to this, we demonstrate further advantages unique to our solution,
such as the ability to maintain constant quality results through re-encoding,
even when performed multiple times. To the best of our knowledge, this is the
first work to explore the opportunities for leveraging normalizing flows for
lossy image compression.

negative:
JPEG is one of the most commonly used standards among lossy image compression
methods. However, JPEG compression inevitably introduces various kinds of
artifacts, especially at high compression rates, which could greatly affect the
Quality of Experience (QoE). Recently, convolutional neural network (CNN) based
methods have shown excellent performance for removing the JPEG artifacts. Lots
of efforts have been made to deepen the CNNs and extract deeper features, while
relatively few works pay attention to the receptive field of the network. In
this paper, we illustrate that the quality of output images can be
significantly improved by enlarging the receptive fields in many cases. One
step further, we propose a Dual-domain Multi-scale CNN (DMCNN) to take full
advantage of redundancies on both the pixel and DCT domains. Experiments show
that DMCNN sets a new state-of-the-art for the task of JPEG artifact removal.

negative:
The JPEG image compression algorithm is the most popular method of image
compression because of its ability for large compression ratios. However, to
achieve such high compression, information is lost. For aggressive quantization
settings, this leads to a noticeable reduction in image quality. Artifact
correction has been studied in the context of deep neural networks for some
time, but the current state-of-the-art methods require a different model to be
trained for each quality setting, greatly limiting their practical application.
We solve this problem by creating a novel architecture which is parameterized
by the JPEG files quantization matrix. This allows our single model to achieve
state-of-the-art performance over models trained for specific quality settings.

negative:
Synthesizing high dynamic range (HDR) images from multiple low-dynamic range
(LDR) exposures in dynamic scenes is challenging. There are two major problems
caused by the large motions of foreground objects. One is the severe
misalignment among the LDR images. The other is the missing content due to the
over-/under-saturated regions caused by the moving objects, which may not be
easily compensated for by the multiple LDR exposures. Thus, it requires the HDR
generation model to be able to properly fuse the LDR images and restore the
missing details without introducing artifacts. To address these two problems,
we propose in this paper a novel GAN-based model, HDR-GAN, for synthesizing HDR
images from multi-exposed LDR images. To our best knowledge, this work is the
first GAN-based approach for fusing multi-exposed LDR images for HDR
reconstruction. By incorporating adversarial learning, our method is able to
produce faithful information in the regions with missing content. In addition,
we also propose a novel generator network, with a reference-based residual
merging block for aligning large object motions in the feature domain, and a
deep HDR supervision scheme for eliminating artifacts of the reconstructed HDR
images. Experimental results demonstrate that our model achieves
state-of-the-art reconstruction performance over the prior HDR methods on
diverse scenes.

negative:
This work evaluates six state-of-the-art deep neural network (DNN)
architectures applied to the problem of enhancing camera-captured document
images. The results from each network were evaluated both qualitatively and
quantitatively using Image Quality Assessment (IQA) metrics, and also compared
with an existing approach based on traditional computer vision techniques. The
best performing architectures generally produced good enhancement compared to
the existing algorithm, showing that it is possible to use DNNs for document
image enhancement. Furthermore, the best performing architectures could work as
a baseline for future investigations on document enhancement using deep
learning techniques. The main contributions of this paper are: a baseline of
deep learning techniques that can be further improved to provide better
results, and a evaluation methodology using IQA metrics for quantitatively
comparing the produced images from the neural networks to a ground truth.

negative:
Document image enhancement and binarization methods are often used to improve
the accuracy and efficiency of document image analysis tasks such as text
recognition. Traditional non-machine-learning methods are constructed on
low-level features in an unsupervised manner but have difficulty with
binarization on documents with severely degraded backgrounds. Convolutional
neural network-based methods focus only on grayscale images and on local
textual features. In this paper, we propose a two-stage color document image
enhancement and binarization method using generative adversarial neural
networks. In the first stage, four color-independent adversarial networks are
trained to extract color foreground information from an input image for
document image enhancement. In the second stage, two independent adversarial
networks with global and local features are trained for image binarization of
documents of variable size. For the adversarial neural networks, we formulate
loss functions between a discriminator and generators having an encoder-decoder
structure. Experimental results show that the proposed method achieves better
performance than many classical and state-of-the-art algorithms over the
Document Image Binarization Contest (DIBCO) datasets, the LRDE Document
Binarization Dataset (LRDE DBD), and our shipping label image dataset. We plan
to release the shipping label dataset as well as our implementation code at
github.com/opensuh/DocumentBinarization/.

negative:
Image compression helps in storing the transmitted data in proficient way by
decreasing its redundancy. This technique helps in transferring more digital or
multimedia data over internet as it increases the storage space. It is
important to maintain the image quality even if it is compressed to certain
extent. Depend upon this the image compression is classified into two
categories : lossy and lossless image compression. There are many lossy digital
image compression techniques exists. Among this Incremental Self Organizing Map
is a familiar one. The good pictures quality can be retrieved if image
denoising technique is used for compression and also provides better
compression ratio. Image denoising is an important pre-processing step for many
image analysis and computer vision system. It refers to the task of recovering
a good estimate of the true image from a degraded observation without altering
and changing useful structure in the image such as discontinuities and edges.
Many approaches have been proposed to remove the noise effectively while
preserving the original image details and features as much as possible. This
paper proposes a technique for image compression using Incremental Self
Organizing Map (ISOM) with Discret Wavelet Transform (DWT) by applying
filtering techniques which play a crucial role in enhancing the quality of a
reconstructed image. The experimental result shows that the proposed technique
obtained better compression ratio value.

negative:
Document Image Analysis, like any Digital Image Analysis requires
identification and extraction of proper features, which are generally extracted
from uncompressed images, though in reality images are made available in
compressed form for the reasons such as transmission and storage efficiency.
However, this implies that the compressed image should be decompressed, which
indents additional computing resources. This limitation induces the motivation
to research in extracting features directly from the compressed image. In this
research, we propose to extract essential features such as projection profile,
run-histogram and entropy for text document analysis directly from run-length
compressed text-documents. The experimentation illustrates that features are
extracted directly from the compressed image without going through the stage of
decompression, because of which the computing time is reduced. The feature
values so extracted are exactly identical to those extracted from uncompressed
images.

negative:
The inherent noise in the observed (e.g., scanned) binary document image
degrades the image quality and harms the compression ratio through breaking the
pattern repentance and adding entropy to the document images. In this paper, we
design a cost function in Bayesian framework with dictionary learning.
Minimizing our cost function produces a restored image which has better quality
than that of the observed noisy image, and a dictionary for representing and
encoding the image. After the restoration, we use this dictionary (from the
same cost function) to encode the restored image following the
symbol-dictionary framework by JBIG2 standard with the lossless mode.
Experimental results with a variety of document images demonstrate that our
method improves the image quality compared with the observed image, and
simultaneously improves the compression ratio. For the test images with
synthetic noise, our method reduces the number of flipped pixels by 48.2% and
improves the compression ratio by 36.36% as compared with the best encoding
methods. For the test images with real noise, our method visually improves the
image quality, and outperforms the cutting-edge method by 28.27% in terms of
the compression ratio.

negative:
For lossy image compression, we develop a neural-based system which learns a
nonlinear estimator for decoding from quantized representations. The system
links two recurrent networks that \help" each other reconstruct same target
image patches using complementary portions of spatial context that communicate
via gradient signals. This dual agent system builds upon prior work that
proposed the iterative refinement algorithm for recurrent neural network
(RNN)based decoding which improved image reconstruction compared to standard
decoding techniques. Our approach, which works with any encoder, neural or
non-neural, This system progressively reduces image patch reconstruction error
over a fixed number of steps. Experiment with variants of RNN memory cells,
with and without future information, find that our model consistently creates
lower distortion images of higher perceptual quality compared to other
approaches. Specifically, on the Kodak Lossless True Color Image Suite, we
observe as much as a 1:64 decibel (dB) gain over JPEG, a 1:46 dB gain over JPEG
2000, a 1:34 dB gain over the GOOG neural baseline, 0:36 over E2E (a modern
competitive neural compression model), and 0:37 over a single iterative neural
decoder.

negative:
Convolutional Neural Networks (CNNs) are state-of-the-art models for document
image classification tasks. However, many of these approaches rely on
parameters and architectures designed for classifying natural images, which
differ from document images. We question whether this is appropriate and
conduct a large empirical study to find what aspects of CNNs most affect
performance on document images. Among other results, we exceed the
state-of-the-art on the RVL-CDIP dataset by using shear transform data
augmentation and an architecture designed for a larger input image.
Additionally, we analyze the learned features and find evidence that CNNs
trained on RVL-CDIP learn region-specific layout features.

negative:
High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid
development of capture technologies. Nevertheless, the dynamic range of common
display devices is still limited, therefore tone mapping (TM) remains a key
challenge for image visualization. Recent work has demonstrated that neural
networks can achieve remarkable performance in this task when compared to
traditional methods, however, the quality of the results of these
learning-based methods is limited by the training data. Most existing works use
as training set a curated selection of best-performing results from existing
traditional tone mapping operators (often guided by a quality metric),
therefore, the quality of newly generated results is fundamentally limited by
the performance of such operators. This quality might be even further limited
by the pool of HDR content that is used for training. In this work we propose a
learning-based self-supervised tone mapping operator that is trained at test
time specifically for each HDR image and does not need any data labeling. The
key novelty of our approach is a carefully designed loss function built upon
fundamental knowledge on contrast perception that allows for directly comparing
the content in the HDR and tone mapped images. We achieve this goal by
reformulating classic VGG feature maps into feature contrast maps that
normalize local feature differences by their average magnitude in a local
neighborhood, allowing our loss to account for contrast masking effects. We
perform extensive ablation studies and exploration of parameters and
demonstrate that our solution outperforms existing approaches with a single set
of fixed parameters, as confirmed by both objective and subjective metrics.

negative:
Spiking neural networks (SNNs) offer an inherent ability to process
spatial-temporal data, or in other words, realworld sensory data, but suffer
from the difficulty of training high accuracy models. A major thread of
research on SNNs is on converting a pre-trained convolutional neural network
(CNN) to an SNN of the same structure. State-of-the-art conversion methods are
approaching the accuracy limit, i.e., the near-zero accuracy loss of SNN
against the original CNN. However, we note that this is made possible only when
significantly more energy is consumed to process an input. In this paper, we
argue that this trend of "energy for accuracy" is not necessary -- a little
energy can go a long way to achieve the near-zero accuracy loss. Specifically,
we propose a novel CNN-to-SNN conversion method that is able to use a
reasonably short spike train (e.g., 256 timesteps for CIFAR10 images) to
achieve the near-zero accuracy loss. The new conversion method, named as
explicit current control (ECC), contains three techniques (current
normalisation, thresholding for residual elimination, and consistency
maintenance for batch-normalisation), in order to explicitly control the
currents flowing through the SNN when processing inputs. We implement ECC into
a tool nicknamed SpKeras, which can conveniently import Keras CNN models and
convert them into SNNs. We conduct an extensive set of experiments with the
tool -- working with VGG16 and various datasets such as CIFAR10 and CIFAR100 --
and compare with state-of-the-art conversion methods. Results show that ECC is
a promising method that can optimise over energy consumption and accuracy loss
simultaneously.

negative:
High dynamic range (HDR) imaging is a technique that allows an extensive
dynamic range of exposures, which is important in image processing, computer
graphics, and computer vision. In recent years, there has been a significant
advancement in HDR imaging using deep learning (DL). This study conducts a
comprehensive and insightful survey and analysis of recent developments in deep
HDR imaging methodologies. We hierarchically and structurally group existing
deep HDR imaging methods into five categories based on (1) number/domain of
input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel
learning strategies, and (5) applications. Importantly, we provide a
constructive discussion on each category regarding its potential and
challenges. Moreover, we review some crucial aspects of deep HDR imaging, such
as datasets and evaluation metrics. Finally, we highlight some open problems
and point out future research directions.

negative:
In this paper, we propose a deep snapshot high dynamic range (HDR) imaging
framework that can effectively reconstruct an HDR image from the RAW data
captured using a multi-exposure color filter array (ME-CFA), which consists of
a mosaic pattern of RGB filters with different exposure levels. To effectively
learn the HDR image reconstruction network, we introduce the idea of luminance
normalization that simultaneously enables effective loss computation and input
data normalization by considering relative local contrasts in the
"normalized-by-luminance" HDR domain. This idea makes it possible to equally
handle the errors in both bright and dark areas regardless of absolute
luminance levels, which significantly improves the visual image quality in a
tone-mapped domain. Experimental results using two public HDR image datasets
demonstrate that our framework outperforms other snapshot methods and produces
high-quality HDR images with fewer visual artifacts.

negative:
Segmentation of additive manufacturing (AM) defects in X-ray Computed
Tomography (XCT) images is challenging, due to the poor contrast, small sizes
and variation in appearance of defects. Automatic segmentation can, however,
provide quality control for additive manufacturing. Over recent years,
three-dimensional convolutional neural networks (3D CNNs) have performed well
in the volumetric segmentation of medical images. In this work, we leverage
techniques from the medical imaging domain and propose training a 3D U-Net
model to automatically segment defects in XCT images of AM samples. This work
not only contributes to the use of machine learning for AM defect detection but
also demonstrates for the first time 3D volumetric segmentation in AM. We train
and test with three variants of the 3D U-Net on an AM dataset, achieving a mean
intersection of union (IOU) value of 88.4%.

negative:
Recently, experiments have been reported where researchers were able to
perform high dynamic range (HDR) tomography in a heuristic fashion, by fusing
multiple tomographic projections. This approach to HDR tomography has been
inspired by HDR photography and inherits the same disadvantages. Taking a
computational imaging approach to the HDR tomography problem, we here suggest a
new model based on the Modulo Radon Transform (MRT), which we rigorously
introduce and analyze. By harnessing a joint design between hardware and
algorithms, we present a single-shot HDR tomography approach, which to our
knowledge, is the only approach that is backed by mathematical guarantees.
  On the hardware front, instead of recording the Radon Transform projections
that my potentially saturate, we propose to measure modulo values of the same.
This ensures that the HDR measurements are folded into a lower dynamic range.
On the algorithmic front, our recovery algorithms reconstruct the HDR images
from folded measurements. Beyond mathematical aspects such as injectivity and
inversion of the MRT for different scenarios including band-limited and
approximately compactly supported images, we also provide a first
proof-of-concept demonstration. To do so, we implement MRT by experimentally
folding tomographic measurements available as an open source data set using our
custom designed modulo hardware. Our reconstruction clearly shows the
advantages of our approach for experimental data. In this way, our MRT based
solution paves a path for HDR acquisition in a number of related imaging
problems.

negative:
Image correction aims to adjust an input image into a visually pleasing one.
Existing approaches are proposed mainly from the perspective of image pixel
manipulation. They are not effective to recover the details in the under/over
exposed regions. In this paper, we revisit the image formation procedure and
notice that the missing details in these regions exist in the corresponding
high dynamic range (HDR) data. These details are well perceived by the human
eyes but diminished in the low dynamic range (LDR) domain because of the tone
mapping process. Therefore, we formulate the image correction task as an HDR
transformation process and propose a novel approach called Deep Reciprocating
HDR Transformation (DRHT). Given an input LDR image, we first reconstruct the
missing details in the HDR domain. We then perform tone mapping on the
predicted HDR data to generate the output LDR image with the recovered details.
To this end, we propose a united framework consisting of two CNNs for HDR
reconstruction and tone mapping. They are integrated end-to-end for joint
training and prediction. Experiments on the standard benchmarks demonstrate
that the proposed method performs favorably against state-of-the-art image
correction methods.

negative:
Tone mapping is a commonly used technique that maps the set of colors in
high-dynamic-range (HDR) images to another set of colors in low-dynamic-range
(LDR) images, to fit the need for print-outs, LCD monitors and projectors.
Unfortunately, during the compression of dynamic range, the overall contrast
and local details generally cannot be preserved simultaneously. Recently, with
the increased use of stereoscopic devices, the notion of binocular tone mapping
has been proposed in the existing research study. However, the existing
research lacks the binocular perception study and is unable to generate the
optimal binocular pair that presents the most visual content. In this paper, we
propose a novel perception-based binocular tone mapping method, that can
generate an optimal binocular image pair (generating left and right images
simultaneously) from an HDR image that presents the most visual content by
designing a binocular perception metric. Our method outperforms the existing
method in terms of both visual and time performance.

negative:
Stack-based high dynamic range (HDR) imaging is a technique for achieving a
larger dynamic range in an image by combining several low dynamic range images
acquired at different exposures. Minimizing the set of images to combine, while
ensuring that the resulting HDR image fully captures the scene's irradiance, is
important to avoid long image acquisition and post-processing times. The
problem of selecting the set of images has received much attention. However,
existing methods either are not fully automatic, can be slow, or can fail to
fully capture more challenging scenes. In this paper, we propose a fully
automatic method for selecting the set of exposures to acquire that is both
fast and more accurate. We show on an extensive set of benchmark scenes that
our proposed method leads to improved HDR images as measured against ground
truth using the mean squared error, a pixel-based metric, and a visible
difference predictor and a quality score, both perception-based metrics.

negative:
High dynamic range (HDR) photography is becoming increasingly popular and
available by DSLR and mobile-phone cameras. While deep neural networks (DNN)
have greatly impacted other domains of image manipulation, their use for HDR
tone-mapping is limited due to the lack of a definite notion of ground-truth
solution, which is needed for producing training data.
  In this paper we describe a new tone-mapping approach guided by the distinct
goal of producing low dynamic range (LDR) renditions that best reproduce the
visual characteristics of native LDR images. This goal enables the use of an
unpaired adversarial training based on unrelated sets of HDR and LDR images,
both of which are widely available and easy to acquire.
  In order to achieve an effective training under this minimal requirements, we
introduce the following new steps and components: (i) a range-normalizing
pre-process which estimates and applies a different level of curve-based
compression, (ii) a loss that preserves the input content while allowing the
network to achieve its goal, and (iii) the use of a more concise discriminator
network, designed to promote the reproduction of low-level attributes native
LDR possess.
  Evaluation of the resulting network demonstrates its ability to produce
photo-realistic artifact-free tone-mapped images, and state-of-the-art
performance on different image fidelity indices and visual distances.

negative:
We proposed a lossless two-layer HDR coding method using a histogram packing
technique. The proposed method was demonstrated to outperform the normative
JPEG XT encoder, under the use of the default tone-mapping operator. However,
the performance under various tone-mapping operators has not been discussed. In
this paper, we aim to compare the performance of the proposed method with that
of the JPEG XT encoder under the use of various tone-mapping operators to
clearly show the characteristic difference between them.

negative:
With the development of human communications the usage of Visual
Communications has also increased. The advancement of image compression methods
is one of the main reasons for the enhancement. This paper first presents main
modes of image compression methods such as JPEG and JPEG2000 without
mathematical details. Also, the paper describes gradient Haar wavelet
transforms in order to construct a preliminary image compression algorithm.
Then, a new image compression method is proposed based on the preliminary image
compression algorithm that can improve standards of image compression. The new
method is compared with original modes of JPEG and JPEG2000 (based on Haar
wavelet) by image quality measures such as MAE, PSNAR, and SSIM. The image
quality and statistical results confirm that can boost image compression
standards. It is suggested that the new method is used in a part or all of an
image compression standard.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
This paper study the reconstruction of High Dynamic Range (HDR) video from
snapshot-coded LDR video. Constructing an HDR video requires restoring the HDR
values for each frame and maintaining the consistency between successive
frames. HDR image acquisition from single image capture, also known as snapshot
HDR imaging, can be achieved in several ways. For example, the reconfigurable
snapshot HDR camera is realized by introducing an optical element into the
optical stack of the camera; by placing a coded mask at a small standoff
distance in front of the sensor. High-quality HDR image can be recovered from
the captured coded image using deep learning methods. This study utilizes
3D-CNNs to perform a joint demosaicking, denoising, and HDR video
reconstruction from coded LDR video. We enforce more temporally consistent HDR
video reconstruction by introducing a temporal loss function that considers the
short-term and long-term consistency. The obtained results are promising and
could lead to affordable HDR video capture using conventional cameras.

negative:
Since the low quality of document images will greatly undermine the chances
of success in automatic text recognition and analysis, it is necessary to
assess the quality of document images uploaded in online business process, so
as to reject those images of low quality. In this paper, we attempt to achieve
document image quality assessment and our contributions are twofold. Firstly,
since document image quality assessment is more interested in text, we propose
a text line based framework to estimate document image quality, which is
composed of three stages: text line detection, text line quality prediction,
and overall quality assessment. Text line detection aims to find potential text
lines with a detector. In the text line quality prediction stage, the quality
score is computed for each text line with a CNN-based prediction model. The
overall quality of document images is finally assessed with the ensemble of all
text line quality. Secondly, to train the prediction model, a large-scale
dataset, comprising 52,094 text line images, is synthesized with diverse
attributes. For each text line image, a quality label is computed with a
piece-wise function. To demonstrate the effectiveness of the proposed
framework, comprehensive experiments are evaluated on two popular document
image quality assessment benchmarks. Our framework significantly outperforms
the state-of-the-art methods by large margins on the large and complicated
dataset.

negative:
Sparse decomposition has been extensively used for different applications
including signal compression and denoising and document analysis. In this
paper, sparse decomposition is used for image segmentation. The proposed
algorithm separates the background and foreground using a sparse-smooth
decomposition technique such that the smooth and sparse components correspond
to the background and foreground respectively. This algorithm is tested on
several test images from HEVC test sequences and is shown to have superior
performance over other methods, such as the hierarchical k-means clustering in
DjVu. This segmentation algorithm can also be used for text extraction, video
compression and medical image segmentation.

negative:
The standard JPEG format is almost the optimum format in image compression.
The compression ratio in JPEG sometimes reaches 30:1. The compression ratio of
JPEG could be increased by embedding the Five Modulus Method (FMM) into the
JPEG algorithm. The novel algorithm gives twice the time as the standard JPEG
algorithm or more. The novel algorithm was called FJPEG (Five-JPEG). The
quality of the reconstructed image after compression is approximately
approaches the JPEG. Standard test images have been used to support and
implement the suggested idea in this paper and the error metrics have been
computed and compared with JPEG.

negative:
Recently, high dynamic range (HDR) image reconstruction based on the multiple
exposure stack from a given single exposure utilizes a deep learning framework
to generate high-quality HDR images. These conventional networks focus on the
exposure transfer task to reconstruct the multi-exposure stack. Therefore, they
often fail to fuse the multi-exposure stack into a perceptually pleasant HDR
image as the inversion artifacts occur. We tackle the problem in stack
reconstruction-based methods by proposing a novel framework with a fully
differentiable high dynamic range imaging (HDRI) process. By explicitly using
the loss, which compares the network's output with the ground truth HDR image,
our framework enables a neural network that generates the multiple exposure
stack for HDRI to train stably. In other words, our differentiable HDR
synthesis layer helps the deep neural network to train to create multi-exposure
stacks while reflecting the precise correlations between multi-exposure images
in the HDRI process. In addition, our network uses the image decomposition and
the recursive process to facilitate the exposure transfer task and to
adaptively respond to recursion frequency. The experimental results show that
the proposed network outperforms the state-of-the-art quantitative and
qualitative results in terms of both the exposure transfer tasks and the whole
HDRI process.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
Digital camera and mobile document image acquisition are new trends arising
in the world of Optical Character Recognition and text detection. In some
cases, such process integrates many distortions and produces poorly scanned
text or text-photo images and natural images, leading to an unreliable OCR
digitization. In this paper, we present a novel nonparametric and unsupervised
method to compensate for undesirable document image distortions aiming to
optimally improve OCR accuracy. Our approach relies on a very efficient stack
of document image enhancing techniques to recover deformation of the entire
document image. First, we propose a local brightness and contrast adjustment
method to effectively handle lighting variations and the irregular distribution
of image illumination. Second, we use an optimized greyscale conversion
algorithm to transform our document image to greyscale level. Third, we sharpen
the useful information in the resulting greyscale image using Un-sharp Masking
method. Finally, an optimal global binarization approach is used to prepare the
final document image to OCR recognition. The proposed approach can
significantly improve text detection rate and optical character recognition
accuracy. To demonstrate the efficiency of our approach, an exhaustive
experimentation on a standard dataset is presented.

negative:
Estimating the primary quantization matrix of double JPEG compressed images
is a problem of relevant importance in image forensics since it allows to infer
important information about the past history of an image. In addition, the
inconsistencies of the primary quantization matrices across different image
regions can be used to localize splicing in double JPEG tampered images.
Traditional model-based approaches work under specific assumptions on the
relationship between the first and second compression qualities and on the
alignment of the JPEG grid. Recently, a deep learning-based estimator capable
to work under a wide variety of conditions has been proposed, that outperforms
tailored existing methods in most of the cases. The method is based on a
Convolutional Neural Network (CNN) that is trained to solve the estimation as a
standard regression problem. By exploiting the integer nature of the
quantization coefficients, in this paper, we propose a deep learning technique
that performs the estimation by resorting to a simil-classification
architecture. The CNN is trained with a loss function that takes into account
both the accuracy and the Mean Square Error (MSE) of the estimation. Results
confirm the superior performance of the proposed technique, compared to the
state-of-the art methods based on statistical analysis and, in particular, deep
learning regression. Moreover, the capability of the method to work under
general operative conditions, regarding the alignment of the second compression
grid with the one of first compression and the combinations of the JPEG
qualities of former and second compression, is very relevant in practical
applications, where these information are unknown a priori.

negative:
Machine learning is penetrating various domains virtually, thereby
proliferating excellent results. It has also found an outlet in digital
forensics, wherein it is becoming the prime driver of computational efficiency.
A prominent feature that exhibits the effectiveness of ML algorithms is feature
extraction that can be instrumental in the applications for digital forensics.
Convolutional Neural Networks are further used to identify parts of the file.
To this end, we observed that the literature does not include sufficient
information about the identification of the algorithms used to compress file
fragments. With this research, we attempt to address this gap as compression
algorithms are beneficial in generating higher entropy comparatively as they
make the data more compact. We used a base dataset, compressed every file with
various algorithms, and designed a model based on that. The used model was
accurately able to identify files compressed using compress, lzip and bzip2.

negative:
Document image quality assessment (DIQA) is an important and challenging
problem in real applications. In order to predict the quality scores of
document images, this paper proposes a novel no-reference DIQA method based on
character gradient, where the OCR accuracy is used as a ground-truth quality
metric. Character gradient is computed on character patches detected with the
maximally stable extremal regions (MSER) based method. Character patches are
essentially significant to character recognition and therefore suitable for use
in estimating document image quality. Experiments on a benchmark dataset show
that the proposed method outperforms the state-of-the-art methods in estimating
the quality score of document images.

negative:
Recent improvements to Generative Adversarial Networks (GANs) have made it
possible to generate realistic images in high resolution based on natural
language descriptions such as image captions. Furthermore, conditional GANs
allow us to control the image generation process through labels or even natural
language descriptions. However, fine-grained control of the image layout, i.e.
where in the image specific objects should be located, is still difficult to
achieve. This is especially true for images that should contain multiple
distinct objects at different spatial locations. We introduce a new approach
which allows us to control the location of arbitrarily many objects within an
image by adding an object pathway to both the generator and the discriminator.
Our approach does not need a detailed semantic layout but only bounding boxes
and the respective labels of the desired objects are needed. The object pathway
focuses solely on the individual objects and is iteratively applied at the
locations specified by the bounding boxes. The global pathway focuses on the
image background and the general image layout. We perform experiments on the
Multi-MNIST, CLEVR, and the more complex MS-COCO data set. Our experiments show
that through the use of the object pathway we can control object locations
within images and can model complex scenes with multiple objects at various
locations. We further show that the object pathway focuses on the individual
objects and learns features relevant for these, while the global pathway
focuses on global image characteristics and the image background.

negative:
Despite significant progress on current state-of-the-art image generation
models, synthesis of document images containing multiple and complex object
layouts is a challenging task. This paper presents a novel approach, called
DocSynth, to automatically synthesize document images based on a given layout.
In this work, given a spatial layout (bounding boxes with object categories) as
a reference by the user, our proposed DocSynth model learns to generate a set
of realistic document images consistent with the defined layout. Also, this
framework has been adapted to this work as a superior baseline model for
creating synthetic document image datasets for augmenting real data during
training for document layout analysis tasks. Different sets of learning
objectives have been also used to improve the model performance.
Quantitatively, we also compare the generated results of our model with real
data using standard evaluation metrics. The results highlight that our model
can successfully generate realistic and diverse document images with multiple
objects. We also present a comprehensive qualitative analysis summary of the
different scopes of synthetic image generation tasks. Lastly, to our knowledge
this is the first work of its kind.

negative:
Documents often exhibit various forms of degradation, which make it hard to
be read and substantially deteriorate the performance of an OCR system. In this
paper, we propose an effective end-to-end framework named Document Enhancement
Generative Adversarial Networks (DE-GAN) that uses the conditional GANs (cGANs)
to restore severely degraded document images. To the best of our knowledge,
this practice has not been studied within the context of generative adversarial
deep networks. We demonstrate that, in different tasks (document clean up,
binarization, deblurring and watermark removal), DE-GAN can produce an enhanced
version of the degraded document with a high quality. In addition, our approach
provides consistent improvements compared to state-of-the-art methods over the
widely used DIBCO 2013, DIBCO 2017 and H-DIBCO 2018 datasets, proving its
ability to restore a degraded document image to its ideal condition. The
obtained results on a wide variety of degradation reveal the flexibility of the
proposed model to be exploited in other document enhancement problems.

negative:
This work analyses the impact of self-supervised pre-training on document
images in the context of document image classification. While previous
approaches explore the effect of self-supervision on natural images, we show
that patch-based pre-training performs poorly on document images because of
their different structural properties and poor intra-sample semantic
information. We propose two context-aware alternatives to improve performance
on the Tobacco-3482 image classification task. We also propose a novel method
for self-supervision, which makes use of the inherent multi-modality of
documents (image and text), which performs better than other popular
self-supervised methods, including supervised ImageNet pre-training, on
document image classification scenarios with a limited amount of data.

negative:
Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.

negative:
Recognition of document images have important applications in restoring old
and classical texts. The problem involves quality improvement before passing it
to a properly trained OCR to get accurate recognition of the text. The image
enhancement and quality improvement constitute important steps as subsequent
recognition depends upon the quality of the input image. There are scenarios
when high resolution images are not available and our experiments show that the
OCR accuracy reduces significantly with decrease in the spatial resolution of
document images. Thus the only option is to improve the resolution of such
document images. The goal is to construct a high resolution image, given a
single low resolution binary image, which constitutes the problem of single
image super-resolution. Most of the previous work in super-resolution deal with
natural images which have more information-content than the document images.
Here, we use Convolution Neural Network to learn the mapping between low and
the corresponding high resolution images. We experiment with different number
of layers, parameter settings and non-linear functions to build a fast
end-to-end framework for document image super-resolution. Our proposed model
shows a very good PSNR improvement of about 4 dB on 75 dpi Tamil images,
resulting in a 3 % improvement of word level accuracy by the OCR. It takes less
time than the recent sparse based natural image super-resolution technique,
making it useful for real-time document recognition applications.

negative:
One of the major prerequisites for any deep learning approach is the
availability of large-scale training data. When dealing with scanned document
images in real world scenarios, the principal information of its content is
stored in the layout itself. In this work, we have proposed an automated deep
generative model using Graph Neural Networks (GNNs) to generate synthetic data
with highly variable and plausible document layouts that can be used to train
document interpretation systems, in this case, specially in digital mailroom
applications. It is also the first graph-based approach for document layout
generation task experimented on administrative document images, in this case,
invoices.

negative:
Line separators are used to segregate text-lines from one another in document
image analysis. Finding the separator points at every line terminal in a
document image would enable text-line segmentation. In particular, identifying
the separators in handwritten text could be a thrilling exercise. Obviously it
would be challenging to perform this in the compressed version of a document
image and that is the proposed objective in this research. Such an effort would
prevent the computational burden of decompressing a document for text-line
segmentation. Since document images are generally compressed using run length
encoding (RLE) technique as per the CCITT standards, the first column in the
RLE will be a white column. The value (depth) in the white column is very low
when a particular line is a text line and the depth could be larger at the
point of text line separation. A longer consecutive sequence of such larger
depth should indicate the gap between the text lines, which provides the
separator region. In case of over separation and under separation issues,
corrective actions such as deletion and insertion are suggested respectively.
An extensive experimentation is conducted on the compressed images of the
benchmark datasets of ICDAR13 and Alireza et al [17] to demonstrate the
efficacy.

negative:
Document image binarization is often a challenging task due to various forms
of degradation. Although there exist several binarization techniques in
literature, the binarized image is typically sensitive to control parameter
settings of the employed technique. This paper presents an automatic document
image binarization algorithm to segment the text from heavily degraded document
images. The proposed technique uses a two band-pass filtering approach for
background noise removal, and Bayesian optimization for automatic
hyperparameter selection for optimal results. The effectiveness of the proposed
binarization technique is empirically demonstrated on the Document Image
Binarization Competition (DIBCO) and the Handwritten Document Image
Binarization Competition (H-DIBCO) datasets.

negative:
While the importance of automatic image analysis is continuously increasing,
recent meta-research revealed major flaws with respect to algorithm validation.
Performance metrics are particularly key for meaningful, objective, and
transparent performance assessment and validation of the used automatic
algorithms, but relatively little attention has been given to the practical
pitfalls when using specific metrics for a given image analysis task. These are
typically related to (1) the disregard of inherent metric properties, such as
the behaviour in the presence of class imbalance or small target structures,
(2) the disregard of inherent data set properties, such as the non-independence
of the test cases, and (3) the disregard of the actual biomedical domain
interest that the metrics should reflect. This living dynamically document has
the purpose to illustrate important limitations of performance metrics commonly
applied in the field of image analysis. In this context, it focuses on
biomedical image analysis problems that can be phrased as image-level
classification, semantic segmentation, instance segmentation, or object
detection task. The current version is based on a Delphi process on metrics
conducted by an international consortium of image analysis experts from more
than 60 institutions worldwide.

negative:
The digital camera captured document images may often be warped and distorted
due to different camera angles or document surfaces. A robust technique is
needed to solve this kind of distortion. The research on dewarping of the
document suffers due to the limited availability of benchmark public dataset.
In recent times, deep learning based approaches are used to solve the problems
accurately. To train most of the deep neural networks a large number of
document images is required and generating such a large volume of document
images manually is difficult. In this paper, we propose a technique to generate
a synthetic warped image from a flat-bedded scanned document image. It is done
by calculating warping factors for each pixel position using two warping
position parameters (WPP) and eight warping control parameters (WCP). These
parameters can be specified as needed depending upon the desired warping. The
results are compared with similar real captured images both qualitative and
quantitative way.

negative:
Single-image high dynamic range (SI-HDR) reconstruction has recently emerged
as a problem well-suited for deep learning methods. Each successive technique
demonstrates an improvement over existing methods by reporting higher image
quality scores. This paper, however, highlights that such improvements in
objective metrics do not necessarily translate to visually superior images. The
first problem is the use of disparate evaluation conditions in terms of data
and metric parameters, calling for a standardized protocol to make it possible
to compare between papers. The second problem, which forms the main focus of
this paper, is the inherent difficulty in evaluating SI-HDR reconstructions
since certain aspects of the reconstruction problem dominate objective
differences, thereby introducing a bias. Here, we reproduce a typical
evaluation using existing as well as simulated SI-HDR methods to demonstrate
how different aspects of the problem affect objective quality metrics.
Surprisingly, we found that methods that do not even reconstruct HDR
information can compete with state-of-the-art deep learning methods. We show
how such results are not representative of the perceived quality and that
SI-HDR reconstruction needs better evaluation protocols.

negative:
We introduce the Brno Mobile OCR Dataset (B-MOD) for document Optical
Character Recognition from low-quality images captured by handheld mobile
devices. While OCR of high-quality scanned documents is a mature field where
many commercial tools are available, and large datasets of text in the wild
exist, no existing datasets can be used to develop and test document OCR
methods robust to non-uniform lighting, image blur, strong noise, built-in
denoising, sharpening, compression and other artifacts present in many
photographs from mobile devices.
  This dataset contains 2 113 unique pages from random scientific papers, which
were photographed by multiple people using 23 different mobile devices. The
resulting 19 728 photographs of various visual quality are accompanied by
precise positions and text annotations of 500k text lines. We further provide
an evaluation methodology, including an evaluation server and a testset with
non-public annotations.
  We provide a state-of-the-art text recognition baseline build on
convolutional and recurrent neural networks trained with Connectionist Temporal
Classification loss. This baseline achieves 2 %, 22 % and 73 % word error rates
on easy, medium and hard parts of the dataset, respectively, confirming that
the dataset is challenging.
  The presented dataset will enable future development and evaluation of
document analysis for low-quality images. It is primarily intended for
line-level text recognition, and can be further used for line localization,
layout analysis, image restoration and text binarization.

negative:
In this work, a region-based Deep Convolutional Neural Network framework is
proposed for document structure learning. The contribution of this work
involves efficient training of region based classifiers and effective
ensembling for document image classification. A primary level of `inter-domain'
transfer learning is used by exporting weights from a pre-trained VGG16
architecture on the ImageNet dataset to train a document classifier on whole
document images. Exploiting the nature of region based influence modelling, a
secondary level of `intra-domain' transfer learning is used for rapid training
of deep learning models for image segments. Finally, stacked generalization
based ensembling is utilized for combining the predictions of the base deep
neural network models. The proposed method achieves state-of-the-art accuracy
of 92.2% on the popular RVL-CDIP document image dataset, exceeding benchmarks
set by existing algorithms.

negative:
This paper presents a novel iterative deep learning framework and apply it
for document enhancement and binarization. Unlike the traditional methods which
predict the binary label of each pixel on the input image, we train the neural
network to learn the degradations in document images and produce the uniform
images of the degraded input images, which allows the network to refine the
output iteratively. Two different iterative methods have been studied in this
paper: recurrent refinement (RR) which uses the same trained neural network in
each iteration for document enhancement and stacked refinement (SR) which uses
a stack of different neural networks for iterative output refinement. Given the
learned uniform and enhanced image, the binarization map can be easy to obtain
by a global or local threshold. The experimental results on several public
benchmark data sets show that our proposed methods provide a new clean version
of the degraded image which is suitable for visualization and promising results
of binarization using the global Otsu's threshold based on the enhanced images
learned iteratively by the neural network.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.

negative:
We propose an automatic method to infer high dynamic range illumination from
a single, limited field-of-view, low dynamic range photograph of an indoor
scene. In contrast to previous work that relies on specialized image capture,
user input, and/or simple scene models, we train an end-to-end deep neural
network that directly regresses a limited field-of-view photo to HDR
illumination, without strong assumptions on scene geometry, material
properties, or lighting. We show that this can be accomplished in a three step
process: 1) we train a robust lighting classifier to automatically annotate the
location of light sources in a large dataset of LDR environment maps, 2) we use
these annotations to train a deep neural network that predicts the location of
lights in a scene from a single limited field-of-view photo, and 3) we
fine-tune this network using a small dataset of HDR environment maps to predict
light intensities. This allows us to automatically recover high-quality HDR
illumination estimates that significantly outperform previous state-of-the-art
methods. Consequently, using our illumination estimates for applications like
3D object insertion, we can achieve results that are photo-realistic, which is
validated via a perceptual user study.

negative:
In this work, we adhere to explore a Multi-Tasking learning (MTL) based
network to perform document attribute classification such as the font type,
font size, font emphasis and scanning resolution classification of a document
image. To accomplish these tasks, we operate on either segmented word level or
on uniformed size patches randomly cropped out of the document. Furthermore, a
hybrid convolution neural network (CNN) architecture "MTL+MI", which is based
on the combination of MTL and Multi-Instance (MI) of patch and word is used to
accomplish joint learning for the classification of the same document
attributes. The contribution of this paper are three fold: firstly, based on
segmented word images and patches, we present a MTL based network for the
classification of a full document image. Secondly, we propose a MTL and MI
(using segmented words and patches) based combined CNN architecture ("MTL+MI")
for the classification of same document attributes. Thirdly, based on the
multi-tasking classifications of the words and/or patches, we propose an
intelligent voting system which is based on the posterior probabilities of each
words and/or patches to perform the classification of document's attributes of
complete document image.

negative:
The demand for on-device document recognition systems increases in
conjunction with the emergence of more strict privacy and security
requirements. In such systems, there is no data transfer from the end device to
a third-party information processing servers. The response time is vital to the
user experience of on-device document recognition. Combined with the
unavailability of discrete GPUs, powerful CPUs, or a large RAM capacity on
consumer-grade end devices such as smartphones, the time limitations put
significant constraints on the computational complexity of the applied
algorithms for on-device execution.
  In this work, we consider document location in an image without prior
knowledge of the document content or its internal structure. In accordance with
the published works, at least 5 systems offer solutions for on-device document
location. All these systems use a location method which can be considered
Hough-based. The precision of such systems seems to be lower than that of the
state-of-the-art solutions which were not designed to account for the limited
computational resources.
  We propose an advanced Hough-based method. In contrast with other approaches,
it accounts for the geometric invariants of the central projection model and
combines both edge and color features for document boundary detection. The
proposed method allowed for the second best result for SmartDoc dataset in
terms of precision, surpassed by U-net like neural network. When evaluated on a
more challenging MIDV-500 dataset, the proposed algorithm guaranteed the best
precision compared to published methods. Our method retained the applicability
to on-device computations.

negative:
We present a framework for learning an efficient holistic representation for
handwritten word images. The proposed method uses a deep convolutional neural
network with traditional classification loss. The major strengths of our work
lie in: (i) the efficient usage of synthetic data to pre-train a deep network,
(ii) an adapted version of the ResNet-34 architecture with the region of
interest pooling (referred to as HWNet v2) which learns discriminative features
for variable sized word images, and (iii) a realistic augmentation of training
data with multiple scales and distortions which mimics the natural process of
handwriting. We further investigate the process of transfer learning to reduce
the domain gap between synthetic and real domain, and also analyze the
invariances learned at different layers of the network using visualization
techniques proposed in the literature.
  Our representation leads to a state-of-the-art word spotting performance on
standard handwritten datasets and historical manuscripts in different languages
with minimal representation size. On the challenging IAM dataset, our method is
first to report an mAP of around 0.90 for word spotting with a representation
size of just 32 dimensions. Furthermore, we also present results on printed
document datasets in English and Indic scripts which validates the generic
nature of the proposed framework for learning word image representation.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
One major drawback of state of the art Neural Networks (NN)-based approaches
for document classification purposes is the large number of training samples
required to obtain an efficient classification. The minimum required number is
around one thousand annotated documents for each class. In many cases it is
very difficult, if not impossible, to gather this number of samples in real
industrial processes. In this paper, we analyse the efficiency of NN-based
document classification systems in a sub-optimal training case, based on the
situation of a company document stream. We evaluated three different
approaches, one based on image content and two on textual content. The
evaluation was divided into four parts: a reference case, to assess the
performance of the system in the lab; two cases that each simulate a specific
difficulty linked to document stream processing; and a realistic case that
combined all of these difficulties. The realistic case highlighted the fact
that there is a significant drop in the efficiency of NN-Based document
classification systems. Although they remain efficient for well represented
classes (with an over-fitting of the system for those classes), it is
impossible for them to handle appropriately less well represented classes.
NN-Based document classification systems need to be adapted to resolve these
two problems before they can be considered for use in a company document
stream.

negative:
Graphical elements: particularly tables and figures contain a visual summary
of the most valuable information contained in a document. Therefore,
localization of such graphical objects in the document images is the initial
step to understand the content of such graphical objects or document images. In
this paper, we present a novel end-to-end trainable deep learning based
framework to localize graphical objects in the document images called as
Graphical Object Detection (GOD). Our framework is data-driven and does not
require any heuristics or meta-data to locate graphical objects in the document
images. The GOD explores the concept of transfer learning and domain adaptation
to handle scarcity of labeled training images for graphical object detection
task in the document images. Performance analysis carried out on the various
public benchmark data sets: ICDAR-2013, ICDAR-POD2017,and UNLV shows that our
model yields promising results as compared to state-of-the-art techniques.

negative:
Identity documents recognition is an important sub-field of document
analysis, which deals with tasks of robust document detection, type
identification, text fields recognition, as well as identity fraud prevention
and document authenticity validation given photos, scans, or video frames of an
identity document capture. Significant amount of research has been published on
this topic in recent years, however a chief difficulty for such research is
scarcity of datasets, due to the subject matter being protected by security
requirements. A few datasets of identity documents which are available lack
diversity of document types, capturing conditions, or variability of document
field values. In addition, the published datasets were typically designed only
for a subset of document recognition problems, not for a complex identity
document analysis. In this paper, we present a dataset MIDV-2020 which consists
of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock
identity documents, each with unique text field values and unique artificially
generated faces, with rich annotation. For the presented benchmark dataset
baselines are provided for such tasks as document location and identification,
text fields recognition, and face detection. With 72409 annotated images in
total, to the date of publication the proposed dataset is the largest publicly
available identity documents dataset with variable artificially generated data,
and we believe that it will prove invaluable for advancement of the field of
document analysis and recognition. The dataset is available for download at
ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr .

negative:
Today all kind of information is getting digitized and along with all this
digitization, the huge archive of various kinds of documents is being digitized
too. We know that, Optical Character Recognition is the method through which,
newspapers and other paper documents convert into digital resources. But, it is
a fact that this method works on texts only. As a result, if we try to process
any document which contains non-textual zones, then we will get garbage texts
as output. That is why; in order to digitize documents properly they should be
prepossessed carefully. And while preprocessing, segmenting document in
different regions according to the category properly is most important. But,
the Optical Character Recognition processes available for Bangla language have
no such algorithm that can categorize a newspaper/book page fully. So we worked
to decompose a document into its several parts like headlines, sub headlines,
columns, images etc. And if the input is skewed and rotated, then the input was
also deskewed and de-rotated. To decompose any Bangla document we found out the
edges of the input image. Then we find out the horizontal and vertical area of
every pixel where it lies in. Later on the input image was cut according to
these areas. Then we pick each and every sub image and found out their
height-width ratio, line height. Then according to these values the sub images
were categorized. To deskew the image we found out the skew angle and de skewed
the image according to this angle. To de-rotate the image we used the line
height, matra line, pixel ratio of matra line.

negative:
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.

negative:
This paper presents results of Document Visual Question Answering Challenge
organized as part of "Text and Documents in the Deep Learning Era" workshop, in
CVPR 2020. The challenge introduces a new problem - Visual Question Answering
on document images. The challenge comprised two tasks. The first task concerns
with asking questions on a single document image. On the other hand, the second
task is set as a retrieval task where the question is posed over a collection
of images. For the task 1 a new dataset is introduced comprising 50,000
questions-answer(s) pairs defined over 12,767 document images. For task 2
another dataset has been created comprising 20 questions over 14,362 document
images which share the same document template.

negative:
We introduce a novel type of text representation that preserves the 2D layout
of a document. This is achieved by encoding each document page as a
two-dimensional grid of characters. Based on this representation, we present a
generic document understanding pipeline for structured documents. This pipeline
makes use of a fully convolutional encoder-decoder network that predicts a
segmentation mask and bounding boxes. We demonstrate its capabilities on an
information extraction task from invoices and show that it significantly
outperforms approaches based on sequential text or document images.

negative:
We introduce FontCode, an information embedding technique for text documents.
Provided a text document with specific fonts, our method embeds user-specified
information in the text by perturbing the glyphs of text characters while
preserving the text content. We devise an algorithm to chooses unobtrusive yet
machine-recognizable glyph perturbations, leveraging a recently developed
generative model that alters the glyphs of each character continuously on a
font manifold. We then introduce an algorithm that embeds a user-provided
message in the text document and produces an encoded document whose appearance
is minimally perturbed from the original document. We also present a glyph
recognition method that recovers the embedded information from an encoded
document stored as a vector graphic or pixel image, or even on a printed paper.
In addition, we introduce a new error-correction coding scheme that rectifies a
certain number of recognition errors. Lastly, we demonstrate that our technique
enables a wide array of applications, using it as a text document metadata
holder, an unobtrusive optical barcode, a cryptographic message embedding
scheme, and a text document signature.

negative:
When digitizing a document into an image, it is common to include a
surrounding border region to visually indicate that the entire document is
present in the image. However, this border should be removed prior to automated
processing. In this work, we present a deep learning based system, PageNet,
which identifies the main page region in an image in order to segment content
from both textual and non-textual border noise. In PageNet, a Fully
Convolutional Network obtains a pixel-wise segmentation which is post-processed
into the output quadrilateral region. We evaluate PageNet on 4 collections of
historical handwritten documents and obtain over 94% mean intersection over
union on all datasets and approach human performance on 2 of these collections.
Additionally, we show that PageNet can segment documents that are overlayed on
top of other documents.

negative:
Document segmentation is a method of rending the document into distinct
regions. A document is an assortment of information and a standard mode of
conveying information to others. Pursuance of data from documents involves ton
of human effort, time intense and might severely prohibit the usage of data
systems. So, automatic information pursuance from the document has become a big
issue. It is been shown that document segmentation will facilitate to beat such
problems. This paper proposes a new approach to segment and classify the
document regions as text, image, drawings and table. Document image is divided
into blocks using Run length smearing rule and features are extracted from
every blocks. Discipulus tool has been used to construct the Genetic
programming based classifier model and located 97.5% classification accuracy.

negative:
This report explores the latest advances in the field of digital document
recognition. With the focus on printed document imagery, we discuss the major
developments in optical character recognition (OCR) and document image
enhancement/restoration in application to Latin and non-Latin scripts. In
addition, we review and discuss the available technologies for hand-written
document recognition. In this report, we also provide some company-accumulated
benchmark results on available OCR engines.