query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am currently engaged in a project aimed at identifying critical areas within medical images, such as X-rays, to aid in disease diagnosis. For instance, in lung X-rays, the region with a tumor is of greater significance, while other areas may not be as relevant. I am seeking a method to pinpoint these crucial areas in the images. Subsequently, we can enhance efficiency by capturing high-resolution images of these important areas. I am interested in models that can execute this concept by initially analyzing a low-resolution image, detecting significant features, and then focusing on more localized but detailed aspects. Additionally, I require techniques that can deliver the same performance level but with reduced computational power and storage, considering the vast quantity of medical images. I need a newer and larger dataset to train and test the model. I would also appreciate recommendations for other image analysis models that are more efficient yet still yield accurate results.

positive:
To enable a deep learning-based system to be used in the medical domain as a
computer-aided diagnosis system, it is essential to not only classify diseases
but also present the locations of the diseases. However, collecting
instance-level annotations for various thoracic diseases is expensive.
Therefore, weakly supervised localization methods have been proposed that use
only image-level annotation. While the previous methods presented the disease
location as the most discriminative part for classification, this causes a deep
network to localize wrong areas for indistinguishable X-ray images. To solve
this issue, we propose a spatial attention method using disease masks that
describe the areas where diseases mainly occur. We then apply the spatial
attention to find the precise disease area by highlighting the highest
probability of disease occurrence. Meanwhile, the various sizes, rotations and
noise in chest X-ray images make generating the disease masks challenging. To
reduce the variation among images, we employ an alignment module to transform
an input X-ray image into a generalized image. Through extensive experiments on
the NIH-Chest X-ray dataset with eight kinds of diseases, we show that the
proposed method results in superior localization performances compared to
state-of-the-art methods.

positive:
COVID-19 is a virus with high transmission rate that demands rapid
identification of the infected patients to reduce the spread of the disease.
The current gold-standard test, Reverse-Transcription Polymerase Chain Reaction
(RT-PCR), has a high rate of false negatives. Diagnosing from CT-scan images as
a more accurate alternative has the challenge of distinguishing COVID-19 from
other pneumonia diseases. Artificial intelligence can help radiologists and
physicians to accelerate the process of diagnosis, increase its accuracy, and
measure the severity of the disease. We designed a new interpretable deep
neural network to distinguish healthy people, patients with COVID-19, and
patients with other pneumonia diseases from axial lung CT-scan images. Our
model also detects the infected areas and calculates the percentage of the
infected lung volume. We first preprocessed the images to eliminate the batch
effects of different devices, and then adopted a weakly supervised method to
train the model without having any tags for the infected parts. We trained and
evaluated the model on a large dataset of 3359 samples from 6 different medical
centers. The model reached sensitivities of 97.75% and 98.15%, and
specificities of 87% and 81.03% in separating healthy people from the diseased
and COVID-19 from other diseases, respectively. It also demonstrated similar
performance for 1435 samples from 6 different medical centers which proves its
generalizability. The performance of the model on a large diverse dataset, its
generalizability, and interpretability makes it suitable to be used as a
reliable diagnostic system.

positive:
Medical images differ from natural images in significantly higher resolutions
and smaller regions of interest. Because of these differences, neural network
architectures that work well for natural images might not be applicable to
medical image analysis. In this work, we extend the globally-aware multiple
instance classifier, a framework we proposed to address these unique properties
of medical images. This model first uses a low-capacity, yet memory-efficient,
network on the whole image to identify the most informative regions. It then
applies another higher-capacity network to collect details from chosen regions.
Finally, it employs a fusion module that aggregates global and local
information to make a final prediction. While existing methods often require
lesion segmentation during training, our model is trained with only image-level
labels and can generate pixel-level saliency maps indicating possible malignant
findings. We apply the model to screening mammography interpretation:
predicting the presence or absence of benign and malignant lesions. On the NYU
Breast Cancer Screening Dataset, consisting of more than one million images,
our model achieves an AUC of 0.93 in classifying breasts with malignant
findings, outperforming ResNet-34 and Faster R-CNN. Compared to ResNet-34, our
model is 4.1x faster for inference while using 78.4% less GPU memory.
Furthermore, we demonstrate, in a reader study, that our model surpasses
radiologist-level AUC by a margin of 0.11. The proposed model is available
online: https://github.com/nyukat/GMIC.

positive:
Medical image analysis practitioners have embraced big data methodologies.
This has created a need for large annotated datasets. The source of big data is
typically large image collections and clinical reports recorded for these
images. In many cases, however, building algorithms aimed at segmentation and
detection of disease requires a training dataset with markings of the areas of
interest on the image that match with the described anomalies. This process of
annotation is expensive and needs the involvement of clinicians. In this work
we propose two separate deep neural network architectures for automatic marking
of a region of interest (ROI) on the image best representing a finding
location, given a textual report or a set of keywords. One architecture
consists of LSTM and CNN components and is trained end to end with images,
matching text, and markings of ROIs for those images. The output layer
estimates the coordinates of the vertices of a polygonal region. The second
architecture uses a network pre-trained on a large dataset of the same image
types for learning feature representations of the findings of interest. We show
that for a variety of findings from chest X-ray images, both proposed
architectures learn to estimate the ROI, as validated by clinical annotations.
There is a clear advantage obtained from the architecture with pre-trained
imaging network. The centroids of the ROIs marked by this network were on
average at a distance equivalent to 5.1% of the image width from the centroids
of the ground truth ROIs.

positive:
The automatic detection of critical findings in chest X-rays (CXR), such as
pneumothorax, is important for assisting radiologists in their clinical
workflow like triaging time-sensitive cases and screening for incidental
findings. While deep learning (DL) models has become a promising predictive
technology with near-human accuracy, they commonly suffer from a lack of
explainability, which is an important aspect for clinical deployment of DL
models in the highly regulated healthcare industry. For example, localizing
critical findings in an image is useful for explaining the predictions of DL
classification algorithms. While there have been a host of joint classification
and localization methods for computer vision, the state-of-the-art DL models
require locally annotated training data in the form of pixel level labels or
bounding box coordinates. In the medical domain, this requires an expensive
amount of manual annotation by medical experts for each critical finding. This
requirement becomes a major barrier for training models that can rapidly scale
to various findings. In this work, we address these shortcomings with an
interpretable DL algorithm based on multi-instance learning that jointly
classifies and localizes critical findings in CXR without the need for local
annotations. We show competitive classification results on three different
critical findings (pneumothorax, pneumonia, and pulmonary edema) from three
different CXR datasets.

positive:
Advances in deep learning for natural images have prompted a surge of
interest in applying similar techniques to medical images. The majority of the
initial attempts focused on replacing the input of a deep convolutional neural
network with a medical image, which does not take into consideration the
fundamental differences between these two types of images. Specifically, fine
details are necessary for detection in medical images, unlike in natural images
where coarse structures matter most. This difference makes it inadequate to use
the existing network architectures developed for natural images, because they
work on heavily downscaled images to reduce the memory requirements. This hides
details necessary to make accurate predictions. Additionally, a single exam in
medical imaging often comes with a set of views which must be fused in order to
reach a correct conclusion. In our work, we propose to use a multi-view deep
convolutional neural network that handles a set of high-resolution medical
images. We evaluate it on large-scale mammography-based breast cancer screening
(BI-RADS prediction) using 886,000 images. We focus on investigating the impact
of the training set size and image size on the prediction accuracy. Our results
highlight that performance increases with the size of training set, and that
the best performance can only be achieved using the original resolution. In the
reader study, performed on a random subset of the test set, we confirmed the
efficacy of our model, which achieved performance comparable to a committee of
radiologists when presented with the same data.

positive:
Radiologists usually observe anatomical regions of chest X-ray images as well
as the overall image before making a decision. However, most existing deep
learning models only look at the entire X-ray image for classification, failing
to utilize important anatomical information. In this paper, we propose a novel
multi-label chest X-ray classification model that accurately classifies the
image finding and also localizes the findings to their correct anatomical
regions. Specifically, our model consists of two modules, the detection module
and the anatomical dependency module. The latter utilizes graph convolutional
networks, which enable our model to learn not only the label dependency but
also the relationship between the anatomical regions in the chest X-ray. We
further utilize a method to efficiently create an adjacency matrix for the
anatomical regions using the correlation of the label across the different
regions. Detailed experiments and analysis of our results show the
effectiveness of our method when compared to the current state-of-the-art
multi-label chest X-ray image classification methods while also providing
accurate location information.

positive:
Deep learning (DL) has drawn tremendous attention in object localization and
recognition for both natural and medical images. U-Net segmentation models have
demonstrated superior performance compared to conventional handcrafted
feature-based methods. Medical image modality-specific DL models are better at
transferring domain knowledge to a relevant target task than those that are
pretrained on stock photography images. This helps improve model adaptation,
generalization, and class-specific region of interest (ROI) localization. In
this study, we train chest X-ray (CXR) modality-specific U-Nets and other
state-of-the-art U-Net models for semantic segmentation of tuberculosis
(TB)-consistent findings. Automated segmentation of such manifestations could
help radiologists reduce errors and supplement decision-making while improving
patient care and productivity. Our approach uses the publicly available TBX11K
CXR dataset with weak TB annotations, typically provided as bounding boxes, to
train a set of U-Net models. Next, we improve the results by augmenting the
training data with weak localizations, post-processed into an ROI mask, from a
DL classifier that is trained to classify CXRs as showing normal lungs or
suspected TB manifestations. Test data are individually derived from the TBX11K
CXR training distribution and other cross-institutional collections including
the Shenzhen TB and Montgomery TB CXR datasets. We observe that our augmented
training strategy helped the CXR modality-specific U-Net models achieve
superior performance with test data derived from the TBX11K CXR training
distribution as well as from cross-institutional collections (p < 0.05).

positive:
The deployment of automated systems to diagnose diseases from medical images
is challenged by the requirement to localise the diagnosed diseases to justify
or explain the classification decision. This requirement is hard to fulfil
because most of the training sets available to develop these systems only
contain global annotations, making the localisation of diseases a weakly
supervised approach. The main methods designed for weakly supervised disease
classification and localisation rely on saliency or attention maps that are not
specifically trained for localisation, or on region proposals that can not be
refined to produce accurate detections. In this paper, we introduce a new model
that combines region proposal and saliency detection to overcome both
limitations for weakly supervised disease classification and localisation.
Using the ChestX-ray14 data set, we show that our proposed model establishes
the new state-of-the-art for weakly-supervised disease diagnosis and
localisation.

positive:
Building a highly accurate predictive model for classification and
localization of abnormalities in chest X-rays usually requires a large number
of manually annotated labels and pixel regions (bounding boxes) of
abnormalities. However, it is expensive to acquire such annotations, especially
the bounding boxes. Recently, contrastive learning has shown strong promise in
leveraging unlabeled natural images to produce highly generalizable and
discriminative features. However, extending its power to the medical image
domain is under-explored and highly non-trivial, since medical images are much
less amendable to data augmentations. In contrast, their prior knowledge, as
well as radiomic features, is often crucial. To bridge this gap, we propose an
end-to-end semi-supervised knowledge-augmented contrastive learning framework,
that simultaneously performs disease classification and localization tasks. The
key knob of our framework is a unique positive sampling approach tailored for
the medical images, by seamlessly integrating radiomic features as a knowledge
augmentation. Specifically, we first apply an image encoder to classify the
chest X-rays and to generate the image features. We next leverage Grad-CAM to
highlight the crucial (abnormal) regions for chest X-rays (even when
unannotated), from which we extract radiomic features. The radiomic features
are then passed through another dedicated encoder to act as the positive sample
for the image features generated from the same chest X-ray. In this way, our
framework constitutes a feedback loop for image and radiomic modality features
to mutually reinforce each other. Their contrasting yields knowledge-augmented
representations that are both robust and interpretable. Extensive experiments
on the NIH Chest X-ray dataset demonstrate that our approach outperforms
existing baselines in both classification and localization tasks.

positive:
In the last few years, deep learning classifiers have shown promising results
in image-based medical diagnosis. However, interpreting the outputs of these
models remains a challenge. In cancer diagnosis, interpretability can be
achieved by localizing the region of the input image responsible for the
output, i.e. the location of a lesion. Alternatively, segmentation or detection
models can be trained with pixel-wise annotations indicating the locations of
malignant lesions. Unfortunately, acquiring such labels is labor-intensive and
requires medical expertise. To overcome this difficulty, weakly-supervised
localization can be utilized. These methods allow neural network classifiers to
output saliency maps highlighting the regions of the input most relevant to the
classification task (e.g. malignant lesions in mammograms) using only
image-level labels (e.g. whether the patient has cancer or not) during
training. When applied to high-resolution images, existing methods produce
low-resolution saliency maps. This is problematic in applications in which
suspicious lesions are small in relation to the image size. In this work, we
introduce a novel neural network architecture to perform weakly-supervised
segmentation of high-resolution images. The proposed model selects regions of
interest via coarse-level localization, and then performs fine-grained
segmentation of those regions. We apply this model to breast cancer diagnosis
with screening mammography, and validate it on a large clinically-realistic
dataset. Measured by Dice similarity score, our approach outperforms existing
methods by a large margin in terms of localization performance of benign and
malignant lesions, relatively improving the performance by 39.6% and 20.0%,
respectively. Code and the weights of some of the models are available at
https://github.com/nyukat/GLAM

positive:
Recently, chest X-ray report generation, which aims to automatically generate
descriptions of given chest X-ray images, has received growing research
interests. The key challenge of chest X-ray report generation is to accurately
capture and describe the abnormal regions. In most cases, the normal regions
dominate the entire chest X-ray image, and the corresponding descriptions of
these normal regions dominate the final report. Due to such data bias,
learning-based models may fail to attend to abnormal regions. In this work, to
effectively capture and describe abnormal regions, we propose the Contrastive
Attention (CA) model. Instead of solely focusing on the current input image,
the CA model compares the current input image with normal images to distill the
contrastive information. The acquired contrastive information can better
represent the visual features of abnormal regions. According to the experiments
on the public IU-X-ray and MIMIC-CXR datasets, incorporating our CA into
several existing models can boost their performance across most metrics. In
addition, according to the analysis, the CA model can help existing models
better attend to the abnormal regions and provide more accurate descriptions
which are crucial for an interpretable diagnosis. Specifically, we achieve the
state-of-the-art results on the two public datasets.

positive:
In recent years, besides the medical treatment methods in medical field,
Computer Aided Diagnosis (CAD) systems which can facilitate the decision making
phase of the physician and can detect the disease at an early stage have
started to be used frequently. The diagnosis of Idiopathic Pulmonary Fibrosis
(IPF) disease by using CAD systems is very important in that it can be followed
by doctors and radiologists. It has become possible to diagnose and follow up
the disease with the help of CAD systems by the development of high resolution
computed imaging scanners and increasing size of computation power. The purpose
of this project is to design a tool that will help specialists diagnose and
follow up the IPF disease by identifying areas of honeycombing and ground glass
patterns in High Resolution Computed Tomography (HRCT) lung images. Creating a
program module that segments the lung pair and creating a self-learner deep
learning model from given Computed Tomography (CT) images for the specific
diseased regions thanks to doctors are the main purposes of this work. Through
the created model, program module will be able to find special regions in given
new CT images. In this study, the performance of lung segmentation was tested
by the S{\o}rensen-Dice coefficient method and the mean performance was
measured as 90.7%, testing of the created model was performed with data not
used in the training stage of the CNN network, and the average performance was
measured as 87.8% for healthy regions, 73.3% for ground-glass areas and 69.1%
for honeycombing zones.

positive:
We consider the problem of abnormality localization for clinical
applications. While deep learning has driven much recent progress in medical
imaging, many clinical challenges are not fully addressed, limiting its broader
usage. While recent methods report high diagnostic accuracies, physicians have
concerns trusting these algorithm results for diagnostic decision-making
purposes because of a general lack of algorithm decision reasoning and
interpretability. One potential way to address this problem is to further train
these models to localize abnormalities in addition to just classifying them.
However, doing this accurately will require a large amount of disease
localization annotations by clinical experts, a task that is prohibitively
expensive to accomplish for most applications. In this work, we take a step
towards addressing these issues by means of a new attention-driven weakly
supervised algorithm comprising a hierarchical attention mining framework that
unifies activation- and gradient-based visual attention in a holistic manner.
Our key algorithmic innovations include the design of explicit ordinal
attention constraints, enabling principled model training in a
weakly-supervised fashion, while also facilitating the generation of
visual-attention-driven model explanations by means of localization cues. On
two large-scale chest X-ray datasets (NIH ChestX-ray14 and CheXpert), we
demonstrate significant localization performance improvements over the current
state of the art while also achieving competitive classification performance.
Our code is available on https://github.com/oyxhust/HAM.

positive:
Landmark Localization plays a very important role in processing medical
images as well as in disease identification. However, In medical field, it's a
challenging task because of the complexity of medical images and the high
requirement of accuracy for disease identification and treatment.There are two
dominant ways to regress landmark coordination, one using the full
convolutional network to regress the heatmaps of landmarks , which is a complex
way and heatmap post-process strategies are needed, and the other way is to
regress the coordination using CNN + Full Connective Network directly, which is
very simple and faster training , but larger dataset and deeper model are
needed to achieve higher accuracy. Though with data augmentation and deeper
network it can reach a reasonable accuracy, but the accuracy still not reach
the requirement of medical field. In addition, a deeper networks also means
larger space consumption. To achieve a higher accuracy, we contrived a new
landmark regression method which combing heatmap regression and direct
coordinate regression base on probability methods and system control theory.

positive:
Telementoring surgeons as they perform surgery can be essential in the
treatment of patients when in situ expertise is not available. Nonetheless,
expert mentors are often unavailable to provide trainees with real-time medical
guidance. When mentors are unavailable, a fallback autonomous mechanism should
provide medical practitioners with the required guidance. However,
AI/autonomous mentoring in medicine has been limited by the availability of
generalizable prediction models, and surgical procedures datasets to train
those models with. This work presents the initial steps towards the development
of an intelligent artificial system for autonomous medical mentoring.
Specifically, we present the first Database for AI Surgical Instruction
(DAISI). DAISI leverages on images and instructions to provide step-by-step
demonstrations of how to perform procedures from various medical disciplines.
The dataset was acquired from real surgical procedures and data from academic
textbooks. We used DAISI to train an encoder-decoder neural network capable of
predicting medical instructions given a current view of the surgery.
Afterwards, the instructions predicted by the network were evaluated using
cumulative BLEU scores and input from expert physicians. According to the BLEU
scores, the predicted and ground truth instructions were as high as 67%
similar. Additionally, expert physicians subjectively assessed the algorithm
using Likert scale, and considered that the predicted descriptions were related
to the images. This work provides a baseline for AI algorithms to assist in
autonomous medical mentoring.

positive:
Deep learning models designed for visual classification tasks on natural
images have become prevalent in medical image analysis. However, medical images
differ from typical natural images in many ways, such as significantly higher
resolutions and smaller regions of interest. Moreover, both the global
structure and local details play important roles in medical image analysis
tasks. To address these unique properties of medical images, we propose a
neural network that is able to classify breast cancer lesions utilizing
information from both a global saliency map and multiple local patches. The
proposed model outperforms the ResNet-based baseline and achieves
radiologist-level performance in the interpretation of screening mammography.
Although our model is trained only with image-level labels, it is able to
generate pixel-level saliency maps that provide localization of possible
malignant findings.

positive:
The identification and localization of diseases in medical images using deep
learning models have recently attracted significant interest. Existing methods
only consider training the networks with each image independently and most
leverage an activation map for disease localization. In this paper, we propose
an alternative approach that learns discriminative features among triplets of
images and cyclically trains on region features to verify whether attentive
regions contain information indicative of a disease. Concretely, we adapt a
distance learning framework for multi-label disease classification to
differentiate subtle disease features. Additionally, we feed back the features
of the predicted class-specific regions to a separate classifier during
training to better verify the localized diseases. Our model can achieve
state-of-the-art classification performance on the challenging ChestX-ray14
dataset, and our ablation studies indicate that both distance learning and
region verification contribute to overall classification performance. Moreover,
the distance learning and region verification modules can capture essential
information for better localization than baseline models without these modules.

positive:
The field of medical diagnostics contains a wealth of challenges which
closely resemble classical machine learning problems; practical constraints,
however, complicate the translation of these endpoints naively into classical
architectures. Many tasks in radiology, for example, are largely problems of
multi-label classification wherein medical images are interpreted to indicate
multiple present or suspected pathologies. Clinical settings drive the
necessity for high accuracy simultaneously across a multitude of pathological
outcomes and greatly limit the utility of tools which consider only a subset.
This issue is exacerbated by a general scarcity of training data and maximizes
the need to extract clinically relevant features from available samples --
ideally without the use of pre-trained models which may carry forward
undesirable biases from tangentially related tasks. We present and evaluate a
partial solution to these constraints in using LSTMs to leverage
interdependencies among target labels in predicting 14 pathologic patterns from
chest x-rays and establish state of the art results on the largest publicly
available chest x-ray dataset from the NIH without pre-training. Furthermore,
we propose and discuss alternative evaluation metrics and their relevance in
clinical practice.

positive:
One issue with computer based histopathology image analysis is that the size
of the raw image is usually very large. Taking the raw image as input to the
deep learning model would be computationally expensive while resizing the raw
image to low resolution would incur information loss. In this paper, we present
a novel deep hybrid attention approach to breast cancer classification. It
first adaptively selects a sequence of coarse regions from the raw image by a
hard visual attention algorithm, and then for each such region it is able to
investigate the abnormal parts based on a soft-attention mechanism. A recurrent
network is then built to make decisions to classify the image region and also
to predict the location of the image region to be investigated at the next time
step. As the region selection process is non-differentiable, we optimize the
whole network through a reinforcement approach to learn an optimal policy to
classify the regions. Based on this novel Look, Investigate and Classify
approach, we only need to process a fraction of the pixels in the raw image
resulting in significant saving in computational resources without sacrificing
performances. Our approach is evaluated on a public breast cancer
histopathology database, where it demonstrates superior performance to the
state-of-the-art deep learning approaches, achieving around 96\% classification
accuracy while only 15% of raw pixels are used.

positive:
Lung cancer is the leading cause of cancer death worldwide. The critical
reason for the deaths is delayed diagnosis and poor prognosis. With the
accelerated development of deep learning techniques, it has been successfully
applied extensively in many real-world applications, including health sectors
such as medical image interpretation and disease diagnosis. By combining more
modalities that being engaged in the processing of information, multimodal
learning can extract better features and improve predictive ability. The
conventional methods for lung cancer survival analysis normally utilize
clinical data and only provide a statistical probability. To improve the
survival prediction accuracy and help prognostic decision-making in clinical
practice for medical experts, we for the first time propose a multimodal deep
learning method for non-small cell lung cancer (NSCLC) survival analysis, named
DeepMMSA. This method leverages CT images in combination with clinical data,
enabling the abundant information hold within medical images to be associate
with lung cancer survival information. We validate our method on the data of
422 NSCLC patients from The Cancer Imaging Archive (TCIA). Experimental results
support our hypothesis that there is an underlying relationship between
prognostic information and radiomic images. Besides, quantitative results
showing that the established multimodal model can be applied to traditional
method and has the potential to break bottleneck of existing methods and
increase the the percentage of concordant pairs(right predicted pairs) in
overall population by 4%.

positive:
Generative Adversarial Networks (GAN) have many potential medical imaging
applications, including data augmentation, domain adaptation, and model
explanation. Due to the limited memory of Graphical Processing Units (GPUs),
most current 3D GAN models are trained on low-resolution medical images, these
models either cannot scale to high-resolution or are prone to patchy artifacts.
In this work, we propose a novel end-to-end GAN architecture that can generate
high-resolution 3D images. We achieve this goal by using different
configurations between training and inference. During training, we adopt a
hierarchical structure that simultaneously generates a low-resolution version
of the image and a randomly selected sub-volume of the high-resolution image.
The hierarchical design has two advantages: First, the memory demand for
training on high-resolution images is amortized among sub-volumes. Furthermore,
anchoring the high-resolution sub-volumes to a single low-resolution image
ensures anatomical consistency between sub-volumes. During inference, our model
can directly generate full high-resolution images. We also incorporate an
encoder with a similar hierarchical structure into the model to extract
features from the images. Experiments on 3D thorax CT and brain MRI demonstrate
that our approach outperforms state of the art in image generation. We also
demonstrate clinical applications of the proposed model in data augmentation
and clinical-relevant feature extraction.

positive:
In the Gastric Histopathology Image Classification (GHIC) tasks, which are
usually weakly supervised learning missions, there is inevitably redundant
information in the images. Therefore, designing networks that can focus on
effective distinguishing features has become a popular research topic. In this
paper, to accomplish the tasks of GHIC superiorly and to assist pathologists in
clinical diagnosis, an intelligent Hierarchical Conditional Random Field based
Attention Mechanism (HCRF-AM) model is proposed. The HCRF-AM model consists of
an Attention Mechanism (AM) module and an Image Classification (IC) module. In
the AM module, an HCRF model is built to extract attention regions. In the IC
module, a Convolutional Neural Network (CNN) model is trained with the
attention regions selected and then an algorithm called Classification
Probability-based Ensemble Learning is applied to obtain the image-level
results from patch-level output of the CNN. In the experiment, a classification
specificity of 96.67% is achieved on a gastric histopathology dataset with 700
images. Our HCRF-AM model demonstrates high classification performance and
shows its effectiveness and future potential in the GHIC field.

positive:
Brain aging, and more specifically the difference between the chronological
and the biological age of a person, may be a promising biomarker for
identifying neurodegenerative diseases. For this purpose accurate prediction is
important but the localisation of the areas that play a significant role in the
prediction is also crucial, in order to gain clinicians' trust and reassurance
about the performance of a prediction model. Most interpretability methods are
focused on classification tasks and cannot be directly transferred to
regression tasks. In this study, we focus on the task of brain age regression
from 3D brain Magnetic Resonance (MR) images using a Convolutional Neural
Network, termed prediction model. We interpret its predictions by extracting
importance maps, which discover the parts of the brain that are the most
important for brain age. In order to do so, we assume that voxels that are not
useful for the regression are resilient to noise addition. We implement a noise
model which aims to add as much noise as possible to the input without harming
the performance of the prediction model. We average the importance maps of the
subjects and end up with a population-based importance map, which displays the
regions of the brain that are influential for the task. We test our method on
13,750 3D brain MR images from the UK Biobank, and our findings are consistent
with the existing neuropathology literature, highlighting that the hippocampus
and the ventricles are the most relevant regions for brain aging.

positive:
Applying deep learning methods to mammography assessment has remained a
challenging topic. Dense noise with sparse expressions, mega-pixel raw data
resolution, lack of diverse examples have all been factors affecting
performance. The lack of pixel-level ground truths have especially limited
segmentation methods in pushing beyond approximately bounding regions. We
propose a classification approach grounded in high performance tissue
assessment as an alternative to all-in-one localization and assessment models
that is also capable of pinpointing the causal pixels. First, the objective of
the mammography assessment task is formalized in the context of local tissue
classifiers. Then, the accuracy of a convolutional neural net is evaluated on
classifying patches of tissue with suspicious findings at varying scales, where
highest obtained AUC is above $0.9$. The local evaluations of one such expert
tissue classifier is used to augment the results of a heatmap regression model
and additionally recover the exact causal regions at high resolution as a
saliency image suitable for clinical settings.

positive:
Given image labels as the only supervisory signal, we focus on harvesting, or
mining, thoracic disease localizations from chest X-ray images. Harvesting such
localizations from existing datasets allows for the creation of improved data
sources for computer-aided diagnosis and retrospective analyses. We train a
convolutional neural network (CNN) for image classification and propose an
attention mining (AM) strategy to improve the model's sensitivity or saliency
to disease patterns. The intuition of AM is that once the most salient disease
area is blocked or hidden from the CNN model, it will pay attention to
alternative image regions, while still attempting to make correct predictions.
However, the model requires to be properly constrained during AM, otherwise, it
may overfit to uncorrelated image parts and forget the valuable knowledge that
it has learned from the original image classification task. To alleviate such
side effects, we then design a knowledge preservation (KP) loss, which
minimizes the discrepancy between responses for X-ray images from the original
and the updated networks. Furthermore, we modify the CNN model to include
multi-scale aggregation (MSA), improving its localization ability on
small-scale disease findings, e.g., lung nodules. We experimentally validate
our method on the publicly-available ChestX-ray14 dataset, outperforming a
class activation map (CAM)-based approach, and demonstrating the value of our
novel framework for mining disease locations.

positive:
Medical image datasets can have large number of images representing patients
with different health conditions and various disease severity. When dealing
with raw unlabeled image datasets, the large number of samples often makes it
hard for experts and non-experts to understand the variety of images present in
a dataset. Supervised learning methods rely on labeled images which requires a
considerable effort by medical experts to first understand the communities of
images present in the data and then labeling the images. Here, we propose an
algorithm to facilitate the automatic identification of communities in medical
image datasets. We further demonstrate that such analysis can be insightful in
a supervised setting when the images are already labeled. Such insights are
useful because, health and disease severity can be considered a continuous
spectrum, and within each class, there usually are finer communities worthy of
investigation, especially when they have similarities to communities in other
classes. In our approach, we use wavelet decomposition of images in tandem with
spectral methods. We show that the eigenvalues of a graph Laplacian can reveal
the number of notable communities in an image dataset. Moreover, analyzing the
similarities may be used to infer a spectrum representing the severity of the
disease. In our experiments, we use a dataset of images labeled with different
conditions for COVID patients. We detect 25 communities in the dataset and then
observe that only 6 of those communities contain patients with pneumonia. We
also investigate the contents of a colorectal cancer histology dataset.

positive:
Convolutional neural networks (CNNs) show impressive performance for image
classification and detection, extending heavily to the medical image domain.
Nevertheless, medical experts are sceptical in these predictions as the
nonlinear multilayer structure resulting in a classification outcome is not
directly graspable. Recently, approaches have been shown which help the user to
understand the discriminative regions within an image which are decisive for
the CNN to conclude to a certain class. Although these approaches could help to
build trust in the CNNs predictions, they are only slightly shown to work with
medical image data which often poses a challenge as the decision for a class
relies on different lesion areas scattered around the entire image. Using the
DiaretDB1 dataset, we show that on retina images different lesion areas
fundamental for diabetic retinopathy are detected on an image level with high
accuracy, comparable or exceeding supervised methods. On lesion level, we
achieve few false positives with high sensitivity, though, the network is
solely trained on image-level labels which do not include information about
existing lesions. Classifying between diseased and healthy images, we achieve
an AUC of 0.954 on the DiaretDB1.

positive:
With the increasing usage of radiograph images as a most common medical
imaging system for diagnosis, treatment planning, and clinical studies, it is
increasingly becoming a vital factor to use machine learning-based systems to
provide reliable information for surgical pre-planning. Segmentation of pelvic
bone in radiograph images is a critical preprocessing step for some
applications such as automatic pose estimation and disease detection. However,
the encoder-decoder style network known as U-Net has demonstrated limited
results due to the challenging complexity of the pelvic shapes, especially in
severe patients. In this paper, we propose a novel multi-task segmentation
method based on Mask R-CNN architecture. For training, the network weights were
initialized by large non-medical dataset and fine-tuned with radiograph images.
Furthermore, in the training process, augmented data was generated to improve
network performance. Our experiments show that Mask R-CNN utilizing multi-task
learning, transfer learning, and data augmentation techniques achieve 0.96 DICE
coefficient, which significantly outperforms the U-Net. Notably, for a fair
comparison, the same transfer learning and data augmentation techniques have
been used for U-net training.

positive:
Medical imaging is an essential tool in many areas of medical applications,
used for both diagnosis and treatment. However, reading medical images and
making diagnosis or treatment recommendations require specially trained medical
specialists. The current practice of reading medical images is labor-intensive,
time-consuming, costly, and error-prone. It would be more desirable to have a
computer-aided system that can automatically make diagnosis and treatment
recommendations. Recent advances in deep learning enable us to rethink the ways
of clinician diagnosis based on medical images. In this thesis, we will
introduce 1) mammograms for detecting breast cancers, the most frequently
diagnosed solid cancer for U.S. women, 2) lung CT images for detecting lung
cancers, the most frequently diagnosed malignant cancer, and 3) head and neck
CT images for automated delineation of organs at risk in radiotherapy. First,
we will show how to employ the adversarial concept to generate the hard
examples improving mammogram mass segmentation. Second, we will demonstrate how
to use the weakly labeled data for the mammogram breast cancer diagnosis by
efficiently design deep learning for multi-instance learning. Third, the thesis
will walk through DeepLung system which combines deep 3D ConvNets and GBM for
automated lung nodule detection and classification. Fourth, we will show how to
use weakly labeled data to improve existing lung nodule detection system by
integrating deep learning with a probabilistic graphic model. Lastly, we will
demonstrate the AnatomyNet which is thousands of times faster and more accurate
than previous methods on automated anatomy segmentation.

positive:
The recent progress of computing, machine learning, and especially deep
learning, for image recognition brings a meaningful effect for automatic
detection of various diseases from chest X-ray images (CXRs). Here efficiency
of lung segmentation and bone shadow exclusion techniques is demonstrated for
analysis of 2D CXRs by deep learning approach to help radiologists identify
suspicious lesions and nodules in lung cancer patients. Training and validation
was performed on the original JSRT dataset (dataset #01), BSE-JSRT dataset,
i.e. the same JSRT dataset, but without clavicle and rib shadows (dataset #02),
original JSRT dataset after segmentation (dataset #03), and BSE-JSRT dataset
after segmentation (dataset #04). The results demonstrate the high efficiency
and usefulness of the considered pre-processing techniques in the simplified
configuration even. The pre-processed dataset without bones (dataset #02)
demonstrates the much better accuracy and loss results in comparison to the
other pre-processed datasets after lung segmentation (datasets #02 and #03).

positive:
This study aims to automatically diagnose thoracic diseases depicted on the
chest x-ray (CXR) images using deep convolutional neural networks. The existing
methods generally used the entire CXR images for training purposes, but this
strategy may suffer from two drawbacks. First, potential misalignment or the
existence of irrelevant objects in the entire CXR images may cause unnecessary
noise and thus limit the network performance. Second, the relatively low image
resolution caused by the resizing operation, which is a common preprocessing
procedure for training neural networks, may lead to the loss of image details,
making it difficult to detect pathologies with small lesion regions. To address
these issues, we present a novel method termed as segmentation-based deep
fusion network (SDFN), which leverages the domain knowledge and the
higherresolution information of local lung regions. Specifically, the local
lung regions were identified and cropped by the Lung Region Generator (LRG).
Two CNN-based classification models were then used as feature extractors to
obtain the discriminative features of the entire CXR images and the cropped
lung region images. Lastly, the obtained features were fused by the feature
fusion module for disease classification. Evaluated by the NIH benchmark split
on the Chest X-ray 14 Dataset, our experimental result demonstrated that the
developed method achieved more accurate disease classification compared with
the available approaches via the receiver operating characteristic (ROC)
analyses. It was also found that the SDFN could localize the lesion regions
more precisely as compared to the traditional method.

positive:
Early detection of lung cancer is essential in reducing mortality. Recent
studies have demonstrated the clinical utility of low-dose computed tomography
(CT) to detect lung cancer among individuals selected based on very limited
clinical information. However, this strategy yields high false positive rates,
which can lead to unnecessary and potentially harmful procedures. To address
such challenges, we established a pipeline that co-learns from detailed
clinical demographics and 3D CT images. Toward this end, we leveraged data from
the Consortium for Molecular and Cellular Characterization of Screen-Detected
Lesions (MCL), which focuses on early detection of lung cancer. A 3D
attention-based deep convolutional neural net (DCNN) is proposed to identify
lung cancer from the chest CT scan without prior anatomical location of the
suspicious nodule. To improve upon the non-invasive discrimination between
benign and malignant, we applied a random forest classifier to a dataset
integrating clinical information to imaging data. The results show that the AUC
obtained from clinical demographics alone was 0.635 while the attention network
alone reached an accuracy of 0.687. In contrast when applying our proposed
pipeline integrating clinical and imaging variables, we reached an AUC of 0.787
on the testing dataset. The proposed network both efficiently captures
anatomical information for classification and also generates attention maps
that explain the features that drive performance.

negative:
Convolutional Neural Networks (CNNs) are state-of-the-art models for document
image classification tasks. However, many of these approaches rely on
parameters and architectures designed for classifying natural images, which
differ from document images. We question whether this is appropriate and
conduct a large empirical study to find what aspects of CNNs most affect
performance on document images. Among other results, we exceed the
state-of-the-art on the RVL-CDIP dataset by using shear transform data
augmentation and an architecture designed for a larger input image.
Additionally, we analyze the learned features and find evidence that CNNs
trained on RVL-CDIP learn region-specific layout features.

negative:
Recently deep learning methods, in particular, convolutional neural networks
(CNNs), have led to a massive breakthrough in the range of computer vision.
Also, the large-scale annotated dataset is the essential key to a successful
training procedure. However, it is a huge challenge to get such datasets in the
medical domain. Towards this, we present a data augmentation method for
generating synthetic medical images using cycle-consistency Generative
Adversarial Networks (GANs). We add semi-supervised attention modules to
generate images with convincing details. We treat tumor images and normal
images as two domains. The proposed GANs-based model can generate a tumor image
from a normal image, and in turn, it can also generate a normal image from a
tumor image. Furthermore, we show that generated medical images can be used for
improving the performance of ResNet18 for medical image classification. Our
model is applied to three limited datasets of tumor MRI images. We first
generate MRI images on limited datasets, then we trained three popular
classification models to get the best model for tumor classification. Finally,
we train the classification model using real images with classic data
augmentation methods and classification models using synthetic images. The
classification results between those trained models showed that the proposed
SAG-GAN data augmentation method can boost Accuracy and AUC compare with
classic data augmentation methods. We believe the proposed data augmentation
method can apply to other medical image domains, and improve the accuracy of
computer-assisted diagnosis.

negative:
Segmentation of additive manufacturing (AM) defects in X-ray Computed
Tomography (XCT) images is challenging, due to the poor contrast, small sizes
and variation in appearance of defects. Automatic segmentation can, however,
provide quality control for additive manufacturing. Over recent years,
three-dimensional convolutional neural networks (3D CNNs) have performed well
in the volumetric segmentation of medical images. In this work, we leverage
techniques from the medical imaging domain and propose training a 3D U-Net
model to automatically segment defects in XCT images of AM samples. This work
not only contributes to the use of machine learning for AM defect detection but
also demonstrates for the first time 3D volumetric segmentation in AM. We train
and test with three variants of the 3D U-Net on an AM dataset, achieving a mean
intersection of union (IOU) value of 88.4%.

negative:
The science of solving clinical problems by analyzing images generated in
clinical practice is known as medical image analysis. The aim is to extract
information in an effective and efficient manner for improved clinical
diagnosis. The recent advances in the field of biomedical engineering has made
medical image analysis one of the top research and development area. One of the
reason for this advancement is the application of machine learning techniques
for the analysis of medical images. Deep learning is successfully used as a
tool for machine learning, where a neural network is capable of automatically
learning features. This is in contrast to those methods where traditionally
hand crafted features are used. The selection and calculation of these features
is a challenging task. Among deep learning techniques, deep convolutional
networks are actively used for the purpose of medical image analysis. This
include application areas such as segmentation, abnormality detection, disease
classification, computer aided diagnosis and retrieval. In this study, a
comprehensive review of the current state-of-the-art in medical image analysis
using deep convolutional networks is presented. The challenges and potential of
these techniques are also highlighted.

negative:
Deep learning based medical image classifiers have shown remarkable prowess
in various application areas like ophthalmology, dermatology, pathology, and
radiology. However, the acceptance of these Computer-Aided Diagnosis (CAD)
systems in real clinical setups is severely limited primarily because their
decision-making process remains largely obscure. This work aims at elucidating
a deep learning based medical image classifier by verifying that the model
learns and utilizes similar disease-related concepts as described and employed
by dermatologists. We used a well-trained and high performing neural network
developed by REasoning for COmplex Data (RECOD) Lab for classification of three
skin tumours, i.e. Melanocytic Naevi, Melanoma and Seborrheic Keratosis and
performed a detailed analysis on its latent space. Two well established and
publicly available skin disease datasets, PH2 and derm7pt, are used for
experimentation. Human understandable concepts are mapped to RECOD image
classification model with the help of Concept Activation Vectors (CAVs),
introducing a novel training and significance testing paradigm for CAVs. Our
results on an independent evaluation set clearly shows that the classifier
learns and encodes human understandable concepts in its latent representation.
Additionally, TCAV scores (Testing with CAVs) suggest that the neural network
indeed makes use of disease-related concepts in the correct way when making
predictions. We anticipate that this work can not only increase confidence of
medical practitioners on CAD but also serve as a stepping stone for further
development of CAV-based neural network interpretation methods.

negative:
Chest X-ray is the most common medical imaging exam used to assess multiple
pathologies. Automated algorithms and tools have the potential to support the
reading workflow, improve efficiency, and reduce reading errors. With the
availability of large scale data sets, several methods have been proposed to
classify pathologies on chest X-ray images. However, most methods report
performance based on random image based splitting, ignoring the high
probability of the same patient appearing in both training and test set. In
addition, most methods fail to explicitly incorporate the spatial information
of abnormalities or utilize the high resolution images. We propose a novel
approach based on location aware Dense Networks (DNetLoc), whereby we
incorporate both high-resolution image data and spatial information for
abnormality classification. We evaluate our method on the largest data set
reported in the community, containing a total of 86,876 patients and 297,541
chest X-ray images. We achieve (i) the best average AUC score for published
training and test splits on the single benchmarking data set (ChestX-Ray14),
and (ii) improved AUC scores when the pathology location information is
explicitly used. To foster future research we demonstrate the limitations of
the current benchmarking setup and provide new reference patient-wise splits
for the used data sets. This could support consistent and meaningful
benchmarking of future methods on the largest publicly available data sets.

negative:
The recent thrust on digital agriculture (DA) has renewed significant
research interest in the automated delineation of agricultural fields. Most
prior work addressing this problem have focused on detecting medium to large
fields, while there is strong evidence that around 40\% of the fields
world-wide and 70% of the fields in Asia and Africa are small. The lack of
adequate labeled images for small fields, huge variations in their color,
texture, and shape, and faint boundary lines separating them make it difficult
to develop an end-to-end learning model for detecting such fields. Hence, in
this paper, we present a multi-stage approach that uses a combination of
machine learning and image processing techniques. In the first stage, we
leverage state-of-the-art edge detection algorithms such as holistically-nested
edge detection (HED) to extract first-level contours and polygons. In the
second stage, we propose image-processing techniques to identify polygons that
are non-fields, over-segmentations, or noise and eliminate them. The next stage
tackles under-segmentations using a combination of a novel ``cut-point'' based
technique and localized second-level edge detection to obtain individual
parcels. Since a few small, non-cropped but vegetated or constructed pockets
can be interspersed in areas that are predominantly croplands, in the final
stage, we train a classifier for identifying each parcel from the previous
stage as an agricultural field or not. In an evaluation using high-resolution
imagery, we show that our approach has a high F-Score of 0.84 in areas with
large fields and reasonable accuracy with an F-Score of 0.73 in areas with
small fields, which is encouraging.

negative:
Document image classification remains a popular research area because it can
be commercialized in many enterprise applications across different industries.
Recent advancements in large pre-trained computer vision and language models
and graph neural networks has lent document image classification many tools.
However using large pre-trained models usually requires substantial computing
resources which could defeat the cost-saving advantages of automatic document
image classification. In the paper we propose an efficient document image
classification framework that uses graph convolution neural networks and
incorporates textual, visual and layout information of the document. We have
rigorously benchmarked our proposed algorithm against several state-of-art
vision and language models on both publicly available dataset and a real-life
insurance document classification dataset. Empirical results on both publicly
available and real-world data show that our methods achieve near SOTA
performance yet require much less computing resources and time for model
training and inference. This results in solutions than offer better cost
advantages, especially in scalable deployment for enterprise applications. The
results showed that our algorithm can achieve classification performance quite
close to SOTA. We also provide comprehensive comparisons of computing
resources, model sizes, train and inference time between our proposed methods
and baselines. In addition we delineate the cost per image using our method and
other baselines.

negative:
Deep neural network models used for medical image segmentation are large
because they are trained with high-resolution three-dimensional (3D) images.
Graphics processing units (GPUs) are widely used to accelerate the trainings.
However, the memory on a GPU is not large enough to train the models. A popular
approach to tackling this problem is patch-based method, which divides a large
image into small patches and trains the models with these small patches.
However, this method would degrade the segmentation quality if a target object
spans multiple patches. In this paper, we propose a novel approach for 3D
medical image segmentation that utilizes the data-swapping, which swaps out
intermediate data from GPU memory to CPU memory to enlarge the effective GPU
memory size, for training high-resolution 3D medical images without patching.
We carefully tuned parameters in the data-swapping method to obtain the best
training performance for 3D U-Net, a widely used deep neural network model for
medical image segmentation. We applied our tuning to train 3D U-Net with
full-size images of 192 x 192 x 192 voxels in brain tumor dataset. As a result,
communication overhead, which is the most important issue, was reduced by
17.1%. Compared with the patch-based method for patches of 128 x 128 x 128
voxels, our training for full-size images achieved improvement on the mean Dice
score by 4.48% and 5.32 % for detecting whole tumor sub-region and tumor core
sub-region, respectively. The total training time was reduced from 164 hours to
47 hours, resulting in 3.53 times of acceleration.

negative:
We present in this paper a new approach for the automatic annotation of
medical images, using the approach of "bag-of-words" to represent the visual
content of the medical image combined with text descriptors based approach
tf.idf and reduced by latent semantic to extract the co-occurrence between
terms and visual terms. A medical report is composed of a text describing a
medical image. First, we are interested to index the text and extract all
relevant terms using a thesaurus containing MeSH medical concepts. In a second
phase, the medical image is indexed while recovering areas of interest which
are invariant to change in scale, light and tilt. To annotate a new medical
image, we use the approach of "bagof-words" to recover the feature vector.
Indeed, we use the vector space model to retrieve similar medical image from
the database training. The calculation of the relevance value of an image to
the query image is based on the cosine function. We conclude with an experiment
carried out on five types of radiological imaging to evaluate the performance
of our system of medical annotation. The results showed that our approach works
better with more images from the radiology of the skull.

negative:
With the ongoing rise of machine learning, the need for methods for
explaining decisions made by artificial intelligence systems is becoming a more
and more important topic. Especially for image classification tasks, many
state-of-the-art tools to explain such classifiers rely on visual highlighting
of important areas of the input data. Contrary, counterfactual explanation
systems try to enable a counterfactual reasoning by modifying the input image
in a way such that the classifier would have made a different prediction. By
doing so, the users of counterfactual explanation systems are equipped with a
completely different kind of explanatory information. However, methods for
generating realistic counterfactual explanations for image classifiers are
still rare. Especially in medical contexts, where relevant information often
consists of textural and structural information, high-quality counterfactual
images have the potential to give meaningful insights into decision processes.
In this work, we present GANterfactual, an approach to generate such
counterfactual image explanations based on adversarial image-to-image
translation techniques. Additionally, we conduct a user study to evaluate our
approach in an exemplary medical use case. Our results show that, in the chosen
medical use-case, counterfactual explanations lead to significantly better
results regarding mental models, explanation satisfaction, trust, emotions, and
self-efficacy than two state-of-the-art systems that work with saliency maps,
namely LIME and LRP.

negative:
Convolutional Neural Networks (CNNs) have shown to be powerful medical image
segmentation models. In this study, we address some of the main unresolved
issues regarding these models. Specifically, training of these models on small
medical image datasets is still challenging, with many studies promoting
techniques such as transfer learning. Moreover, these models are infamous for
producing over-confident predictions and for failing silently when presented
with out-of-distribution (OOD) data at test time. In this paper, we advocate
for multi-task learning, i.e., training a single model on several different
datasets, spanning several different organs of interest and different imaging
modalities. We show that not only a single CNN learns to automatically
recognize the context and accurately segment the organ of interest in each
context, but also that such a joint model often has more accurate and
better-calibrated predictions than dedicated models trained separately on each
dataset. Our experiments show that multi-task learning can outperform transfer
learning in medical image segmentation tasks. For detecting OOD data, we
propose a method based on spectral analysis of CNN feature maps. We show that
different datasets, representing different imaging modalities and/or different
organs of interest, have distinct spectral signatures, which can be used to
identify whether or not a test image is similar to the images used to train a
model. We show that this approach is far more accurate than OOD detection based
on prediction uncertainty. The methods proposed in this paper contribute
significantly to improving the accuracy and reliability of CNN-based medical
image segmentation models.

negative:
Celiac Disease (CD) and Environmental Enteropathy (EE) are common causes of
malnutrition and adversely impact normal childhood development. Both conditions
require a tissue biopsy for diagnosis and a major challenge of interpreting
clinical biopsy images to differentiate between these gastrointestinal diseases
is striking histopathologic overlap between them. In the current study, we
propose four diagnosis techniques for these diseases and address their
limitations and advantages. First, the diagnosis between CD, EE, and Normal
biopsies is considered, but the main challenge with this diagnosis technique is
the staining problem. The dataset used in this research is collected from
different centers with different staining standards. To solve this problem, we
use color balancing in order to train our model with a varying range of colors.
Random Multimodel Deep Learning (RMDL) architecture has been used as another
approach to mitigate the effects of the staining problem. RMDL combines
different architectures and structures of deep learning and the final output of
the model is based on the majority vote. CD is a chronic autoimmune disease
that affects the small intestine genetically predisposed children and adults.
Typically, CD rapidly progress from Marsh I to IIIa. Marsh III is sub-divided
into IIIa (partial villus atrophy), Marsh IIIb (subtotal villous atrophy), and
Marsh IIIc (total villus atrophy) to explain the spectrum of villus atrophy
along with crypt hypertrophy and increased intraepithelial lymphocytes. In the
second part of this study, we proposed two ways for diagnosing different stages
of CD. Finally, in the third part of this study, these two steps are combined
as Hierarchical Medical Image Classification (HMIC) to have a model to diagnose
the disease data hierarchically.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
Segmentation is one of the most important and popular tasks in medical image
analysis, which plays a critical role in disease diagnosis, surgical planning,
and prognosis evaluation. During the past five years, on the one hand,
thousands of medical image segmentation methods have been proposed for various
organs and lesions in different medical images, which become more and more
challenging to fairly compare different methods. On the other hand,
international segmentation challenges can provide a transparent platform to
fairly evaluate and compare different methods. In this paper, we present a
comprehensive review of the top methods in ten 3D medical image segmentation
challenges during 2020, covering a variety of tasks and datasets. We also
identify the "happy-families" practices in the cutting-edge segmentation
methods, which are useful for developing powerful segmentation approaches.
Finally, we discuss open research problems that should be addressed in the
future. We also maintain a list of cutting-edge segmentation methods at
\url{https://github.com/JunMa11/SOTA-MedSeg}.

negative:
Direct automatic segmentation of objects from 3D medical imaging, such as
magnetic resonance (MR) imaging, is challenging as it often involves accurately
identifying a number of individual objects with complex geometries within a
large volume under investigation. To address these challenges, most deep
learning approaches typically enhance their learning capability by
substantially increasing the complexity or the number of trainable parameters
within their models. Consequently, these models generally require long
inference time on standard workstations operating clinical MR systems and are
restricted to high-performance computing hardware due to their large memory
requirement. Further, to fit 3D dataset through these large models using
limited computer memory, trade-off techniques such as patch-wise training are
often used which sacrifice the fine-scale geometric information from input
images which could be clinically significant for diagnostic purposes. To
address these challenges, we present a compact convolutional neural network
with a shallow memory footprint to efficiently reduce the number of model
parameters required for state-of-art performance. This is critical for
practical employment as most clinical environments only have low-end hardware
with limited computing power and memory. The proposed network can maintain data
integrity by directly processing large full-size 3D input volumes with no
patches required and significantly reduces the computational time required for
both training and inference. We also propose a novel loss function with extra
shape constraint to improve the accuracy for imbalanced classes in 3D MR
images.

negative:
In this work, a region-based Deep Convolutional Neural Network framework is
proposed for document structure learning. The contribution of this work
involves efficient training of region based classifiers and effective
ensembling for document image classification. A primary level of `inter-domain'
transfer learning is used by exporting weights from a pre-trained VGG16
architecture on the ImageNet dataset to train a document classifier on whole
document images. Exploiting the nature of region based influence modelling, a
secondary level of `intra-domain' transfer learning is used for rapid training
of deep learning models for image segments. Finally, stacked generalization
based ensembling is utilized for combining the predictions of the base deep
neural network models. The proposed method achieves state-of-the-art accuracy
of 92.2% on the popular RVL-CDIP document image dataset, exceeding benchmarks
set by existing algorithms.

negative:
Content-based image retrieval (CBIR) of medical images is a crucial task that
can contribute to a more reliable diagnosis if applied to big data. Recent
advances in feature extraction and classification have enormously improved CBIR
results for digital images. However, considering the increasing accessibility
of big data in medical imaging, we are still in need of reducing both memory
requirements and computational expenses of image retrieval systems. This work
proposes to exclude the features of image blocks that exhibit a low encoding
error when learned by a $n/p/n$ autoencoder ($p\!<\!n$). We examine the
histogram of autoendcoding errors of image blocks for each image class to
facilitate the decision which image regions, or roughly what percentage of an
image perhaps, shall be declared relevant for the retrieval task. This leads to
reduction of feature dimensionality and speeds up the retrieval process. To
validate the proposed scheme, we employ local binary patterns (LBP) and support
vector machines (SVM) which are both well-established approaches in CBIR
research community. As well, we use IRMA dataset with 14,410 x-ray images as
test data. The results show that the dimensionality of annotated feature
vectors can be reduced by up to 50% resulting in speedups greater than 27% at
expense of less than 1% decrease in the accuracy of retrieval when validating
the precision and recall of the top 20 hits.

negative:
Deep learning techniques have led to state-of-the-art image super resolution
with natural images. Normally, pairs of high-resolution and low-resolution
images are used to train the deep learning models. These techniques have also
been applied to medical image super-resolution. The characteristics of medical
images differ significantly from natural images in several ways. First, it is
difficult to obtain high-resolution images for training in real clinical
applications due to the limitations of imaging systems and clinical
requirements. Second, other modal high-resolution images are available (e.g.,
high-resolution T1-weighted images are available for enhancing low-resolution
T2-weighted images). In this paper, we propose an unsupervised image
super-resolution technique based on simple prior knowledge of the human
anatomy. This technique does not require target T2WI high-resolution images for
training. Furthermore, we present a guided residual dense network, which
incorporates a residual dense network with a guided deep convolutional neural
network for enhancing the resolution of low-resolution images by referring to
different modal high-resolution images of the same subject. Experiments on a
publicly available brain MRI database showed that our proposed method achieves
better performance than the state-of-the-art methods.

negative:
In safety-critical applications like medical diagnosis, certainty associated
with a model's prediction is just as important as its accuracy. Consequently,
uncertainty estimation and reduction play a crucial role. Uncertainty in
predictions can be attributed to noise or randomness in data (aleatoric) and
incorrect model inferences (epistemic). While model uncertainty can be reduced
with more data or bigger models, aleatoric uncertainty is more intricate. This
work proposes a novel approach that interprets data uncertainty estimated from
a self-supervised task as noise inherent to the data and utilizes it to reduce
aleatoric uncertainty in another task related to the same dataset via data
augmentation. The proposed method was evaluated on a benchmark medical imaging
dataset with image reconstruction as the self-supervised task and segmentation
as the image analysis task. Our findings demonstrate the effectiveness of the
proposed approach in significantly reducing the aleatoric uncertainty in the
image segmentation task while achieving better or on-par performance compared
to the standard augmentation techniques.

negative:
Spatial attention has been introduced to convolutional neural networks (CNNs)
for improving both their performance and interpretability in visual tasks
including image classification. The essence of the spatial attention is to
learn a weight map which represents the relative importance of activations
within the same layer or channel. All existing attention mechanisms are local
attentions in the sense that weight maps are image-specific. However, in the
medical field, there are cases that all the images should share the same weight
map because the set of images record the same kind of symptom related to the
same object and thereby share the same structural content. In this paper, we
thus propose a novel global spatial attention mechanism in CNNs mainly for
medical image classification. The global weight map is instantiated by a
decision boundary between important pixels and unimportant pixels. And we
propose to realize the decision boundary by a binary classifier in which the
intensities of all images at a pixel are the features of the pixel. The binary
classification is integrated into an image classification CNN and is to be
optimized together with the CNN. Experiments on two medical image datasets and
one facial expression dataset showed that with the proposed attention, not only
the performance of four powerful CNNs which are GoogleNet, VGG, ResNet, and
DenseNet can be improved, but also meaningful attended regions can be obtained,
which is beneficial for understanding the content of images of a domain.

negative:
We propose a causal hidden Markov model to achieve robust prediction of
irreversible disease at an early stage, which is safety-critical and vital for
medical treatment in early stages. Specifically, we introduce the hidden
variables which propagate to generate medical data at each time step. To avoid
learning spurious correlation (e.g., confounding bias), we explicitly separate
these hidden variables into three parts: a) the disease (clinical)-related
part; b) the disease (non-clinical)-related part; c) others, with only a),b)
causally related to the disease however c) may contain spurious correlations
(with the disease) inherited from the data provided. With personal attributes
and the disease label respectively provided as side information and
supervision, we prove that these disease-related hidden variables can be
disentangled from others, implying the avoidance of spurious correlation for
generalization to medical data from other (out-of-) distributions. Guaranteed
by this result, we propose a sequential variational auto-encoder with a
reformulated objective function. We apply our model to the early prediction of
peripapillary atrophy and achieve promising results on out-of-distribution test
data. Further, the ablation study empirically shows the effectiveness of each
component in our method. And the visualization shows the accurate
identification of lesion regions from others.

negative:
Coronavirus disease (COVID-19) is an infectious disease caused by a newly
discovered coronavirus. The disease presents with symptoms such as shortness of
breath, fever, dry cough, and chronic fatigue, amongst others. Sometimes the
symptoms of the disease increase so much they lead to the death of the
patients. The disease may be asymptomatic in some patients in the early stages,
which can lead to increased transmission of the disease to others. Many studies
have tried to use medical imaging for early diagnosis of COVID-19. This study
attempts to review papers on automatic methods for medical image analysis and
diagnosis of COVID-19. For this purpose, PubMed, Google Scholar, arXiv and
medRxiv were searched to find related studies by the end of April 2020, and the
essential points of the collected studies were summarised. The contribution of
this study is four-fold: 1) to use as a tutorial of the field for both
clinicians and technologists, 2) to comprehensively review the characteristics
of COVID-19 as presented in medical images, 3) to examine automated artificial
intelligence-based approaches for COVID-19 diagnosis based on the accuracy and
the method used, 4) to express the research limitations in this field and the
methods used to overcome them. COVID-19 reveals signs in medical images can be
used for early diagnosis of the disease even in asymptomatic patients. Using
automated machine learning-based methods can diagnose the disease with high
accuracy from medical images and reduce time, cost and error of diagnostic
procedure. It is recommended to collect bulk imaging data from patients in the
shortest possible time to improve the performance of COVID-19 automated
diagnostic methods.

negative:
Medical images can be a valuable resource for reliable information to support
medical diagnosis. However, the large volume of medical images makes it
challenging to retrieve relevant information given a particular scenario. To
solve this challenge, content-based image retrieval (CBIR) attempts to
characterize images (or image regions) with invariant content information in
order to facilitate image search. This work presents a feature extraction
technique for medical images using stacked autoencoders, which encode images to
binary vectors. The technique is applied to the IRMA dataset, a collection of
14,410 x-ray images in order to demonstrate the ability of autoencoders to
retrieve similar x-rays given test queries. Using IRMA dataset as a benchmark,
it was found that stacked autoencoders gave excellent results with a retrieval
error of 376 for 1,733 test images with a compression of 74.61%.

negative:
Image-to-image translation is an ill-posed problem as unique one-to-one
mapping may not exist between the source and target images. Learning-based
methods proposed in this context often evaluate the performance on test data
that is similar to the training data, which may be impractical. This demands
robust methods that can quantify uncertainty in the prediction for making
informed decisions, especially for critical areas such as medical imaging.
Recent works that employ conditional generative adversarial networks (GANs)
have shown improved performance in learning photo-realistic image-to-image
mappings between the source and the target images. However, these methods do
not focus on (i)~robustness of the models to out-of-distribution (OOD)-noisy
data and (ii)~uncertainty quantification. This paper proposes a GAN-based
framework that (i)~models an adaptive loss function for robustness to OOD-noisy
data that automatically tunes the spatially varying norm for penalizing the
residuals and (ii)~estimates the per-voxel uncertainty in the predictions. We
demonstrate our method on two key applications in medical imaging:
(i)~undersampled magnetic resonance imaging (MRI) reconstruction (ii)~MRI
modality propagation. Our experiments with two different real-world datasets
show that the proposed method (i)~is robust to OOD-noisy test data and provides
improved accuracy and (ii)~quantifies voxel-level uncertainty in the
predictions.

negative:
Content-based image retrieval (CBIR) of medical images in large datasets to
identify similar images when a query image is given can be very useful in
improving the diagnostic decision of the clinical experts and as well in
educational scenarios. In this paper, we used two stage classification and
retrieval approach to retrieve similar images. First, the Gabor filters are
applied to Radon-transformed images to extract features and to train a
multi-class SVM. Then based on the classification results and using an
extracted Gabor barcode, similar images are retrieved. The proposed method was
tested on IRMA dataset which contains more than 14,000 images. Experimental
results show the efficiency of our approach in retrieving similar images
compared to other Gabor-Radon-oriented methods.

negative:
Deep learning-based computer-aided diagnosis is gradually deployed to review
and analyze medical images. However, this paradigm is restricted in real-world
clinical applications due to the poor robustness and generalization. The issue
is more sinister with a lack of training data. In this paper, we address the
challenge from the transfer learning point of view. Different from the common
setting that transferring knowledge from the natural image domain to the
medical image domain, we find the knowledge from the same domain further boosts
the model robustness and generalization. Therefore, we propose a novel
two-stage framework for robust generalized medical image segmentation. Firstly,
an unsupervised tile-wise autoencoder pretraining architecture is proposed to
learn local and global knowledge. Secondly, the downstream segmentation model
coupled with an auxiliary reconstruction network is designed. The
reconstruction branch encourages the model to capture more general semantic
features. Experiments of lung segmentation on multi chest X-ray datasets are
conducted. Comprehensive results demonstrate the superior robustness of the
proposed framework to corruption and high generalization performance on unseen
datasets, especially under the scenario of the limited training data.

negative:
Since the low quality of document images will greatly undermine the chances
of success in automatic text recognition and analysis, it is necessary to
assess the quality of document images uploaded in online business process, so
as to reject those images of low quality. In this paper, we attempt to achieve
document image quality assessment and our contributions are twofold. Firstly,
since document image quality assessment is more interested in text, we propose
a text line based framework to estimate document image quality, which is
composed of three stages: text line detection, text line quality prediction,
and overall quality assessment. Text line detection aims to find potential text
lines with a detector. In the text line quality prediction stage, the quality
score is computed for each text line with a CNN-based prediction model. The
overall quality of document images is finally assessed with the ensemble of all
text line quality. Secondly, to train the prediction model, a large-scale
dataset, comprising 52,094 text line images, is synthesized with diverse
attributes. For each text line image, a quality label is computed with a
piece-wise function. To demonstrate the effectiveness of the proposed
framework, comprehensive experiments are evaluated on two popular document
image quality assessment benchmarks. Our framework significantly outperforms
the state-of-the-art methods by large margins on the large and complicated
dataset.

negative:
There is a need for information retrieval from large collections of
low-resolution (LR) binary document images, which can be found in digital
libraries across the world, where the high-resolution (HR) counterpart is not
available. This gives rise to the problem of binary document image
super-resolution (BDISR). The objective of this paper is to address the
interesting and challenging problem of super resolution of binary Tamil
document images for improved readability and better optical character
recognition (OCR). We propose multiple deep neural network architectures to
address this problem and analyze their performance. The proposed models are all
single image super-resolution techniques, which learn a generalized spatial
correspondence between the LR and HR binary document images. We employ
convolutional layers for feature extraction followed by transposed convolution
and sub-pixel convolution layers for upscaling the features. Since the outputs
of the neural networks are gray scale, we utilize the advantage of power law
transformation as a post-processing technique to improve the character level
pixel connectivity. The performance of our models is evaluated by comparing the
OCR accuracies and the mean opinion scores given by human evaluators on LR
images and the corresponding model-generated HR images.

negative:
We consider the problem of accurately and efficiently querying a remote
server to retrieve information about images captured by a mobile device. In
addition to reduced transmission overhead and computational complexity, the
retrieval protocol should be robust to variations in the image acquisition
process, such as translation, rotation, scaling, and sensor-related
differences. We propose to extract scale-invariant image features and then
perform clustering to reduce the number of features needed for image matching.
Principal Component Analysis (PCA) and Non-negative Matrix Factorization (NMF)
are investigated as candidate clustering approaches. The image matching
complexity at the database server is quadratic in the (small) number of
clusters, not in the (very large) number of image features. We employ an
image-dependent information content metric to approximate the model order,
i.e., the number of clusters, needed for accurate matching, which is preferable
to setting the model order using trial and error. We show how to combine the
hypotheses provided by PCA and NMF factor loadings, thereby obtaining more
accurate retrieval than using either approach alone. In experiments on a
database of urban images, we obtain a top-1 retrieval accuracy of 89% and a
top-3 accuracy of 92.5%.

negative:
Face recognition approaches often rely on equal image resolution for
verifying faces on two images. However, in practical applications, those image
resolutions are usually not in the same range due to different image capture
mechanisms or sources. In this work, we first analyze the impact of image
resolutions on face verification performance with a state-of-the-art face
recognition model. For images synthetically reduced to $5\,\times\,5$ px
resolution, the verification performance drops from $99.23\%$ increasingly down
to almost $55\%$. Especially for cross-resolution image pairs (one high- and
one low-resolution image), the verification accuracy decreases even further. We
investigate this behavior more in-depth by looking at the feature distances for
every 2-image test pair. To tackle this problem, we propose the following two
methods: 1) Train a state-of-the-art face-recognition model straightforwardly
with $50\%$ low-resolution images directly within each batch. 2) Train a
siamese-network structure and add a cosine distance feature loss between high-
and low-resolution features. Both methods show an improvement for
cross-resolution scenarios and can increase the accuracy at very low resolution
to approximately $70\%$. However, a disadvantage is that a specific model needs
to be trained for every resolution pair. Thus, we extend the aforementioned
methods by training them with multiple image resolutions at once. The
performances for particular testing image resolutions are slightly worse, but
the advantage is that this model can be applied to arbitrary resolution images
and achieves an overall better performance ($97.72\%$ compared to $96.86\%$).
Due to the lack of a benchmark for arbitrary resolution images for the
cross-resolution and equal-resolution task, we propose an evaluation protocol
for five well-known datasets, focusing on high, mid, and low-resolution images.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
This work analyses the impact of self-supervised pre-training on document
images in the context of document image classification. While previous
approaches explore the effect of self-supervision on natural images, we show
that patch-based pre-training performs poorly on document images because of
their different structural properties and poor intra-sample semantic
information. We propose two context-aware alternatives to improve performance
on the Tobacco-3482 image classification task. We also propose a novel method
for self-supervision, which makes use of the inherent multi-modality of
documents (image and text), which performs better than other popular
self-supervised methods, including supervised ImageNet pre-training, on
document image classification scenarios with a limited amount of data.

negative:
For more than two decades, research has been performed on content-based image
retrieval (CBIR). By combining Radon projections and the support vector
machines (SVM), a content-based medical image retrieval method is presented in
this work. The proposed approach employs the normalized Radon projections with
corresponding image category labels to build an SVM classifier, and the Radon
barcode database which encodes every image in a binary format is also generated
simultaneously to tag all images. To retrieve similar images when a query image
is given, Radon projections and the barcode of the query image are generated.
Subsequently, the k-nearest neighbor search method is applied to find the
images with minimum Hamming distance of the Radon barcode within the same class
predicted by the trained SVM classifier that uses Radon features. The
performance of the proposed method is validated by using the IRMA 2009 dataset
with 14,410 x-ray images in 57 categories. The results demonstrate that our
method has the capacity to retrieve similar responses for the correctly
identified query image and even for those mistakenly classified by SVM. The
approach further is very fast and has low memory requirement.

negative:
Medical images constitute a source of information essential for disease
diagnosis, treatment and follow-up. In addition, due to its patient-specific
nature, imaging information represents a critical component required for
advancing precision medicine into clinical practice. This manuscript describes
recently developed technologies for better handling of image information:
photorealistic visualization of medical images with Cinematic Rendering,
artificial agents for in-depth image understanding, support for minimally
invasive procedures, and patient-specific computational models with enhanced
predictive power. Throughout the manuscript we will analyze the capabilities of
such technologies and extrapolate on their potential impact to advance the
quality of medical care, while reducing its cost.

negative:
Indeed, these are exciting times. We are in the heart of a digital
renaissance. Automation and computer technology allow engineers and scientists
to fabricate processes that amalgamate quality of life. We anticipate much
growth in medical image interpretation and understanding, due to the influx of
computer technologies. This work should serve as a guide to introduce the
reader to core themes in theoretical computer science, as well as imaging
applications for understanding vocal-fold vibrations. In this work, we motivate
the use of automation, review some mathematical models of computation. We
present a proof of a classical problem in image analysis that cannot be
automated by means of algorithms. Furthermore, discuss some applications for
processing medical images of the vocal folds, and discuss some of the
exhilarating directions the art of automation will take vocal-fold image
interpretation and quite possibly other areas of biomedical image analysis.

negative:
Recognition of document images have important applications in restoring old
and classical texts. The problem involves quality improvement before passing it
to a properly trained OCR to get accurate recognition of the text. The image
enhancement and quality improvement constitute important steps as subsequent
recognition depends upon the quality of the input image. There are scenarios
when high resolution images are not available and our experiments show that the
OCR accuracy reduces significantly with decrease in the spatial resolution of
document images. Thus the only option is to improve the resolution of such
document images. The goal is to construct a high resolution image, given a
single low resolution binary image, which constitutes the problem of single
image super-resolution. Most of the previous work in super-resolution deal with
natural images which have more information-content than the document images.
Here, we use Convolution Neural Network to learn the mapping between low and
the corresponding high resolution images. We experiment with different number
of layers, parameter settings and non-linear functions to build a fast
end-to-end framework for document image super-resolution. Our proposed model
shows a very good PSNR improvement of about 4 dB on 75 dpi Tamil images,
resulting in a 3 % improvement of word level accuracy by the OCR. It takes less
time than the recent sparse based natural image super-resolution technique,
making it useful for real-time document recognition applications.

negative:
One of the major prerequisites for any deep learning approach is the
availability of large-scale training data. When dealing with scanned document
images in real world scenarios, the principal information of its content is
stored in the layout itself. In this work, we have proposed an automated deep
generative model using Graph Neural Networks (GNNs) to generate synthetic data
with highly variable and plausible document layouts that can be used to train
document interpretation systems, in this case, specially in digital mailroom
applications. It is also the first graph-based approach for document layout
generation task experimented on administrative document images, in this case,
invoices.

negative:
Landmark detection algorithms trained on high resolution images perform
poorly on datasets containing low resolution images. This deters the
performance of algorithms relying on quality landmarks, for example, face
recognition. To the best of our knowledge, there does not exist any dataset
consisting of low resolution face images along with their annotated landmarks,
making supervised training infeasible. In this paper, we present a
semi-supervised approach to predict landmarks on low resolution images by
learning them from labeled high resolution images. The objective of this work
is to show that predicting landmarks directly on low resolution images is more
effective than the current practice of aligning images after rescaling or
superresolution. In a two-step process, the proposed approach first learns to
generate low resolution images by modeling the distribution of target low
resolution images. In the second stage, the roles of generated images and real
low resolution images are switched and the model learns to predict landmarks
for real low resolution images from generated low resolution images. With
extensive experimentation, we study the impact of each of the design choices
and also show that prediction of landmarks directly on low resolution images
improves the performance of important tasks such as face recognition in low
resolution images.

negative:
This paper is on image and face super-resolution. The vast majority of prior
work for this problem focus on how to increase the resolution of low-resolution
images which are artificially generated by simple bilinear down-sampling (or in
a few cases by blurring followed by down-sampling).We show that such methods
fail to produce good results when applied to real-world low-resolution, low
quality images. To circumvent this problem, we propose a two-stage process
which firstly trains a High-to-Low Generative Adversarial Network (GAN) to
learn how to degrade and downsample high-resolution images requiring, during
training, only unpaired high and low-resolution images. Once this is achieved,
the output of this network is used to train a Low-to-High GAN for image
super-resolution using this time paired low- and high-resolution images. Our
main result is that this network can be now used to efectively increase the
quality of real-world low-resolution images. We have applied the proposed
pipeline for the problem of face super-resolution where we report large
improvement over baselines and prior work although the proposed method is
potentially applicable to other object categories.

negative:
Many applications for the automated diagnosis of plant disease have been
developed based on the success of deep learning techniques. However, these
applications often suffer from overfitting, and the diagnostic performance is
drastically decreased when used on test datasets from new environments. In this
paper, we propose LeafGAN, a novel image-to-image translation system with own
attention mechanism. LeafGAN generates a wide variety of diseased images via
transformation from healthy images, as a data augmentation tool for improving
the performance of plant disease diagnosis. Thanks to its own attention
mechanism, our model can transform only relevant areas from images with a
variety of backgrounds, thus enriching the versatility of the training images.
Experiments with five-class cucumber disease classification show that data
augmentation with vanilla CycleGAN cannot help to improve the generalization,
i.e., disease diagnostic performance increased by only 0.7% from the baseline.
In contrast, LeafGAN boosted the diagnostic performance by 7.4%. We also
visually confirmed the generated images by our LeafGAN were much better quality
and more convincing than those generated by vanilla CycleGAN. The code is
available publicly at: https://github.com/IyatomiLab/LeafGAN.

negative:
Additive Manufacturing (AM) is a crucial component of the smart industry. In
this paper, we propose an automated quality grading system for the AM process
using a deep convolutional neural network (CNN) model. The CNN model is trained
offline using the images of the internal and surface defects in the
layer-by-layer deposition of materials and tested online by studying the
performance of detecting and classifying the failure in AM process at different
extruder speeds and temperatures. The model demonstrates the accuracy of 94%
and specificity of 96%, as well as above 75% in three classifier measures of
the Fscore, the sensitivity, and precision for classifying the quality of the
printing process in five grades in real-time. The proposed online model adds an
automated, consistent, and non-contact quality control signal to the AM process
that eliminates the manual inspection of parts after they are entirely built.
The quality monitoring signal can also be used by the machine to suggest
remedial actions by adjusting the parameters in real-time. The proposed quality
predictive model serves as a proof-of-concept for any type of AM machines to
produce reliable parts with fewer quality hiccups while limiting the waste of
both time and materials.

negative:
Document Image Analysis, like any Digital Image Analysis requires
identification and extraction of proper features, which are generally extracted
from uncompressed images, though in reality images are made available in
compressed form for the reasons such as transmission and storage efficiency.
However, this implies that the compressed image should be decompressed, which
indents additional computing resources. This limitation induces the motivation
to research in extracting features directly from the compressed image. In this
research, we propose to extract essential features such as projection profile,
run-histogram and entropy for text document analysis directly from run-length
compressed text-documents. The experimentation illustrates that features are
extracted directly from the compressed image without going through the stage of
decompression, because of which the computing time is reduced. The feature
values so extracted are exactly identical to those extracted from uncompressed
images.

negative:
Obtaining datasets labeled to facilitate model development is a challenge for
most machine learning tasks. The difficulty is heightened for medical imaging,
where data itself is limited in accessibility and labeling requires costly time
and effort by trained medical specialists. Medical imaging studies, however,
are often accompanied by a medical report produced by a radiologist,
identifying important features on the corresponding scan for other physicians
not specifically trained in radiology. We propose a methodology for
approximating image-level labels for radiology studies from associated reports
using a general purpose language processing tool for medical concept extraction
and sentiment analysis, and simple manually crafted heuristics for false
positive reduction. Using this approach, we label more than 175,000 Head CT
studies for the presence of 33 features indicative of 11 clinically relevant
conditions. For 27 of the 30 keywords that yielded positive results (3 had no
occurrences), the lower bound of the confidence intervals created to estimate
the percentage of accurately labeled reports was above 85%, with the average
being above 95%. Though noisier then manual labeling, these results suggest
this method to be a viable means of labeling medical images at scale.

negative:
In this paper we address the problem of matching two images with two
different resolutions: a high-resolution image and a low-resolution one. The
difference in resolution between the two images is not known and without loss
of generality one of the images is assumed to be the high-resolution one. On
the premise that changes in resolution act as a smoothing equivalent to changes
in scale, a scale-space representation of the high-resolution image is
produced. Hence the one-to-one classical image matching paradigm becomes
one-to-many because the low-resolution image is compared with all the
scale-space representations of the high-resolution one. Key to the success of
such a process is the proper representation of the features to be matched in
scale-space. We show how to represent and extract interest points at variable
scales and we devise a method allowing the comparison of two images at two
different resolutions. The method comprises the use of photometric- and
rotation-invariant descriptors, a geometric model mapping the high-resolution
image onto a low-resolution image region, and an image matching strategy based
on local constraints and on the robust estimation of this geometric model.
Extensive experiments show that our matching method can be used for scale
changes up to a factor of 6.

negative:
Digital camera and mobile document image acquisition are new trends arising
in the world of Optical Character Recognition and text detection. In some
cases, such process integrates many distortions and produces poorly scanned
text or text-photo images and natural images, leading to an unreliable OCR
digitization. In this paper, we present a novel nonparametric and unsupervised
method to compensate for undesirable document image distortions aiming to
optimally improve OCR accuracy. Our approach relies on a very efficient stack
of document image enhancing techniques to recover deformation of the entire
document image. First, we propose a local brightness and contrast adjustment
method to effectively handle lighting variations and the irregular distribution
of image illumination. Second, we use an optimized greyscale conversion
algorithm to transform our document image to greyscale level. Third, we sharpen
the useful information in the resulting greyscale image using Un-sharp Masking
method. Finally, an optimal global binarization approach is used to prepare the
final document image to OCR recognition. The proposed approach can
significantly improve text detection rate and optical character recognition
accuracy. To demonstrate the efficiency of our approach, an exhaustive
experimentation on a standard dataset is presented.

negative:
Graphical elements: particularly tables and figures contain a visual summary
of the most valuable information contained in a document. Therefore,
localization of such graphical objects in the document images is the initial
step to understand the content of such graphical objects or document images. In
this paper, we present a novel end-to-end trainable deep learning based
framework to localize graphical objects in the document images called as
Graphical Object Detection (GOD). Our framework is data-driven and does not
require any heuristics or meta-data to locate graphical objects in the document
images. The GOD explores the concept of transfer learning and domain adaptation
to handle scarcity of labeled training images for graphical object detection
task in the document images. Performance analysis carried out on the various
public benchmark data sets: ICDAR-2013, ICDAR-POD2017,and UNLV shows that our
model yields promising results as compared to state-of-the-art techniques.

negative:
Despite significant progress on current state-of-the-art image generation
models, synthesis of document images containing multiple and complex object
layouts is a challenging task. This paper presents a novel approach, called
DocSynth, to automatically synthesize document images based on a given layout.
In this work, given a spatial layout (bounding boxes with object categories) as
a reference by the user, our proposed DocSynth model learns to generate a set
of realistic document images consistent with the defined layout. Also, this
framework has been adapted to this work as a superior baseline model for
creating synthetic document image datasets for augmenting real data during
training for document layout analysis tasks. Different sets of learning
objectives have been also used to improve the model performance.
Quantitatively, we also compare the generated results of our model with real
data using standard evaluation metrics. The results highlight that our model
can successfully generate realistic and diverse document images with multiple
objects. We also present a comprehensive qualitative analysis summary of the
different scopes of synthetic image generation tasks. Lastly, to our knowledge
this is the first work of its kind.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Automatic annotation of images with descriptive words is a challenging
problem with vast applications in the areas of image search and retrieval. This
problem can be viewed as a label-assignment problem by a classifier dealing
with a very large set of labels, i.e., the vocabulary set. We propose a novel
annotation method that employs two layers of sparse coding and performs
coarse-to-fine labeling. Themes extracted from the training data are treated as
coarse labels. Each theme is a set of training images that share a common
subject in their visual and textual contents. Our system extracts coarse labels
for training and test images without requiring any prior knowledge. Vocabulary
words are the fine labels to be associated with images. Most of the annotation
methods achieve low recall due to the large number of available fine labels,
i.e., vocabulary words. These systems also tend to achieve high precision for
highly frequent words only while relatively rare words are more important for
search and retrieval purposes. Our system not only outperforms various
previously proposed annotation systems, but also achieves symmetric response in
terms of precision and recall. Our system scores and maintains high precision
for words with a wide range of frequencies. Such behavior is achieved by
intelligently reducing the number of available fine labels or words for each
image based on coarse labels assigned to it.

negative:
Significant progress has been made in boundary detection with the help of
convolutional neural networks. Recent boundary detection models not only focus
on real object boundary detection but also "crisp" boundaries (precisely
localized along the object's contour). There are two methods to evaluate crisp
boundary performance. One uses more strict tolerance to measure the distance
between the ground truth and the detected contour. The other focuses on
evaluating the contour map without any postprocessing. In this study, we
analyze both methods and conclude that both methods are two aspects of crisp
contour evaluation. Accordingly, we propose a novel network named deep
refinement network (DRNet) that stacks multiple refinement modules to achieve
richer feature representation and a novel loss function, which combines
cross-entropy and dice loss through effective adaptive fusion. Experimental
results demonstrated that we achieve state-of-the-art performance for several
available datasets.

negative:
With increasing urbanization, in recent years there has been a growing
interest and need in monitoring and analyzing urban flood events. Social media,
as a new data source, can provide real-time information for flood monitoring.
The social media posts with locations are often referred to as Volunteered
Geographic Information (VGI), which can reveal the spatial pattern of such
events. Since more images are shared on social media than ever before, recent
research focused on the extraction of flood-related posts by analyzing images
in addition to texts. Apart from merely classifying posts as flood relevant or
not, more detailed information, e.g. the flood severity, can also be extracted
based on image interpretation. However, it has been less tackled and has not
yet been applied for flood severity mapping.
  In this paper, we propose a novel three-step process to extract and map flood
severity information. First, flood relevant images are retrieved with the help
of pre-trained convolutional neural networks as feature extractors. Second, the
images containing people are further classified into four severity levels by
observing the relationship between body parts and their partial inundation,
i.e. images are classified according to the water level with respect to
different body parts, namely ankle, knee, hip, and chest. Lastly, locations of
the Tweets are used for generating a map of estimated flood extent and
severity. This process was applied to an image dataset collected during
Hurricane Harvey in 2017, as a proof of concept. The results show that VGI can
be used as a supplement to remote sensing observations for flood extent mapping
and is beneficial, especially for urban areas, where the infrastructure is
often occluding water. Based on the extracted water level information, an
integrated overview of flood severity can be provided for the early stages of
emergency response.

negative:
One image processing application that is very helpful for humans is to
improve image quality, poor image quality makes the image more difficult to
interpret because the information conveyed by the image is reduced. In the
process of the acquisition of medical images, the resulting image has decreased
quality (degraded) due to external factors and medical equipment used. For this
reason, it is necessary to have an image processing process to improve the
quality of medical images, so that later it is expected to help facilitate
medical personnel in analyzing and translating medical images, which will lead
to an improvement in the quality of diagnosis. In this study, an analysis will
be carried out to improve the quality of medical images with noise reduction
with the Gaussian Filter Method. Next, it is carried out, and tested against
medical images, in this case, the lung photo image. The test image is given
noise in the form of impulse salt & pepper and adaptive Gaussian then analyzed
its performance qualitatively by comparing the output filter image, noise
image, and the original image by naked eye.

negative:
Visual complexity identifies the level of intricacy and details in an image
or the level of difficulty to describe the image. It is an important concept in
a variety of areas such as cognitive psychology, computer vision and
visualization, and advertisement. Yet, efforts to create large, downloadable
image datasets with diverse content and unbiased groundtruthing are lacking. In
this work, we introduce Savoias, a visual complexity dataset that compromises
of more than 1,400 images from seven image categories relevant to the above
research areas, namely Scenes, Advertisements, Visualization and infographics,
Objects, Interior design, Art, and Suprematism. The images in each category
portray diverse characteristics including various low-level and high-level
features, objects, backgrounds, textures and patterns, text, and graphics. The
ground truth for Savoias is obtained by crowdsourcing more than 37,000 pairwise
comparisons of images using the forced-choice methodology and with more than
1,600 contributors. The resulting relative scores are then converted to
absolute visual complexity scores using the Bradley-Terry method and matrix
completion. When applying five state-of-the-art algorithms to analyze the
visual complexity of the images in the Savoias dataset, we found that the
scores obtained from these baseline tools only correlate well with crowdsourced
labels for abstract patterns in the Suprematism category (Pearson correlation
r=0.84). For the other categories, in particular, the objects and advertisement
categories, low correlation coefficients were revealed (r=0.3 and 0.56,
respectively). These findings suggest that (1) state-of-the-art approaches are
mostly insufficient and (2) Savoias enables category-specific method
development, which is likely to improve the impact of visual complexity
analysis on specific application areas, including computer vision.

negative:
"If I provide you a face image of mine (without telling you the actual age
when I took the picture) and a large amount of face images that I crawled
(containing labeled faces of different ages but not necessarily paired), can
you show me what I would look like when I am 80 or what I was like when I was
5?" The answer is probably a "No." Most existing face aging works attempt to
learn the transformation between age groups and thus would require the paired
samples as well as the labeled query image. In this paper, we look at the
problem from a generative modeling perspective such that no paired samples is
required. In addition, given an unlabeled image, the generative model can
directly produce the image with desired age attribute. We propose a conditional
adversarial autoencoder (CAAE) that learns a face manifold, traversing on which
smooth age progression and regression can be realized simultaneously. In CAAE,
the face is first mapped to a latent vector through a convolutional encoder,
and then the vector is projected to the face manifold conditional on age
through a deconvolutional generator. The latent vector preserves personalized
face features (i.e., personality) and the age condition controls progression
vs. regression. Two adversarial networks are imposed on the encoder and
generator, respectively, forcing to generate more photo-realistic faces.
Experimental results demonstrate the appealing performance and flexibility of
the proposed framework by comparing with the state-of-the-art and ground truth.

negative:
Traditionally, the main focus of image super-resolution techniques is on
recovering the most likely high-quality images from low-quality images, using a
one-to-one low- to high-resolution mapping. Proceeding that way, we ignore the
fact that there are generally many valid versions of high-resolution images
that map to a given low-resolution image. We are tackling in this work the
problem of obtaining different high-resolution versions from the same
low-resolution image using Generative Adversarial Models. Our learning approach
makes use of high frequencies available in the training high-resolution images
for preserving and exploring in an unsupervised manner the structural
information available within these images. Experimental results on the CelebA
dataset confirm the effectiveness of the proposed method, which allows the
generation of both realistic and diverse high-resolution images from
low-resolution images.

negative:
Generating realistic images is difficult, and many formulations for this task
have been proposed recently. If we restrict the task to that of generating a
particular class of images, however, the task becomes more tractable. That is
to say, instead of generating an arbitrary image as a sample from the manifold
of natural images, we propose to sample images from a particular "subspace" of
natural images, directed by a low-resolution image from the same subspace. The
problem we address, while close to the formulation of the single-image
super-resolution problem, is in fact rather different. Single image
super-resolution is the task of predicting the image closest to the ground
truth from a relatively low resolution image. We propose to produce samples of
high resolution images given extremely small inputs with a new method called
Latent Adversarial Generator (LAG). In our generative sampling framework, we
only use the input (possibly of very low-resolution) to direct what class of
samples the network should produce. As such, the output of our algorithm is not
a unique image that relates to the input, but rather a possible se} of related
images sampled from the manifold of natural images. Our method learns
exclusively in the latent space of the adversary using perceptual loss -- it
does not have a pixel loss.

negative:
Document Layout Analysis is a fundamental step in Handwritten Text Processing
systems, from the extraction of the text lines to the type of zone it belongs
to. We present a system based on artificial neural networks which is able to
determine not only the baselines of text lines present in the document, but
also performs geometric and logic layout analysis of the document. Experiments
in three different datasets demonstrate the potential of the method and show
competitive results with respect to state-of-the-art methods.

negative:
Achieving a symbiotic blending between reality and virtuality is a dream that
has been lying in the minds of many people for a long time. Advances in various
domains constantly bring us closer to making that dream come true. Augmented
reality as well as virtual reality are in fact trending terms and are expected
to further progress in the years to come.
  This master's thesis aims to explore these areas and starts by defining
necessary terms such as augmented reality (AR) or virtual reality (VR). Usual
taxonomies to classify and compare the corresponding experiences are then
discussed.
  In order to enable those applications, many technical challenges need to be
tackled, such as accurate motion tracking with 6 degrees of freedom (positional
and rotational), that is necessary for compelling experiences and to prevent
user sickness. Additionally, augmented reality experiences typically rely on
image processing to position the superimposed content. To do so, "paper"
markers or features extracted from the environment are often employed. Both
sets of techniques are explored and common solutions and algorithms are
presented.
  After investigating those technical aspects, I carry out an objective
comparison of the existing state-of-the-art and state-of-the-practice in those
domains, and I discuss present and potential applications in these areas. As a
practical validation, I present the results of an application that I have
developed using Microsoft HoloLens, one of the more advanced affordable
technologies for augmented reality that is available today. Based on the
experience and lessons learned during this development, I discuss the
limitations of current technologies and present some avenues of future
research.

negative:
Identity documents recognition is an important sub-field of document
analysis, which deals with tasks of robust document detection, type
identification, text fields recognition, as well as identity fraud prevention
and document authenticity validation given photos, scans, or video frames of an
identity document capture. Significant amount of research has been published on
this topic in recent years, however a chief difficulty for such research is
scarcity of datasets, due to the subject matter being protected by security
requirements. A few datasets of identity documents which are available lack
diversity of document types, capturing conditions, or variability of document
field values. In addition, the published datasets were typically designed only
for a subset of document recognition problems, not for a complex identity
document analysis. In this paper, we present a dataset MIDV-2020 which consists
of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock
identity documents, each with unique text field values and unique artificially
generated faces, with rich annotation. For the presented benchmark dataset
baselines are provided for such tasks as document location and identification,
text fields recognition, and face detection. With 72409 annotated images in
total, to the date of publication the proposed dataset is the largest publicly
available identity documents dataset with variable artificially generated data,
and we believe that it will prove invaluable for advancement of the field of
document analysis and recognition. The dataset is available for download at
ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr .

negative:
Documents often exhibit various forms of degradation, which make it hard to
be read and substantially deteriorate the performance of an OCR system. In this
paper, we propose an effective end-to-end framework named Document Enhancement
Generative Adversarial Networks (DE-GAN) that uses the conditional GANs (cGANs)
to restore severely degraded document images. To the best of our knowledge,
this practice has not been studied within the context of generative adversarial
deep networks. We demonstrate that, in different tasks (document clean up,
binarization, deblurring and watermark removal), DE-GAN can produce an enhanced
version of the degraded document with a high quality. In addition, our approach
provides consistent improvements compared to state-of-the-art methods over the
widely used DIBCO 2013, DIBCO 2017 and H-DIBCO 2018 datasets, proving its
ability to restore a degraded document image to its ideal condition. The
obtained results on a wide variety of degradation reveal the flexibility of the
proposed model to be exploited in other document enhancement problems.

negative:
Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.

negative:
This paper presents results of Document Visual Question Answering Challenge
organized as part of "Text and Documents in the Deep Learning Era" workshop, in
CVPR 2020. The challenge introduces a new problem - Visual Question Answering
on document images. The challenge comprised two tasks. The first task concerns
with asking questions on a single document image. On the other hand, the second
task is set as a retrieval task where the question is posed over a collection
of images. For the task 1 a new dataset is introduced comprising 50,000
questions-answer(s) pairs defined over 12,767 document images. For task 2
another dataset has been created comprising 20 questions over 14,362 document
images which share the same document template.

negative:
We introduce a novel type of text representation that preserves the 2D layout
of a document. This is achieved by encoding each document page as a
two-dimensional grid of characters. Based on this representation, we present a
generic document understanding pipeline for structured documents. This pipeline
makes use of a fully convolutional encoder-decoder network that predicts a
segmentation mask and bounding boxes. We demonstrate its capabilities on an
information extraction task from invoices and show that it significantly
outperforms approaches based on sequential text or document images.

negative:
There is a large collection of Handwritten English paper documents of
Historical and Scientific importance. But paper documents are not recognized
directly by computer. Hence the closest way of indexing these documents is by
storing their document digital image. Hence a large database of document images
can replace the paper documents. But the document and data corresponding to
each image cannot be directly recognized by the computer.
  This paper applies the technique of word spotting using Modified Character
Shape Code to Handwritten English document images for quick and efficient query
search of words on a database of document images. It is different from other
Word Spotting techniques as it implements two level of selection for word
segments to match search query. First based on word size and then based on
character shape code of query. It makes the process faster and more efficient
and reduces the need of multiple pre-processing.

negative:
Today all kind of information is getting digitized and along with all this
digitization, the huge archive of various kinds of documents is being digitized
too. We know that, Optical Character Recognition is the method through which,
newspapers and other paper documents convert into digital resources. But, it is
a fact that this method works on texts only. As a result, if we try to process
any document which contains non-textual zones, then we will get garbage texts
as output. That is why; in order to digitize documents properly they should be
prepossessed carefully. And while preprocessing, segmenting document in
different regions according to the category properly is most important. But,
the Optical Character Recognition processes available for Bangla language have
no such algorithm that can categorize a newspaper/book page fully. So we worked
to decompose a document into its several parts like headlines, sub headlines,
columns, images etc. And if the input is skewed and rotated, then the input was
also deskewed and de-rotated. To decompose any Bangla document we found out the
edges of the input image. Then we find out the horizontal and vertical area of
every pixel where it lies in. Later on the input image was cut according to
these areas. Then we pick each and every sub image and found out their
height-width ratio, line height. Then according to these values the sub images
were categorized. To deskew the image we found out the skew angle and de skewed
the image according to this angle. To de-rotate the image we used the line
height, matra line, pixel ratio of matra line.

negative:
Nowadays, as cameras are rapidly adopted in our daily routine, images of
documents are becoming both abundant and prevalent. Unlike natural images that
capture physical objects, document-images contain a significant amount of text
with critical semantics and complicated layouts. In this work, we devise a
generic unsupervised technique to learn multimodal affinities between textual
entities in a document-image, considering their visual style, the content of
their underlying text and their geometric context within the image. We then use
these learned affinities to automatically cluster the textual entities in the
image into different semantic groups. The core of our approach is a deep
optimization scheme dedicated for an image provided by the user that detects
and leverages reliable pairwise connections in the multimodal representation of
the textual elements in order to properly learn the affinities. We show that
our technique can operate on highly varying images spanning a wide range of
documents and demonstrate its applicability for various editing operations
manipulating the content, appearance and geometry of the image.

negative:
The accuracy of OCR is usually affected by the quality of the input document
image and different kinds of marred document images hamper the OCR results.
Among these scenarios, the low-resolution image is a common and challenging
case. In this paper, we propose the cascaded networks for document image
super-resolution. Our model is composed by the Detail-Preserving Networks with
small magnification. The loss function with perceptual terms is designed to
simultaneously preserve the original patterns and enhance the edge of the
characters. These networks are trained with the same architecture and different
parameters and then assembled into a pipeline model with a larger
magnification. The low-resolution images can upscale gradually by passing
through each Detail-Preserving Network until the final high-resolution images.
Through extensive experiments on two scanning document image datasets, we
demonstrate that the proposed approach outperforms recent state-of-the-art
image super-resolution methods, and combining it with standard OCR system lead
to signification improvements on the recognition results.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.

negative:
One major drawback of state of the art Neural Networks (NN)-based approaches
for document classification purposes is the large number of training samples
required to obtain an efficient classification. The minimum required number is
around one thousand annotated documents for each class. In many cases it is
very difficult, if not impossible, to gather this number of samples in real
industrial processes. In this paper, we analyse the efficiency of NN-based
document classification systems in a sub-optimal training case, based on the
situation of a company document stream. We evaluated three different
approaches, one based on image content and two on textual content. The
evaluation was divided into four parts: a reference case, to assess the
performance of the system in the lab; two cases that each simulate a specific
difficulty linked to document stream processing; and a realistic case that
combined all of these difficulties. The realistic case highlighted the fact
that there is a significant drop in the efficiency of NN-Based document
classification systems. Although they remain efficient for well represented
classes (with an over-fitting of the system for those classes), it is
impossible for them to handle appropriately less well represented classes.
NN-Based document classification systems need to be adapted to resolve these
two problems before they can be considered for use in a company document
stream.

negative:
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.

negative:
The digital camera captured document images may often be warped and distorted
due to different camera angles or document surfaces. A robust technique is
needed to solve this kind of distortion. The research on dewarping of the
document suffers due to the limited availability of benchmark public dataset.
In recent times, deep learning based approaches are used to solve the problems
accurately. To train most of the deep neural networks a large number of
document images is required and generating such a large volume of document
images manually is difficult. In this paper, we propose a technique to generate
a synthetic warped image from a flat-bedded scanned document image. It is done
by calculating warping factors for each pixel position using two warping
position parameters (WPP) and eight warping control parameters (WCP). These
parameters can be specified as needed depending upon the desired warping. The
results are compared with similar real captured images both qualitative and
quantitative way.

negative:
Document segmentation is a method of rending the document into distinct
regions. A document is an assortment of information and a standard mode of
conveying information to others. Pursuance of data from documents involves ton
of human effort, time intense and might severely prohibit the usage of data
systems. So, automatic information pursuance from the document has become a big
issue. It is been shown that document segmentation will facilitate to beat such
problems. This paper proposes a new approach to segment and classify the
document regions as text, image, drawings and table. Document image is divided
into blocks using Run length smearing rule and features are extracted from
every blocks. Discipulus tool has been used to construct the Genetic
programming based classifier model and located 97.5% classification accuracy.