query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am working on autonomous driving research and I need to minimize the risk of autonomous vehicles to the public, especially pedestrians. Therefore, I am trying to build a model that will inform my autonomous vehicles to avoid crowded streets where lots of people are walking. This model has to be deployed in real-time. My current idea is to use a computer vision model that can extract and count the number of pedestrians from surveillance footage videos. It needs to be a fast pedestrian detection model. How should I go about doing this? Should my model be able to process videos or simply just images ? If I use videos, the computational load would be higher but I could get a more accurate estimate of the walking speed of these pedestrians during a time interval.

positive:
Detecting and predicting the behavior of pedestrians is extremely crucial for
self-driving vehicles to plan and interact with them safely. Although there
have been several research works in this area, it is important to have fast and
memory efficient models such that it can operate in embedded hardware in these
autonomous machines. In this work, we propose a novel architecture using
spatial-temporal multi-tasking to do camera based pedestrian detection and
intention prediction. Our approach significantly reduces the latency by being
able to detect and predict all pedestrians' intention in a single shot manner
while also being able to attain better accuracy by sharing features with
relevant object level information and interactions.

positive:
Studies of object detection and localization, particularly pedestrian
detection have received considerable attention in recent times due to its
several prospective applications such as surveillance, driving assistance,
autonomous cars, etc. Also, a significant trend of latest research studies in
related problem areas is the use of sophisticated Deep Learning based
approaches to improve the benchmark performance on various standard datasets. A
trade-off between the speed (number of video frames processed per second) and
detection accuracy has often been reported in the existing literature. In this
article, we present a new but simple deep learning based strategy for
pedestrian detection that improves this trade-off. Since training of similar
models using publicly available sample datasets failed to improve the detection
performance to some significant extent, particularly for the instances of
pedestrians of smaller sizes, we have developed a new sample dataset consisting
of more than 80K annotated pedestrian figures in videos recorded under varying
traffic conditions. Performance of the proposed model on the test samples of
the new dataset and two other existing datasets, namely Caltech Pedestrian
Dataset (CPD) and CityPerson Dataset (CD) have been obtained. Our proposed
system shows nearly 16\% improvement over the existing state-of-the-art result.

positive:
With the rise of self-driving vehicles comes the risk of accidents and the
need for higher safety, and protection for pedestrian detection in the
following scenarios: imminent crashes, thus the car should crash into an object
and avoid the pedestrian, and in the case of road intersections, where it is
important for the car to stop when pedestrians are crossing. Currently, a
special topology of deep neural networks called Fused Deep Neural Network
(F-DNN) is considered to be the state of the art in pedestrian detection, as it
has the lowest miss rate, yet it is very slow. Therefore, acceleration is
needed to speed up the performance. This project proposes two contributions to
address this problem, by using a deep neural network used for object detection,
called Single Shot Multi-Box Detector (SSD). The first contribution is training
and tuning the hyperparameters of SSD to improve pedestrian detection. The
second contribution is a new FPGA design for accelerating the model on the
Altera Arria 10 platform. The final system will be used in self-driving
vehicles in real-time. Preliminary results of the improved SSD shows 3% higher
miss-rate than F-DNN on Caltech pedestrian detection benchmark, but 4x
performance improvement. The acceleration design is expected to achieve an
additional performance improvement significantly outweighing the minimal
difference in accuracy.

positive:
In this paper, we present a real-time robust multi-view pedestrian detection
and tracking system for video surveillance using neural networks which can be
used in dynamic environments. The proposed system consists of two phases:
multi-view pedestrian detection and tracking. First, pedestrian detection
utilizes background subtraction to segment the foreground blob. An adaptive
background subtraction method where each of the pixel of input image models as
a mixture of Gaussians and uses an on-line approximation to update the model
applies to extract the foreground region. The Gaussian distributions are then
evaluated to determine which are most likely to result from a background
process. This method produces a steady, real-time tracker in outdoor
environment that consistently deals with changes of lighting condition, and
long-term scene change. Second, the Tracking is performed at two phases:
pedestrian classification and tracking the individual subject. A sliding window
is applied on foreground binary image to select an input window which is used
for selecting the input image patches from actually input frame. The neural
networks is used for classification with PHOG features. Finally, a Kalman
filter is applied to calculate the subsequent step for tracking that aims at
finding the exact position of pedestrians in an input image. The experimental
result shows that the proposed approach yields promising performance on
multi-view pedestrian detection and tracking on different benchmark datasets.

positive:
Studies on microscopic pedestrian requires large amounts of trajectory data
from real-world pedestrian crowds. Such data collection, if done manually,
needs tremendous effort and is very time consuming. Though many studies have
asserted the possibility of automating this task using video cameras, we found
that only a few have demonstrated good performance in very crowded situations
or from a top-angled view scene. This paper deals with tracking pedestrian
crowd under heavy occlusions from an angular scene. Our automated tracking
system consists of two modules that perform sequentially. The first module
detects moving objects as blobs. The second module is a tracking system. We
employ probability distribution from the detection of each pedestrian and use
Bayesian update to track the next position. The result of such tracking is a
database of pedestrian trajectories over time and space. With certain prior
information, we showed that the system can track a large number of people under
occlusion and clutter scene.

positive:
Vehicle-to-Pedestrian (V2P) communication can significantly improve
pedestrian safety at a signalized intersection. It is unlikely that pedestrians
will carry a low latency communication enabled device and activate a pedestrian
safety application in their hand-held device all the time. Because of this
limitation, multiple traffic cameras at the signalized intersection can be used
to accurately detect and locate pedestrians using deep learning and broadcast
safety alerts related to pedestrians to warn connected and automated vehicles
around a signalized intersection. However, unavailability of high-performance
computing infrastructure at the roadside and limited network bandwidth between
traffic cameras and the computing infrastructure limits the ability of
real-time data streaming and processing for pedestrian detection. In this
paper, we develop an edge computing based real-time pedestrian detection
strategy combining pedestrian detection algorithm using deep learning and an
efficient data communication approach to reduce bandwidth requirements while
maintaining a high object detection accuracy. We utilize a lossy compression
technique on traffic camera data to determine the tradeoff between the
reduction of the communication bandwidth requirements and a defined object
detection accuracy. The performance of the pedestrian-detection strategy is
measured in terms of pedestrian classification accuracy with varying peak
signal-to-noise ratios. The analyses reveal that we detect pedestrians by
maintaining a defined detection accuracy with a peak signal-to-noise ratio
(PSNR) 43 dB while reducing the communication bandwidth from 9.82 Mbits/sec to
0.31 Mbits/sec, a 31x reduction.

positive:
Detecting pedestrian has been arguably addressed as a special topic beyond
general object detection. Although recent deep learning object detectors such
as Fast/Faster R-CNN [1, 2] have shown excellent performance for general object
detection, they have limited success for detecting pedestrian, and previous
leading pedestrian detectors were in general hybrid methods combining
hand-crafted and deep convolutional features. In this paper, we investigate
issues involving Faster R-CNN [2] for pedestrian detection. We discover that
the Region Proposal Network (RPN) in Faster R-CNN indeed performs well as a
stand-alone pedestrian detector, but surprisingly, the downstream classifier
degrades the results. We argue that two reasons account for the unsatisfactory
accuracy: (i) insufficient resolution of feature maps for handling small
instances, and (ii) lack of any bootstrapping strategy for mining hard negative
examples. Driven by these observations, we propose a very simple but effective
baseline for pedestrian detection, using an RPN followed by boosted forests on
shared, high-resolution convolutional feature maps. We comprehensively evaluate
this method on several benchmarks (Caltech, INRIA, ETH, and KITTI), presenting
competitive accuracy and good speed. Code will be made publicly available.

positive:
As camera quality improves and their deployment moves to areas with limited
bandwidth, communication bottlenecks can impair real-time constraints of an ITS
application, such as video-based real-time pedestrian detection. Video
compression reduces the bandwidth requirement to transmit the video but
degrades the video quality. As the quality level of the video decreases, it
results in the corresponding decreases in the accuracy of the vision-based
pedestrian detection model. Furthermore, environmental conditions (e.g., rain
and darkness) alter the compression ratio and can make maintaining a high
pedestrian detection accuracy more difficult. The objective of this study is to
develop a real-time error-bounded lossy compression (EBLC) strategy to
dynamically change the video compression level depending on different
environmental conditions in order to maintain a high pedestrian detection
accuracy. We conduct a case study to show the efficacy of our dynamic EBLC
strategy for real-time vision-based pedestrian detection under adverse
environmental conditions. Our strategy selects the error tolerances dynamically
for lossy compression that can maintain a high detection accuracy across a
representative set of environmental conditions. Analyses reveal that our
strategy increases pedestrian detection accuracy up to 14% and reduces the
communication bandwidth up to 14x for adverse environmental conditions compared
to the same conditions but without our dynamic EBLC strategy. Our dynamic EBLC
strategy is independent of detection models and environmental conditions
allowing other detection models and environmental conditions to be easily
incorporated in our strategy.

positive:
Effective fusion of complementary information captured by multi-modal sensors
(visible and infrared cameras) enables robust pedestrian detection under
various surveillance situations (e.g. daytime and nighttime). In this paper, we
present a novel box-level segmentation supervised learning framework for
accurate and real-time multispectral pedestrian detection by incorporating
features extracted in visible and infrared channels. Specifically, our method
takes pairs of aligned visible and infrared images with easily obtained
bounding box annotations as input and estimates accurate prediction maps to
highlight the existence of pedestrians. It offers two major advantages over the
existing anchor box based multispectral detection methods. Firstly, it
overcomes the hyperparameter setting problem occurred during the training phase
of anchor box based detectors and can obtain more accurate detection results,
especially for small and occluded pedestrian instances. Secondly, it is capable
of generating accurate detection results using small-size input images, leading
to improvement of computational efficiency for real-time autonomous driving
applications. Experimental results on KAIST multispectral dataset show that our
proposed method outperforms state-of-the-art approaches in terms of both
accuracy and speed.

positive:
Pedestrians and drivers interact closely in a wide range of environments.
Autonomous vehicles (AVs) correspondingly face the need to predict pedestrians'
future trajectories in these same environments. Traditional model-based
prediction methods have been limited to making predictions in highly structured
scenes with signalized intersections, marked crosswalks, or curbs. Deep
learning methods have instead leveraged datasets to learn predictive features
that generalize across scenes, at the cost of model interpretability. This
paper aims to achieve both widely applicable and interpretable predictions by
proposing a risk-based attention mechanism to learn when pedestrians yield, and
a model of vehicle influence to learn how yielding affects motion. A novel
probabilistic method, Off the Sidewalk Predictions (OSP), uses these to achieve
accurate predictions in both shared spaces and traditional scenes. Experiments
on urban datasets demonstrate that the realtime method achieves
state-of-the-art performance.

positive:
In this paper, we present an end-to-end future-prediction model that focuses
on pedestrian safety. Specifically, our model uses previous video frames,
recorded from the perspective of the vehicle, to predict if a pedestrian will
cross in front of the vehicle. The long term goal of this work is to design a
fully autonomous system that acts and reacts as a defensive human driver would
--- predicting future events and reacting to mitigate risk. We focus on
pedestrian-vehicle interactions because of the high risk of harm to the
pedestrian if their actions are miss-predicted. Our end-to-end model consists
of two stages: the first stage is an encoder/decoder network that learns to
predict future video frames. The second stage is a deep spatio-temporal network
that utilizes the predicted frames of the first stage to predict the
pedestrian's future action. Our system achieves state-of-the-art accuracy on
pedestrian behavior prediction and future frames prediction on the Joint
Attention for Autonomous Driving (JAAD) dataset.

positive:
Social distancing can reduce the infection rates in respiratory pandemics
such as COVID-19. Traffic intersections are particularly suitable for
monitoring and evaluation of social distancing behavior in metropolises. We
propose and evaluate a privacy-preserving social distancing analysis system
(B-SDA), which uses bird's-eye view video recordings of pedestrians who cross
traffic intersections. We devise algorithms for video pre-processing, object
detection and tracking which are rooted in the known computer-vision and deep
learning techniques, but modified to address the problem of detecting very
small objects/pedestrians captured by a highly elevated camera. We propose a
method for incorporating pedestrian grouping for detection of social distancing
violations. B-SDA is used to compare pedestrian behavior based on pre-pandemic
and pandemic videos in a major metropolitan area. The accomplished pedestrian
detection performance is $63.0\%$ $AP_{50}$ and the tracking performance is
$47.6\%$ MOTA. The social distancing violation rate of $15.6\%$ during the
pandemic is notably lower than $31.4\%$ pre-pandemic baseline, indicating that
pedestrians followed CDC-prescribed social distancing recommendations. The
proposed system is suitable for deployment in real-world applications.

positive:
Generative adversarial networks (GANs) have been promising for many computer
vision problems due to their powerful capabilities to enhance the data for
training and test. In this paper, we leveraged GANs and proposed a new
architecture with a cascaded Single Shot Detector (SSD) for pedestrian
detection at distance, which is yet a challenge due to the varied sizes of
pedestrians in videos at distance. To overcome the low-resolution issues in
pedestrian detection at distance, DCGAN is employed to improve the resolution
first to reconstruct more discriminative features for a SSD to detect objects
in images or videos. A crucial advantage of our method is that it learns a
multi-scale metric to distinguish multiple objects at different distances under
one image, while DCGAN serves as an encoder-decoder platform to generate parts
of an image that contain better discriminative information. To measure the
effectiveness of our proposed method, experiments were carried out on the
Canadian Institute for Advanced Research (CIFAR) dataset, and it was
demonstrated that the proposed new architecture achieved a much better
detection rate, particularly on vehicles and pedestrians at distance, making it
highly suitable for smart cities applications that need to discover key objects
or pedestrians at distance.

positive:
In recent years, road safety has attracted significant attention from
researchers and practitioners in the intelligent transport systems domain. As
one of the most common and vulnerable groups of road users, pedestrians cause
great concerns due to their unpredictable behavior and movement, as subtle
misunderstandings in vehicle-pedestrian interaction can easily lead to risky
situations or collisions. Existing methods use either predefined
collision-based models or human-labeling approaches to estimate the
pedestrians' risks. These approaches are usually limited by their poor
generalization ability and lack of consideration of interactions between the
ego vehicle and a pedestrian. This work tackles the listed problems by
proposing a Pedestrian Risk Level Prediction system. The system consists of
three modules. Firstly, vehicle-perspective pedestrian data are collected.
Since the data contains information regarding the movement of both the ego
vehicle and pedestrian, it can simplify the prediction of spatiotemporal
features in an interaction-aware fashion. Using the long short-term memory
model, the pedestrian trajectory prediction module predicts their
spatiotemporal features in the subsequent five frames. As the predicted
trajectory follows certain interaction and risk patterns, a hybrid clustering
and classification method is adopted to explore the risk patterns in the
spatiotemporal features and train a risk level classifier using the learned
patterns. Upon predicting the spatiotemporal features of pedestrians and
identifying the corresponding risk level, the risk patterns between the ego
vehicle and pedestrians are determined. Experimental results verified the
capability of the PRLP system to predict the risk level of pedestrians, thus
supporting the collision risk assessment of intelligent vehicles and providing
safety warnings to both vehicles and pedestrians.

negative:
This paper presents a planning system for autonomous driving among many
pedestrians. A key ingredient of our approach is PORCA, a pedestrian motion
prediction model that accounts for both a pedestrian's global navigation
intention and local interactions with the vehicle and other pedestrians.
Unfortunately, the autonomous vehicle does not know the pedestrian's intention
a priori and requires a planning algorithm that hedges against the uncertainty
in pedestrian intentions. Our planning system combines a POMDP algorithm with
the pedestrian motion model and runs in near real time. Experiments show that
it enables a robot vehicle to drive safely, efficiently, and smoothly among a
crowd with a density of nearly one person per square meter.

negative:
A major bottleneck of pedestrian detection lies on the sharp performance
deterioration in the presence of small-size pedestrians that are relatively far
from the camera. Motivated by the observation that pedestrians of disparate
spatial scales exhibit distinct visual appearances, we propose in this paper an
active pedestrian detector that explicitly operates over multiple-layer
neuronal representations of the input still image. More specifically,
convolutional neural nets such as ResNet and faster R-CNNs are exploited to
provide a rich and discriminative hierarchy of feature representations as well
as initial pedestrian proposals. Here each pedestrian observation of distinct
size could be best characterized in terms of the ResNet feature representation
at a certain layer of the hierarchy; Meanwhile, initial pedestrian proposals
are attained by faster R-CNNs techniques, i.e. region proposal network and
follow-up region of interesting pooling layer employed right after the specific
ResNet convolutional layer of interest, to produce joint predictions on the
bounding-box proposals' locations and categories (i.e. pedestrian or not). This
is engaged as input to our active detector where for each initial pedestrian
proposal, a sequence of coordinate transformation actions is carried out to
determine its proper x-y 2D location and layer of feature representation, or
eventually terminated as being background. Empirically our approach is
demonstrated to produce overall lower detection errors on widely-used
benchmarks, and it works particularly well with far-scale pedestrians. For
example, compared with 60.51% log-average miss rate of the state-of-the-art
MS-CNN for far-scale pedestrians (those below 80 pixels in bounding-box height)
of the Caltech benchmark, the miss rate of our approach is 41.85%, with a
notable reduction of 18.68%.

negative:
We develop predictive models of pedestrian dynamics by encoding the coupled
nature of multi-pedestrian interaction using game theory, and deep
learning-based visual analysis to estimate person-specific behavior parameters.
Building predictive models for multi-pedestrian interactions however, is very
challenging due to two reasons: (1) the dynamics of interaction are complex
interdependent processes, where the predicted behavior of one pedestrian can
affect the actions taken by others and (2) dynamics are variable depending on
an individuals physical characteristics (e.g., an older person may walk slowly
while the younger person may walk faster). To address these challenges, we (1)
utilize concepts from game theory to model the interdependent decision making
process of multiple pedestrians and (2) use visual classifiers to learn a
mapping from pedestrian appearance to behavior parameters. We evaluate our
proposed model on several public multiple pedestrian interaction video
datasets. Results show that our strategic planning model explains human
interactions 25% better when compared to state-of-the-art methods.

negative:
As autonomous vehicles and autonomous racing rise in popularity, so does the
need for faster and more accurate detectors. While our naked eyes are able to
extract contextual information almost instantly, even from far away, image
resolution and computational resources limitations make detecting smaller
objects (that is, objects that occupy a small pixel area in the input image) a
genuinely challenging task for machines and a wide-open research field. This
study explores how the popular YOLOv5 object detector can be modified to
improve its performance in detecting smaller objects, with a particular
application in autonomous racing. To achieve this, we investigate how replacing
certain structural elements of the model (as well as their connections and
other parameters) can affect performance and inference time. In doing so, we
propose a series of models at different scales, which we name `YOLO-Z', and
which display an improvement of up to 6.9% in mAP when detecting smaller
objects at 50% IOU, at the cost of just a 3ms increase in inference time
compared to the original YOLOv5. Our objective is to inform future research on
the potential of adjusting a popular detector such as YOLOv5 to address
specific tasks and provide insights on how specific changes can impact small
object detection. Such findings, applied to the broader context of autonomous
vehicles, could increase the amount of contextual information available to such
systems.

negative:
It is challenging for a mobile robot to navigate through human crowds.
Existing approaches usually assume that pedestrians follow a predefined
collision avoidance strategy, like social force model (SFM) or optimal
reciprocal collision avoidance (ORCA). However, their performances commonly
need to be further improved for practical applications, where pedestrians
follow multiple different collision avoidance strategies. In this paper, we
propose a map-based deep reinforcement learning approach for crowd-aware robot
navigation with various pedestrians. We use the sensor map to represent the
environmental information around the robot, including its shape and observable
appearances of obstacles. We also introduce the pedestrian map that specifies
the movements of pedestrians around the robot. By applying both maps as inputs
of the neural network, we show that a navigation policy can be trained to
better interact with pedestrians following different collision avoidance
strategies. We evaluate our approach under multiple scenarios both in the
simulator and on an actual robot. The results show that our approach allows the
robot to successfully interact with various pedestrians and outperforms
compared methods in terms of the success rate.

negative:
Pedestrian detection is a popular research topic due to its paramount
importance for a number of applications, especially in the fields of
automotive, surveillance and robotics. Despite the significant improvements,
pedestrian detection is still an open challenge that calls for more and more
accurate algorithms. In the last few years, deep learning and in particular
convolutional neural networks emerged as the state of the art in terms of
accuracy for a number of computer vision tasks such as image classification,
object detection and segmentation, often outperforming the previous gold
standards by a large margin. In this paper, we propose a pedestrian detection
system based on deep learning, adapting a general-purpose convolutional network
to the task at hand. By thoroughly analyzing and optimizing each step of the
detection pipeline we propose an architecture that outperforms traditional
methods, achieving a task accuracy close to that of state-of-the-art
approaches, while requiring a low computational time. Finally, we tested the
system on an NVIDIA Jetson TK1, a 192-core platform that is envisioned to be a
forerunner computational brain of future self-driving cars.

negative:
Pedestrian detection is a research hotspot and a difficult issue in the
computer vision such as the Intelligent Surveillance System, the Intelligent
Transport System, robotics, and automotive safety. However, the human body's
position, angle, and dress in a video scene are complicated and changeable,
which have a great influence on the detection accuracy. In this paper, through
the analysis on the pros and cons of Census Transform Histogram (CENTRIST), a
novel feature is presented for human detection Ternary CENTRIST (T-CENTRIST).
The T-CENTRIST feature takes the relationship between each pixel and its
neighborhood pixels into account. Meanwhile, it also considers the relevancy
among these neighborhood pixels. Therefore, the proposed feature description
method can reflect the silhouette of pedestrian more adequately and accurately
than that of CENTRIST. Second, we propose a fast pedestrian detection framework
based on T-CENTRIST in infrared image, which introduces the idea of extended
blocks and the integral image. Finally, experimental results verify the
effectiveness of the proposed pedestrian detection method.

negative:
A real-time Deep Learning based method for Pedestrian Detection (PD) is
applied to the Human-Aware robot navigation problem. The pedestrian detector
combines the Aggregate Channel Features (ACF) detector with a deep
Convolutional Neural Network (CNN) in order to obtain fast and accurate
performance. Our solution is firstly evaluated using a set of real images taken
from onboard and offboard cameras and, then, it is validated in a typical robot
navigation environment with pedestrians (two distinct experiments are
conducted). The results on both tests show that our pedestrian detector is
robust and fast enough to be used on robot navigation applications.

negative:
To obtain effective pedestrian detection results in surveillance video, there
have been many methods proposed to handle the problems from severe occlusion,
pose variation, clutter background, \emph{etc}. Besides detection accuracy, a
robust surveillance video system should be stable to video quality degradation
by network transmission, environment variation, etc. In this study, we conduct
the research on the robustness of pedestrian detection algorithms to video
quality degradation. The main contribution of this work includes the following
three aspects. First, a large-scale Distorted Surveillance Video Data Set
(DSurVD) is constructed from high-quality video sequences and their
corresponding distorted versions. Second, we design a method to evaluate
detection stability and a robustness measure called Robustness Quadrangle,
which can be adopted to visualize detection accuracy of pedestrian detection
algorithms on high-quality video sequences and stability with video quality
degradation. Third, the robustness of seven existing pedestrian detection
algorithms is evaluated by the built DSurVD. Experimental results show that the
robustness can be further improved for existing pedestrian detection
algorithms. Additionally, we provide much in-depth discussion on how different
distortion types influence the performance of pedestrian detection algorithms,
which is important to design effective pedestrian detection algorithms for
surveillance. The DSurVD data set can be download from BaiduYunDisk,
https://pan.baidu.com/s/1I9Kqj8rmubOYu7bkBfkUpA, Password: lqmc

negative:
Pedestrian behavior prediction is one of the major challenges for intelligent
driving systems in urban environments. Pedestrians often exhibit a wide range
of behaviors and adequate interpretations of those depend on various sources of
information such as pedestrian appearance, states of other road users, the
environment layout, etc. To address this problem, we propose a novel
multi-modal prediction algorithm that incorporates different sources of
information captured from the environment to predict future crossing actions of
pedestrians. The proposed model benefits from a hybrid learning architecture
consisting of feedforward and recurrent networks for analyzing visual features
of the environment and dynamics of the scene. Using the existing 2D pedestrian
behavior benchmarks and a newly annotated 3D driving dataset, we show that our
proposed model achieves state-of-the-art performance in pedestrian crossing
prediction.

negative:
Pedestrian detection plays an important role in many applications such as
autonomous driving. We propose a method that explores semantic segmentation
results as self-attention cues to significantly improve the pedestrian
detection performance. Specifically, a multi-task network is designed to
jointly learn semantic segmentation and pedestrian detection from image
datasets with weak box-wise annotations. The semantic segmentation feature maps
are concatenated with corresponding convolution features maps to provide more
discriminative features for pedestrian detection and pedestrian classification.
By jointly learning segmentation and detection, our proposed pedestrian
self-attention mechanism can effectively identify pedestrian regions and
suppress backgrounds. In addition, we propose to incorporate semantic attention
information from multi-scale layers into deep convolution neural network to
boost pedestrian detection. Experiment results show that the proposed method
achieves the best detection performance with MR of 6.27% on Caltech dataset and
obtain competitive performance on CityPersons dataset while maintaining high
computational efficiency.

negative:
Detection of pedestrians in aerial imagery captured by drones has many
applications including intersection monitoring, patrolling, and surveillance,
to name a few. However, the problem is involved due to continuouslychanging
camera viewpoint and object appearance as well as the need for lightweight
algorithms to run on on-board embedded systems. To address this issue, the
paper proposes a framework for pedestrian detection in videos based on the YOLO
object detection network [6] while having a high throughput of more than 5 FPS
on the Jetson TX2 embedded board. The framework exploits deep learning for
robust operation and uses a pre-trained model without the need for any
additional training which makes it flexible to apply on different setups with
minimum amount of tuning. The method achieves ~81 mAP when applied on a sample
video from the Embedded Real-Time Inference (ERTI) Challenge where pedestrians
are monitored by a UAV.

negative:
Pedestrian detection is an important component for safety of autonomous
vehicles, as well as for traffic and street surveillance. There are extensive
benchmarks on this topic and it has been shown to be a challenging problem when
applied on real use-case scenarios. In purely image-based pedestrian detection
approaches, the state-of-the-art results have been achieved with convolutional
neural networks (CNN) and surprisingly few detection frameworks have been built
upon multi-cue approaches. In this work, we develop a new pedestrian detector
for autonomous vehicles that exploits LiDAR data, in addition to visual
information. In the proposed approach, LiDAR data is utilized to generate
region proposals by processing the three dimensional point cloud that it
provides. These candidate regions are then further processed by a
state-of-the-art CNN classifier that we have fine-tuned for pedestrian
detection. We have extensively evaluated the proposed detection process on the
KITTI dataset. The experimental results show that the proposed LiDAR space
clustering approach provides a very efficient way of generating region
proposals leading to higher recall rates and fewer misses for pedestrian
detection. This indicates that LiDAR data can provide auxiliary information for
CNN-based approaches.

negative:
With the unprecedented shift towards automated urban environments in recent
years, a new paradigm is required to study pedestrian behaviour. Studying
pedestrian behaviour in futuristic scenarios requires modern data sources that
consider both the Automated Vehicle (AV) and pedestrian perspectives. Current
open datasets on AVs predominantly fail to account for the latter, as they do
not include an adequate number of events and associated details that involve
pedestrian and vehicle interactions. To address this issue, we propose using
Virtual Reality (VR) data as a complementary resource to current datasets,
which can be designed to measure pedestrian behaviour under specific
conditions. In this research, we focus on the context-aware pedestrian
trajectory prediction framework for automated vehicles at mid-block
unsignalized crossings. For this purpose, we develop a novel multi-input
network of Long Short-Term Memory (LSTM) and fully connected dense layers. In
addition to past trajectories, the proposed framework incorporates pedestrian
head orientations and distance to the upcoming vehicles as sequential input
data. By merging the sequential data with contextual information of the
environment, we train a model to predict the future pedestrian trajectory. Our
results show that the prediction error and overfitting to the training data are
reduced by considering contextual information in the model. To analyze the
application of the methods to real AV data, the proposed framework is trained
and applied to pedestrian trajectory extracted from an open-access video
dataset. Finally, by implementing a game theory-based model interpretability
method, we provide detailed insights and propose recommendations to improve the
current automated vehicle sensing systems from a pedestrian-oriented point of
view.

negative:
Action and intention recognition of pedestrians in urban settings are
challenging problems for Advanced Driver Assistance Systems as well as future
autonomous vehicles to maintain smooth and safe traffic. This work investigates
a number of feature extraction methods in combination with several machine
learning algorithms to build knowledge on how to automatically detect the
action and intention of pedestrians in urban traffic. We focus on the motion
and head orientation to predict whether the pedestrian is about to cross the
street or not. The work is based on the Joint Attention for Autonomous Driving
(JAAD) dataset, which contains 346 videoclips of various traffic scenarios
captured with cameras mounted in the windshield of a car. An accuracy of 72%
for head orientation estimation and 85% for motion detection is obtained in our
experiments.

negative:
One of the major challenges for autonomous vehicles in urban environments is
to understand and predict other road users' actions, in particular, pedestrians
at the point of crossing. The common approach to solving this problem is to use
the motion history of the agents to predict their future trajectories. However,
pedestrians exhibit highly variable actions most of which cannot be understood
without visual observation of the pedestrians themselves and their
surroundings. To this end, we propose a solution for the problem of pedestrian
action anticipation at the point of crossing. Our approach uses a novel stacked
RNN architecture in which information collected from various sources, both
scene dynamics and visual features, is gradually fused into the network at
different levels of processing. We show, via extensive empirical evaluations,
that the proposed algorithm achieves a higher prediction accuracy compared to
alternative recurrent network architectures. We conduct experiments to
investigate the impact of the length of observation, time to event and types of
features on the performance of the proposed method. Finally, we demonstrate how
different data fusion strategies impact prediction accuracy.

negative:
Pedestrians in videos have a wide range of appearances such as body poses,
occlusions, and complex backgrounds, and there exists the proposal shift
problem in pedestrian detection that causes the loss of body parts such as head
and legs. To address it, we propose part-level convolutional neural networks
(CNN) for pedestrian detection using saliency and boundary box alignment in
this paper. The proposed network consists of two sub-networks: detection and
alignment. We use saliency in the detection sub-network to remove false
positives such as lamp posts and trees. We adopt bounding box alignment on
detection proposals in the alignment sub-network to address the proposal shift
problem. First, we combine FCN and CAM to extract deep features for pedestrian
detection. Then, we perform part-level CNN to recall the lost body parts.
Experimental results on various datasets demonstrate that the proposed method
remarkably improves accuracy in pedestrian detection and outperforms existing
state-of-the-arts in terms of log average miss rate at false position per image
(FPPI).

negative:
Predicting the collective motion of a group of pedestrians (a crowd) under
the vehicle influence is essential for the development of autonomous vehicles
to deal with mixed urban scenarios where interpersonal interaction and
vehicle-crowd interaction (VCI) are significant. This usually requires a model
that can describe individual pedestrian motion under the influence of nearby
pedestrians and the vehicle. This study proposed two pedestrian trajectory
datasets, CITR dataset and DUT dataset, so that the pedestrian motion models
can be further calibrated and verified, especially when vehicle influence on
pedestrians plays an important role. CITR dataset consists of experimentally
designed fundamental VCI scenarios (front, back, and lateral VCIs) and provides
unique ID for each pedestrian, which is suitable for exploring a specific
aspect of VCI. DUT dataset gives two ordinary and natural VCI scenarios in
crowded university campus, which can be used for more general purpose VCI
exploration. The trajectories of pedestrians, as well as vehicles, were
extracted by processing video frames that come from a down-facing camera
mounted on a hovering drone as the recording equipment. The final trajectories
of pedestrians and vehicles were refined by Kalman filters with linear
point-mass model and nonlinear bicycle model, respectively, in which
xy-velocity of pedestrians and longitudinal speed and orientation of vehicles
were estimated. The statistics of the velocity magnitude distribution
demonstrated the validity of the proposed dataset. In total, there are
approximate 340 pedestrian trajectories in CITR dataset and 1793 pedestrian
trajectories in DUT dataset. The dataset is available at GitHub.

negative:
Vehicle-pedestrian interaction (VPI) is one of the most challenging tasks for
automated driving systems. The design of driving strategies for such systems
usually starts with verifying VPI in simulation. This work proposed an improved
framework for the study of VPI in uncontrolled pedestrian crossing scenarios.
The framework admits the mutual effect between the pedestrian and the vehicle.
A multi-state social force based pedestrian motion model was designed to
describe the microscopic motion of the pedestrian crossing behavior. The
pedestrian model considers major interaction factors such as the accepted gap
of the pedestrian's decision on when to start crossing, the desired speed of
the pedestrian, and the effect of the vehicle on the pedestrian while the
pedestrian is crossing the road. Vehicle driving strategies focus on the
longitudinal motion control, for which the feedback obstacle avoidance control
and the model predictive control were tested and compared in the framework. The
simulation results verified that the proposed framework can generate a variety
of VPI scenarios, consisting of either the pedestrian yielding to the vehicle
or the vehicle yielding to the pedestrian. The framework can be easily extended
to apply different approaches to the VPI problems.

negative:
We present a novel, realtime algorithm to compute the trajectory of each
pedestrian in moderately dense crowd scenes. Our formulation is based on an
adaptive particle filtering scheme that uses a multi-agent motion model based
on velocity-obstacles, and takes into account local interactions as well as
physical and personal constraints of each pedestrian. Our method dynamically
changes the number of particles allocated to each pedestrian based on different
confidence metrics. Additionally, we use a new high-definition crowd video
dataset, which is used to evaluate the performance of different pedestrian
tracking algorithms. This dataset consists of videos of indoor and outdoor
scenes, recorded at different locations with 30-80 pedestrians. We highlight
the performance benefits of our algorithm over prior techniques using this
dataset. In practice, our algorithm can compute trajectories of tens of
pedestrians on a multi-core desktop CPU at interactive rates (27-30 frames per
second). To the best of our knowledge, our approach is 4-5 times faster than
prior methods, which provide similar accuracy.

negative:
To deal with many pedestrian data, automatic data collection is needed. This
paper describes how to automate the microscopic pedestrian flow data collection
from video files. The study is restricted only to pedestrians without
considering vehicular - pedestrian interaction. Pedestrian tracking system
consists of three sub-systems, which calculates the image processing, object
tracking and traffic flow variables. The system receives input of stacks of
images and parameters. The first sub-system performs Image Processing analysis
while the second sub-system carries out the tracking of pedestrians by matching
the features and tracing the pedestrian numbers frame by frame. The last
sub-system deals with a NTXY database to calculate the pedestrian traffic-flow
characteristic such as flow rate, speed and area module. Comparison with manual
data collection method confirmed that the procedures described have significant
potential to automate the data collection of both microscopic and macroscopic
pedestrian flow variables.

negative:
Various methods to automate traffic data collection have recently been
developed by many researchers. A macroscopic data collection through image
processing has been proposed. For microscopic traffic flow data, such as
individual speed and time or distance headway, tracking of individual movement
is needed. The tracking algorithms for pedestrian or vehicle have been
developed to trace the movement of one or two pedestrians based on sign
pattern, and feature detection. No research has been done to track many
pedestrians or vehicles at once. This paper describes a new and fast algorithm
to track the movement of many individual vehicles or pedestrians

negative:
Pedestrians and vehicles often share the road in complex inner city traffic.
This leads to interactions between the vehicle and pedestrians, with each
affecting the other's motion. In order to create robust methods to reason about
pedestrian behavior and to design interfaces of communication between
self-driving cars and pedestrians we need to better understand such
interactions. In this paper, we present a data-driven approach to implicitly
model pedestrians' interactions with vehicles, to better predict pedestrian
behavior. We propose a LSTM model that takes as input the past trajectories of
the pedestrian and ego-vehicle, and pedestrian head orientation, and predicts
the future positions of the pedestrian. Our experiments based on a real-world,
inner city dataset captured with vehicle mounted cameras, show that the usage
of such cues improve pedestrian prediction when compared to a baseline that
purely uses the past trajectory of the pedestrian.

negative:
Risk is traditionally described as the expected likelihood of an undesirable
outcome, such as collisions for autonomous vehicles. Accurately predicting risk
or potentially risky situations is critical for the safe operation of
autonomous vehicles. In our previous work, we showed that risk could be
characterized by two components: 1) the probability of an undesirable outcome
and 2) an estimate of how undesirable the outcome is (loss). This paper is an
extension to our previous work. In this paper, using our trained deep
reinforcement learning model for navigating around crowds, we developed a
risk-based decision-making framework for the autonomous vehicle that integrates
the high-level risk-based path planning with the reinforcement learning-based
low-level control. We evaluated our method in a high-fidelity simulation such
as CARLA. This work can improve safety by allowing an autonomous vehicle to one
day avoid and react to risky situations.

negative:
Predicting the future trajectories of pedestrians is a challenging problem
that has a range of application, from crowd surveillance to autonomous driving.
In literature, methods to approach pedestrian trajectory prediction have
evolved, transitioning from physics-based models to data-driven models based on
recurrent neural networks. In this work, we propose a new approach to
pedestrian trajectory prediction, with the introduction of a novel 2D
convolutional model. This new model outperforms recurrent models, and it
achieves state-of-the-art results on the ETH and TrajNet datasets. We also
present an effective system to represent pedestrian positions and powerful data
augmentation techniques, such as the addition of Gaussian noise and the use of
random rotations, which can be applied to any model. As an additional
exploratory analysis, we present experimental results on the inclusion of
occupancy methods to model social information, which empirically show that
these methods are ineffective in capturing social interaction.

negative:
Autonomous vehicles (AVs) must share space with pedestrians, both in
carriageway cases such as cars at pedestrian crossings and off-carriageway
cases such as delivery vehicles navigating through crowds on pedestrianized
high-streets. Unlike static obstacles, pedestrians are active agents with
complex, interactive motions. Planning AV actions in the presence of
pedestrians thus requires modelling of their probable future behaviour as well
as detecting and tracking them. This narrative review article is Part II of a
pair, together surveying the current technology stack involved in this process,
organising recent research into a hierarchical taxonomy ranging from low-level
image detection to high-level psychological models, from the perspective of an
AV designer. This self-contained Part II covers the higher levels of this
stack, consisting of models of pedestrian behaviour, from prediction of
individual pedestrians' likely destinations and paths, to game-theoretic models
of interactions between pedestrians and autonomous vehicles. This survey
clearly shows that, although there are good models for optimal walking
behaviour, high-level psychological and social modelling of pedestrian
behaviour still remains an open research question that requires many conceptual
issues to be clarified. Early work has been done on descriptive and qualitative
models of behaviour, but much work is still needed to translate them into
quantitative algorithms for practical AV control.

negative:
We present a Pedestrian Dominance Model (PDM) to identify the dominance
characteristics of pedestrians for robot navigation. Through a perception study
on a simulated dataset of pedestrians, PDM models the perceived dominance
levels of pedestrians with varying motion behaviors corresponding to
trajectory, speed, and personal space. At runtime, we use PDM to identify the
dominance levels of pedestrians to facilitate socially-aware navigation for the
robots. PDM can predict dominance levels from trajectories with ~85% accuracy.
Prior studies in psychology literature indicate that when interacting with
humans, people are more comfortable around people that exhibit complementary
movement behaviors. Our algorithm leverages this by enabling the robots to
exhibit complementing responses to pedestrian dominance. We also present an
application of PDM for generating dominance-based collision-avoidance behaviors
in the navigation of autonomous vehicles among pedestrians. We demonstrate the
benefits of our algorithm for robots navigating among tens of pedestrians in
simulated environments.

negative:
As autonomous vehicles become an every-day reality, high-accuracy pedestrian
detection is of paramount practical importance. Pedestrian detection is a
highly researched topic with mature methods, but most datasets focus on common
scenes of people engaged in typical walking poses on sidewalks. But performance
is most crucial for dangerous scenarios, such as children playing in the street
or people using bicycles/skateboards in unexpected ways. Such "in-the-tail"
data is notoriously hard to observe, making both training and testing
difficult. To analyze this problem, we have collected a novel annotated dataset
of dangerous scenarios called the Precarious Pedestrian dataset. Even given a
dedicated collection effort, it is relatively small by contemporary standards
(around 1000 images). To allow for large-scale data-driven learning, we explore
the use of synthetic data generated by a game engine. A significant challenge
is selected the right "priors" or parameters for synthesis: we would like
realistic data with poses and object configurations that mimic true Precarious
Pedestrians. Inspired by Generative Adversarial Networks (GANs), we generate a
massive amount of synthetic data and train a discriminative classifier to
select a realistic subset, which we deem the Adversarial Imposters. We
demonstrate that this simple pipeline allows one to synthesize realistic
training data by making use of rendering/animation engines within a GAN
framework. Interestingly, we also demonstrate that such data can be used to
rank algorithms, suggesting that Adversarial Imposters can also be used for
"in-the-tail" validation at test-time, a notoriously difficult challenge for
real-world deployment.

negative:
Pedestrian detection in a crowd is a challenging task due to a high number of
mutually-occluding human instances, which brings ambiguity and optimization
difficulties to the current IoU-based ground truth assignment procedure in
classical object detection methods. In this paper, we develop a unique
perspective of pedestrian detection as a variational inference problem. We
formulate a novel and efficient algorithm for pedestrian detection by modeling
the dense proposals as a latent variable while proposing a customized Auto
Encoding Variational Bayes (AEVB) algorithm. Through the optimization of our
proposed algorithm, a classical detector can be fashioned into a variational
pedestrian detector. Experiments conducted on CrowdHuman and CityPersons
datasets show that the proposed algorithm serves as an efficient solution to
handle the dense pedestrian detection problem for the case of single-stage
detectors. Our method can also be flexibly applied to two-stage detectors,
achieving notable performance enhancement.

negative:
We present a novel, real-time algorithm to track the trajectory of each
pedestrian in moderately dense crowded scenes. Our formulation is based on an
adaptive particle-filtering scheme that uses a combination of various
multi-agent heterogeneous pedestrian simulation models. We automatically
compute the optimal parameters for each of these different models based on
prior tracked data and use the best model as motion prior for our
particle-filter based tracking algorithm. We also use our "mixture of motion
models" for adaptive particle selection and accelerate the performance of the
online tracking algorithm. The motion model parameter estimation is formulated
as an optimization problem, and we use an approach that solves this
combinatorial optimization problem in a model independent manner and hence
scalable to any multi-agent pedestrian motion model. We evaluate the
performance of our approach on different crowd video datasets and highlight the
improvement in accuracy over homogeneous motion models and a baseline
mean-shift based tracker. In practice, our formulation can compute trajectories
of tens of pedestrians on a multi-core desktop CPU in in real time and offer
higher accuracy as compared to prior real time pedestrian tracking algorithms.

negative:
To ensure pedestrian friendly streets in the era of automated vehicles,
reassessment of current policies, practices, design, rules and regulations of
urban areas is of importance. This study investigates pedestrian crossing
behaviour, as an important element of urban dynamics that is expected to be
affected by the presence of automated vehicles. For this purpose, an
interpretable machine learning framework is proposed to explore factors
affecting pedestrians' wait time before crossing mid-block crosswalks in the
presence of automated vehicles. To collect rich behavioural data, we developed
a dynamic and immersive virtual reality experiment, with 180 participants from
a heterogeneous population in 4 different locations in the Greater Toronto Area
(GTA). Pedestrian wait time behaviour is then analyzed using a data-driven Cox
Proportional Hazards (CPH) model, in which the linear combination of the
covariates is replaced by a flexible non-linear deep neural network. The
proposed model achieved a 5% improvement in goodness of fit, but more
importantly, enabled us to incorporate a richer set of covariates. A game
theoretic based interpretability method is used to understand the contribution
of different covariates to the time pedestrians wait before crossing. Results
show that the presence of automated vehicles on roads, wider lane widths, high
density on roads, limited sight distance, and lack of walking habits are the
main contributing factors to longer wait times. Our study suggested that, to
move towards pedestrian-friendly urban areas, national level educational
programs for children, enhanced safety measures for seniors, promotion of
active modes of transportation, and revised traffic rules and regulations
should be considered.

negative:
In mixed traffic scenarios, a certain number of pedestrians might coexist in
a small area while interacting with vehicles. In this situation, every
pedestrian must simultaneously react to the surrounding pedestrians and
vehicles. Analytical modeling of such collective pedestrian motion can benefit
intelligent transportation practices like shared space design and urban
autonomous driving. This work proposed the sub-goal social force model (SG-SFM)
to describe the collective pedestrian motion under vehicle influence. The
proposed model introduced a new design of vehicle influence on pedestrian
motion, which was smoothly combined with the influence of surrounding
pedestrians using the sub-goal concept. This model aims to describe generalized
pedestrian motion, i.e., it is applicable to various vehicle-pedestrian
interaction patterns. The generalization was verified by both quantitative and
qualitative evaluation. The quantitative evaluation was conducted to reproduce
pedestrian motion in three different datasets, HBS, CITR, and DUT. It also
compared two different ways of calibrating the model parameters. The
qualitative evaluation examined the simulation of collective pedestrian motion
in a series of fundamental vehicle-pedestrian interaction scenarios. The above
evaluation results demonstrated the effectiveness of the proposed model.

negative:
In order to operate safely on the road, autonomous vehicles need not only to
be able to identify objects in front of them, but also to be able to estimate
the risk level of the object in front of the vehicle automatically. It is
obvious that different objects have different levels of danger to autonomous
vehicles. An evaluation system is needed to automatically determine the danger
level of the object for the autonomous vehicle. It would be too subjective and
incomplete if the system were completely defined by humans. Based on this, we
propose a framework based on nonparametric Bayesian learning method -- a sticky
hierarchical Dirichlet process hidden Markov model(sticky HDP-HMM), and
discover the relationship between driving scenarios and driving styles. We use
the analysis of driving styles of autonomous vehicles to reflect the risk
levels of driving scenarios to the vehicles. In this framework, we firstly use
sticky HDP-HMM to extract driving styles from the dataset and get different
clusters, then an evaluation system is proposed to evaluate and rank the
urgency levels of the clusters. Finally, we map the driving scenarios to the
ranking results and thus get clusters of driving scenarios in different risk
levels. More importantly, we find the relationship between driving scenarios
and driving styles. The experiment shows that our framework can cluster and
rank driving styles of different urgency levels and find the relationship
between driving scenarios and driving styles and the conclusions also fit
people's common sense when driving. Furthermore, this framework can be used for
autonomous vehicles to estimate risk levels of driving scenarios and help them
make precise and safe decisions.

negative:
Prediction of pedestrian behavior is critical for fully autonomous vehicles
to drive in busy city streets safely and efficiently. The future autonomous
cars need to fit into mixed conditions with not only technical but also social
capabilities. As more algorithms and datasets have been developed to predict
pedestrian behaviors, these efforts lack the benchmark labels and the
capability to estimate the temporal-dynamic intent changes of the pedestrians,
provide explanations of the interaction scenes, and support algorithms with
social intelligence. This paper proposes and shares another benchmark dataset
called the IUPUI-CSRC Pedestrian Situated Intent (PSI) data with two innovative
labels besides comprehensive computer vision labels. The first novel label is
the dynamic intent changes for the pedestrians to cross in front of the
ego-vehicle, achieved from 24 drivers with diverse backgrounds. The second one
is the text-based explanations of the driver reasoning process when estimating
pedestrian intents and predicting their behaviors during the interaction
period. These innovative labels can enable several computer vision tasks,
including pedestrian intent/behavior prediction, vehicle-pedestrian interaction
segmentation, and video-to-language mapping for explainable algorithms. The
released dataset can fundamentally improve the development of pedestrian
behavior prediction models and develop socially intelligent autonomous cars to
interact with pedestrians efficiently. The dataset has been evaluated with
different tasks and is released to the public to access.

negative:
Pedestrian detection methods have been significantly improved with the
development of deep convolutional neural networks. Nevertheless, detecting
small-scaled pedestrians and occluded pedestrians remains a challenging
problem. In this paper, we propose a pedestrian detection method with a
couple-network to simultaneously address these two issues. One of the
sub-networks, the gated multi-layer feature extraction sub-network, aims to
adaptively generate discriminative features for pedestrian candidates in order
to robustly detect pedestrians with large variations on scales. The second
sub-network targets in handling the occlusion problem of pedestrian detection
by using deformable regional RoI-pooling. We investigate two different gate
units for the gated sub-network, namely, the channel-wise gate unit and the
spatio-wise gate unit, which can enhance the representation ability of the
regional convolutional features among the channel dimensions or across the
spatial domain, repetitively. Ablation studies have validated the effectiveness
of both the proposed gated multi-layer feature extraction sub-network and the
deformable occlusion handling sub-network. With the coupled framework, our
proposed pedestrian detector achieves state-of-the-art results on the Caltech
and the CityPersons pedestrian detection benchmarks.

negative:
In this paper, we present an efficient pedestrian detection system, designed
by fusion of multiple deep neural network (DNN) systems. Pedestrian candidates
are first generated by a single shot convolutional multi-box detector at
different locations with various scales and aspect ratios. The candidate
generator is designed to provide the majority of ground truth pedestrian
annotations at the cost of a large number of false positives. Then, a
classification system using the idea of ensemble learning is deployed to
improve the detection accuracy. The classification system further classifies
the generated candidates based on opinions of multiple deep verification
networks and a fusion network which utilizes a novel soft-rejection fusion
method to adjust the confidence in the detection results. To improve the
training of the deep verification networks, a novel soft-label method is
devised to assign floating point labels to the generated pedestrian candidates.
A deep context aggregation semantic segmentation network also provides
pixel-level classification of the scene and its results are softly fused with
the detection results by the single shot detector. Our pedestrian detector
compared favorably to state-of-art methods on all popular pedestrian detection
datasets. For example, our fused DNN has better detection accuracy on the
Caltech Pedestrian dataset than all previous state of art methods, while also
being the fastest. We significantly improved the log-average miss rate on the
Caltech pedestrian dataset to 7.67% and achieved the new state-of-the-art.

negative:
Autonomous vehicles (AVs) must share space with pedestrians, both in
carriageway cases such as cars at pedestrian crossings and off-carriageway
cases such as delivery vehicles navigating through crowds on pedestrianized
high-streets. Unlike static obstacles, pedestrians are active agents with
complex, interactive motions. Planning AV actions in the presence of
pedestrians thus requires modelling of their probable future behaviour as well
as detecting and tracking them. This narrative review article is Part I of a
pair, together surveying the current technology stack involved in this process,
organising recent research into a hierarchical taxonomy ranging from low-level
image detection to high-level psychology models, from the perspective of an AV
designer. This self-contained Part I covers the lower levels of this stack,
from sensing, through detection and recognition, up to tracking of pedestrians.
Technologies at these levels are found to be mature and available as
foundations for use in high-level systems, such as behaviour modelling,
prediction and interaction control.

negative:
The main essence of this paper is to investigate the performance of RetinaNet
based object detectors on pedestrian detection. Pedestrian detection is an
important research topic as it provides a baseline for general object detection
and has a great number of practical applications like autonomous car, robotics
and Security camera. Though extensive research has made huge progress in
pedestrian detection, there are still many issues and open for more research
and improvement. Recent deep learning based methods have shown state-of-the-art
performance in computer vision tasks such as image classification, object
detection, and segmentation. Wider pedestrian detection challenge aims at
finding improve solutions for pedestrian detection problem. In this paper, We
propose a pedestrian detection system based on RetinaNet. Our solution has
scored 0.4061 mAP. The code is available at
https://github.com/miltonbd/ECCV_2018_pedestrian_detection_challenege.

negative:
Anticipating the intentions of vulnerable road users (VRUs) such as
pedestrians and cyclists is critical for performing safe and comfortable
driving maneuvers. This is the case for human driving and, thus, should be
taken into account by systems providing any level of driving assistance, from
advanced driver assistant systems (ADAS) to fully autonomous vehicles (AVs). In
this paper, we show how the latest advances on monocular vision-based human
pose estimation, i.e. those relying on deep Convolutional Neural Networks
(CNNs), enable to recognize the intentions of such VRUs. In the case of
cyclists, we assume that they follow traffic rules to indicate future maneuvers
with arm signals. In the case of pedestrians, no indications can be assumed.
Instead, we hypothesize that the walking pattern of a pedestrian allows to
determine if he/she has the intention of crossing the road in the path of the
ego-vehicle, so that the ego-vehicle must maneuver accordingly (e.g. slowing
down or stopping). In this paper, we show how the same methodology can be used
for recognizing pedestrians and cyclists' intentions. For pedestrians, we
perform experiments on the JAAD dataset. For cyclists, we did not found an
analogous dataset, thus, we created our own one by acquiring and annotating
videos which we share with the research community. Overall, the proposed
pipeline provides new state-of-the-art results on the intention recognition of
VRUs.

negative:
As autonomous vehicles (AVs) inch closer to reality, a central requirement
for acceptance will be earning the trust of humans in everyday driving
situations. In particular, the interaction between AVs and pedestrians is of
high importance, as every human is a pedestrian at some point of the day. This
paper considers the interaction of a pedestrian and an autonomous vehicle at a
mid-block, unsignalized intersection where there is ambiguity over when the
pedestrian should cross and when and how the vehicle should yield. By modeling
pedestrian behavior through the concept of gap acceptance, the authors show
that a hybrid controller with just four distinct modes allows an autonomous
vehicle to successfully interact with a pedestrian across a continuous spectrum
of possible crosswalk entry behaviors. The controller is validated through
extensive simulation and compared to an alternate POMDP solution and
experimental results are provided on a research vehicle for a virtual
pedestrian.

negative:
Pedestrian trajectory prediction is essential for collision avoidance in
autonomous driving and robot navigation. However, predicting a pedestrian's
trajectory in crowded environments is non-trivial as it is influenced by other
pedestrians' motion and static structures that are present in the scene. Such
human-human and human-space interactions lead to non-linearities in the
trajectories. In this paper, we present a new spatio-temporal graph based Long
Short-Term Memory (LSTM) network for predicting pedestrian trajectory in
crowded environments, which takes into account the interaction with static
(physical objects) and dynamic (other pedestrians) elements in the scene. Our
results are based on two widely-used datasets to demonstrate that the proposed
method outperforms the state-of-the-art approaches in human trajectory
prediction. In particular, our method leads to a reduction in Average
Displacement Error (ADE) and Final Displacement Error (FDE) of up to 55% and
61% respectively over state-of-the-art approaches.

negative:
Social acceptance is a major hurdle for autonomous vehicle technology,
central to which is ensuring both passengers and nearby pedestrians feel safe.
This idea of `feeling safe' and perceived safety is highly subjective and
rooted in human intuition. As such, traditional analytical approaches to
autonomous navigation often fail to cater for the social expectations of
individuals. Therefore, this paper proposes an approach to capture the
complexity of social expectations and integrate this complexity into a
3-layered Contextual Speed Controller. The layers were; the legal road speed
limit, the socially acceptable speed given the number of nearby pedestrians,
and the socially acceptable speed based on proximity to nearby pedestrians. An
implementation of this layered approach was tested in areas of both low and
high vehicle-pedestrian interactions. From the experiments conducted, the lower
two layers were seen working in tandem to modulate the vehicle speed to
appropriate levels that mimicked conservative human driver behaviour. In
summary, this work quantified the relationship between pedestrian context and
socially acceptable vehicle speeds, allowing for more perceivably safe
autonomous driving. Furthermore, the need for different driving schemes for
navigating different road environments was identified.

negative:
Pedestrian trajectory prediction is a prominent research track that has
advanced towards modelling of crowd social and contextual interactions, with
extensive usage of Long Short-Term Memory (LSTM) for temporal representation of
walking trajectories.
  Existing approaches use virtual neighborhoods as a fixed grid for pooling
social states of pedestrians with tuning process that controls how social
interactions are being captured. This entails performance customization to
specific scenes but lowers the generalization capability of the approaches. In
our work, we deploy \textit{Grid-LSTM}, a recent extension of LSTM, which
operates over multidimensional feature inputs. We present a new perspective to
interaction modeling by proposing that pedestrian neighborhoods can become
adaptive in design. We use \textit{Grid-LSTM} as an encoder to learn about
potential future neighborhoods and their influence on pedestrian motion given
the visual and the spatial boundaries. Our model outperforms state-of-the-art
approaches that collate resembling features over several publicly-tested
surveillance videos. The experiment results clearly illustrate the
generalization of our approach across datasets that varies in scene features
and crowd dynamics.

negative:
Thermal images are mainly used to detect the presence of people at night or
in bad lighting conditions, but perform poorly at daytime. To solve this
problem, most state-of-the-art techniques employ a fusion network that uses
features from paired thermal and color images. Instead, we propose to augment
thermal images with their saliency maps, to serve as an attention mechanism for
the pedestrian detector especially during daytime. We investigate how such an
approach results in improved performance for pedestrian detection using only
thermal images, eliminating the need for paired color images. For our
experiments, we train the Faster R-CNN for pedestrian detection and report the
added effect of saliency maps generated using static and deep methods (PiCA-Net
and R3-Net). Our best performing model results in an absolute reduction of miss
rate by 13.4% and 19.4% over the baseline in day and night images respectively.
We also annotate and release pixel level masks of pedestrians on a subset of
the KAIST Multispectral Pedestrian Detection dataset, which is a first publicly
available dataset for salient pedestrian detection.

negative:
Currently, studying the vehicle-human interactive behavior in the emergency
needs a large amount of datasets in the actual emergent situations that are
almost unavailable. Existing public data sources on autonomous vehicles (AVs)
mainly focus either on the normal driving scenarios or on emergency situations
without human involvement. To fill this gap and facilitate related research,
this paper provides a new yet convenient way to extract the interactive
behavior data (i.e., the trajectories of vehicles and humans) from actual
accident videos that were captured by both the surveillance cameras and driving
recorders. The main challenge for data extraction from real-time accident video
lies in the fact that the recording cameras are un-calibrated and the angles of
surveillance are unknown. The approach proposed in this paper employs image
processing to obtain a new perspective which is different from the original
video's perspective. Meanwhile, we manually detect and mark object feature
points in each image frame. In order to acquire a gradient of reference ratios,
a geometric model is implemented in the analysis of reference pixel value, and
the feature points are then scaled to the object trajectory based on the
gradient of ratios. The generated trajectories not only restore the object
movements completely but also reflect changes in vehicle velocity and rotation
based on the feature points distributions.

negative:
In order to be globally deployed, autonomous cars must guarantee the safety
of pedestrians. This is the reason why forecasting pedestrians' intentions
sufficiently in advance is one of the most critical and challenging tasks for
autonomous vehicles. This work tries to solve this problem by jointly
predicting the intention and visual states of pedestrians. In terms of visual
states, whereas previous work focused on x-y coordinates, we will also predict
the size and indeed the whole bounding box of the pedestrian. The method is a
recurrent neural network in a multi-task learning approach. It has one head
that predicts the intention of the pedestrian for each one of its future
position and another one predicting the visual states of the pedestrian.
Experiments on the JAAD dataset show the superiority of the performance of our
method compared to previous works for intention prediction. Also, although its
simple architecture (more than 2 times faster), the performance of the bounding
box prediction is comparable to the ones yielded by much more complex
architectures. Our code is available online.

negative:
Autonomous landing of Unmanned Aerial Vehicles (UAVs) in crowded scenarios is
crucial for successful deployment of UAVs in populated areas, particularly in
emergency landing situations where the highest priority is to avoid hurting
people. In this work, a new visual-based algorithm for identifying Safe Landing
Zones (SLZ) in crowded scenarios is proposed, considering a camera mounted on
an UAV, where the people in the scene move with unknown dynamics. To do so, a
density map is generated for each image frame using a Deep Neural Network, from
where a binary occupancy map is obtained aiming to overestimate the people's
location for security reasons. Then, the occupancy map is projected to the
head's plane, and the SLZ candidates are obtained as circular regions in the
head's plane with a minimum security radius. Finally, to keep track of the SLZ
candidates, a multiple instance tracking algorithm is implemented using Kalman
Filters along with the Hungarian algorithm for data association. Several
scenarios were studied to prove the validity of the proposed strategy,
including public datasets and real uncontrolled scenarios with people moving in
public squares, taken from an UAV in flight. The study showed promising results
in the search of preventing the UAV from hurting people during emergency
landing.

negative:
Pedestrian detection has achieved significant progress with the availability
of existing benchmark datasets. However, there is a gap in the diversity and
density between real world requirements and current pedestrian detection
benchmarks: 1) most of existing datasets are taken from a vehicle driving
through the regular traffic scenario, usually leading to insufficient
diversity; 2) crowd scenarios with highly occluded pedestrians are still under
represented, resulting in low density. To narrow this gap and facilitate future
pedestrian detection research, we introduce a large and diverse dataset named
WiderPerson for dense pedestrian detection in the wild. This dataset involves
five types of annotations in a wide range of scenarios, no longer limited to
the traffic scenario. There are a total of $13,382$ images with $399,786$
annotations, i.e., $29.87$ annotations per image, which means this dataset
contains dense pedestrians with various kinds of occlusions. Hence, pedestrians
in the proposed dataset are extremely challenging due to large variations in
the scenario and occlusion, which is suitable to evaluate pedestrian detectors
in the wild. We introduce an improved Faster R-CNN and the vanilla RetinaNet to
serve as baselines for the new pedestrian detection benchmark. Several
experiments are conducted on previous datasets including Caltech-USA and
CityPersons to analyze the generalization capabilities of the proposed dataset
and we achieve state-of-the-art performances on these previous datasets without
bells and whistles. Finally, we analyze common failure cases and find the
classification ability of pedestrian detector needs to be improved to reduce
false alarm and miss detection rates. The proposed dataset is available at
http://www.cbsr.ia.ac.cn/users/sfzhang/WiderPerson

negative:
Pedestrian detection is used in many vision based applications ranging from
video surveillance to autonomous driving. Despite achieving high performance,
it is still largely unknown how well existing detectors generalize to unseen
data. This is important because a practical detector should be ready to use in
various scenarios in applications. To this end, we conduct a comprehensive
study in this paper, using a general principle of direct cross-dataset
evaluation. Through this study, we find that existing state-of-the-art
pedestrian detectors, though perform quite well when trained and tested on the
same dataset, generalize poorly in cross dataset evaluation. We demonstrate
that there are two reasons for this trend. Firstly, their designs (e.g. anchor
settings) may be biased towards popular benchmarks in the traditional
single-dataset training and test pipeline, but as a result largely limit their
generalization capability. Secondly, the training source is generally not dense
in pedestrians and diverse in scenarios. Under direct cross-dataset evaluation,
surprisingly, we find that a general purpose object detector, without
pedestrian-tailored adaptation in design, generalizes much better compared to
existing state-of-the-art pedestrian detectors. Furthermore, we illustrate that
diverse and dense datasets, collected by crawling the web, serve to be an
efficient source of pre-training for pedestrian detection. Accordingly, we
propose a progressive training pipeline and find that it works well for
autonomous-driving oriented pedestrian detection. Consequently, the study
conducted in this paper suggests that more emphasis should be put on
cross-dataset evaluation for the future design of generalizable pedestrian
detectors. Code and models can be accessed at
https://github.com/hasanirtiza/Pedestron.

negative:
In this work, we consider the problem of pedestrian detection in natural
scenes. Intuitively, instances of pedestrians with different spatial scales may
exhibit dramatically different features. Thus, large variance in instance
scales, which results in undesirable large intra-category variance in features,
may severely hurt the performance of modern object instance detection methods.
We argue that this issue can be substantially alleviated by the
divide-and-conquer philosophy. Taking pedestrian detection as an example, we
illustrate how we can leverage this philosophy to develop a Scale-Aware Fast
R-CNN (SAF R-CNN) framework. The model introduces multiple built-in
sub-networks which detect pedestrians with scales from disjoint ranges. Outputs
from all the sub-networks are then adaptively combined to generate the final
detection results that are shown to be robust to large variance in instance
scales, via a gate function defined over the sizes of object proposals.
Extensive evaluations on several challenging pedestrian detection datasets well
demonstrate the effectiveness of the proposed SAF R-CNN. Particularly, our
method achieves state-of-the-art performance on Caltech, INRIA, and ETH, and
obtains competitive results on KITTI.

negative:
Smart video sensors for applications related to surveillance and security are
IOT-based as they use Internet for various purposes. Such applications include
crowd behaviour monitoring and advanced decision support systems operating and
transmitting information over internet. The analysis of crowd and pedestrian
behaviour is an important task for smart IoT cameras and in particular video
processing. In order to provide related behavioural models, simulation and
tracking approaches have been considered in the literature. In both cases
ground truth is essential to train deep models and provide a meaningful
quantitative evaluation. We propose a framework for crowd simulation and
automatic data generation and annotation that supports multiple cameras and
multiple targets. The proposed approach is based on synthetically generated
human agents, augmented frames and compositing techniques combined with path
finding and planning methods. A number of popular crowd and pedestrian data
sets were used to validate the model, and scenarios related to annotation and
simulation were considered.

negative:
Pedestrian detection in crowded scenes is a challenging problem, because
occlusion happens frequently among different pedestrians. In this paper, we
propose an effective and efficient detection network to hunt pedestrians in
crowd scenes. The proposed method, namely PedHunter, introduces strong
occlusion handling ability to existing region-based detection networks without
bringing extra computations in the inference stage. Specifically, we design a
mask-guided module to leverage the head information to enhance the feature
representation learning of the backbone network. Moreover, we develop a strict
classification criterion by improving the quality of positive samples during
training to eliminate common false positives of pedestrian detection in crowded
scenes. Besides, we present an occlusion-simulated data augmentation to enrich
the pattern and quantity of occlusion samples to improve the occlusion
robustness. As a consequent, we achieve state-of-the-art results on three
pedestrian detection datasets including CityPersons, Caltech-USA and
CrowdHuman. To facilitate further studies on the occluded pedestrian detection
in surveillance scenes, we release a new pedestrian dataset, called SUR-PED,
with a total of over 162k high-quality manually labeled instances in 10k
images. The proposed dataset, source codes and trained models will be released.

negative:
Pedestrian trajectory prediction for surveillance video is one of the
important research topics in the field of computer vision and a key technology
of intelligent surveillance systems. Social relationship among pedestrians is a
key factor influencing pedestrian walking patterns but was mostly ignored in
the literature. Pedestrians with different social relationships play different
roles in the motion decision of target pedestrian. Motivated by this idea, we
propose a Social Relationship Attention LSTM (SRA-LSTM) model to predict future
trajectories. We design a social relationship encoder to obtain the
representation of their social relationship through the relative position
between each pair of pedestrians. Afterwards, the social relationship feature
and latent movements are adopted to acquire the social relationship attention
of this pair of pedestrians. Social interaction modeling is achieved by
utilizing social relationship attention to aggregate movement information from
neighbor pedestrians. Experimental results on two public walking pedestrian
video datasets (ETH and UCY), our model achieves superior performance compared
with state-of-the-art methods. Contrast experiments with other attention
methods also demonstrate the effectiveness of social relationship attention.

negative:
Monitoring vehicle flows in cities is crucial to improve the urban
environment and quality of life of citizens. Images are the best sensing
modality to perceive and assess the flow of vehicles in large areas. Current
technologies for vehicle counting in images hinge on large quantities of
annotated data, preventing their scalability to city-scale as new cameras are
added to the system. This is a recurrent problem when dealing with physical
systems and a key research area in Machine Learning and AI. We propose and
discuss a new methodology to design image-based vehicle density estimators with
few labeled data via multiple camera domain adaptations.

negative:
Autonomous valet parking is a specific application for autonomous vehicles.
In this task, vehicles need to navigate in narrow, crowded and GPS-denied
parking lots. Accurate localization ability is of great importance. Traditional
visual-based methods suffer from tracking lost due to texture-less regions,
repeated structures, and appearance changes. In this paper, we exploit robust
semantic features to build the map and localize vehicles in parking lots.
Semantic features contain guide signs, parking lines, speed bumps, etc, which
typically appear in parking lots. Compared with traditional features, these
semantic features are long-term stable and robust to the perspective and
illumination change. We adopt four surround-view cameras to increase the
perception range. Assisting by an IMU (Inertial Measurement Unit) and wheel
encoders, the proposed system generates a global visual semantic map. This map
is further used to localize vehicles at the centimeter level. We analyze the
accuracy and recall of our system and compare it against other methods in real
experiments. Furthermore, we demonstrate the practicability of the proposed
system by the autonomous parking application.

negative:
Deep learning-based methods for video pedestrian detection and tracking
require large volumes of training data to achieve good performance. However,
data acquisition in crowded public environments raises data privacy concerns --
we are not allowed to simply record and store data without the explicit consent
of all participants. Furthermore, the annotation of such data for computer
vision applications usually requires a substantial amount of manual effort,
especially in the video domain. Labeling instances of pedestrians in highly
crowded scenarios can be challenging even for human annotators and may
introduce errors in the training data. In this paper, we study how we can
advance different aspects of multi-person tracking using solely synthetic data.
To this end, we generate MOTSynth, a large, highly diverse synthetic dataset
for object detection and tracking using a rendering game engine. Our
experiments show that MOTSynth can be used as a replacement for real data on
tasks such as pedestrian detection, re-identification, segmentation, and
tracking.

negative:
We developed a machine vision system to automatically capture the dynamics of
pedestrians under four different traffic scenarios. By considering the overhead
view of each pedestrian as a digital object, the system processes the image
sequences to track the pedestrians. Considering the perspective effect of the
camera lens and the projected area of the hallway at the top-view scene, the
distance of each tracked object from its original position to its current
position is approximated every video frame. Using the approximated distance and
the video frame rate (30 frames per second), the respective velocity and
acceleration of each tracked object are later derived. The quantified motion
characteristics of the pedestrians are displayed by the system through
2-dimensional graphs of the kinematics of motion. The system also outputs video
images of the pedestrians with superimposed markers for tracking. These visual
markers were used to visually describe and quantify the behavior of the
pedestrians under different traffic scenarios.

negative:
In applications such as autonomous driving, it is important to understand,
infer, and anticipate the intention and future behavior of pedestrians. This
ability allows vehicles to avoid collisions and improve ride safety and
quality. This paper proposes a biomechanically inspired recurrent neural
network (Bio-LSTM) that can predict the location and 3D articulated body pose
of pedestrians in a global coordinate frame, given 3D poses and locations
estimated in prior frames with inaccuracy. The proposed network is able to
predict poses and global locations for multiple pedestrians simultaneously, for
pedestrians up to 45 meters from the cameras (urban intersection scale). The
outputs of the proposed network are full-body 3D meshes represented in Skinned
Multi-Person Linear (SMPL) model parameters. The proposed approach relies on a
novel objective function that incorporates the periodicity of human walking
(gait), the mirror symmetry of the human body, and the change of ground
reaction forces in a human gait cycle. This paper presents prediction results
on the PedX dataset, a large-scale, in-the-wild data set collected at real
urban intersections with heavy pedestrian traffic. Results show that the
proposed network can successfully learn the characteristics of pedestrian gait
and produce accurate and consistent 3D pose predictions.

negative:
Accurate pedestrian orientation estimation of autonomous driving helps the
ego vehicle obtain the intentions of pedestrians in the related environment,
which are the base of safety measures such as collision avoidance and
prewarning. However, because of relatively small sizes and high-level
deformation of pedestrians, common pedestrian orientation estimation models
fail to extract sufficient and comprehensive information from them, thus having
their performance restricted, especially monocular ones which fail to obtain
depth information of objects and related environment. In this paper, a novel
monocular pedestrian orientation estimation model, called FFNet, is proposed.
Apart from camera captures, the model adds the 2D and 3D dimensions of
pedestrians as two other inputs according to the logic relationship between
orientation and them. The 2D and 3D dimensions of pedestrians are determined
from the camera captures and further utilized through two feedforward links
connected to the orientation estimator. The feedforward links strengthen the
logicality and interpretability of the network structure of the proposed model.
Experiments show that the proposed model has at least 1.72% AOS increase than
most state-of-the-art models after identical training processes. The model also
has competitive results in orientation estimation evaluation on KITTI dataset.

negative:
In urban or crowded environments, humans rely on eye contact for fast and
efficient communication with nearby people. Autonomous agents also need to
detect eye contact to interact with pedestrians and safely navigate around
them. In this paper, we focus on eye contact detection in the wild, i.e.,
real-world scenarios for autonomous vehicles with no control over the
environment or the distance of pedestrians. We introduce a model that leverages
semantic keypoints to detect eye contact and show that this high-level
representation (i) achieves state-of-the-art results on the publicly-available
dataset JAAD, and (ii) conveys better generalization properties than leveraging
raw images in an end-to-end network. To study domain adaptation, we create
LOOK: a large-scale dataset for eye contact detection in the wild, which
focuses on diverse and unconstrained scenarios for real-world generalization.
The source code and the LOOK dataset are publicly shared towards an open
science mission.

negative:
Interactions between vehicles and pedestrians have always been a major
problem in traffic safety. Experienced human drivers are able to analyze the
environment and choose driving strategies that will help them avoid crashes.
What is not yet clear, however, is how automated vehicles will interact with
pedestrians. This paper proposes a new method for evaluating the safety and
feasibility of the driving strategy of automated vehicles when encountering
unsignalized crossings. MobilEye sensors installed on buses in Ann Arbor,
Michigan, collected data on 2,973 valid crossing events. A stochastic
interaction model was then created using a multivariate Gaussian mixture model.
This model allowed us to simulate the movements of pedestrians reacting to an
oncoming vehicle when approaching unsignalized crossings, and to evaluate the
passing strategies of automated vehicles. A simulation was then conducted to
demonstrate the evaluation procedure.

negative:
Studies predict that demand for autonomous vehicles will increase tenfold
between 2019 and 2026. However, recent high-profile accidents have
significantly impacted consumer confidence in this technology. The cause for
many of these accidents can be traced back to the inability of these vehicles
to correctly sense the impending danger. In response, manufacturers have been
improving the already extensive on-vehicle sensor packages to ensure that the
system always has access to the data necessary to ensure safe navigation.
However, these sensor packages only provide a view from the vehicle's
perspective and, as a result, autonomous vehicles still require frequent human
intervention to ensure safety.
  To address this issue, I developed a system, called Horus, that combines
on-vehicle and infrastructure-based sensors to provide a more complete view of
the environment, including areas not visible from the vehicle. I built a
small-scale experimental testbed as a proof of concept. My measurements of the
impact of sensor failures showed that even short outages (1 second) at slow
speeds (25 km/hr scaled velocity) prevents vehicles that rely on on-vehicle
sensors from navigating properly. My experiments also showed that Horus
dramatically improves driving safety and that the sensor fusion algorithm
selected plays a significant role in the quality of the navigation. With just a
pair of infrastructure sensors, Horus could tolerate sensors that fail 40% of
the time and still navigate safely. These results are a promising first step
towards safer autonomous vehicles.

negative:
Automated vehicles require a comprehensive understanding of traffic
situations to ensure safe and anticipatory driving. In this context, the
prediction of pedestrians is particularly challenging as pedestrian behavior
can be influenced by multiple factors. In this paper, we thoroughly analyze the
requirements on pedestrian behavior prediction for automated driving via a
system-level approach. To this end we investigate real-world pedestrian-vehicle
interactions with human drivers. Based on human driving behavior we then derive
appropriate reaction patterns of an automated vehicle and determine
requirements for the prediction of pedestrians. This includes a novel metric
tailored to measure prediction performance from a system-level perspective. The
proposed metric is evaluated on a large-scale dataset comprising thousands of
real-world pedestrian-vehicle interactions. We furthermore conduct an ablation
study to evaluate the importance of different contextual cues and compare these
results to ones obtained using established performance metrics for pedestrian
prediction. Our results highlight the importance of a system-level approach to
pedestrian behavior prediction.

negative:
State-of-the-art pedestrian detection models have achieved great success in
many benchmarks. However, these models require lots of annotation information
and the labeling process usually takes much time and efforts. In this paper, we
propose a method to generate labeled pedestrian data and adapt them to support
the training of pedestrian detectors. The proposed framework is built on the
Generative Adversarial Network (GAN) with multiple discriminators, trying to
synthesize realistic pedestrians and learn the background context
simultaneously. To handle the pedestrians of different sizes, we adopt the
Spatial Pyramid Pooling (SPP) layer in the discriminator. We conduct
experiments on two benchmarks. The results show that our framework can smoothly
synthesize pedestrians on background images of variations and different levels
of details. To quantitatively evaluate our approach, we add the generated
samples into training data of the baseline pedestrian detectors and show the
synthetic images are able to improve the detectors' performance.

negative:
One of the major challenges that autonomous cars are facing today is driving
in urban environments. To make it a reality, autonomous vehicles require the
ability to communicate with other road users and understand their intentions.
Such interactions are essential between the vehicles and pedestrians as the
most vulnerable road users. Understanding pedestrian behavior, however, is not
intuitive and depends on various factors such as demographics of the
pedestrians, traffic dynamics, environmental conditions, etc. In this paper, we
identify these factors by surveying pedestrian behavior studies, both the
classical works on pedestrian-driver interaction and the modern ones that
involve autonomous vehicles. To this end, we will discuss various methods of
studying pedestrian behavior, and analyze how the factors identified in the
literature are interrelated. We will also review the practical applications
aimed at solving the interaction problem including design approaches for
autonomous vehicles that communicate with pedestrians and visual perception and
reasoning algorithms tailored to understanding pedestrian intention. Based on
our findings, we will discuss the open problems and propose future research
directions.

negative:
While human and group analysis have become an important area in last decades,
some current and relevant applications involve to estimate future motion of
pedestrians in real video sequences. This paper presents a method to provide
motion estimation of real pedestrians in next seconds, using crowd simulation.
Our method is based on Physics and heuristics and use BioCrowds as crowd
simulation methodology to estimate future positions of people in video
sequences. Results show that our method for estimation works well even for
complex videos where events can happen. The maximum achieved average error is
$2.72$cm when estimating the future motion of 32 pedestrians with more than 2
seconds in advance. This paper discusses this and other results.

negative:
Understanding how people view and interact with autonomous vehicles is
important to guide future directions of research. One such way of aiding
understanding is through simulations of virtual environments involving people
and autonomous vehicles. We present a simulation model that incorporates people
and autonomous vehicles in a shared urban space. The model is able to simulate
many thousands of people and vehicles in real-time. This is achieved by use of
GPU hardware, and through a novel linear program solver optimized for large
numbers of problems on the GPU. The model is up to 30 times faster than the
equivalent multi-core CPU model.

negative:
This manuscript presents an efficient approach to map pedestrian surveillance
footage to an aerial view for global assessment of features. The analysis of
the footages relies on low level computer vision and enable real-time
surveillance. While we neglect object tracking, we introduce cumulative grids
on top view scene flow visualization to highlight situations of interest in the
footage. Our approach is tested on multiview footage both from RGB cameras and,
for the first time in the field, on RGB-D-sensors.

negative:
Autonomous surveillance unmanned aerial vehicles (UAVs) are deployed to
observe the streets of the city for any suspicious activities. This paper
utilizes surveillance UAVs for the purpose of detecting the presence of a fire
in the streets. An extensive database is collected from UAV surveillance
drones. With the aid of artificial intelligence (AI), fire stations can swiftly
identify the presence of a fire emerging in the neighborhood. Spatio-temporal
split learning is applied to this scenario to preserve privacy and globally
train a fire classification model. Fires are hazardous natural disasters that
can spread very quickly. Swift identification of fire is required to deploy
firefighters to the scene. In order to do this, strong communication between
the UAV and the central server where the deep learning process occurs is
required. Improving communication resilience is integral to enhancing a safe
experience on the roads. Therefore, this paper explores the adequate number of
clients and data ratios for split learning in this UAV setting, as well as the
required network infrastructure.

negative:
If autonomous vehicles are to be widely accepted, we need to ensure their
safe operation. For this reason, verification and validation (V&V) approaches
must be developed that are suitable for this domain. Model checking is a formal
technique which allows us to exhaustively explore the paths of an abstract
model of a system. Using a probabilistic model checker such as PRISM, we may
determine properties such as the expected time for a mission, or the
probability that a specific mission failure occurs. However, model checking of
complex systems is difficult due to the loss of information during abstraction.
This is especially so when considering systems such as autonomous vehicles
which are subject to external influences. An alternative solution is the use of
Monte Carlo simulation to explore the results of a continuous-time model of the
system. The main disadvantage of this approach is that the approach is not
exhaustive as not all executions of the system are analysed. We are therefore
interested in developing a framework for formal verification of autonomous
vehicles, using Monte Carlo simulation to inform and validate our symbolic
models during the initial stages of development. In this paper, we present a
continuous-time model of a quadrotor unmanned aircraft undertaking an
autonomous mission. We employ this model in Monte Carlo simulation to obtain
specific mission properties which will inform the symbolic models employed in
formal verification.

negative:
Building trajectories for biped robot walking is a complex task considering
all degrees of freedom (DOFs) commonly bound within the mechanical structure. A
typical problem for such robots is the instability produced by violent
transitions between walking phases in particular when a swinging leg impacts
the surface. Although extensive research on novel efficient walking algorithms
has been conducted, falls commonly appear as the walking speed increases or as
the terrain condition changes. This paper presents a polynomial trajectory
generation algorithm (PTA) to implement the walking on biped robots following
the cubic Hermitian polynomial interpolation between initial and final
conditions. The proposed algorithm allows smooth transitions between walking
phases, significantly reducing the possibility of falling. The algorithm has
been successfully tested by generating walking trajectories under different
terrain conditions on a biped robot of 10 DOFs. PTA has shown to be simple and
suitable to generate real time walking trajectories, despite reduced computing
resources of a commercial embedded microcontroller. Experimental evidence and
comparisons to other state-of-the-art methods demonstrates a better performance
of the proposed method in generating walking trajectories under different
ground conditions.

negative:
Despite growing attention in autonomy, there are still many open problems,
including how autonomous vehicles will interact and communicate with other
agents, such as human drivers and pedestrians. Unlike most approaches that
focus on pedestrian detection and planning for collision avoidance, this paper
considers modeling the interaction between human drivers and pedestrians and
how it might influence map estimation, as a proxy for detection. We take a
mapping inspired approach and incorporate people as sensors into mapping
frameworks. By taking advantage of other agents' actions, we demonstrate how we
can impute portions of the map that would otherwise be occluded. We evaluate
our framework in human driving experiments and on real-world data, using
occupancy grids and landmark-based mapping approaches. Our approach
significantly improves overall environment awareness and out-performs standard
mapping techniques.

negative:
Paper-by-paper results make it easy to miss the forest for the trees.We
analyse the remarkable progress of the last decade by discussing the main ideas
explored in the 40+ detectors currently present in the Caltech pedestrian
detection benchmark. We observe that there exist three families of approaches,
all currently reaching similar detection quality. Based on our analysis, we
study the complementarity of the most promising ideas by combining multiple
published strategies. This new decision forest detector achieves the current
best known performance on the challenging Caltech-USA dataset.

negative:
Machine learning algorithms have found several applications in the field of
robotics and control systems. The control systems community has started to show
interest towards several machine learning algorithms from the sub-domains such
as supervised learning, imitation learning and reinforcement learning to
achieve autonomous control and intelligent decision making. Amongst many
complex control problems, stable bipedal walking has been the most challenging
problem. In this paper, we present an architecture to design and simulate a
planar bipedal walking robot(BWR) using a realistic robotics simulator, Gazebo.
The robot demonstrates successful walking behaviour by learning through several
of its trial and errors, without any prior knowledge of itself or the world
dynamics. The autonomous walking of the BWR is achieved using reinforcement
learning algorithm called Deep Deterministic Policy Gradient(DDPG). DDPG is one
of the algorithms for learning controls in continuous action spaces. After
training the model in simulation, it was observed that, with a proper shaped
reward function, the robot achieved faster walking or even rendered a running
gait with an average speed of 0.83 m/s. The gait pattern of the bipedal walker
was compared with the actual human walking pattern. The results show that the
bipedal walking pattern had similar characteristics to that of a human walking
pattern. The video presenting our experiment is available at
https://goo.gl/NHXKqR.

negative:
Typical methods for pedestrian detection focus on either tackling mutual
occlusions between crowded pedestrians, or dealing with the various scales of
pedestrians. Detecting pedestrians with substantial appearance diversities such
as different pedestrian silhouettes, different viewpoints or different
dressing, remains a crucial challenge. Instead of learning each of these
diverse pedestrian appearance features individually as most existing methods
do, we propose to perform contrastive learning to guide the feature learning in
such a way that the semantic distance between pedestrians with different
appearances in the learned feature space is minimized to eliminate the
appearance diversities, whilst the distance between pedestrians and background
is maximized. To facilitate the efficiency and effectiveness of contrastive
learning, we construct an exemplar dictionary with representative pedestrian
appearances as prior knowledge to construct effective contrastive training
pairs and thus guide contrastive learning. Besides, the constructed exemplar
dictionary is further leveraged to evaluate the quality of pedestrian proposals
during inference by measuring the semantic distance between the proposal and
the exemplar dictionary. Extensive experiments on both daytime and nighttime
pedestrian detection validate the effectiveness of the proposed method.

negative:
Identity documents recognition is an important sub-field of document
analysis, which deals with tasks of robust document detection, type
identification, text fields recognition, as well as identity fraud prevention
and document authenticity validation given photos, scans, or video frames of an
identity document capture. Significant amount of research has been published on
this topic in recent years, however a chief difficulty for such research is
scarcity of datasets, due to the subject matter being protected by security
requirements. A few datasets of identity documents which are available lack
diversity of document types, capturing conditions, or variability of document
field values. In addition, the published datasets were typically designed only
for a subset of document recognition problems, not for a complex identity
document analysis. In this paper, we present a dataset MIDV-2020 which consists
of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock
identity documents, each with unique text field values and unique artificially
generated faces, with rich annotation. For the presented benchmark dataset
baselines are provided for such tasks as document location and identification,
text fields recognition, and face detection. With 72409 annotated images in
total, to the date of publication the proposed dataset is the largest publicly
available identity documents dataset with variable artificially generated data,
and we believe that it will prove invaluable for advancement of the field of
document analysis and recognition. The dataset is available for download at
ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr .

negative:
My overarching research goal is to provide robots with perceptional abilities
that allow interactions with humans in a human-like manner. To develop these
perceptional abilities, I believe that it is useful to study the principles of
the human visual system. I use these principles to develop new computer vision
algorithms and validate their effectiveness in intelligent robotic systems. I
am enthusiastic about this approach as it offers the dual benefit of uncovering
principles inherent in the human visual system, as well as applying these
principles to its artificial counterpart. Fig. 1 contains a depiction of my
research.

negative:
Multi-pedestrian tracking in aerial imagery has several applications such as
large-scale event monitoring, disaster management, search-and-rescue missions,
and as input into predictive crowd dynamic models. Due to the challenges such
as the large number and the tiny size of the pedestrians (e.g., 4 x 4 pixels)
with their similar appearances as well as different scales and atmospheric
conditions of the images with their extremely low frame rates (e.g., 2 fps),
current state-of-the-art algorithms including the deep learning-based ones are
unable to perform well. In this paper, we propose AerialMPTNet, a novel
approach for multi-pedestrian tracking in geo-referenced aerial imagery by
fusing appearance features from a Siamese Neural Network, movement predictions
from a Long Short-Term Memory, and pedestrian interconnections from a GraphCNN.
In addition, to address the lack of diverse aerial pedestrian tracking
datasets, we introduce the Aerial Multi-Pedestrian Tracking (AerialMPT) dataset
consisting of 307 frames and 44,740 pedestrians annotated. We believe that
AerialMPT is the largest and most diverse dataset to this date and will be
released publicly. We evaluate AerialMPTNet on AerialMPT and KIT AIS, and
benchmark with several state-of-the-art tracking methods. Results indicate that
AerialMPTNet significantly outperforms other methods on accuracy and
time-efficiency.

negative:
Convolututional Neural Networks have achieved state of the art in image
classification, object detection and other image related tasks. In this paper I
present another use of CNNs i.e. if given a set of images and then giving a
single test image the network identifies that the test image is part of which
image from the images given before. This is a task somehow similar to measuring
image similarity and can be done using a simple CNN. Doing this task manually
by looping can be quite a time consuming problem and won't be a generalizable
solution. The task is quite similar to doing object detection but for that lots
training data should be given or in the case of sliding window it takes lot of
time and my algorithm can work with much fewer examples, is totally
unsupervised and works much efficiently. Also, I explain that how unsupervised
algorithm like K-Means or supervised algorithm like K-NN are not good enough to
perform this task. The basic idea is that image encodings are collected for
each image from a CNN, when a test image comes it is replaced by a part of
original image, the encoding is generated using the same network, the frobenius
norm is calculated and if it comes under a tolerance level then the test image
is said to be the part of the original image.

negative:
Deep convolutional neural networks continue to advance the state-of-the-art
in many domains as they grow bigger and more complex. It has been observed that
many of the parameters of a large network are redundant, allowing for the
possibility of learning a smaller network that mimics the outputs of the large
network through a process called Knowledge Distillation. We show, however, that
standard Knowledge Distillation is not effective for learning small models for
the task of pedestrian detection. To improve this process, we introduce a
higher-dimensional hint layer to increase information flow. We also estimate
the variance in the outputs of the large network and propose a loss function to
incorporate this uncertainty. Finally, we attempt to boost the complexity of
the small network without increasing its size by using as input hand-designed
features that have been demonstrated to be effective for pedestrian detection.
We succeed in training a model that contains $400\times$ fewer parameters than
the large network while outperforming AlexNet on the Caltech Pedestrian
Dataset.

negative:
One of the objectives of understanding pedestrian behavior is to predict the
effect of proposed changes in the design or evaluation of pedestrian
facilities. We want to know the impact to the user of the facilities, as the
design of the facilities change. That impact was traditionally evaluated by
level of service standards. Another design criterion to measure the impact of
design change is measured by the pedestrian flow performance index. This paper
describes the determination of pedestrian flow performance based video tracking
or any microscopic pedestrian simulation models. Most of pedestrian researches
have been done on a macroscopic level, which is an aggregation of all
pedestrian movement in pedestrian areas into flow, average speed and area
module. Macroscopic level, however, does not consider the interaction between
pedestrians. It is also not well suited for prediction of pedestrian flow
performance in pedestrian areas or in buildings with some obstruction, that
reduces the effective width of the walkways. On the other hand, the microscopic
level has a more general usage and considers detail in the design. More
efficient pedestrian flow can even be reached with less space. Those results
have rejected the linearity assumption of space and flow in the macroscopic
level.

negative:
Deep learning-based computer vision is usually data-hungry. Many researchers
attempt to augment datasets with synthesized data to improve model robustness.
However, the augmentation of popular pedestrian datasets, such as Caltech and
Citypersons, can be extremely challenging because real pedestrians are commonly
in low quality. Due to the factors like occlusions, blurs, and low-resolution,
it is significantly difficult for existing augmentation approaches, which
generally synthesize data using 3D engines or generative adversarial networks
(GANs), to generate realistic-looking pedestrians. Alternatively, to access
much more natural-looking pedestrians, we propose to augment pedestrian
detection datasets by transforming real pedestrians from the same dataset into
different shapes. Accordingly, we propose the Shape Transformation-based
Dataset Augmentation (STDA) framework. The proposed framework is composed of
two subsequent modules, i.e. the shape-guided deformation and the environment
adaptation. In the first module, we introduce a shape-guided warping field to
help deform the shape of a real pedestrian into a different shape. Then, in the
second stage, we propose an environment-aware blending map to better adapt the
deformed pedestrians into surrounding environments, obtaining more
realistic-looking pedestrians and more beneficial augmentation results for
pedestrian detection. Extensive empirical studies on different pedestrian
detection benchmarks show that the proposed STDA framework consistently
produces much better augmentation results than other pedestrian synthesis
approaches using low-quality pedestrians. By augmenting the original datasets,
our proposed framework also improves the baseline pedestrian detector by up to
38% on the evaluated benchmarks, achieving state-of-the-art performance.

negative:
Early detection of psychological distress is key to effective treatment.
Automatic detection of distress, such as depression, is an active area of
research. Current approaches utilise vocal, facial, and bodily modalities. Of
these, the bodily modality is the least investigated, partially due to the
difficulty in extracting bodily representations from videos, and partially due
to the lack of viable datasets. Existing body modality approaches use automatic
categorization of expressions to represent body language as a series of
specific expressions, much like words within natural language. In this
dissertation I present a new type of feature, within the body modality, that
represents meta information of gestures, such as speed, and use it to predict a
non-clinical depression label. This differs to existing work by representing
overall behaviour as a small set of aggregated meta features derived from a
person's movement. In my method I extract pose estimation from videos, detect
gestures within body parts, extract meta information from individual gestures,
and finally aggregate these features to generate a small feature vector for use
in prediction tasks. I introduce a new dataset of 65 video recordings of
interviews with self-evaluated distress, personality, and demographic labels.
This dataset enables the development of features utilising the whole body in
distress detection tasks. I evaluate my newly introduced meta-features for
predicting depression, anxiety, perceived stress, somatic stress, five standard
personality measures, and gender. A linear regression based classifier using
these features achieves a 82.70% F1 score for predicting depression within my
novel dataset.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
For the simulation of pedestrians a method is introduced to find routing
alternatives from any origin position to a given destination area in a given
geometry composed of walking areas and obstacles. The method includes a
parameter which sets a threshold for the approximate minimum size of obstacles
to generate routing alternatives. The resulting data structure for navigation
is constructed such that it does not introduce artifacts to the movement of
simulated pedestrians and that locally pedestrians prefer to walk on the
shortest path. The generated set of routes can be used with iterating static or
dynamic assignment methods.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.

negative:
Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.