  Classifying time series data using neural networks is a challenging problem
when the length of the data varies. Video object trajectories, which are key to
many of the visual surveillance applications, are often found to be of varying
length. If such trajectories are used to understand the behavior (normal or
anomalous) of moving objects, they need to be represented correctly. In this
paper, we propose video object trajectory classification and anomaly detection
using a hybrid Convolutional Neural Network (CNN) and Variational Autoencoder
(VAE) architecture. First, we introduce a high level representation of object
trajectories using color gradient form. In the next stage, a semi-supervised
way to annotate moving object trajectories extracted using Temporal Unknown
Incremental Clustering (TUIC), has been applied for trajectory class labeling.
Anomalous trajectories are separated using t-Distributed Stochastic Neighbor
Embedding (t-SNE). Finally, a hybrid CNN-VAE architecture has been used for
trajectory classification and anomaly detection. The results obtained using
publicly available surveillance video datasets reveal that the proposed method
can successfully identify some of the important traffic anomalies such as
vehicles not following lane driving, sudden speed variations, abrupt
termination of vehicle movement, and vehicles moving in wrong directions. The
proposed method is able to detect above anomalies at higher accuracy as
compared to existing anomaly detection methods.

Computer Vision and Pattern Recognition