query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : In the context of time series forecasting, it's observed that anomalies in certain data sources, such as seismic activity, heart rate monitors, and nuclear plant cooling system temperatures, occur infrequently but often result in catastrophic outcomes like earthquakes, heart attacks, and nuclear meltdowns. My objective is to devise a time series forecasting model capable of predicting these rare anomalies with high precision. This model must be equipped to manage extremely sparse supervised signals. To train my model effectively, I believe it's necessary to employ some form of sampling or bootstrapping to simulate various event pathways leading to these anomalies. Essentially, this involves data augmentation to generate more positive signals for training. Consequently, I require a simulation agent capable of learning the underlying time series data distribution. This simulation agent (model) must also be generative to create new pathways and simulate new events. I propose that the simulation agent should incorporate a reinforcement learning component. However, I am open to any literature that discusses rare event simulation, regardless of whether the method is statistical or deep learning-based. Ultimately, I am confident that my model will be robust enough for deployment in real-world scenarios for disaster prevention and early warning systems.

positive:
One of the limiting factors in training data-driven, rare-event prediction
algorithms is the scarcity of the events of interest resulting in an extreme
imbalance in the data. There have been many methods introduced in the
literature for overcoming this issue; simple data manipulation through
undersampling and oversampling, utilizing cost-sensitive learning algorithms,
or by generating synthetic data points following the distribution of the
existing data. While synthetic data generation has recently received a great
deal of attention, there are real challenges involved in doing so for
high-dimensional data such as multivariate time series. In this study, we
explore the usefulness of the conditional generative adversarial network (CGAN)
as a means to perform data-informed oversampling in order to balance a large
dataset of multivariate time series. We utilize a flare forecasting benchmark
dataset, named SWAN-SF, and design two verification methods to both
quantitatively and qualitatively evaluate the similarity between the generated
minority and the ground-truth samples. We further assess the quality of the
generated samples by training a classical, supervised machine learning
algorithm on synthetic data, and testing the trained model on the unseen, real
data. The results show that the classifier trained on the data augmented with
the synthetic multivariate time series achieves a significant improvement
compared with the case where no augmentation is used. The popular flare
forecasting evaluation metrics, TSS and HSS, report 20-fold and 5-fold
improvements, respectively, indicating the remarkable statistical similarities,
and the usefulness of CGAN-based data generation for complicated tasks such as
flare forecasting.

positive:
Generative Adversarial Net (GAN) has been proven to be a powerful machine
learning tool in image data analysis and generation. In this paper, we propose
to use Conditional Generative Adversarial Net (CGAN) to learn and simulate time
series data. The conditions can be both categorical and continuous variables
containing different kinds of auxiliary information. Our simulation studies
show that CGAN is able to learn different kinds of normal and heavy tail
distributions, as well as dependent structures of different time series and it
can further generate conditional predictive distributions consistent with the
training data distributions. We also provide an in-depth discussion on the
rationale of GAN and the neural network as hierarchical splines to draw a clear
connection with the existing statistical method for distribution generation. In
practice, CGAN has a wide range of applications in the market risk and
counterparty risk analysis: it can be applied to learn the historical data and
generate scenarios for the calculation of Value-at-Risk (VaR) and Expected
Shortfall (ES) and predict the movement of the market risk factors. We present
a real data analysis including a backtesting to demonstrate CGAN is able to
outperform the Historic Simulation, a popular method in market risk analysis
for the calculation of VaR. CGAN can also be applied in the economic time
series modeling and forecasting, and an example of hypothetical shock analysis
for economic models and the generation of potential CCAR scenarios by CGAN is
given at the end of the paper.

positive:
Time series data are often corrupted by outliers or other kinds of anomalies.
Identifying the anomalous points can be a goal on its own (anomaly detection),
or a means to improving performance of other time series tasks (e.g.
forecasting). Recent deep-learning-based approaches to anomaly detection and
forecasting commonly assume that the proportion of anomalies in the training
data is small enough to ignore, and treat the unlabeled data as coming from the
nominal data distribution. We present a simple yet effective technique for
augmenting existing time series models so that they explicitly account for
anomalies in the training data. By augmenting the training data with a latent
anomaly indicator variable whose distribution is inferred while training the
underlying model using Monte Carlo EM, our method simultaneously infers
anomalous points while improving model performance on nominal data. We
demonstrate the effectiveness of the approach by combining it with a simple
feed-forward forecasting model. We investigate how anomalies in the train set
affect the training of forecasting models, which are commonly used for time
series anomaly detection, and show that our method improves the training of the
model.

positive:
To predict rare extreme events using deep neural networks, one encounters the
so-called small data problem because even long-term observations often contain
few extreme events. Here, we investigate a model-assisted framework where the
training data is obtained from numerical simulations, as opposed to
observations, with adequate samples from extreme events. However, to ensure the
trained networks are applicable in practice, the training is not performed on
the full simulation data; instead we only use a small subset of observable
quantities which can be measured in practice. We investigate the feasibility of
this model-assisted framework on three different dynamical systems (Rossler
attractor, FitzHugh-Nagumo model, and a turbulent fluid flow) and three
different deep neural network architectures (feedforward, long short-term
memory, and reservoir computing). In each case, we study the prediction
accuracy, robustness to noise, reproducibility under repeated training, and
sensitivity to the type of input data. In particular, we find long short-term
memory networks to be most robust to noise and to yield relatively accurate
predictions, while requiring minimal fine-tuning of the hyperparameters.

negative:
Robust Anomaly Detection (AD) on time series data is a key component for
monitoring many complex modern systems. These systems typically generate
high-dimensional time series that can be highly noisy, seasonal, and
inter-correlated. This paper explores some of the challenges in such data, and
proposes a new approach that makes inroads towards increased robustness on
seasonal and contaminated data, while providing a better root cause
identification of anomalies. In particular, we propose the use of Robust
Seasonal Multivariate Generative Adversarial Network (RSM-GAN) that extends
recent advancements in GAN with the adoption of convolutional-LSTM layers and
attention mechanisms to produce excellent performance on various settings. We
conduct extensive experiments in which not only do this model displays more
robust behavior on complex seasonality patterns, but also shows increased
resistance to training data contamination. We compare it with existing
classical and deep-learning AD models, and show that this architecture is
associated with the lowest false positive rate and improves precision by 30%
and 16% in real-world and synthetic data, respectively.

negative:
Today's Cyber-Physical Systems (CPSs) are large, complex, and affixed with
networked sensors and actuators that are targets for cyber-attacks.
Conventional detection techniques are unable to deal with the increasingly
dynamic and complex nature of the CPSs. On the other hand, the networked
sensors and actuators generate large amounts of data streams that can be
continuously monitored for intrusion events. Unsupervised machine learning
techniques can be used to model the system behaviour and classify deviant
behaviours as possible attacks. In this work, we proposed a novel Generative
Adversarial Networks-based Anomaly Detection (GAN-AD) method for such complex
networked CPSs. We used LSTM-RNN in our GAN to capture the distribution of the
multivariate time series of the sensors and actuators under normal working
conditions of a CPS. Instead of treating each sensor's and actuator's time
series independently, we model the time series of multiple sensors and
actuators in the CPS concurrently to take into account of potential latent
interactions between them. To exploit both the generator and the discriminator
of our GAN, we deployed the GAN-trained discriminator together with the
residuals between generator-reconstructed data and the actual samples to detect
possible anomalies in the complex CPS. We used our GAN-AD to distinguish
abnormal attacked situations from normal working conditions for a complex
six-stage Secure Water Treatment (SWaT) system. Experimental results showed
that the proposed strategy is effective in identifying anomalies caused by
various attacks with high detection rate and low false positive rate as
compared to existing methods.

negative:
Although aviation accidents are rare, safety incidents occur more frequently
and require a careful analysis to detect and mitigate risks in a timely manner.
Analyzing safety incidents using operational data and producing event-based
explanations is invaluable to airline companies as well as to governing
organizations such as the Federal Aviation Administration (FAA) in the United
States. However, this task is challenging because of the complexity involved in
mining multi-dimensional heterogeneous time series data, the lack of
time-step-wise annotation of events in a flight, and the lack of scalable tools
to perform analysis over a large number of events. In this work, we propose a
precursor mining algorithm that identifies events in the multidimensional time
series that are correlated with the safety incident. Precursors are valuable to
systems health and safety monitoring and in explaining and forecasting safety
incidents. Current methods suffer from poor scalability to high dimensional
time series data and are inefficient in capturing temporal behavior. We propose
an approach by combining multiple-instance learning (MIL) and deep recurrent
neural networks (DRNN) to take advantage of MIL's ability to learn using weakly
supervised data and DRNN's ability to model temporal behavior. We describe the
algorithm, the data, the intuition behind taking a MIL approach, and a
comparative analysis of the proposed algorithm with baseline models. We also
discuss the application to a real-world aviation safety problem using data from
a commercial airline company and discuss the model's abilities and
shortcomings, with some final remarks about possible deployment directions.

negative:
Large network logs, recording multivariate time series generated from
heterogeneous devices and sensors in a network, can often reveal important
information about abnormal activities, such as network intrusions and device
malfunctions. Existing machine learning methods for anomaly detection on
multivariate time series typically assume that 1) normal sequences would have
consistent behavior for training unsupervised models, or 2) require a large set
of labeled normal and abnormal sequences for supervised models. However, in
practice, normal network activities can demonstrate significantly varying
sequence patterns (e.g., before and after rerouting partial network traffic).
Also, the recorded abnormal events can be sparse. This paper presents a novel
semi-supervised method that efficiently captures dependencies between network
time series and across time points to generate meaningful representations of
network activities for predicting abnormal events. The method can use the
limited labeled data to explicitly learn separable embedding space for normal
and abnormal samples and effectively leverage unlabeled data to handle training
data scarcity. The experiments demonstrate that our approach significantly
outperformed state-of-the-art approaches for event detection on a large
real-world network log.

negative:
Machine learning and in particular deep learning algorithms are the emerging
approaches to data analysis. These techniques have transformed traditional data
mining-based analysis radically into a learning-based model in which existing
data sets along with their cluster labels (i.e., train set) are learned to
build a supervised learning model and predict the cluster labels of unseen data
(i.e., test set). In particular, deep learning techniques are capable of
capturing and learning hidden features in a given data sets and thus building a
more accurate prediction model for clustering and labeling problem. However,
the major problem is that time series data are often unlabeled and thus
supervised learning-based deep learning algorithms cannot be directly adapted
to solve the clustering problems for these special and complex types of data
sets. To address this problem, this paper introduces a two-stage method for
clustering time series data. First, a novel technique is introduced to utilize
the characteristics (e.g., volatility) of given time series data in order to
create labels and thus be able to transform the problem from unsupervised
learning into supervised learning. Second, an autoencoder-based deep learning
model is built to learn and model both known and hidden features of time series
data along with their created labels to predict the labels of unseen time
series data. The paper reports a case study in which financial and stock time
series data of selected 70 stock indices are clustered into distinct groups
using the introduced two-stage procedure. The results show that the proposed
procedure is capable of achieving 87.5\% accuracy in clustering and predicting
the labels for unseen time series data.

negative:
Long-tail and rare event problems become crucial when autonomous driving
algorithms are applied in the real world. For the purpose of evaluating systems
in challenging settings, we propose a generative framework to create
safety-critical scenarios for evaluating specific task algorithms. We first
represent the traffic scenarios with a series of autoregressive building blocks
and generate diverse scenarios by sampling from the joint distribution of these
blocks. We then train the generative model as an agent (or a generator) to
investigate the risky distribution parameters for a given driving algorithm
being evaluated. We regard the task algorithm as an environment (or a
discriminator) that returns a reward to the agent when a risky scenario is
generated. Through the experiments conducted on several scenarios in the
simulation, we demonstrate that the proposed framework generates
safety-critical scenarios more efficiently than grid search or human design
methods. Another advantage of this method is its adaptiveness to the routes and
parameters.

negative:
This work contributes to the development of neural forecasting models with
novel randomization-based learning methods. These methods improve the fitting
abilities of the neural model, in comparison to the standard method, by
generating network parameters in accordance with the data and target function
features. A pattern-based representation of time series makes the proposed
approach useful for forecasting time series with multiple seasonality. In the
simulation study, we evaluate the performance of the proposed models and find
that they can compete in terms of forecasting accuracy with fully-trained
networks. Extremely fast and easy training, simple architecture, ease of
implementation, high accuracy as well as dealing with nonstationarity and
multiple seasonality in time series make the proposed model very attractive for
a wide range of complex time series forecasting problems.

negative:
This paper presents a time series forecasting framework which combines
standard forecasting methods and a machine learning model. The inputs to the
machine learning model are not lagged values or regular time series features,
but instead forecasts produced by standard methods. The machine learning model
can be either a convolutional neural network model or a recurrent neural
network model. The intuition behind this approach is that forecasts of a time
series are themselves good features characterizing the series, especially when
the modelling purpose is forecasting. It can also be viewed as a weighted
ensemble method. Tested on the M4 competition dataset, this approach
outperforms all submissions for quarterly series, and is more accurate than all
but the winning algorithm for monthly series.

negative:
Empirically observed time series in physics, biology, or medicine, are
commonly generated by some underlying dynamical system (DS) which is the target
of scientific interest. There is an increasing interest to harvest machine
learning methods to reconstruct this latent DS in a data-driven, unsupervised
way. In many areas of science it is common to sample time series observations
from many data modalities simultaneously, e.g. electrophysiological and
behavioral time series in a typical neuroscience experiment. However, current
machine learning tools for reconstructing DSs usually focus on just one data
modality. Here we propose a general framework for multi-modal data integration
for the purpose of nonlinear DS reconstruction and the analysis of cross-modal
relations. This framework is based on dynamically interpretable recurrent
neural networks as general approximators of nonlinear DSs, coupled to sets of
modality-specific decoder models from the class of generalized linear models.
Both an expectation-maximization and a variational inference algorithm for
model training are advanced and compared. We show on nonlinear DS benchmarks
that our algorithms can efficiently compensate for too noisy or missing
information in one data channel by exploiting other channels, and demonstrate
on experimental neuroscience data how the algorithm learns to link different
data domains to the underlying dynamics.

negative:
In recent years, deep learning techniques have outperformed traditional
models in many machine learning tasks. Deep neural networks have successfully
been applied to address time series forecasting problems, which is a very
important topic in data mining. They have proved to be an effective solution
given their capacity to automatically learn the temporal dependencies present
in time series. However, selecting the most convenient type of deep neural
network and its parametrization is a complex task that requires considerable
expertise. Therefore, there is a need for deeper studies on the suitability of
all existing architectures for different forecasting tasks. In this work, we
face two main challenges: a comprehensive review of the latest works using deep
learning for time series forecasting; and an experimental study comparing the
performance of the most popular architectures. The comparison involves a
thorough analysis of seven types of deep learning models in terms of accuracy
and efficiency. We evaluate the rankings and distribution of results obtained
with the proposed models under many different architecture configurations and
training hyperparameters. The datasets used comprise more than 50000 time
series divided into 12 different forecasting problems. By training more than
38000 models on these data, we provide the most extensive deep learning study
for time series forecasting. Among all studied models, the results show that
long short-term memory (LSTM) and convolutional networks (CNN) are the best
alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve
comparable performance with less variability of results under different
parameter configurations, while also being more efficient.

negative:
In recent times, deep artificial neural networks have achieved many successes
in pattern recognition. Part of this success can be attributed to the reliance
on big data to increase generalization. However, in the field of time series
recognition, many datasets are often very small. One method of addressing this
problem is through the use of data augmentation. In this paper, we survey data
augmentation techniques for time series and their application to time series
classification with neural networks. We propose a taxonomy and outline the four
families in time series data augmentation, including transformation-based
methods, pattern mixing, generative models, and decomposition methods.
Furthermore, we empirically evaluate 12 time series data augmentation methods
on 128 time series classification datasets with six different types of neural
networks. Through the results, we are able to analyze the characteristics,
advantages and disadvantages, and recommendations of each data augmentation
method. This survey aims to help in the selection of time series data
augmentation for neural network applications.

negative:
With the wider availability of sensor technology, a number of Structural
Health Monitoring (SHM) systems are deployed to monitor civil infrastructure.
The continuous monitoring provides valuable information about the structure
that can help in providing a decision support system for retrofits and other
structural modifications. However, when the sensors are exposed to harsh
environmental conditions, the data measured by the SHM systems tend to be
affected by multiple anomalies caused by faulty or broken sensors. Given a
deluge of high-dimensional data collected continuously over time, research into
using machine learning methods to detect anomalies are a topic of great
interest to the SHM community. This paper contributes to this effort by
proposing the use of a relatively new time series representation named Shapelet
Transform in combination with a Random Forest classifier to autonomously
identify anomalies in SHM data. The shapelet transform is a unique time series
representation that is solely based on the shape of the time series data. In
consideration of the individual characteristics unique to every anomaly, the
application of this transform yields a new shape-based feature representation
that can be combined with any standard machine learning algorithm to detect
anomalous data with no manual intervention. For the present study, the anomaly
detection framework consists of three steps: identifying unique shapes from
anomalous data, using these shapes to transform the SHM data into a local-shape
space and training machine learning algorithm on this transformed data to
identify anomalies. The efficacy of this method is demonstrated by the
identification of anomalies in acceleration data from a SHM system installed on
a long-span bridge in China. The results show that multiple data anomalies in
SHM data can be automatically detected with high accuracy using the proposed
method.

negative:
A new variational mode decomposition (VMD) based deep learning approach is
proposed in this paper for time series forecasting problem. Firstly, VMD is
adopted to decompose the original time series into several sub-signals. Then, a
convolutional neural network (CNN) is applied to learn the reconstruction
patterns on the decomposed sub-signals to obtain several reconstructed
sub-signals. Finally, a long short term memory (LSTM) network is employed to
forecast the time series with the decomposed sub-signals and the reconstructed
sub-signals as inputs. The proposed VMD-CNN-LSTM approach is originated from
the decomposition-reconstruction-ensemble framework, and innovated by embedding
the reconstruction, single forecasting, and ensemble steps in a unified deep
learning approach. To verify the forecasting performance of the proposed
approach, four typical time series datasets are introduced for empirical
analysis. The empirical results demonstrate that the proposed approach
outperforms consistently the benchmark approaches in terms of forecasting
accuracy, and also indicate that the reconstructed sub-signals obtained by CNN
is of importance for further improving the forecasting performance.

negative:
Forecasting models that are trained across sets of many time series, known as
Global Forecasting Models (GFM), have shown recently promising results in
forecasting competitions and real-world applications, outperforming many
state-of-the-art univariate forecasting techniques. In most cases, GFMs are
implemented using deep neural networks, and in particular Recurrent Neural
Networks (RNN), which require a sufficient amount of time series to estimate
their numerous model parameters. However, many time series databases have only
a limited number of time series. In this study, we propose a novel, data
augmentation based forecasting framework that is capable of improving the
baseline accuracy of the GFM models in less data-abundant settings. We use
three time series augmentation techniques: GRATIS, moving block bootstrap
(MBB), and dynamic time warping barycentric averaging (DBA) to synthetically
generate a collection of time series. The knowledge acquired from these
augmented time series is then transferred to the original dataset using two
different approaches: the pooled approach and the transfer learning approach.
When building GFMs, in the pooled approach, we train a model on the augmented
time series alongside the original time series dataset, whereas in the transfer
learning approach, we adapt a pre-trained model to the new dataset. In our
evaluation on competition and real-world time series datasets, our proposed
variants can significantly improve the baseline accuracy of GFM models and
outperform state-of-the-art univariate forecasting methods.

negative:
We introduce Neural Contextual Anomaly Detection (NCAD), a framework for
anomaly detection on time series that scales seamlessly from the unsupervised
to supervised setting, and is applicable to both univariate and multivariate
time series. This is achieved by effectively combining recent developments in
representation learning for multivariate time series, with techniques for deep
anomaly detection originally developed for computer vision that we tailor to
the time series setting. Our window-based approach facilitates learning the
boundary between normal and anomalous classes by injecting generic synthetic
anomalies into the available data. Moreover, our method can effectively take
advantage of all the available information, be it as domain knowledge, or as
training labels in the semi-supervised setting. We demonstrate empirically on
standard benchmark datasets that our approach obtains a state-of-the-art
performance in these settings.

negative:
Forecasting on sparse multivariate time series (MTS) aims to model the
predictors of future values of time series given their incomplete past, which
is important for many emerging applications. However, most existing methods
process MTS's individually, and do not leverage the dynamic distributions
underlying the MTS's, leading to sub-optimal results when the sparsity is high.
To address this challenge, we propose a novel generative model, which tracks
the transition of latent clusters, instead of isolated feature representations,
to achieve robust modeling. It is characterized by a newly designed dynamic
Gaussian mixture distribution, which captures the dynamics of clustering
structures, and is used for emitting timeseries. The generative model is
parameterized by neural networks. A structured inference network is also
designed for enabling inductive analysis. A gating mechanism is further
introduced to dynamically tune the Gaussian mixture distributions. Extensive
experimental results on a variety of real-life datasets demonstrate the
effectiveness of our method.

negative:
Most current anomaly detection methods suffer from the curse of
dimensionality when dealing with high-dimensional data. We propose an anomaly
detection algorithm that can scale to high-dimensional data using concepts from
the theory of large deviations. The proposed Large Deviations Anomaly Detection
(LAD) algorithm is shown to outperform state of art anomaly detection methods
on a variety of large and high-dimensional benchmark data sets. Exploiting the
ability of the algorithm to scale to high-dimensional data, we propose an
online anomaly detection method to identify anomalies in a collection of
multivariate time series. We demonstrate the applicability of the online
algorithm in identifying counties in the United States with anomalous trends in
terms of COVID-19 related cases and deaths. Several of the identified anomalous
counties correlate with counties with documented poor response to the COVID
pandemic.

negative:
Recently, with the development of deep learning, end-to-end neural network
architectures have been increasingly applied to condition monitoring signals.
They have demonstrated superior performance for fault detection and
classification, in particular using convolutional neural networks. Even more
recently, an extension of the concept of convolution to the concept of
kervolution has been proposed with some promising results in image
classification tasks. In this paper, we explore the potential of kervolutional
neural networks applied to time series data. We demonstrate that using a
mixture of convolutional and kervolutional layers improves the model
performance. The mixed model is first applied to a classification task in time
series, as a benchmark dataset. Subsequently, the proposed mixed architecture
is used to detect anomalies in time series data recorded by accelerometers on
helicopters. We propose a residual-based anomaly detection approach using a
temporal auto-encoder. We demonstrate that mixing kervolutional with
convolutional layers in the encoder is more sensitive to variations in the
input data and is able to detect anomalous time series in a better way.

negative:
A challenging part of dynamic probabilistic risk assessment for nuclear power
plants is the need for large amounts of temporal simulations given various
initiating events and branching conditions from which representative feature
extraction becomes complicated for subsequent applications. Artificial
Intelligence techniques have been shown to be powerful tools in time-dependent
sequential data processing to automatically extract and yield complex features
from large data. An advanced temporal neural network referred to as the
Transformer is used within a supervised learning fashion to model the
time-dependent NPP simulation data and to infer whether a given sequence of
events leads to core damage or not. The training and testing datasets for the
Transformer are obtained by running 10,000 RELAP5-3D NPP blackout simulations
with the list of variables obtained from the RAVEN software. Each simulation is
classified as "OK" or "CORE DAMAGE" based on the consequence. The results show
that the Transformer can learn the characteristics of the sequential data and
yield promising performance with approximately 99% classification accuracy on
the testing dataset.

negative:
Many problems in science and engineering require making predictions based on
few observations. To build a robust predictive model, these sparse data may
need to be augmented with simulated data, especially when the design space is
multi-dimensional. Simulations, however, often suffer from an inherent bias.
Estimation of this bias may be poorly constrained not only because of data
sparsity, but also because traditional predictive models fit only one type of
observed outputs, such as scalars or images, instead of all available output
data modalities, which might have been acquired and simulated at great cost. To
break this limitation and open up the path for multi-modal calibration, we
propose to combine a novel, transfer learning technique for suppressing the
bias with recent developments in deep learning, which allow building predictive
models with multi-modal outputs. First, we train an initial neural network
model on simulated data to learn important correlations between different
output modalities and between simulation inputs and outputs. Then, the model is
partially retrained, or transfer learned, to fit the experiments; a method that
has never been implemented in this type of architecture. Using fewer than 10
inertial confinement fusion experiments for training, transfer learning
systematically improves the simulation predictions while a simple output
calibration, which we design as a baseline, makes the predictions worse. We
also offer extensive cross-validation with real and carefully designed
synthetic data. The method described in this paper can be applied to a wide
range of problems that require transferring knowledge from simulations to the
domain of experiments.

negative:
Having access to a forward model enables the use of planning algorithms such
as Monte Carlo Tree Search and Rolling Horizon Evolution. Where a model is
unavailable, a natural aim is to learn a model that reflects accurately the
dynamics of the environment. In many situations it might not be possible and
minimal glitches in the model may lead to poor performance and failure. This
paper explores the problem of model misspecification through uncertainty-aware
reinforcement learning agents. We propose a bootstrapped multi-headed neural
network that learns the distribution of future states and rewards. We
experiment with a number of schemes to extract the most likely predictions.
Moreover, we also introduce a global error correction filter that applies
high-level constraints guided by the context provided through the predictive
distribution. We illustrate our approach on Minipacman. The evaluation
demonstrates that when dealing with imperfect models, our methods exhibit
increased performance and stability, both in terms of model accuracy and in its
use within a planning algorithm.

negative:
Irregularly sampled time series data arise naturally in many application
domains including biology, ecology, climate science, astronomy, and health.
Such data represent fundamental challenges to many classical models from
machine learning and statistics due to the presence of non-uniform intervals
between observations. However, there has been significant progress within the
machine learning community over the last decade on developing specialized
models and architectures for learning from irregularly sampled univariate and
multivariate time series data. In this survey, we first describe several axes
along which approaches to learning from irregularly sampled time series differ
including what data representations they are based on, what modeling primitives
they leverage to deal with the fundamental problem of irregular sampling, and
what inference tasks they are designed to perform. We then survey the recent
literature organized primarily along the axis of modeling primitives. We
describe approaches based on temporal discretization, interpolation,
recurrence, attention and structural invariance. We discuss similarities and
differences between approaches and highlight primary strengths and weaknesses.

negative:
Deep probabilistic forecasting techniques have recently been proposed for
modeling large collections of time-series. However, these techniques explicitly
assume either complete independence (local model) or complete dependence
(global model) between time-series in the collection. This corresponds to the
two extreme cases where every time-series is disconnected from every other
time-series in the collection or likewise, that every time-series is related to
every other time-series resulting in a completely connected graph. In this
work, we propose a deep hybrid probabilistic graph-based forecasting framework
called Graph Deep Factors (GraphDF) that goes beyond these two extremes by
allowing nodes and their time-series to be connected to others in an arbitrary
fashion. GraphDF is a hybrid forecasting framework that consists of a
relational global and relational local model. In particular, we propose a
relational global model that learns complex non-linear time-series patterns
globally using the structure of the graph to improve both forecasting accuracy
and computational efficiency. Similarly, instead of modeling every time-series
independently, we learn a relational local model that not only considers its
individual time-series but also the time-series of nodes that are connected in
the graph. The experiments demonstrate the effectiveness of the proposed deep
hybrid graph-based forecasting model compared to the state-of-the-art methods
in terms of its forecasting accuracy, runtime, and scalability. Our case study
reveals that GraphDF can successfully generate cloud usage forecasts and
opportunistically schedule workloads to increase cloud cluster utilization by
47.5% on average.

negative:
Ensemble models are powerful model building tools that are developed with a
focus to improve the accuracy of model predictions. They find applications in
time series forecasting in varied scenarios including but not limited to
process industries, health care, and economics where a single model might not
provide optimal performance. It is known that if models selected for data
modelling are distinct (linear/non-linear, static/dynamic) and independent
(minimally correlated models), the accuracy of the predictions is improved.
Various approaches suggested in the literature to weigh the ensemble models use
a static set of weights. Due to this limitation, approaches using a static set
of weights for weighing ensemble models cannot capture the dynamic changes or
local features of the data effectively. To address this issue, a Reinforcement
Learning (RL) approach to dynamically assign and update weights of each of the
models at different time instants depending on the nature of data and the
individual model predictions is proposed in this work. The RL method
implemented online, essentially learns to update the weights and reduce the
errors as the time progresses. Simulation studies on time series data showed
that the dynamic weighted approach using RL learns the weight better than
existing approaches. The accuracy of the proposed method is compared with an
existing approach of online Neural Network tuning quantitatively through
normalized mean square error(NMSE) values.

negative:
Forecasting financial time series is considered to be a difficult task due to
the chaotic feature of the series. Statistical approaches have shown solid
results in some specific problems such as predicting market direction and
single-price of stocks; however, with the recent advances in deep learning and
big data techniques, new promising options have arises to tackle financial time
series forecasting. Moreover, recent literature has shown that employing a
combination of statistics and machine learning may improve accuracy in the
forecasts in comparison to single solutions. Taking into consideration the
mentioned aspects, in this work, we proposed the MegazordNet, a framework that
explores statistical features within a financial series combined with a
structured deep learning model for time series forecasting. We evaluated our
approach predicting the closing price of stocks in the S&P 500 using different
metrics, and we were able to beat single statistical and machine learning
methods.

negative:
This work presents an introduction to feature-based time-series analysis. The
time series as a data type is first described, along with an overview of the
interdisciplinary time-series analysis literature. I then summarize the range
of feature-based representations for time series that have been developed to
aid interpretable insights into time-series structure. Particular emphasis is
given to emerging research that facilitates wide comparison of feature-based
representations that allow us to understand the properties of a time-series
dataset that make it suited to a particular feature-based representation or
analysis algorithm. The future of time-series analysis is likely to embrace
approaches that exploit machine learning methods to partially automate human
learning to aid understanding of the complex dynamical patterns in the time
series we measure from the world.

negative:
With the increase in the learning capability of deep convolution-based
architectures, various applications of such models have been proposed over
time. In the field of anomaly detection, improvements in deep learning opened
new prospects of exploration for the researchers whom tried to automate the
labor-intensive features of data collection. First, in terms of data
collection, it is impossible to anticipate all the anomalies that might exist
in a given environment. Second, assuming we limit the possibilities of
anomalies, it will still be hard to record all these scenarios for the sake of
training a model. Third, even if we manage to record a significant amount of
abnormal data, it's laborious to annotate this data on pixel or even frame
level. Various approaches address the problem by proposing one-class
classification using generative models trained on only normal data. In such
methods, only the normal data is used, which is abundantly available and
doesn't require significant human input. However, these are trained with only
normal data and at the test time, given abnormal data as input, may often
generate normal-looking output. This happens due to the hallucination
characteristic of generative models. Next, these systems are designed to not
use abnormal examples during the training. In this paper, we propose anomaly
detection with negative learning (ADNL), which employs the negative learning
concept for the enhancement of anomaly detection by utilizing a very small
number of labeled anomaly data as compared with the normal data during
training. The idea is to limit the reconstruction capability of a generative
model using the given a small amount of anomaly examples. This way, the network
not only learns to reconstruct normal data but also encloses the normal
distribution far from the possible distribution of anomalies.

negative:
A real-world dataset is provided from a pulp-and-paper manufacturing
industry. The dataset comes from a multivariate time series process. The data
contains a rare event of paper break that commonly occurs in the industry. The
data contains sensor readings at regular time-intervals (x's) and the event
label (y). The primary purpose of the data is thought to be building a
classification model for early prediction of the rare event. However, it can
also be used for multivariate time series data exploration and building other
supervised and unsupervised models.

negative:
We address a three-tier numerical framework based on manifold learning for
the forecasting of high-dimensional time series. At the first step, we embed
the time series into a reduced low-dimensional space using a nonlinear manifold
learning algorithm such as Locally Linear Embedding and Diffusion Maps. At the
second step, we construct reduced-order regression models on the manifold, in
particular Multivariate Autoregressive (MVAR) and Gaussian Process Regression
(GPR) models, to forecast the embedded dynamics. At the final step, we lift the
embedded time series back to the original high-dimensional space using Radial
Basis Functions interpolation and Geometric Harmonics. For our illustrations,
we test the forecasting performance of the proposed numerical scheme with four
sets of time series: three synthetic stochastic ones resembling EEG signals
produced from linear and nonlinear stochastic models with different model
orders, and one real-world data set containing daily time series of 10 key
foreign exchange rates (FOREX) spanning the time period 03/09/2001-29/10/2020.
The forecasting performance of the proposed numerical scheme is assessed using
the combinations of manifold learning, modelling and lifting approaches. We
also provide a comparison with the Principal Component Analysis algorithm as
well as with the naive random walk model and the MVAR and GPR models trained
and implemented directly in the high-dimensional space.

negative:
This paper addresses the problem of multi-step time series forecasting for
non-stationary signals that can present sudden changes. Current
state-of-the-art deep learning forecasting methods, often trained with variants
of the MSE, lack the ability to provide sharp predictions in deterministic and
probabilistic contexts. To handle these challenges, we propose to incorporate
shape and temporal criteria in the training objective of deep models. We define
shape and temporal similarities and dissimilarities, based on a smooth
relaxation of Dynamic Time Warping (DTW) and Temporal Distortion Index (TDI),
that enable to build differentiable loss functions and positive semi-definite
(PSD) kernels. With these tools, we introduce DILATE (DIstortion Loss including
shApe and TimE), a new objective for deterministic forecasting, that explicitly
incorporates two terms supporting precise shape and temporal change detection.
For probabilistic forecasting, we introduce STRIPE++ (Shape and Time diverRsIty
in Probabilistic forEcasting), a framework for providing a set of sharp and
diverse forecasts, where the structured shape and time diversity is enforced
with a determinantal point process (DPP) diversity loss. Extensive experiments
and ablations studies on synthetic and real-world datasets confirm the benefits
of leveraging shape and time features in time series forecasting.

negative:
Time series data is ubiquitous in the real-world problems across various
domains including healthcare, social media, and crime surveillance. Detecting
anomalies, or irregular and rare events, in time series data, can enable us to
find abnormal events in any natural phenomena, which may require special
treatment. Moreover, labeled instances of anomaly are hard to get in time
series data. On the other hand, time series data, due to its nature, often
exhibits localized expansions and compressions in the time dimension which is
called warping. These two challenges make it hard to detect anomalies in time
series as often such warpings could get detected as anomalies erroneously. Our
objective is to build an anomaly detection model that is robust to such warping
variations. In this paper, we propose a novel unsupervised time series anomaly
detection method, WaRTEm-AD, that operates in two stages. Within the key stage
of representation learning, we employ data augmentation through bespoke time
series operators which are passed through a twin autoencoder architecture to
learn warping-robust representations for time series data. Second, adaptations
of state-of-the-art anomaly detection methods are employed on the learnt
representations to identify anomalies. We will illustrate that WaRTEm-AD is
designed to detect two types of time series anomalies: point and sequence
anomalies. We compare WaRTEm-AD with the state-of-the-art baselines and
establish the effectiveness of our method both in terms of anomaly detection
performance and computational efficiency.

negative:
Analysis of water and environmental data is an important aspect of many
intelligent water and environmental system applications where inference from
such analysis plays a significant role in decision making. Quite often these
data that are collected through sensible sensors can be anomalous due to
different reasons such as systems breakdown, malfunctioning of sensor
detectors, and more. Regardless of their root causes, such data severely affect
the results of the subsequent analysis. This paper demonstrates data cleaning
and preparation for time-series data and further proposes cost-sensitive
machine learning algorithms as a solution to detect anomalous data points in
time-series data. The following models: Logistic Regression, Random Forest,
Support Vector Machines have been modified to support the cost-sensitive
learning which penalizes misclassified samples thereby minimizing the total
misclassification cost. Our results showed that Random Forest outperformed the
rest of the models at predicting the positive class (i.e anomalies). Applying
predictive model improvement techniques like data oversampling seems to provide
little or no improvement to the Random Forest model. Interestingly, with
recursive feature elimination, we achieved a better model performance thereby
reducing the dimensions in the data. Finally, with Influxdb and Kapacitor the
data was ingested and streamed to generate new data points to further evaluate
the model performance on unseen data, this will allow for early recognition of
undesirable changes in the drinking water quality and will enable the water
supply companies to rectify on a timely basis whatever undesirable changes
abound.

negative:
While classical time series forecasting considers individual time series in
isolation, recent advances based on deep learning showed that jointly learning
from a large pool of related time series can boost the forecasting accuracy.
However, the accuracy of these methods suffers greatly when modeling
out-of-sample time series, significantly limiting their applicability compared
to classical forecasting methods. To bridge this gap, we adopt a meta-learning
view of the time series forecasting problem. We introduce a novel forecasting
method, called Meta Global-Local Auto-Regression (Meta-GLAR), that adapts to
each time series by learning in closed-form the mapping from the
representations produced by a recurrent neural network (RNN) to one-step-ahead
forecasts. Crucially, the parameters ofthe RNN are learned across multiple time
series by backpropagating through the closed-form adaptation mechanism. In our
extensive empirical evaluation we show that our method is competitive with the
state-of-the-art in out-of-sample forecasting accuracy reported in earlier
work.

negative:
In complex industrial settings, it is common practice to monitor the
operation of machines in order to detect undesired states, adjust maintenance
schedules, optimize system performance or collect usage statistics of
individual machines. In this work, we focus on estimating the power output of a
Combined Heat and Power (CHP) machine of a medium-sized company facility by
analyzing the total facility power consumption. We formulate the problem as a
time-series classification problem where the class label represents the CHP
power output. As the facility is fully instrumented and sensor measurements
from the CHP are available, we generate the training labels in an automated
fashion from the CHP sensor readings. However, sensor failures result in
mislabeled training data samples which are hard to detect and remove from the
dataset. Therefore, we propose a novel multi-task deep learning approach that
jointly trains a classifier and an autoencoder with a shared embedding
representation. The proposed approach targets to gradually correct the
mislabelled data samples during training in a self-supervised fashion, without
any prior assumption on the amount of label noise. We benchmark our approach on
several time-series classification datasets and find it to be comparable and
sometimes better than state-of-the-art methods. On the real-world use-case of
predicting the CHP power output, we thoroughly evaluate the architectural
design choices and show that the final architecture considerably increases the
robustness of the learning process and consistently beats other recent
state-of-the-art algorithms in the presence of unstructured as well as
structured label noise.

negative:
The monitoring and management of numerous and diverse time series data at
Alibaba Group calls for an effective and scalable time series anomaly detection
service. In this paper, we propose RobustTAD, a Robust Time series Anomaly
Detection framework by integrating robust seasonal-trend decomposition and
convolutional neural network for time series data. The seasonal-trend
decomposition can effectively handle complicated patterns in time series, and
meanwhile significantly simplifies the architecture of the neural network,
which is an encoder-decoder architecture with skip connections. This
architecture can effectively capture the multi-scale information from time
series, which is very useful in anomaly detection. Due to the limited labeled
data in time series anomaly detection, we systematically investigate data
augmentation methods in both time and frequency domains. We also introduce
label-based weight and value-based weight in the loss function by utilizing the
unbalanced nature of the time series anomaly detection problem. Compared with
the widely used forecasting-based anomaly detection algorithms,
decomposition-based algorithms, traditional statistical algorithms, as well as
recent neural network based algorithms, RobustTAD performs significantly better
on public benchmark datasets. It is deployed as a public online service and
widely adopted in different business scenarios at Alibaba Group.

negative:
Anomaly detection has been a challenging task given high-dimensional
multivariate time series data generated by networked sensors and actuators in
Cyber-Physical Systems (CPS). Besides the highly nonlinear, complex, and
dynamic natures of such time series, the lack of labeled data impedes data
exploitation in a supervised manner and thus prevents an accurate detection of
abnormal phenomenons. On the other hand, the collected data at the edge of the
network is often privacy sensitive and large in quantity, which may hinder the
centralized training at the main server. To tackle these issues, we propose an
unsupervised time series anomaly detection framework in a federated fashion to
continuously monitor the behaviors of interconnected devices within a network
and alerts for abnormal incidents so that countermeasures can be taken before
undesired consequences occur. To be specific, we leave the training data
distributed at the edge to learn a shared Variational Autoencoder (VAE) based
on Convolutional Gated Recurrent Unit (ConvGRU) model, which jointly captures
feature and temporal dependencies in the multivariate time series data for
representation learning and downstream anomaly detection tasks. Experiments on
three real-world networked sensor datasets illustrate the advantage of our
approach over other state-of-the-art models. We also conduct extensive
experiments to demonstrate the effectiveness of our detection framework under
non-federated and federated settings in terms of overall performance and
detection latency.

negative:
Cyber-physical systems (CPS) greatly benefit by using machine learning
components that can handle the uncertainty and variability of the real-world.
Typical components such as deep neural networks, however, introduce new types
of hazards that may impact system safety. The system behavior depends on data
that are available only during runtime and may be different than the data used
for training. Out-of-distribution data may lead to a large error and compromise
safety. The paper considers the problem of efficiently detecting
out-of-distribution data in CPS control systems. Detection must be robust and
limit the number of false alarms while being computational efficient for
real-time monitoring. The proposed approach leverages inductive conformal
prediction and anomaly detection for developing a method that has a
well-calibrated false alarm rate. We use variational autoencoders and deep
support vector data description to learn models that can be used efficiently
compute the nonconformity of new inputs relative to the training set and enable
real-time detection of out-of-distribution high-dimensional inputs. We
demonstrate the method using an advanced emergency braking system and a
self-driving end-to-end controller implemented in an open source simulator for
self-driving cars. The simulation results show very small number of false
positives and detection delay while the execution time is comparable to the
execution time of the original machine learning components.

negative:
A robust model for time series forecasting is highly important in many
domains, including but not limited to financial forecast, air temperature and
electricity consumption. To improve forecasting performance, traditional
approaches usually require additional feature sets. However, adding more
feature sets from different sources of data is not always feasible due to its
accessibility limitation. In this paper, we propose a novel self-boosted
mechanism in which the original time series is decomposed into multiple time
series. These time series played the role of additional features in which the
closely related time series group is used to feed into multi-task learning
model, and the loosely related group is fed into multi-view learning part to
utilize its complementary information. We use three real-world datasets to
validate our model and show the superiority of our proposed method over
existing state-of-the-art baseline methods.

negative:
This paper introduces a new methodology for detecting anomalies in time
series data, with a primary application to monitoring the health of (micro-)
services and cloud resources. The main novelty in our approach is that instead
of modeling time series consisting of real values or vectors of real values, we
model time series of probability distributions over real values (or vectors).
This extension to time series of probability distributions allows the technique
to be applied to the common scenario where the data is generated by requests
coming in to a service, which is then aggregated at a fixed temporal frequency.
Our method is amenable to streaming anomaly detection and scales to monitoring
for anomalies on millions of time series. We show the superior accuracy of our
method on synthetic and public real-world data. On the Yahoo Webscope data set,
we outperform the state of the art in 3 out of 4 data sets and we show that we
outperform popular open-source anomaly detection tools by up to 17% average
improvement for a real-world data set.

negative:
Accurate and fast simulation of particle physics processes is crucial for the
high-energy physics community. Simulating particle interactions with detectors
is both time consuming and computationally expensive. With the proton-proton
collision energy of 13 TeV, the Large Hadron Collider is uniquely positioned to
detect and measure the rare phenomena that can shape our knowledge of new
interactions. The High-Luminosity Large Hadron Collider (HL-LHC) upgrade will
put a significant strain on the computing infrastructure due to increased event
rate and levels of pile-up. Simulation of high-energy physics collisions needs
to be significantly faster without sacrificing the physics accuracy. Machine
learning approaches can offer faster solutions, while maintaining a high level
of fidelity. We discuss a graph generative model that provides effective
reconstruction of LHC events, paving the way for full detector level fast
simulation for HL-LHC.

negative:
Time-series forecasting is important for many applications. Forecasting
models are usually trained using time-series data in a specific target task.
However, sufficient data in the target task might be unavailable, which leads
to performance degradation. In this paper, we propose a few-shot learning
method that forecasts a future value of a time-series in a target task given a
few time-series in the target task. Our model is trained using time-series data
in multiple training tasks that are different from target tasks. Our model uses
a few time-series to build a forecasting function based on a recurrent neural
network with an attention mechanism. With the attention mechanism, we can
retrieve useful patterns in a small number of time-series for the current
situation. Our model is trained by minimizing an expected test error of
forecasting next timestep values. We demonstrate the effectiveness of the
proposed method using 90 time-series datasets.

negative:
In this work we propose for the first time a transformer-based framework for
unsupervised representation learning of multivariate time series. Pre-trained
models can be potentially used for downstream tasks such as regression and
classification, forecasting and missing value imputation. By evaluating our
models on several benchmark datasets for multivariate time series regression
and classification, we show that not only does our modeling approach represent
the most successful method employing unsupervised learning of multivariate time
series presented to date, but also that it exceeds the current state-of-the-art
performance of supervised methods; it does so even when the number of training
samples is very limited, while offering computational efficiency. Finally, we
demonstrate that unsupervised pre-training of our transformer models offers a
substantial performance benefit over fully supervised learning, even without
leveraging additional unlabeled data, i.e., by reusing the same data samples
through the unsupervised objective.

negative:
Event time models predict occurrence times of an event of interest based on
known features. Recent work has demonstrated that neural networks achieve
state-of-the-art event time predictions in a variety of settings. However,
standard event time models suppose that the event occurs, eventually, in all
cases. Consequently, no distinction is made between a) the probability of event
occurrence, and b) the predicted time of occurrence. This distinction is
critical when predicting medical diagnoses, equipment defects, social media
posts, and other events that or may not occur, and for which the features
affecting a) may be different from those affecting b). In this work, we develop
a conditional event time model that distinguishes between these components,
implement it as a neural network with a binary stochastic layer representing
finite event occurrence, and show how it may be learned from right-censored
event times via maximum likelihood estimation. Results demonstrate superior
event occurrence and event time predictions on synthetic data, medical events
(MIMIC-III), and social media posts (Reddit), comprising 21 total prediction
tasks.

negative:
Detecting earthquake events from seismic time series has proved itself a
challenging task. Manual detection can be expensive and tedious due to the
intensive labor and large scale data set. In recent years, automatic detection
methods based on machine learning have been developed to improve accuracy and
efficiency. However, the accuracy of those methods relies on a sufficient
amount of high-quality training data, which itself can be expensive to obtain
due to the requirement of domain knowledge and subject matter expertise. This
paper is to resolve this dilemma by answering two questions: (1) provided with
a limited number of reliable labels, can we use them to generate more synthetic
labels; (2) Can we use those synthetic labels to improve the detectability?
Among all the existing generative models, the generative adversarial network
(GAN) shows its supreme capability in generating high-quality synthetic samples
in multiple domains. We designed our model based on GAN. In particular, we
studied several different network structures. By comparing the generated
results, our GAN-based generative model yields the highest quality. We further
combine the dataset with synthetic samples generated by our generative model
and show that the detectability of our earthquake classification model is
significantly improved than the one trained without augmenting the training
set.

negative:
Most real-world datasets, and particularly those collected from physical
systems, are full of noise, packet loss, and other imperfections. However, most
specification mining, anomaly detection and other such algorithms assume, or
even require, perfect data quality to function properly. Such algorithms may
work in lab conditions when given clean, controlled data, but will fail in the
field when given imperfect data. We propose a method for accurately
reconstructing discrete temporal or sequential system traces affected by data
loss, using Long Short-Term Memory Networks (LSTMs). The model works by
learning to predict the next event in a sequence of events, and uses its own
output as an input to continue predicting future events. As a result, this
method can be used for data restoration even with streamed data. Such a method
can reconstruct even long sequence of missing events, and can also help
validate and improve data quality for noisy data. The output of the model will
be a close reconstruction of the true data, and can be fed to algorithms that
rely on clean data. We demonstrate our method by reconstructing automotive CAN
traces consisting of long sequences of discrete events. We show that given even
small parts of a CAN trace, our LSTM model can predict future events with an
accuracy of almost 90%, and can successfully reconstruct large portions of the
original trace, greatly outperforming a Markov Model benchmark. We separately
feed the original, lossy, and reconstructed traces into a specification mining
framework to perform downstream analysis of the effect of our method on
state-of-the-art models that use these traces for understanding the behavior of
complex systems.

negative:
In this paper, we present a new approach to time series forecasting. Time
series data are prevalent in many scientific and engineering disciplines. Time
series forecasting is a crucial task in modeling time series data, and is an
important area of machine learning. In this work we developed a novel method
that employs Transformer-based machine learning models to forecast time series
data. This approach works by leveraging self-attention mechanisms to learn
complex patterns and dynamics from time series data. Moreover, it is a generic
framework and can be applied to univariate and multivariate time series data,
as well as time series embeddings. Using influenza-like illness (ILI)
forecasting as a case study, we show that the forecasting results produced by
our approach are favorably comparable to the state-of-the-art.

negative:
Global Forecasting Models (GFM) that are trained across a set of multiple
time series have shown superior results in many forecasting competitions and
real-world applications compared with univariate forecasting approaches. One
aspect of the popularity of statistical forecasting models such as ETS and
ARIMA is their relative simplicity and interpretability (in terms of relevant
lags, trend, seasonality, and others), while GFMs typically lack
interpretability, especially towards particular time series. This reduces the
trust and confidence of the stakeholders when making decisions based on the
forecasts without being able to understand the predictions. To mitigate this
problem, in this work, we propose a novel local model-agnostic interpretability
approach to explain the forecasts from GFMs. We train simpler univariate
surrogate models that are considered interpretable (e.g., ETS) on the
predictions of the GFM on samples within a neighbourhood that we obtain through
bootstrapping or straightforwardly as the one-step-ahead global black-box model
forecasts of the time series which needs to be explained. After, we evaluate
the explanations for the forecasts of the global models in both qualitative and
quantitative aspects such as accuracy, fidelity, stability and
comprehensibility, and are able to show the benefits of our approach.

negative:
Time series forecasting is an important yet challenging task. Though deep
learning methods have recently been developed to give superior forecasting
results, it is crucial to improve the interpretability of time series models.
Previous interpretation methods, including the methods for general neural
networks and attention-based methods, mainly consider the interpretation in the
feature dimension while ignoring the crucial temporal dimension. In this paper,
we present the series saliency framework for temporal interpretation for
multivariate time series forecasting, which considers the forecasting
interpretation in both feature and temporal dimensions. By extracting the
"series images" from the sliding windows of the time series, we apply the
saliency map segmentation following the smallest destroying region principle.
The series saliency framework can be employed to any well-defined deep learning
models and works as a data augmentation to get more accurate forecasts.
Experimental results on several real datasets demonstrate that our framework
generates temporal interpretations for the time series forecasting task while
produces accurate time series forecast.

negative:
The prediction of residential power usage is essential in assisting a smart
grid to manage and preserve energy to ensure efficient use. An accurate energy
forecasting at the customer level will reflect directly into efficiency
improvements across the power grid system, however forecasting building energy
use is a complex task due to many influencing factors, such as meteorological
and occupancy patterns. In addiction, high-dimensional time series increasingly
arise in the Internet of Energy (IoE), given the emergence of multi-sensor
environments and the two way communication between energy consumers and the
smart grid. Therefore, methods that are capable of computing high-dimensional
time series are of great value in smart building and IoE applications. Fuzzy
Time Series (FTS) models stand out as data-driven non-parametric models of easy
implementation and high accuracy. Unfortunately, the existing FTS models can be
unfeasible if all features were used to train the model. We present a new
methodology for handling high-dimensional time series, by projecting the
original high-dimensional data into a low dimensional embedding space and using
multivariate FTS approach in this low dimensional representation. Combining
these techniques enables a better representation of the complex content of
multivariate time series and more accurate forecasts.

negative:
Modeling the spatiotemporal nature of the spread of infectious diseases can
provide useful intuition in understanding the time-varying aspect of the
disease spread and the underlying complex spatial dependency observed in
people's mobility patterns. Besides, the county level multiple related time
series information can be leveraged to make a forecast on an individual time
series. Adding to this challenge is the fact that real-time data often deviates
from the unimodal Gaussian distribution assumption and may show some complex
mixed patterns. Motivated by this, we develop a deep learning-based time-series
model for probabilistic forecasting called Auto-regressive Mixed Density
Dynamic Diffusion Network(ARM3Dnet), which considers both people's mobility and
disease spread as a diffusion process on a dynamic directed graph. The Gaussian
Mixture Model layer is implemented to consider the multimodal nature of the
real-time data while learning from multiple related time series. We show that
our model, when trained with the best combination of dynamic covariate features
and mixture components, can outperform both traditional statistical and deep
learning models in forecasting the number of Covid-19 deaths and cases at the
county level in the United States.

negative:
Some real-world domains, such as Agriculture and Healthcare, comprise
early-stage disease indications whose recording constitutes a rare event, and
yet, whose precise detection at that stage is critical. In this type of highly
imbalanced classification problems, which encompass complex features, deep
learning (DL) is much needed because of its strong detection capabilities. At
the same time, DL is observed in practice to favor majority over minority
classes and consequently suffer from inaccurate detection of the targeted
early-stage indications. To simulate such scenarios, we artificially generate
skewness (99% vs. 1%) for certain plant types out of the PlantVillage dataset
as a basis for classification of scarce visual cues through transfer learning.
By randomly and unevenly picking healthy and unhealthy samples from certain
plant types to form a training set, we consider a base experiment as
fine-tuning ResNet34 and VGG19 architectures and then testing the model
performance on a balanced dataset of healthy and unhealthy images. We
empirically observe that the initial F1 test score jumps from 0.29 to 0.95 for
the minority class upon adding a final Batch Normalization (BN) layer just
before the output layer in VGG19. We demonstrate that utilizing an additional
BN layer before the output layer in modern CNN architectures has a considerable
impact in terms of minimizing the training time and testing error for minority
classes in highly imbalanced data sets. Moreover, when the final BN is
employed, minimizing the loss function may not be the best way to assure a high
F1 test score for minority classes in such problems. That is, the network might
perform better even if it is not confident enough while making a prediction;
leading to another discussion about why softmax output is not a good
uncertainty measure for DL models.

negative:
Neural network based data-driven market simulation unveils a new and flexible
way of modelling financial time series without imposing assumptions on the
underlying stochastic dynamics. Though in this sense generative market
simulation is model-free, the concrete modelling choices are nevertheless
decisive for the features of the simulated paths. We give a brief overview of
currently used generative modelling approaches and performance evaluation
metrics for financial time series, and address some of the challenges to
achieve good results in the latter. We also contrast some classical approaches
of market simulation with simulation based on generative modelling and
highlight some advantages and pitfalls of the new approach. While most
generative models tend to rely on large amounts of training data, we present
here a generative model that works reliably in environments where the amount of
available training data is notoriously small. Furthermore, we show how a rough
paths perspective combined with a parsimonious Variational Autoencoder
framework provides a powerful way for encoding and evaluating financial time
series in such environments where available training data is scarce. Finally,
we also propose a suitable performance evaluation metric for financial time
series and discuss some connections of our Market Generator to deep hedging.

negative:
A highly comparative, feature-based approach to time series classification is
introduced that uses an extensive database of algorithms to extract thousands
of interpretable features from time series. These features are derived from
across the scientific time-series analysis literature, and include summaries of
time series in terms of their correlation structure, distribution, entropy,
stationarity, scaling properties, and fits to a range of time-series models.
After computing thousands of features for each time series in a training set,
those that are most informative of the class structure are selected using
greedy forward feature selection with a linear classifier. The resulting
feature-based classifiers automatically learn the differences between classes
using a reduced number of time-series properties, and circumvent the need to
calculate distances between time series. Representing time series in this way
results in orders of magnitude of dimensionality reduction, allowing the method
to perform well on very large datasets containing long time series or time
series of different lengths. For many of the datasets studied, classification
performance exceeded that of conventional instance-based classifiers, including
one nearest neighbor classifiers using Euclidean distances and dynamic time
warping and, most importantly, the features selected provide an understanding
of the properties of the dataset, insight that can guide further scientific
investigation.

negative:
In the current context of Big Data, the nature of many forecasting problems
has changed from predicting isolated time series to predicting many time series
from similar sources. This has opened up the opportunity to develop competitive
global forecasting models that simultaneously learn from many time series. But,
it still remains unclear when global forecasting models can outperform the
univariate benchmarks, especially along the dimensions of the
homogeneity/heterogeneity of series, the complexity of patterns in the series,
the complexity of forecasting models, and the lengths/number of series. Our
study attempts to address this problem through investigating the effect from
these factors, by simulating a number of datasets that have controllable time
series characteristics. Specifically, we simulate time series from simple data
generating processes (DGP), such as Auto Regressive (AR) and Seasonal AR, to
complex DGPs, such as Chaotic Logistic Map, Self-Exciting Threshold
Auto-Regressive, and Mackey-Glass Equations. The data heterogeneity is
introduced by mixing time series generated from several DGPs into a single
dataset. The lengths and the number of series in the dataset are varied in
different scenarios. We perform experiments on these datasets using global
forecasting models including Recurrent Neural Networks (RNN), Feed-Forward
Neural Networks, Pooled Regression (PR) models and Light Gradient Boosting
Models (LGBM), and compare their performance against standard statistical
univariate forecasting techniques. Our experiments demonstrate that when
trained as global forecasting models, techniques such as RNNs and LGBMs, which
have complex non-linear modelling capabilities, are competitive methods in
general under challenging forecasting scenarios such as series having short
lengths, datasets with heterogeneous series and having minimal prior knowledge
of the patterns of the series.

negative:
For most diseases, building large databases of labeled genetic data is an
expensive and time-demanding task. To address this, we introduce genetic
Generative Adversarial Networks (gGAN), a semi-supervised approach based on an
innovative GAN architecture to create large synthetic genetic data sets
starting with a small amount of labeled data and a large amount of unlabeled
data. Our goal is to determine the propensity of a new individual to develop
the severe form of the illness from their genetic profile alone. The proposed
model achieved satisfactory results using real genetic data from different
datasets and populations, in which the test populations may not have the same
genetic profiles. The proposed model is self-aware and capable of determining
whether a new genetic profile has enough compatibility with the data on which
the network was trained and is thus suitable for prediction. The code and
datasets used can be found at https://github.com/caio-davi/gGAN.

negative:
Automatic event detection from time series signals has wide applications,
such as abnormal event detection in video surveillance and event detection in
geophysical data. Traditional detection methods detect events primarily by the
use of similarity and correlation in data. Those methods can be inefficient and
yield low accuracy. In recent years, because of the significantly increased
computational power, machine learning techniques have revolutionized many
science and engineering domains. In this study, we apply a deep-learning-based
method to the detection of events from time series seismic signals. However, a
direct adaptation of the similar ideas from 2D object detection to our problem
faces two challenges. The first challenge is that the duration of earthquake
event varies significantly; The other is that the proposals generated are
temporally correlated. To address these challenges, we propose a novel cascaded
region-based convolutional neural network to capture earthquake events in
different sizes, while incorporating contextual information to enrich features
for each individual proposal. To achieve a better generalization performance,
we use densely connected blocks as the backbone of our network. Because of the
fact that some positive events are not correctly annotated, we further
formulate the detection problem as a learning-from-noise problem. To verify the
performance of our detection methods, we employ our methods to seismic data
generated from a bi-axial "earthquake machine" located at Rock Mechanics
Laboratory, and we acquire labels with the help of experts. Through our
numerical tests, we show that our novel detection techniques yield high
accuracy. Therefore, our novel deep-learning-based detection methods can
potentially be powerful tools for locating events from time series data in
various applications.

negative:
The availability of large amounts of time series data, paired with the
performance of deep-learning algorithms on a broad class of problems, has
recently led to significant interest in the use of sequence-to-sequence models
for time series forecasting. We provide the first theoretical analysis of this
time series forecasting framework. We include a comparison of
sequence-to-sequence modeling to classical time series models, and as such our
theory can serve as a quantitative guide for practitioners choosing between
different modeling methodologies.

negative:
Autonomous robots need to be able to adapt to unforeseen situations and to
acquire new skills through trial and error. Reinforcement learning in principle
offers a suitable methodological framework for this kind of autonomous
learning. However current computational reinforcement learning agents mostly
learn each individual skill entirely from scratch. How can we enable artificial
agents, such as robots, to acquire some form of generic knowledge, which they
could leverage for the learning of new skills? This paper argues that, like the
brain, the cognitive system of artificial agents has to develop a world model
to support adaptive behavior and learning. Inspiration is taken from two recent
developments in the cognitive science literature: predictive processing
theories of cognition, and the sensorimotor contingencies theory of perception.
Based on these, a hypothesis is formulated about what the content of
information might be that is encoded in an internal world model, and how an
agent could autonomously acquire it. A computational model is described to
formalize this hypothesis, and is evaluated in a series of simulation
experiments.

negative:
Feature-based time series representations have attracted substantial
attention in a wide range of time series analysis methods. Recently, the use of
time series features for forecast model averaging has been an emerging research
focus in the forecasting community. Nonetheless, most of the existing
approaches depend on the manual choice of an appropriate set of features.
Exploiting machine learning methods to extract features from time series
automatically becomes crucial in state-of-the-art time series analysis. In this
paper, we introduce an automated approach to extract time series features based
on time series imaging. We first transform time series into recurrence plots,
from which local features can be extracted using computer vision algorithms.
The extracted features are used for forecast model averaging. Our experiments
show that forecasting based on automatically extracted features, with less
human intervention and a more comprehensive view of the raw time series data,
yields highly comparable performances with the best methods in the largest
forecasting competition dataset (M4) and outperforms the top methods in the
Tourism forecasting competition dataset.

negative:
This paper presents a machine learning approach to model the electric
consumption of electric vehicles at macroscopic level, i.e., in the absence of
a speed profile, while preserving microscopic level accuracy. For this work, we
leveraged a high-performance, agent-based transportation tool to model trips
that occur in the Greater Chicago region under various scenario changes, along
with physics-based modeling and simulation tools to provide high-fidelity
energy consumption values. The generated results constitute a very large
dataset of vehicle-route energy outcomes that capture variability in vehicle
and routing setting, and in which high-fidelity time series of vehicle speed
dynamics is masked. We show that although all internal dynamics that affect
energy consumption are masked, it is possible to learn aggregate-level energy
consumption values quite accurately with a deep learning approach. When
large-scale data is available, and with carefully tailored feature engineering,
a well-designed model can overcome and retrieve latent information. This model
has been deployed and integrated within POLARIS Transportation System
Simulation Tool to support real-time behavioral transportation models for
individual charging decision-making, and rerouting of electric vehicles.

negative:
Fuzzy time series forecasting methods are very popular among researchers for
predicting future values as they are not based on the strict assumptions of
traditional time series forecasting methods. Non-stochastic methods of fuzzy
time series forecasting are preferred by the researchers as they provide more
significant forecasting results. There are generally, four factors that
determine the performance of the forecasting method (1) number of intervals
(NOIs) and length of intervals to partition universe of discourse (UOD) (2)
fuzzification rules or feature representation of crisp time series (3) method
of establishing fuzzy logic rule (FLRs) between input and target values (4)
defuzzification rule to get crisp forecasted value. Considering the first two
factors to improve the forecasting accuracy, we proposed a novel non-stochastic
method fuzzy time series forecasting in which interval index number and
membership value are used as input features to predict future value. We
suggested a simple rounding-off range and suitable step size method to find the
optimal number of intervals (NOIs) and used fuzzy c-means clustering process to
divide UOD into intervals of unequal length. We implement support vector
machine (SVM) to establish FLRs. To test our proposed method we conduct a
simulated study on five widely used real time series and compare the
performance with some recently developed models. We also examine the
performance of the proposed model by using multi-layer perceptron (MLP) instead
of SVM. Two performance measures RSME and SMAPE are used for performance
analysis and observed better forecasting accuracy by the proposed model.

negative:
The field of predictive process monitoring focuses on case-level models to
predict a single specific outcome such as a particular objective, (remaining)
time, or next activity/remaining sequence. Recently, a longer-horizon,
model-wide approach has been proposed in the form of process model forecasting,
which predicts the future state of a whole process model through the
forecasting of all activity-to-activity relations at once using time series
forecasting.
  This paper introduces the concept of \emph{predictive process model
monitoring} which sits in the middle of both predictive process monitoring and
process model forecasting. Concretely, by modelling a process model as a set of
constraints being present between activities over time, we can capture more
detailed information between activities compared to process model forecasting,
while being compatible with typical predictive process monitoring objectives
which are often expressed in the same language as these constraints. To achieve
this, Processes-As-Movies (PAM) is introduced, i.e., a novel technique capable
of jointly mining and predicting declarative process constraints between
activities in various windows of a process' execution. PAM predicts what
declarative rules hold for a trace (objective-based), which also supports the
prediction of all constraints together as a process model (model-based).
Various recurrent neural network topologies inspired by video analysis tailored
to temporal high-dimensional input are used to model the process model
evolution with windows as time steps, including encoder-decoder long short-term
memory networks, and convolutional long short-term memory networks. Results
obtained over real-life event logs show that these topologies are effective in
terms of predictive accuracy and precision.

negative:
Deep learning performs remarkably well on many time series analysis tasks
recently. The superior performance of deep neural networks relies heavily on a
large number of training data to avoid overfitting. However, the labeled data
of many real-world time series applications may be limited such as
classification in medical time series and anomaly detection in AIOps. As an
effective way to enhance the size and quality of the training data, data
augmentation is crucial to the successful application of deep learning models
on time series data. In this paper, we systematically review different data
augmentation methods for time series. We propose a taxonomy for the reviewed
methods, and then provide a structured review for these methods by highlighting
their strengths and limitations. We also empirically compare different data
augmentation methods for different tasks including time series classification,
anomaly detection, and forecasting. Finally, we discuss and highlight five
future directions to provide useful research guidance.

negative:
We participated in the M4 competition for time series forecasting and
describe here our methods for forecasting daily time series. We used an
ensemble of five statistical forecasting methods and a method that we refer to
as the correlator. Our retrospective analysis using the ground truth values
published by the M4 organisers after the competition demonstrates that the
correlator was responsible for most of our gains over the naive constant
forecasting method. We identify data leakage as one reason for its success,
partly due to test data selected from different time intervals, and partly due
to quality issues in the original time series. We suggest that future
forecasting competitions should provide actual dates for the time series so
that some of those leakages could be avoided by the participants.

negative:
Evaluating the reliability of intelligent physical systems against rare
safety-critical events poses a huge testing burden for real-world applications.
Simulation provides a useful platform to evaluate the extremal risks of these
systems before their deployments. Importance Sampling (IS), while proven to be
powerful for rare-event simulation, faces challenges in handling these
learning-based systems due to their black-box nature that fundamentally
undermines its efficiency guarantee, which can lead to under-estimation without
diagnostically detected. We propose a framework called Deep Probabilistic
Accelerated Evaluation (Deep-PrAE) to design statistically guaranteed IS, by
converting black-box samplers that are versatile but could lack guarantees,
into one with what we call a relaxed efficiency certificate that allows
accurate estimation of bounds on the safety-critical event probability. We
present the theory of Deep-PrAE that combines the dominating point concept with
rare-event set learning via deep neural network classifiers, and demonstrate
its effectiveness in numerical examples including the safety-testing of an
intelligent driving algorithm.

negative:
Time Series Forecasting is at the core of many practical applications such as
sales forecasting for business, rainfall forecasting for agriculture and many
others. Though this problem has been extensively studied for years, it is still
considered a challenging problem due to complex and evolving nature of time
series data. Typical methods proposed for time series forecasting modeled
linear or non-linear dependencies between data observations. However it is a
generally accepted notion that no one method is universally effective for all
kinds of time series data. Attempts have been made to use dynamic and weighted
combination of heterogeneous and independent forecasting models and it has been
found to be a promising direction to tackle this problem. This method is based
on the assumption that different forecasters have different specialization and
varying performance for different distribution of data and weights are
dynamically assigned to multiple forecasters accordingly. However in many
practical time series data-set, the distribution of data slowly evolves with
time. We propose to employ a re-weighting based method to adjust the assigned
weights to various forecasters in order to account for such distribution-drift.
An exhaustive testing was performed against both real-world and synthesized
time-series. Experimental results show the competitiveness of the method in
comparison to state-of-the-art approaches for combining forecasters and
handling drift.

negative:
We present Earliness-Aware Deep Convolutional Networks (EA-ConvNets), an
end-to-end deep learning framework, for early classification of time series
data. Unlike most existing methods for early classification of time series
data, that are designed to solve this problem under the assumption of the
availability of a good set of pre-defined (often hand-crafted) features, our
framework can jointly perform feature learning (by learning a deep hierarchy of
\emph{shapelets} capturing the salient characteristics in each time series),
along with a dynamic truncation model to help our deep feature learning
architecture focus on the early parts of each time series. Consequently, our
framework is able to make highly reliable early predictions, outperforming
various state-of-the-art methods for early time series classification, while
also being competitive when compared to the state-of-the-art time series
classification algorithms that work with \emph{fully observed} time series
data. To the best of our knowledge, the proposed framework is the first to
perform data-driven (deep) feature learning in the context of early
classification of time series data. We perform a comprehensive set of
experiments, on several benchmark data sets, which demonstrate that our method
yields significantly better predictions than various state-of-the-art methods
designed for early time series classification. In addition to obtaining high
accuracies, our experiments also show that the learned deep shapelets based
features are also highly interpretable and can help gain better understanding
of the underlying characteristics of time series data.

negative:
Performance and high availability have become increasingly important drivers,
amongst other drivers, for user retention in the context of web services such
as social networks, and web search. Exogenic and/or endogenic factors often
give rise to anomalies, making it very challenging to maintain high
availability, while also delivering high performance. Given that
service-oriented architectures (SOA) typically have a large number of services,
with each service having a large set of metrics, automatic detection of
anomalies is non-trivial.
  Although there exists a large body of prior research in anomaly detection,
existing techniques are not applicable in the context of social network data,
owing to the inherent seasonal and trend components in the time series data.
  To this end, we developed two novel statistical techniques for automatically
detecting anomalies in cloud infrastructure data. Specifically, the techniques
employ statistical learning to detect anomalies in both application, and system
metrics. Seasonal decomposition is employed to filter the trend and seasonal
components of the time series, followed by the use of robust statistical
metrics -- median and median absolute deviation (MAD) -- to accurately detect
anomalies, even in the presence of seasonal spikes.
  We demonstrate the efficacy of the proposed techniques from three different
perspectives, viz., capacity planning, user behavior, and supervised learning.
In particular, we used production data for evaluation, and we report Precision,
Recall, and F-measure in each case.

negative:
Time series forecasting involves collecting and analyzing past observations
to develop a model to extrapolate such observations into the future.
Forecasting of future events is important in many fields to support decision
making as it contributes to reducing the future uncertainty. We propose
explainable boosted linear regression (EBLR) algorithm for time series
forecasting, which is an iterative method that starts with a base model, and
explains the model's errors through regression trees. At each iteration, the
path leading to highest error is added as a new variable to the base model. In
this regard, our approach can be considered as an improvement over general time
series models since it enables incorporating nonlinear features by residuals
explanation. More importantly, use of the single rule that contributes to the
error most allows for interpretable results. The proposed approach extends to
probabilistic forecasting through generating prediction intervals based on the
empirical error distribution. We conduct a detailed numerical study with EBLR
and compare against various other approaches. We observe that EBLR
substantially improves the base model performance through extracted features,
and provide a comparable performance to other well established approaches. The
interpretability of the model predictions and high predictive accuracy of EBLR
makes it a promising method for time series forecasting.

negative:
Classical anomaly detection is principally concerned with point-based
anomalies, those anomalies that occur at a single point in time. Yet, many
real-world anomalies are range-based, meaning they occur over a period of time.
Motivated by this observation, we present a new mathematical model to evaluate
the accuracy of time series classification algorithms. Our model expands the
well-known Precision and Recall metrics to measure ranges, while simultaneously
enabling customization support for domain-specific preferences.

negative:
Choosing the technique that is the best at forecasting your data, is a
problem that arises in any forecasting application. Decades of research have
resulted into an enormous amount of forecasting methods that stem from
statistics, econometrics and machine learning (ML), which leads to a very
difficult and elaborate choice to make in any forecasting exercise. This paper
aims to facilitate this process for high-level tactical sales forecasts by
comparing a large array of techniques for 35 times series that consist of both
industry data from the Coca-Cola Company and publicly available datasets.
However, instead of solely focusing on the accuracy of the resulting forecasts,
this paper introduces a novel and completely automated profit-driven approach
that takes into account the expected profit that a technique can create during
both the model building and evaluation process. The expected profit function
that is used for this purpose, is easy to understand and adaptable to any
situation by combining forecasting accuracy with business expertise.
Furthermore, we examine the added value of ML techniques, the inclusion of
external factors and the use of seasonal models in order to ascertain which
type of model works best in tactical sales forecasting. Our findings show that
simple seasonal time series models consistently outperform other methodologies
and that the profit-driven approach can lead to selecting a different
forecasting model.

negative:
Recent advances in AIoT technologies have led to an increasing popularity of
utilizing machine learning algorithms to detect operational failures for
cyber-physical systems (CPS). In its basic form, an anomaly detection module
monitors the sensor measurements and actuator states from the physical plant,
and detects anomalies in these measurements to identify abnormal operation
status. Nevertheless, building effective anomaly detection models for CPS is
rather challenging as the model has to accurately detect anomalies in presence
of highly complicated system dynamics and unknown amount of sensor noise. In
this work, we propose a novel time series anomaly detection method called
Neural System Identification and Bayesian Filtering (NSIBF) in which a
specially crafted neural network architecture is posed for system
identification, i.e., capturing the dynamics of CPS in a dynamical state-space
model; then a Bayesian filtering algorithm is naturally applied on top of the
"identified" state-space model for robust anomaly detection by tracking the
uncertainty of the hidden state of the system recursively over time. We provide
qualitative as well as quantitative experiments with the proposed method on a
synthetic and three real-world CPS datasets, showing that NSIBF compares
favorably to the state-of-the-art methods with considerable improvements on
anomaly detection in CPS.

negative:
Several techniques for multivariate time series anomaly detection have been
proposed recently, but a systematic comparison on a common set of datasets and
metrics is lacking. This paper presents a systematic and comprehensive
evaluation of unsupervised and semi-supervised deep-learning based methods for
anomaly detection and diagnosis on multivariate time series data from
cyberphysical systems. Unlike previous works, we vary the model and
post-processing of model errors, i.e. the scoring functions independently of
each other, through a grid of 10 models and 4 scoring functions, comparing
these variants to state of the art methods. In time-series anomaly detection,
detecting anomalous events is more important than detecting individual
anomalous time-points. Through experiments, we find that the existing
evaluation metrics either do not take events into account, or cannot
distinguish between a good detector and trivial detectors, such as a random or
an all-positive detector. We propose a new metric to overcome these drawbacks,
namely, the composite F-score ($Fc_1$), for evaluating time-series anomaly
detection.
  Our study highlights that dynamic scoring functions work much better than
static ones for multivariate time series anomaly detection, and the choice of
scoring functions often matters more than the choice of the underlying model.
We also find that a simple, channel-wise model - the Univariate Fully-Connected
Auto-Encoder, with the dynamic Gaussian scoring function emerges as a winning
candidate for both anomaly detection and diagnosis, beating state of the art
algorithms.

negative:
Mechanical devices such as engines, vehicles, aircrafts, etc., are typically
instrumented with numerous sensors to capture the behavior and health of the
machine. However, there are often external factors or variables which are not
captured by sensors leading to time-series which are inherently unpredictable.
For instance, manual controls and/or unmonitored environmental conditions or
load may lead to inherently unpredictable time-series. Detecting anomalies in
such scenarios becomes challenging using standard approaches based on
mathematical models that rely on stationarity, or prediction models that
utilize prediction errors to detect anomalies. We propose a Long Short Term
Memory Networks based Encoder-Decoder scheme for Anomaly Detection (EncDec-AD)
that learns to reconstruct 'normal' time-series behavior, and thereafter uses
reconstruction error to detect anomalies. We experiment with three publicly
available quasi predictable time-series datasets: power demand, space shuttle,
and ECG, and two real-world engine datasets with both predictive and
unpredictable behavior. We show that EncDec-AD is robust and can detect
anomalies from predictable, unpredictable, periodic, aperiodic, and
quasi-periodic time-series. Further, we show that EncDec-AD is able to detect
anomalies from short time-series (length as small as 30) as well as long
time-series (length as large as 500).

negative:
Many software engineering tasks, such as testing, and anomaly detection can
benefit from the ability to infer a behavioral model of the software.Most
existing inference approaches assume access to code to collect execution
sequences. In this paper, we investigate a black-box scenario, where the system
under analysis cannot be instrumented, in this granular fashion.This scenario
is particularly prevalent with control systems' log analysis in the form of
continuous signals. In this situation, an execution trace amounts to a
multivariate time-series of input and output signals, where different states of
the system correspond to different `phases` in the time-series. The main
challenge is to detect when these phase changes take place. Unfortunately, most
existing solutions are either univariate, make assumptions on the data
distribution, or have limited learning power.Therefore, we propose a hybrid
deep neural network that accepts as input a multivariate time series and
applies a set of convolutional and recurrent layers to learn the non-linear
correlations between signals and the patterns over time.We show how this
approach can be used to accurately detect state changes, and how the inferred
models can be successfully applied to transfer-learning scenarios, to
accurately process traces from different products with similar execution
characteristics. Our experimental results on two UAV autopilot case studies
indicate that our approach is highly accurate (over 90% F1 score for state
classification) and significantly improves baselines (by up to 102% for change
point detection).Using transfer learning we also show that up to 90% of the
maximum achievable F1 scores in the open-source case study can be achieved by
reusing the trained models from the industrial case and only fine tuning them
using as low as 5 labeled samples, which reduces the manual labeling effort by
98%.

negative:
Predicting disaster events from seismic data is of paramount importance and
can save thousands of lives, especially in earthquake-prone areas and
habitations around volcanic craters. The drastic rise in the number of seismic
monitoring stations in recent years has allowed the collection of a huge
quantity of data, outpacing the capacity of seismologists. Due to the complex
nature of the seismological data, it is often difficult for seismologists to
detect subtle patterns with major implications. Machine learning algorithms
have been demonstrated to be effective in classification and prediction tasks
for seismic data. It has been widely known that some animals can sense
disasters like earthquakes from seismic signals well before the disaster
strikes. Mel spectrogram has been widely used for speech recognition as it
scales the actual frequencies according to human hearing. In this paper, we
propose a variant of the Mel spectrogram to scale the raw frequencies of
seismic data to the hearing of such animals that can sense disasters from
seismic signals. We are using a Computer vision algorithm along with clustering
that allows for the classification of unlabelled seismic data.

negative:
The explosion of time series data in recent years has brought a flourish of
new time series analysis methods, for forecasting, clustering, classification
and other tasks. The evaluation of these new methods requires either collecting
or simulating a diverse set of time series benchmarking data to enable reliable
comparisons against alternative approaches. We propose GeneRAting TIme Series
with diverse and controllable characteristics, named GRATIS, with the use of
mixture autoregressive (MAR) models. We simulate sets of time series using MAR
models and investigate the diversity and coverage of the generated time series
in a time series feature space. By tuning the parameters of the MAR models,
GRATIS is also able to efficiently generate new time series with controllable
features. In general, as a costless surrogate to the traditional data
collection approach, GRATIS can be used as an evaluation tool for tasks such as
time series forecasting and classification. We illustrate the usefulness of our
time series generation process through a time series forecasting application.

negative:
Autoencoders have been proposed as a powerful tool for model-independent
anomaly detection in high-energy physics. The operating principle is that
events which do not belong to the space of training data will be reconstructed
poorly, thus flagging them as anomalies. We point out that in a variety of
examples of interest, the connection between large reconstruction error and
anomalies is not so clear. In particular, for data sets with nontrivial
topology, there will always be points that erroneously seem anomalous due to
global issues. Conversely, neural networks typically have an inductive bias or
prior to locally interpolate such that undersampled or rare events may be
reconstructed with small error, despite actually being the desired anomalies.
Taken together, these facts are in tension with the simple picture of the
autoencoder as an anomaly detector. Using a series of illustrative
low-dimensional examples, we show explicitly how the intrinsic and extrinsic
topology of the dataset affects the behavior of an autoencoder and how this
topology is manifested in the latent space representation during training. We
ground this analysis in the discussion of a mock "bump hunt" in which the
autoencoder fails to identify an anomalous "signal" for reasons tied to the
intrinsic topology of $n$-particle phase space.

negative:
I describe the rationale for, and design of, an agent-based simulation model
of a contemporary online sports-betting exchange: such exchanges, closely
related to the exchange mechanisms at the heart of major financial markets,
have revolutionized the gambling industry in the past 20 years, but gathering
sufficiently large quantities of rich and temporally high-resolution data from
real exchanges - i.e., the sort of data that is needed in large quantities for
Deep Learning - is often very expensive, and sometimes simply impossible; this
creates a need for a plausibly realistic synthetic data generator, which is
what this simulation now provides. The simulator, named the "Bristol Betting
Exchange" (BBE), is intended as a common platform, a data-source and
experimental test-bed, for researchers studying the application of AI and
machine learning (ML) techniques to issues arising in betting exchanges; and,
as far as I have been able to determine, BBE is the first of its kind: a free
open-source agent-based simulation model consisting not only of a
sports-betting exchange, but also a minimal simulation model of racetrack
sporting events (e.g., horse-races or car-races) about which bets may be made,
and a population of simulated bettors who each form their own private
evaluation of odds and place bets on the exchange before and - crucially -
during the race itself (i.e., so-called "in-play" betting) and whose betting
opinions change second-by-second as each race event unfolds. BBE is offered as
a proof-of-concept system that enables the generation of large high-resolution
data-sets for automated discovery or improvement of profitable strategies for
betting on sporting events via the application of AI/ML and advanced data
analytics techniques. This paper offers an extensive survey of relevant
literature and explains the motivation and design of BBE, and presents brief
illustrative results.

negative:
A time series represents a set of observations collected over time.
Typically, these observations are captured with a uniform sampling frequency
(e.g. daily). When data points are observed in uneven time intervals the time
series is referred to as irregular or intermittent. In such scenarios, the most
common solution is to reconstruct the time series to make it regular, thus
removing its intermittency. We hypothesise that, in irregular time series, the
time at which each observation is collected may be helpful to summarise the
dynamics of the data and improve forecasting performance. We study this idea by
developing a novel automatic feature engineering framework, which focuses on
extracting information from this point of view, i.e., when each instance is
collected. We study how valuable this information is by integrating it in a
time series forecasting workflow and investigate how it compares to or
complements state-of-the-art methods for regular time series forecasting. In
the end, we contribute by providing a novel framework that tackles feature
engineering for time series from an angle previously vastly ignored. We show
that our approach has the potential to further extract more information about
time series that significantly improves forecasting performance.

negative:
Time series data account for a major part of data supply available today.
Time series mining handles several tasks such as classification, clustering,
query-by-content, prediction, and others. Performing data mining tasks on raw
time series is inefficient as these data are high-dimensional by nature.
Instead, time series are first pre-processed using several techniques before
different data mining tasks can be performed on them. In general, there are two
main approaches to reduce time series dimensionality, the first is what we call
landmark methods. These methods are based on finding characteristic features in
the target time series. The second is based on data transformations. These
methods transform the time series from the original space into a reduced space,
where they can be managed more efficiently. The method we present in this paper
applies a third approach, as it projects a time series onto a lower-dimensional
space by selecting important points in the time series. The novelty of our
method is that these points are not chosen according to a geometric criterion,
which is subjective in most cases, but through an optimization process. The
other important characteristic of our method is that these important points are
selected on a dataset-level and not on a single time series-level. The direct
advantage of this strategy is that the distance defined on the low-dimensional
space lower bounds the original distance applied to raw data. This enables us
to apply the popular GEMINI algorithm. The promising results of our experiments
on a wide variety of time series datasets, using different optimizers, and
applied to the two major data mining tasks, validate our new method.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Is critical input information encoded in specific sparse pathways within the
neural network? In this work, we discuss the problem of identifying these
critical pathways and subsequently leverage them for interpreting the network's
response to an input. The pruning objective -- selecting the smallest group of
neurons for which the response remains equivalent to the original network --
has been previously proposed for identifying critical pathways. We demonstrate
that sparse pathways derived from pruning do not necessarily encode critical
input information. To ensure sparse pathways include critical fragments of the
encoded input information, we propose pathway selection via neurons'
contribution to the response. We proceed to explain how critical pathways can
reveal critical input features. We prove that pathways selected via neuron
contribution are locally linear (in an L2-ball), a property that we use for
proposing a feature attribution method: "pathway gradient". We validate our
interpretation method using mainstream evaluation experiments. The validation
of pathway gradient interpretation method further confirms that selected
pathways using neuron contributions correspond to critical input features. The
code is publicly available.

negative:
With the sweeping digitalization of societal, medical, industrial, and
scientific processes, sensing technologies are being deployed that produce
increasing volumes of time series data, thus fueling a plethora of new or
improved applications. In this setting, outlier detection is frequently
important, and while solutions based on neural networks exist, they leave room
for improvement in terms of both accuracy and efficiency. With the objective of
achieving such improvements, we propose a diversity-driven, convolutional
ensemble. To improve accuracy, the ensemble employs multiple basic outlier
detection models built on convolutional sequence-to-sequence autoencoders that
can capture temporal dependencies in time series. Further, a novel
diversity-driven training method maintains diversity among the basic models,
with the aim of improving the ensemble's accuracy. To improve efficiency, the
approach enables a high degree of parallelism during training. In addition, it
is able to transfer some model parameters from one basic model to another,
which reduces training time. We report on extensive experiments using
real-world multivariate time series that offer insight into the design choices
underlying the new approach and offer evidence that it is capable of improved
accuracy and efficiency. This is an extended version of "Unsupervised Time
Series Outlier Detection with Diversity-Driven Convolutional Ensembles", to
appear in PVLDB 2022.

negative:
The emergence of continuous health monitoring and the availability of an
enormous amount of time series data has provided a great opportunity for the
advancement of personal health tracking. In recent years, unsupervised learning
methods have drawn special attention of researchers to tackle the sparse
annotation of health data and real-time detection of anomalies has been a
central problem of interest. However, one problem that has not been well
addressed before is the early prediction of forthcoming negative health events.
Early signs of an event can introduce subtle and gradual changes in the health
signal prior to its onset, detection of which can be invaluable in effective
prevention. In this study, we first demonstrate our observations on the
shortcoming of widely adopted anomaly detection methods in uncovering the
changes prior to a negative health event. We then propose a framework which
relies on online clustering of signal segment representations which are
automatically learned by a specially designed LSTM auto-encoder. We show the
effectiveness of our approach by predicting Bradycardia events in infants using
MIT-PICS dataset 1.3 minutes ahead of time with 68\% AUC score on average,
using no label supervision. Results of our study can indicate the viability of
our approach in the early detection of health events in other applications as
well.

negative:
Extreme events are occurrences whose magnitude and potential cause extensive
damage on people, infrastructure, and the environment. Motivated by the extreme
nature of the current global health landscape, which is plagued by the
coronavirus pandemic, we seek to better understand and model extreme events.
Modeling extreme events is common in practice and plays an important role in
time-series prediction applications. Our goal is to (i) compare and investigate
the effect of some common extreme events modeling methods to explore which
method can be practical in reality and (ii) accelerate the deep learning
training process, which commonly uses deep recurrent neural network (RNN), by
implementing the asynchronous local Stochastic Gradient Descent (SGD) framework
among multiple compute nodes. In order to verify our distributed extreme events
modeling, we evaluate our proposed framework on a stock data set S\&P500, with
a standard recurrent neural network. Our intuition is to explore the (best)
extreme events modeling method which could work well under the distributed deep
learning setting. Moreover, by using asynchronous distributed learning, we aim
to significantly reduce the communication cost among the compute nodes and
central server, which is the main bottleneck of almost all distributed learning
frameworks.
  We implement our proposed work and evaluate its performance on representative
data sets, such as S&P500 stock in $5$-year period. The experimental results
validate the correctness of the design principle and show a significant
training duration reduction upto $8$x, compared to the baseline single compute
node. Our results also show that our proposed work can achieve the same level
of test accuracy, compared to the baseline setting.

negative:
Interactive response time is important in analytical pipelines for users to
explore a sufficient number of possibilities and make informed business
decisions. We consider a forecasting pipeline with large volumes of
high-dimensional time series data. Real-time forecasting can be conducted in
two steps. First, we specify the part of data to be focused on and the measure
to be predicted by slicing, dicing, and aggregating the data. Second, a
forecasting model is trained on the aggregated results to predict the trend of
the specified measure. While there are a number of forecasting models
available, the first step is the performance bottleneck. A natural idea is to
utilize sampling to obtain approximate aggregations in real time as the input
to train the forecasting model. Our scalable real-time forecasting system
FlashP (Flash Prediction) is built based on this idea, with two major
challenges to be resolved in this paper: first, we need to figure out how
approximate aggregations affect the fitting of forecasting models, and
forecasting results; and second, accordingly, what sampling algorithms we
should use to obtain these approximate aggregations and how large the samples
are. We introduce a new sampling scheme, called GSW sampling, and analyze error
bounds for estimating aggregations using GSW samples. We introduce how to
construct compact GSW samples with the existence of multiple measures to be
analyzed. We conduct experiments to evaluate our solution and compare it with
alternatives on real data.

negative:
Deep Learning (DL) models can be used to tackle time series analysis tasks
with great success. However, the performance of DL models can degenerate
rapidly if the data are not appropriately normalized. This issue is even more
apparent when DL is used for financial time series forecasting tasks, where the
non-stationary and multimodal nature of the data pose significant challenges
and severely affect the performance of DL models. In this work, a simple, yet
effective, neural layer, that is capable of adaptively normalizing the input
time series, while taking into account the distribution of the data, is
proposed. The proposed layer is trained in an end-to-end fashion using
back-propagation and leads to significant performance improvements compared to
other evaluated normalization schemes. The proposed method differs from
traditional normalization methods since it learns how to perform normalization
for a given task instead of using a fixed normalization scheme. At the same
time, it can be directly applied to any new time series without requiring
re-training. The effectiveness of the proposed method is demonstrated using a
large-scale limit order book dataset, as well as a load forecasting dataset.

negative:
Time series forecasting is a crucial component of many important
applications, ranging from forecasting the stock markets to energy load
prediction. The high-dimensionality, velocity and variety of the data collected
in these applications pose significant and unique challenges that must be
carefully addressed for each of them. In this work, a novel Temporal Logistic
Neural Bag-of-Features approach, that can be used to tackle these challenges,
is proposed. The proposed method can be effectively combined with deep neural
networks, leading to powerful deep learning models for time series analysis.
However, combining existing BoF formulations with deep feature extractors pose
significant challenges: the distribution of the input features is not
stationary, tuning the hyper-parameters of the model can be especially
difficult and the normalizations involved in the BoF model can cause
significant instabilities during the training process. The proposed method is
capable of overcoming these limitations by a employing a novel adaptive scaling
mechanism and replacing the classical Gaussian-based density estimation
involved in the regular BoF model with a logistic kernel. The effectiveness of
the proposed approach is demonstrated using extensive experiments on a
large-scale financial time series dataset that consists of more than 4 million
limit orders.

negative:
A central server needs to perform statistical inference based on samples that
are distributed over multiple users who can each send a message of limited
length to the center. We study problems of distribution learning and identity
testing in this distributed inference setting and examine the role of shared
randomness as a resource. We propose a general-purpose simulate-and-infer
strategy that uses only private-coin communication protocols and is
sample-optimal for distribution learning. This general strategy turns out to be
sample-optimal even for distribution testing among private-coin protocols.
Interestingly, we propose a public-coin protocol that outperforms
simulate-and-infer for distribution testing and is, in fact, sample-optimal.
Underlying our public-coin protocol is a random hash that when applied to the
samples minimally contracts the chi-squared distance of their distribution to
the uniform distribution.

negative:
The ability to perform causal and counterfactual reasoning are central
properties of human intelligence. Decision-making systems that can perform
these types of reasoning have the potential to be more generalizable and
interpretable. Simulations have helped advance the state-of-the-art in this
domain, by providing the ability to systematically vary parameters (e.g.,
confounders) and generate examples of the outcomes in the case of
counterfactual scenarios. However, simulating complex temporal causal events in
multi-agent scenarios, such as those that exist in driving and vehicle
navigation, is challenging. To help address this, we present a high-fidelity
simulation environment that is designed for developing algorithms for causal
discovery and counterfactual reasoning in the safety-critical context. A core
component of our work is to introduce \textit{agency}, such that it is simple
to define and create complex scenarios using high-level definitions. The
vehicles then operate with agency to complete these objectives, meaning
low-level behaviors need only be controlled if necessary. We perform
experiments with three state-of-the-art methods to create baselines and
highlight the affordances of this environment. Finally, we highlight challenges
and opportunities for future work.

negative:
Generative adversarial networks (GANs) are recently highly successful in
generative applications involving images and start being applied to time series
data. Here we describe EEG-GAN as a framework to generate
electroencephalographic (EEG) brain signals. We introduce a modification to the
improved training of Wasserstein GANs to stabilize training and investigate a
range of architectural choices critical for time series generation (most
notably up- and down-sampling). For evaluation we consider and compare
different metrics such as Inception score, Frechet inception distance and
sliced Wasserstein distance, together showing that our EEG-GAN framework
generated naturalistic EEG examples. It thus opens up a range of new generative
application scenarios in the neuroscientific and neurological context, such as
data augmentation in brain-computer interfacing tasks, EEG super-sampling, or
restoration of corrupted data segments. The possibility to generate signals of
a certain class and/or with specific properties may also open a new avenue for
research into the underlying structure of brain signals.

negative:
In the context of time series forecasting, it is a common practice to
evaluate multiple methods and choose one of these methods or an ensemble for
producing the best forecasts. However, choosing among different ensembles over
multiple methods remains a challenging task that undergoes a combinatorial
explosion as the number of methods increases. In the context of demand
forecasting or revenue forecasting, this challenge is further exacerbated by a
large number of time series as well as limited historical data points available
due to changing business context. Although deep learning forecasting methods
aim to simultaneously forecast large collections of time series, they become
challenging to apply in such scenarios due to the limited history available and
might not yield desirable results. We propose a framework for forecasting short
high-dimensional time series data by combining low-rank temporal matrix
factorization and optimal model selection on latent time series using
cross-validation. We demonstrate that forecasting the latent factors leads to
significant performance gains as compared to directly applying different
uni-variate models on time series. Performance has been validated on a
truncated version of the M4 monthly dataset which contains time series data
from multiple domains showing the general applicability of the method.
Moreover, it is amenable to incorporating the analyst view of the future owing
to the low number of latent factors which is usually impractical when applying
forecasting methods directly to high dimensional datasets.

negative:
Time series forecasting is widely used in business intelligence, e.g.,
forecast stock market price, sales, and help the analysis of data trend. Most
time series of interest are macroscopic time series that are aggregated from
microscopic data. However, instead of directly modeling the macroscopic time
series, rare literature studied the forecasting of macroscopic time series by
leveraging data on the microscopic level. In this paper, we assume that the
microscopic time series follow some unknown mixture probabilistic
distributions. We theoretically show that as we identify the ground truth
latent mixture components, the estimation of time series from each component
could be improved because of lower variance, thus benefitting the estimation of
macroscopic time series as well. Inspired by the power of Seq2seq and its
variants on the modeling of time series data, we propose Mixture of Seq2seq
(MixSeq), an end2end mixture model to cluster microscopic time series, where
all the components come from a family of Seq2seq models parameterized by
different parameters. Extensive experiments on both synthetic and real-world
data show the superiority of our approach.

negative:
Complex computer simulators are increasingly used across fields of science as
generative models tying parameters of an underlying theory to experimental
observations. Inference in this setup is often difficult, as simulators rarely
admit a tractable density or likelihood function. We introduce Adversarial
Variational Optimization (AVO), a likelihood-free inference algorithm for
fitting a non-differentiable generative model incorporating ideas from
generative adversarial networks, variational optimization and empirical Bayes.
We adapt the training procedure of generative adversarial networks by replacing
the differentiable generative network with a domain-specific simulator. We
solve the resulting non-differentiable minimax problem by minimizing
variational upper bounds of the two adversarial objectives. Effectively, the
procedure results in learning a proposal distribution over simulator
parameters, such that the JS divergence between the marginal distribution of
the synthetic data and the empirical distribution of observed data is
minimized. We evaluate and compare the method with simulators producing both
discrete and continuous data.

negative:
Time-series anomaly detection is a popular topic in both academia and
industrial fields. Many companies need to monitor thousands of temporal signals
for their applications and services and require instant feedback and alerts for
potential incidents in time. The task is challenging because of the complex
characteristics of time-series, which are messy, stochastic, and often without
proper labels. This prohibits training supervised models because of lack of
labels and a single model hardly fits different time series. In this paper, we
propose a solution to address these issues. We present an automated model
selection framework to automatically find the most suitable detection model
with proper parameters for the incoming data. The model selection layer is
extensible as it can be updated without too much effort when a new detector is
available to the service. Finally, we incorporate a customized tuning algorithm
to flexibly filter anomalies to meet customers' criteria. Experiments on
real-world datasets show the effectiveness of our solution.

negative:
Research into time series classification has tended to focus on the case of
series of uniform length. However, it is common for real-world time series data
to have unequal lengths. Differing time series lengths may arise from a number
of fundamentally different mechanisms. In this work, we identify and evaluate
two classes of such mechanisms -- variations in sampling rate relative to the
relevant signal and variations between the start and end points of one time
series relative to one another. We investigate how time series generated by
each of these classes of mechanism are best addressed for time series
classification. We perform extensive experiments and provide practical
recommendations on how variations in length should be handled in time series
classification.

negative:
Time series research has gathered lots of interests in the last decade,
especially for Time Series Classification (TSC) and Time Series Forecasting
(TSF). Research in TSC has greatly benefited from the University of California
Riverside and University of East Anglia (UCR/UEA) Time Series Archives. On the
other hand, the advancement in Time Series Forecasting relies on time series
forecasting competitions such as the Makridakis competitions, NN3 and NN5
Neural Network competitions, and a few Kaggle competitions. Each year,
thousands of papers proposing new algorithms for TSC and TSF have utilized
these benchmarking archives. These algorithms are designed for these specific
problems, but may not be useful for tasks such as predicting the heart rate of
a person using photoplethysmogram (PPG) and accelerometer data. We refer to
this problem as Time Series Extrinsic Regression (TSER), where we are
interested in a more general methodology of predicting a single continuous
value, from univariate or multivariate time series. This prediction can be from
the same time series or not directly related to the predictor time series and
does not necessarily need to be a future value or depend heavily on recent
values. To the best of our knowledge, research into TSER has received much less
attention in the time series research community and there are no models
developed for general time series extrinsic regression problems. Most models
are developed for a specific problem. Therefore, we aim to motivate and support
the research into TSER by introducing the first TSER benchmarking archive. This
archive contains 19 datasets from different domains, with varying number of
dimensions, unequal length dimensions, and missing values. In this paper, we
introduce the datasets in this archive and did an initial benchmark on existing
models.

negative:
In reinforcement learning, it is common to let an agent interact for a fixed
amount of time with its environment before resetting it and repeating the
process in a series of episodes. The task that the agent has to learn can
either be to maximize its performance over (i) that fixed period, or (ii) an
indefinite period where time limits are only used during training to diversify
experience. In this paper, we provide a formal account for how time limits
could effectively be handled in each of the two cases and explain why not doing
so can cause state aliasing and invalidation of experience replay, leading to
suboptimal policies and training instability. In case (i), we argue that the
terminations due to time limits are in fact part of the environment, and thus a
notion of the remaining time should be included as part of the agent's input to
avoid violation of the Markov property. In case (ii), the time limits are not
part of the environment and are only used to facilitate learning. We argue that
this insight should be incorporated by bootstrapping from the value of the
state at the end of each partial episode. For both cases, we illustrate
empirically the significance of our considerations in improving the performance
and stability of existing reinforcement learning algorithms, showing
state-of-the-art results on several control tasks.

negative:
The paper is focused on the forecasting method for time series groups with
the use of algorithms for cluster analysis. $K$-means algorithm is suggested to
be a basic one for clustering. The coordinates of the centers of clusters have
been put in correspondence with summarizing time series data the centroids of
the clusters. A description of time series, the centroids of the clusters, is
implemented with the use of forecasting models. They are based on strict binary
trees and a modified clonal selection algorithm. With the help of such
forecasting models, the possibility of forming analytic dependences is shown.
It is suggested to use a common forecasting model, which is constructed for
time series the centroid of the cluster, in forecasting the private
(individual) time series in the cluster. The promising application of the
suggested method for grouped time series forecasting is demonstrated.

negative:
Data augmentation is a key element of deep learning pipelines, as it informs
the network during training about transformations of the input data that keep
the label unchanged. Manually finding adequate augmentation methods and
parameters for a given pipeline is however rapidly cumbersome. In particular,
while intuition can guide this decision for images, the design and choice of
augmentation policies remains unclear for more complex types of data, such as
neuroscience signals. Besides, class-dependent augmentation strategies have
been surprisingly unexplored in the literature, although it is quite intuitive:
changing the color of a car image does not change the object class to be
predicted, but doing the same to the picture of an orange does. This paper
investigates gradient-based automatic data augmentation algorithms amenable to
class-wise policies with exponentially larger search spaces. Motivated by
supervised learning applications using EEG signals for which good augmentation
policies are mostly unknown, we propose a new differentiable relaxation of the
problem. In the class-agnostic setting, results show that our new relaxation
leads to optimal performance with faster training than competing gradient-based
methods, while also outperforming gradient-free methods in the class-wise
setting. This work proposes also novel differentiable augmentation operations
relevant for sleep stage classification.

negative:
We consider the problem of mining signal temporal logical requirements from a
dataset of regular (good) and anomalous (bad) trajectories of a dynamical
system. We assume the training set to be labeled by human experts and that we
have access only to a limited amount of data, typically noisy. We provide a
systematic approach to synthesize both the syntactical structure and the
parameters of the temporal logic formula using a two-steps procedure: first, we
leverage a novel evolutionary algorithm for learning the structure of the
formula; second, we perform the parameter synthesis operating on the
statistical emulation of the average robustness for a candidate formula w.r.t.
its parameters. We compare our results with our previous work [{BufoBSBLB14]
and with a recently proposed decision-tree [bombara_decision_2016] based
method. We present experimental results on two case studies: an anomalous
trajectory detection problem of a naval surveillance system and the
characterization of an Ineffective Respiratory effort, showing the usefulness
of our work.

negative:
Many businesses and industries nowadays rely on large quantities of time
series data making time series forecasting an important research area. Global
forecasting models that are trained across sets of time series have shown a
huge potential in providing accurate forecasts compared with the traditional
univariate forecasting models that work on isolated series. However, there are
currently no comprehensive time series archives for forecasting that contain
datasets of time series from similar sources available for the research
community to evaluate the performance of new global forecasting algorithms over
a wide variety of datasets. In this paper, we present such a comprehensive time
series forecasting archive containing 20 publicly available time series
datasets from varied domains, with different characteristics in terms of
frequency, series lengths, and inclusion of missing values. We also
characterise the datasets, and identify similarities and differences among
them, by conducting a feature analysis. Furthermore, we present the performance
of a set of standard baseline forecasting methods over all datasets across
eight error metrics, for the benefit of researchers using the archive to
benchmark their forecasting algorithms.

negative:
Organizations rely heavily on time series metrics to measure and model key
aspects of operational and business performance. The ability to reliably detect
issues with these metrics is imperative to identifying early indicators of
major problems before they become pervasive. It can be very challenging to
proactively monitor a large number of diverse and constantly changing time
series for anomalies, so there are often gaps in monitoring coverage, disabled
or ignored monitors due to false positive alarms, and teams resorting to manual
inspection of charts to catch problems. Traditionally, variations in the data
generation processes and patterns have required strong modeling expertise to
create models that accurately flag anomalies. In this paper, we describe an
anomaly detection system that overcomes this common challenge by keeping track
of its own performance and making changes as necessary to each model without
requiring manual intervention. We demonstrate that this novel approach
outperforms available alternatives on benchmark datasets in many scenarios.

negative:
In the fields of statistics and unsupervised machine learning a fundamental
and well-studied problem is anomaly detection. Anomalies are difficult to
define, yet many algorithms have been proposed. Underlying the approaches is
the nebulous understanding that anomalies are rare, unusual or inconsistent
with the majority of data. The present work provides a philosophical treatise
to clearly define anomalies and develops an algorithm for their efficient
detection with minimal user intervention. Inspired by the Gestalt School of
Psychology and the Helmholtz principle of human perception, anomalies are
assumed to be observations that are unexpected to occur with respect to certain
groupings made by the majority of the data. Under appropriate random variable
modelling anomalies are directly found in a set of data by a uniform and
independent random assumption of the distribution of constituent elements of
the observations, with anomalies corresponding to those observations where the
expectation of the number of occurrences of the elements in a given view is
$<1$. Starting from fundamental principles of human perception an unsupervised
anomaly detection algorithm is developed that is simple, real-time and
parameter-free. Experiments suggest it as a competing choice for univariate
data with promising results on the detection of global anomalies in
multivariate data.

negative:
Document date is essential for many important tasks, such as document
retrieval, summarization, event detection, etc. While existing approaches for
these tasks assume accurate knowledge of the document date, this is not always
available, especially for arbitrary documents from the Web. Document Dating is
a challenging problem which requires inference over the temporal structure of
the document. Prior document dating systems have largely relied on handcrafted
features while ignoring such document internal structures. In this paper, we
propose NeuralDater, a Graph Convolutional Network (GCN) based document dating
approach which jointly exploits syntactic and temporal graph structures of
document in a principled way. To the best of our knowledge, this is the first
application of deep learning for the problem of document dating. Through
extensive experiments on real-world datasets, we find that NeuralDater
significantly outperforms state-of-the-art baseline by 19% absolute (45%
relative) accuracy points.

negative:
Models for predicting the time of a future event are crucial for risk
assessment, across a diverse range of applications. Existing time-to-event
(survival) models have focused primarily on preserving pairwise ordering of
estimated event times, or relative risk. Model calibration is relatively under
explored, despite its critical importance in time-to-event applications. We
present a survival function estimator for probabilistic predictions in
time-to-event models, based on a neural network model for draws from the
distribution of event times, without explicit assumptions on the form of the
distribution. This is done like in adversarial learning, but we achieve
learning without a discriminator or adversarial objective. The proposed
estimator can be used in practice as a means of estimating and comparing
conditional survival distributions, while accounting for the predictive
uncertainty of probabilistic models. Extensive experiments show that the
proposed model outperforms existing approaches, trained both with and without
adversarial learning, in terms of both calibration and concentration of
time-to-event distributions.

negative:
Training models with discrete latent variables is challenging due to the
difficulty of estimating the gradients accurately. Much of the recent progress
has been achieved by taking advantage of continuous relaxations of the system,
which are not always available or even possible. The Augment-REINFORCE-Merge
(ARM) estimator provides an alternative that, instead of relaxation, uses
continuous augmentation. Applying antithetic sampling over the augmenting
variables yields a relatively low-variance and unbiased estimator applicable to
any model with binary latent variables. However, while antithetic sampling
reduces variance, the augmentation process increases variance. We show that ARM
can be improved by analytically integrating out the randomness introduced by
the augmentation process, guaranteeing substantial variance reduction. Our
estimator, DisARM, is simple to implement and has the same computational cost
as ARM. We evaluate DisARM on several generative modeling benchmarks and show
that it consistently outperforms ARM and a strong independent sample baseline
in terms of both variance and log-likelihood. Furthermore, we propose a local
version of DisARM designed for optimizing the multi-sample variational bound,
and show that it outperforms VIMCO, the current state-of-the-art method.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
Additive Manufacturing (AM) is a manufacturing paradigm that builds
three-dimensional objects from a computer-aided design model by successively
adding material layer by layer. AM has become very popular in the past decade
due to its utility for fast prototyping such as 3D printing as well as
manufacturing functional parts with complex geometries using processes such as
laser metal deposition that would be difficult to create using traditional
machining. As the process for creating an intricate part for an expensive metal
such as Titanium is prohibitive with respect to cost, computational models are
used to simulate the behavior of AM processes before the experimental run.
However, as the simulations are computationally costly and time-consuming for
predicting multiscale multi-physics phenomena in AM, physics-informed
data-driven machine-learning systems for predicting the behavior of AM
processes are immensely beneficial. Such models accelerate not only multiscale
simulation tools but also empower real-time control systems using in-situ data.
In this paper, we design and develop essential components of a scientific
framework for developing a data-driven model-based real-time control system.
Finite element methods are employed for solving time-dependent heat equations
and developing the database. The proposed framework uses extremely randomized
trees - an ensemble of bagged decision trees as the regression algorithm
iteratively using temperatures of prior voxels and laser information as inputs
to predict temperatures of subsequent voxels. The models achieve mean absolute
percentage errors below 1% for predicting temperature profiles for AM
processes. The code is made available for the research community at
https://github.com/paularindam/ml-iter-additive.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
My overarching research goal is to provide robots with perceptional abilities
that allow interactions with humans in a human-like manner. To develop these
perceptional abilities, I believe that it is useful to study the principles of
the human visual system. I use these principles to develop new computer vision
algorithms and validate their effectiveness in intelligent robotic systems. I
am enthusiastic about this approach as it offers the dual benefit of uncovering
principles inherent in the human visual system, as well as applying these
principles to its artificial counterpart. Fig. 1 contains a depiction of my
research.

negative:
This paper describes the short-term competition on the Components
Segmentation Task of Document Photos that was prepared in the context of the
16th International Conference on Document Analysis and Recognition (ICDAR
2021). This competition aims to bring together researchers working in the field
of identification document image processing and provides them a suitable
benchmark to compare their techniques on the component segmentation task of
document images. Three challenge tasks were proposed entailing different
segmentation assignments to be performed on a provided dataset. The collected
data are from several types of Brazilian ID documents, whose personal
information was conveniently replaced. There were 16 participants whose results
obtained for some or all the three tasks show different rates for the adopted
metrics, like Dice Similarity Coefficient ranging from 0.06 to 0.99. Different
Deep Learning models were applied by the entrants with diverse strategies to
achieve the best results in each of the tasks. Obtained results show that the
currently applied methods for solving one of the proposed tasks (document
boundary detection) are already well established. However, for the other two
challenge tasks (text zone and handwritten sign detection) research and
development of more robust approaches are still required to achieve acceptable
results.

negative:
Civil engineers use numerical simulations of a building's responses to
seismic forces to understand the nature of building failures, the limitations
of building codes, and how to determine the latter to prevent the former. Such
simulations generate large ensembles of multivariate, multiattribute time
series. Comprehensive understanding of this data requires techniques that
support the multivariate nature of the time series and can compare behaviors
that are both periodic and non-periodic across multiple time scales and
multiple time series themselves. In this paper, we present a novel technique to
extract such patterns from time series generated from simulations of seismic
responses. The core of our approach is the use of topic modeling, where topics
correspond to interpretable and discriminative features of the earthquakes. We
transform the raw time series data into a time series of topics, and use this
visual summary to compare temporal patterns in earthquakes, query earthquakes
via the topics across arbitrary time scales, and enable details on demand by
linking the topic visualization with the original earthquake data. We show,
through a surrogate task and an expert study, that this technique allows
analysts to more easily identify recurring patterns in such time series. By
integrating this technique in a prototype system, we show how it enables novel
forms of visual interaction.