Anomaly detection in time series data is an important topic in many domains. However, time series are known to be particularly hard to analyze. Based on the recent developments in adversarially learned models, we propose a new approach for anomaly detection in time series data. We build upon the idea to use a combination of a reconstruction error and the output of a Critic network. To this end we propose a cycle-consistent GAN architecture for sequential data and a new way of measuring the reconstruction error. We then show in a detailed evaluation how the different parts of our model contribute to the final anomaly score and demonstrate how the method improves the results on several data sets. We also compare our model to other baseline anomaly detection methods to verify its performance.
The paper trains a GAN on univariate time series data and uses reconstruction errors in combination with the critic's output to predict anomalous subsequences. The method is applied on two real-world data sets and compared to three simple baselines.
I have several reservations about this manuscript:
- The methodology isn't very original: The GAN architecture is essentially a slightly modified CycleGAN trained with Wasserstein loss.
- Many design choices appear to be ad-hoc: there has been no principled selection of the GAN's hyperparameters, nor has their effect on the experimental results been studied. For computing the reconstruction error the authors use the integral over the difference between two time series without taking the absolute value, which is a very unusual choice. (Standard choices would be e.g. L1 or L2 norm.) While this turns out to have worked "surprisingly well" on the studied datasets, it is not difficult to construct scenarios where this choice will fail. 
- Another design choice that should be discussed in more detail is the smoothing of the time series data using a moving average filter; in the context of anomaly detection this can have a significant impact, and it may not work equally well for different data sets, so a principled approach for determining the level of filtering is paramount. Same goes for the de-trending, and the actual parameters both of the moving averages and the de-trending functions should be reported.
- It is not clear to me how exactly the anomaly scores (supposedly for different sequence lengths l?) are used to predict the subsequence(s) containing an anomaly. It seems there is no incentive to keep the predicted subsequences as short as possible, i.e. if the anomaly score indicates there might be an anomaly present, then the safe approach is to just flag the entire sequence as anomalous (it doesn't hurt the sensitivity they way it's computed)?
- The description of the experiments requires more detail. Are both datasets labelled? After dividing the datasets into rolling chunks of length 100, how many samples do the training and test sets contain? How many of the test set samples contain anomalies?
- The baselines need to be described in more detail. What method was used, e.g. to select and fit the ARIMA models? What type of reconstruction loss was used? How were the anomalous subsequences predicted?
- Is there any way to compare the proposed method with Li et al's (who also used a GAN for anomaly detection in time series data)?
Detailed comments:
- abstract: "particular hard" -> "particularly hard"
- p.3, paragraph starting with "To support...": shouldn't this be "E(x) with x ~ P_X"?
- p.4 and p.6: broken citations "?"
------------------
I acknowledge that I have read the authors' response, but it doesn't change my assessment that several major revisions are needed:
- further motivating the design choices and comparing them to the state-of-the-art;
- formalizing (ideally providing a model for) the sort of anomalies that the proposed method aims to detect;
- discussing limitations of the proposed approach.