TSInsight: A local-global attribution framework for interpretability in time-series data
With the rise in employment of deep learning methods in safety-critical scenarios, interpretability is more essential than ever before. Although many different directions regarding interpretability have been explored for visual modalities, time-series data has been neglected with only a handful of methods tested due to their poor intelligibility. We approach the problem of interpretability in a novel way by proposing TSInsight where we attach an auto-encoder with a sparsity-inducing norm on its output to the classifier and fine-tune it based on the gradients from the classifier and a reconstruction penalty. The auto-encoder learns to preserve features that are important for the prediction by the classifier and suppresses the ones that are irrelevant i.e. serves as a feature attribution method to boost interpretability. In other words, we ask the network to only reconstruct parts which are useful for the classifier i.e. are correlated or causal for the prediction. In contrast to most other attribution frameworks, TSInsight is capable of generating both instance-based and model-based explanations. We evaluated TSInsight along with other commonly used attribution methods on a range of different time-series datasets to validate its efficacy. Furthermore, we analyzed the set of properties that TSInsight achieves out of the box including adversarial robustness and output space contraction. The obtained results advocate that TSInsight can be an effective tool for the interpretability of deep time-series models.
The paper presents a new approach for improving the interpretability of deep learning methods used for time series. The is mainly concerned with classification tasks for time series. First, the classifier is learned in a usual way. Subsequently, a sparse auto-encoder is used that encodes the last layer of the classifier. For training the auto-encoder the classifier is fixed and there is a decoding loss as well as a sparsity loss. The sparse encoding of the last layer is supposed to increase the interpretability of the classification as it indicates which features are important for the classification. In general, I think this paper needs to be considerably improved in order to justify a publication. I am not convinced about the interpretability of the sparse extracted feature vector. It is not clear to me why this should be more interpretable then other methods. The paper contains many plots where the compare to other attributes methods, but it is not clear why the presented results should be any better as other methods (for example Fig 3). The paper is missing also a lot of motivation, the results are not well explained (e.g. Fig 3) and it needs to be improved in terms of writing. Equation 1 is not motivated (which is the main part of the paper) and it is not clear how Figure 2a has been generated and why this represented should be * an interesting one, doesn’t help with the interpretability of the model*. The authors have to improve the motivation part as well as the discussion of the results. More comments below: - The method seems to suffer from a severe parameter tuning problem, which makes it hard to use in practise. - It is unclear to me why the discriminator is fixed during training the encoder and decoder. Shouldnt it improve performance to also adapt the discriminator to the new representation. - Why can we not just add a layer with a sparsity constraint one layer before the *encoded* layer such that we have the same architecture and optimize that end to end? At least comparison to such an approach would be needed to justify something more complex. - The plots need to be better explained. While the comparisons seems to be exhaustive, it is already too many plots and it is very easy to get lost. Also, the quality of the plots need to be improved (e.g. font size)
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
TSInsight: A local-global attribution framework for interpretability in time-series data
With the rise in employment of deep learning methods in safety-critical scenarios, interpretability is more essential than ever before. Although many different directions regarding interpretability have been explored for visual modalities, time-series data has been neglected with only a handful of methods tested due to their poor intelligibility. We approach the problem of interpretability in a novel way by proposing TSInsight where we attach an auto-encoder with a sparsity-inducing norm on its output to the classifier and fine-tune it based on the gradients from the classifier and a reconstruction penalty. The auto-encoder learns to preserve features that are important for the prediction by the classifier and suppresses the ones that are irrelevant i.e. serves as a feature attribution method to boost interpretability. In other words, we ask the network to only reconstruct parts which are useful for the classifier i.e. are correlated or causal for the prediction. In contrast to most other attribution frameworks, TSInsight is capable of generating both instance-based and model-based explanations. We evaluated TSInsight along with other commonly used attribution methods on a range of different time-series datasets to validate its efficacy. Furthermore, we analyzed the set of properties that TSInsight achieves out of the box including adversarial robustness and output space contraction. The obtained results advocate that TSInsight can be an effective tool for the interpretability of deep time-series models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper presents a new approach for improving the interpretability of deep learning methods used for time series. The is mainly concerned with classification tasks for time series. First, the classifier is learned in a usual way. Subsequently, a sparse auto-encoder is used that encodes the last layer of the classifier. For training the auto-encoder the classifier is fixed and there is a decoding loss as well as a sparsity loss. The sparse encoding of the last layer is supposed to increase the interpretability of the classification as it indicates which features are important for the classification. In general, I think this paper needs to be considerably improved in order to justify a publication. I am not convinced about the interpretability of the sparse extracted feature vector. It is not clear to me why this should be more interpretable then other methods. The paper contains many plots where the compare to other attributes methods, but it is not clear why the presented results should be any better as other methods (for example Fig 3). The paper is missing also a lot of motivation, the results are not well explained (e.g. Fig 3) and it needs to be improved in terms of writing. Equation 1 is not motivated (which is the main part of the paper) and it is not clear how Figure 2a has been generated and why this represented should be * an interesting one, doesn’t help with the interpretability of the model*. The authors have to improve the motivation part as well as the discussion of the results. More comments below: - The method seems to suffer from a severe parameter tuning problem, which makes it hard to use in practise. - It is unclear to me why the discriminator is fixed during training the encoder and decoder. Shouldnt it improve performance to also adapt the discriminator to the new representation. - Why can we not just add a layer with a sparsity constraint one layer before the *encoded* layer such that we have the same architecture and optimize that end to end? At least comparison to such an approach would be needed to justify something more complex. - The plots need to be better explained. While the comparisons seems to be exhaustive, it is already too many plots and it is very easy to get lost. Also, the quality of the plots need to be improved (e.g. font size)
assistant
The paper presents a new approach for improving the interpretability of deep learning methods used for time series. The is mainly concerned with classification tasks for time series. First, the classifier is learned in a usual way. Subsequently, a sparse auto-encoder is used that encodes the last layer of the classifier. For training the auto-encoder the classifier is fixed and there is a decoding loss as well as a sparsity loss. The sparse encoding of the last layer is supposed to increase the interpretability of the classification as it indicates which features are important for the classification. In general, I think this paper needs to be considerably improved in order to justify a publication. I am not convinced about the interpretability of the sparse extracted feature vector. It is not clear to me why this should be more interpretable then other methods. The paper contains many plots where the compare to other attributes methods, but it is not clear why the presented results should be any better as other methods (for example Fig 3). The paper is missing also a lot of motivation, the results are not well explained (e.g. Fig 3) and it needs to be improved in terms of writing. Equation 1 is not motivated (which is the main part of the paper) and it is not clear how Figure 2a has been generated and why this represented should be * an interesting one, doesn’t help with the interpretability of the model*. The authors have to improve the motivation part as well as the discussion of the results. More comments below: - The method seems to suffer from a severe parameter tuning problem, which makes it hard to use in practise. - It is unclear to me why the discriminator is fixed during training the encoder and decoder. Shouldnt it improve performance to also adapt the discriminator to the new representation. - Why can we not just add a layer with a sparsity constraint one layer before the *encoded* layer such that we have the same architecture and optimize that end to end? At least comparison to such an approach would be needed to justify something more complex. - The plots need to be better explained. While the comparisons seems to be exhaustive, it is already too many plots and it is very easy to get lost. Also, the quality of the plots need to be improved (e.g. font size)