Classification-Based Anomaly Detection for General Data
Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence. Recently, classification-based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method to relax current generalization assumptions. Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The strong performance of our method is extensively validated on multiple datasets from different domains.
This paper proposes a novel approach to classification-based anomaly detection for general data. Classification-based anomaly detection uses auxiliary tasks (transformations) to train a model to extract useful features from the data. This approach is well-known in image data, where auxiliary tasks such as classification of rotated or flipped images have been demonstrated to work effectively. The paper generalizes to the task by using the affine transformation y = Wx+b. A novel distance-based classification is also devised to learn the model in such as way that it generalizes to unseen data. This is achieved by modeling the each auxiliary task subspace by a sphere and by using the distance to the center for the calculation of the loss function. The anomaly score then becomes the product of the probabilities that the transformed samples are in their respective subspaces. The paper provides comparison to SOT methods for both Cifar10 and 4 non-image datasets. The proposed method substantially outperforms SOT on all datasets. A section is devoted to explore the benefits of this approach on adversarial attacks using PGD. It is shown that random transformations (implemented with the affine transformation and a random matrix) do increase the robustness of the models by 50%. Another section is devoted to studying the effect of contamination (anomaly data in the training set). The approach is shown to degrade more gracefully than DAGMM on KDDCUP99. Finally, a section studies the effect of the number of tasks on the performance, showing that after a certain number of task (which is probably problem-dependent), the accuracy stabilizes. PROS: * A general and novel approach to anomaly detection with SOT results. * The method allows for any type of classifier to be used. The authors note that deep models perform well on the large datasets (KDDCUP) while shallower models are sufficient for smaller datasets. * The paper is relatively well written and easy to follow, the math is clearly laid out. CONS: * The lack of a pseudo-code algorithm makes it hard to understand and reproduce the method * Figure 1 (left) has inverted colors (DAGMM should be blue - higher error). * Figure 1 (right) - it is unclear what the scale of the x-axis is since there is only 1 label. Also the tick marks seem spaced logarithmically, which, if i understand correctly, is wrong. * The paragraph *Number of operations* should be renamed *Number of tasks* to be consistent. Also the sentence *From 16 ...* should be clarified, as it seems to contrast accuracy and results, which are the same entity. The concept of *stability of results* is not explained clearly. It would suffice to say: *From 16 tasks and larger, the accuracy remains stable*. * In section 6, the paragraph *Generating many tasks* should be named *Number of tasks*, to be consistent with the corresponding paragraph in section 5.2. Also the first sentence should be: *As illustrated in Figure 1 (right), increasing the number of tasks does result in improved performance but the trend is not linear and beyond a certain threshold, no improvements are made. And again the concept of *stability* is somewhat misleading here. The sentence *...it mainly improves the stability of the results* is wrong. The stability is not improved, it is just that the performance trend is stable. * The study on the number of tasks should be carried on several datasets. Only one dataset is too few to make any claims on the accuracy trends as the number of task is increased. * The authors should coin an acronym to name their methods. Overall this paper provides a novel approach to classification-based semi-supervised anomaly detection of general data. The results are very encouraging, beating SOT methods by a good margin on standard benchmarks.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Classification-Based Anomaly Detection for General Data
Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence. Recently, classification-based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method to relax current generalization assumptions. Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The strong performance of our method is extensively validated on multiple datasets from different domains.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes a novel approach to classification-based anomaly detection for general data. Classification-based anomaly detection uses auxiliary tasks (transformations) to train a model to extract useful features from the data. This approach is well-known in image data, where auxiliary tasks such as classification of rotated or flipped images have been demonstrated to work effectively. The paper generalizes to the task by using the affine transformation y = Wx+b. A novel distance-based classification is also devised to learn the model in such as way that it generalizes to unseen data. This is achieved by modeling the each auxiliary task subspace by a sphere and by using the distance to the center for the calculation of the loss function. The anomaly score then becomes the product of the probabilities that the transformed samples are in their respective subspaces. The paper provides comparison to SOT methods for both Cifar10 and 4 non-image datasets. The proposed method substantially outperforms SOT on all datasets. A section is devoted to explore the benefits of this approach on adversarial attacks using PGD. It is shown that random transformations (implemented with the affine transformation and a random matrix) do increase the robustness of the models by 50%. Another section is devoted to studying the effect of contamination (anomaly data in the training set). The approach is shown to degrade more gracefully than DAGMM on KDDCUP99. Finally, a section studies the effect of the number of tasks on the performance, showing that after a certain number of task (which is probably problem-dependent), the accuracy stabilizes. PROS: * A general and novel approach to anomaly detection with SOT results. * The method allows for any type of classifier to be used. The authors note that deep models perform well on the large datasets (KDDCUP) while shallower models are sufficient for smaller datasets. * The paper is relatively well written and easy to follow, the math is clearly laid out. CONS: * The lack of a pseudo-code algorithm makes it hard to understand and reproduce the method * Figure 1 (left) has inverted colors (DAGMM should be blue - higher error). * Figure 1 (right) - it is unclear what the scale of the x-axis is since there is only 1 label. Also the tick marks seem spaced logarithmically, which, if i understand correctly, is wrong. * The paragraph *Number of operations* should be renamed *Number of tasks* to be consistent. Also the sentence *From 16 ...* should be clarified, as it seems to contrast accuracy and results, which are the same entity. The concept of *stability of results* is not explained clearly. It would suffice to say: *From 16 tasks and larger, the accuracy remains stable*. * In section 6, the paragraph *Generating many tasks* should be named *Number of tasks*, to be consistent with the corresponding paragraph in section 5.2. Also the first sentence should be: *As illustrated in Figure 1 (right), increasing the number of tasks does result in improved performance but the trend is not linear and beyond a certain threshold, no improvements are made. And again the concept of *stability* is somewhat misleading here. The sentence *...it mainly improves the stability of the results* is wrong. The stability is not improved, it is just that the performance trend is stable. * The study on the number of tasks should be carried on several datasets. Only one dataset is too few to make any claims on the accuracy trends as the number of task is increased. * The authors should coin an acronym to name their methods. Overall this paper provides a novel approach to classification-based semi-supervised anomaly detection of general data. The results are very encouraging, beating SOT methods by a good margin on standard benchmarks.
assistant
This paper proposes a novel approach to classification-based anomaly detection for general data. Classification-based anomaly detection uses auxiliary tasks (transformations) to train a model to extract useful features from the data. This approach is well-known in image data, where auxiliary tasks such as classification of rotated or flipped images have been demonstrated to work effectively. The paper generalizes to the task by using the affine transformation y = Wx+b. A novel distance-based classification is also devised to learn the model in such as way that it generalizes to unseen data. This is achieved by modeling the each auxiliary task subspace by a sphere and by using the distance to the center for the calculation of the loss function. The anomaly score then becomes the product of the probabilities that the transformed samples are in their respective subspaces. The paper provides comparison to SOT methods for both Cifar10 and 4 non-image datasets. The proposed method substantially outperforms SOT on all datasets. A section is devoted to explore the benefits of this approach on adversarial attacks using PGD. It is shown that random transformations (implemented with the affine transformation and a random matrix) do increase the robustness of the models by 50%. Another section is devoted to studying the effect of contamination (anomaly data in the training set). The approach is shown to degrade more gracefully than DAGMM on KDDCUP99. Finally, a section studies the effect of the number of tasks on the performance, showing that after a certain number of task (which is probably problem-dependent), the accuracy stabilizes. PROS: * A general and novel approach to anomaly detection with SOT results. * The method allows for any type of classifier to be used. The authors note that deep models perform well on the large datasets (KDDCUP) while shallower models are sufficient for smaller datasets. * The paper is relatively well written and easy to follow, the math is clearly laid out. CONS: * The lack of a pseudo-code algorithm makes it hard to understand and reproduce the method * Figure 1 (left) has inverted colors (DAGMM should be blue - higher error). * Figure 1 (right) - it is unclear what the scale of the x-axis is since there is only 1 label. Also the tick marks seem spaced logarithmically, which, if i understand correctly, is wrong. * The paragraph *Number of operations* should be renamed *Number of tasks* to be consistent. Also the sentence *From 16 ...* should be clarified, as it seems to contrast accuracy and results, which are the same entity. The concept of *stability of results* is not explained clearly. It would suffice to say: *From 16 tasks and larger, the accuracy remains stable*. * In section 6, the paragraph *Generating many tasks* should be named *Number of tasks*, to be consistent with the corresponding paragraph in section 5.2. Also the first sentence should be: *As illustrated in Figure 1 (right), increasing the number of tasks does result in improved performance but the trend is not linear and beyond a certain threshold, no improvements are made. And again the concept of *stability* is somewhat misleading here. The sentence *...it mainly improves the stability of the results* is wrong. The stability is not improved, it is just that the performance trend is stable. * The study on the number of tasks should be carried on several datasets. Only one dataset is too few to make any claims on the accuracy trends as the number of task is increased. * The authors should coin an acronym to name their methods. Overall this paper provides a novel approach to classification-based semi-supervised anomaly detection of general data. The results are very encouraging, beating SOT methods by a good margin on standard benchmarks.