A Multi-modal one-class generative adversarial network for anomaly detection in manufacturing
One class anomaly detection on high-dimensional data is one of the critical issue in both fundamental machine learning research area and manufacturing applica- tions. A good anomaly detection should accurately discriminate anomalies from normal data. Although most previous anomaly detection methods achieve good performances, they do not perform well on high-dimensional imbalanced data- set 1) with a limited amount of data; 2) multi-modal distribution; 3) few anomaly data. In this paper, we develop a multi-modal one-class generative adversarial net- work based detector (MMOC-GAN) to distinguish anomalies from normal data (products). Apart from a domain-specific feature extractor, our model leverage a generative adversarial network(GAN). The generator takes in a modified noise vector using a pseudo latent prior and generate samples at the low-density area of the given normal data to simulate the anomalies. The discriminator then is trained to distinguish the generate samples from the normal samples. Since the generated samples simulate the low density area for each modal, the discriminator could directly detect anomalies from normal data. Experiments demonstrate that our model outperforms the state-of-the-art one-class classification models and other anomaly detection methods on both normal data and anomalies accuracy, as well as the F1 score. Also, the generated samples can fully capture the low density area of different types of products.
 The paper presents an anomaly detection method called MMOCGAN which is claimed to work well on high-dimensional datasets with limited, multimodal data. The proposed idea is to train a GAN generator to simulate anomalies in the data in order to provide the one-class classifier with more negative examples. Overall I find that the paper is not clear and reproducible enough for me to recommend its acceptance: - Results are presented on a single private dataset, and I don’t see any indication that the dataset will be shared with the community. This is problematic because there is no way for the community to reproduce and validate these results. I don’t think results on private datasets should systematically be rejected, but they should at least be presented alongside results on public benchmarks to enable some form of reproducibility. - Given the small number of non-pass products in the dataset (22), it’s unclear to me whether a held-out test set was used, or if hyperparameter selection was performed on the full set of non-pass products. - The use of a pre-trained, domain-specific feature extractor is briefly mentioned but no details (what is the architecture, on which data it’s been trained, etc.) are provided. - The central idea in the paper is to have the generator capture the "complementary distribution" of the data-generating distribution. The way in which this distribution is defined is not specific enough (it depends on hyperparameters C and epsilon for which there is no clear prescribed value). On a conceptual level it seems to me that for a data-generating distribution corresponding to a low-dimensional manifold embedded in a high-dimensional space the complementary distribution will essentially be uniform random noise, and in that case it’s unclear to me how it’s supposed to "simulate anomalies". - The way the proposed method is presented makes it look ad-hoc: several moving parts (InfoGAN, generator loss term encouraging it to learn the "complementary distribution", feature matching regularization term, pull-away loss term, discriminator entropy term) are connected together and their individual inclusion in the final loss is loosely justified. In practice, looking at the results it’s impossible for me to tell which term is necessary and which is not. - The word "modal" is used throughout as a noun. I’m not sure if the authors mean "model", "mode", or "modality", but based on the context I assume they mean "mode" as in "mode of the distribution". - The use of an InfoGAN architecture and loss is not credited clearly enough to Chen et al. and may give the impression to a casual reader that the idea is novel to this paper. The paper also does not make it clear how the number of categories or modes for the latent variable should be chosen, and what was the value used for the experiments. - The paper is legible, but there are several grammatical errors and typos throughout that make it harder to read than necessary.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
A Multi-modal one-class generative adversarial network for anomaly detection in manufacturing
One class anomaly detection on high-dimensional data is one of the critical issue in both fundamental machine learning research area and manufacturing applica- tions. A good anomaly detection should accurately discriminate anomalies from normal data. Although most previous anomaly detection methods achieve good performances, they do not perform well on high-dimensional imbalanced data- set 1) with a limited amount of data; 2) multi-modal distribution; 3) few anomaly data. In this paper, we develop a multi-modal one-class generative adversarial net- work based detector (MMOC-GAN) to distinguish anomalies from normal data (products). Apart from a domain-specific feature extractor, our model leverage a generative adversarial network(GAN). The generator takes in a modified noise vector using a pseudo latent prior and generate samples at the low-density area of the given normal data to simulate the anomalies. The discriminator then is trained to distinguish the generate samples from the normal samples. Since the generated samples simulate the low density area for each modal, the discriminator could directly detect anomalies from normal data. Experiments demonstrate that our model outperforms the state-of-the-art one-class classification models and other anomaly detection methods on both normal data and anomalies accuracy, as well as the F1 score. Also, the generated samples can fully capture the low density area of different types of products.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper presents an anomaly detection method called MMOCGAN which is claimed to work well on high-dimensional datasets with limited, multimodal data. The proposed idea is to train a GAN generator to simulate anomalies in the data in order to provide the one-class classifier with more negative examples. Overall I find that the paper is not clear and reproducible enough for me to recommend its acceptance: - Results are presented on a single private dataset, and I don’t see any indication that the dataset will be shared with the community. This is problematic because there is no way for the community to reproduce and validate these results. I don’t think results on private datasets should systematically be rejected, but they should at least be presented alongside results on public benchmarks to enable some form of reproducibility. - Given the small number of non-pass products in the dataset (22), it’s unclear to me whether a held-out test set was used, or if hyperparameter selection was performed on the full set of non-pass products. - The use of a pre-trained, domain-specific feature extractor is briefly mentioned but no details (what is the architecture, on which data it’s been trained, etc.) are provided. - The central idea in the paper is to have the generator capture the "complementary distribution" of the data-generating distribution. The way in which this distribution is defined is not specific enough (it depends on hyperparameters C and epsilon for which there is no clear prescribed value). On a conceptual level it seems to me that for a data-generating distribution corresponding to a low-dimensional manifold embedded in a high-dimensional space the complementary distribution will essentially be uniform random noise, and in that case it’s unclear to me how it’s supposed to "simulate anomalies". - The way the proposed method is presented makes it look ad-hoc: several moving parts (InfoGAN, generator loss term encouraging it to learn the "complementary distribution", feature matching regularization term, pull-away loss term, discriminator entropy term) are connected together and their individual inclusion in the final loss is loosely justified. In practice, looking at the results it’s impossible for me to tell which term is necessary and which is not. - The word "modal" is used throughout as a noun. I’m not sure if the authors mean "model", "mode", or "modality", but based on the context I assume they mean "mode" as in "mode of the distribution". - The use of an InfoGAN architecture and loss is not credited clearly enough to Chen et al. and may give the impression to a casual reader that the idea is novel to this paper. The paper also does not make it clear how the number of categories or modes for the latent variable should be chosen, and what was the value used for the experiments. - The paper is legible, but there are several grammatical errors and typos throughout that make it harder to read than necessary.
assistant
 The paper presents an anomaly detection method called MMOCGAN which is claimed to work well on high-dimensional datasets with limited, multimodal data. The proposed idea is to train a GAN generator to simulate anomalies in the data in order to provide the one-class classifier with more negative examples. Overall I find that the paper is not clear and reproducible enough for me to recommend its acceptance: - Results are presented on a single private dataset, and I don’t see any indication that the dataset will be shared with the community. This is problematic because there is no way for the community to reproduce and validate these results. I don’t think results on private datasets should systematically be rejected, but they should at least be presented alongside results on public benchmarks to enable some form of reproducibility. - Given the small number of non-pass products in the dataset (22), it’s unclear to me whether a held-out test set was used, or if hyperparameter selection was performed on the full set of non-pass products. - The use of a pre-trained, domain-specific feature extractor is briefly mentioned but no details (what is the architecture, on which data it’s been trained, etc.) are provided. - The central idea in the paper is to have the generator capture the "complementary distribution" of the data-generating distribution. The way in which this distribution is defined is not specific enough (it depends on hyperparameters C and epsilon for which there is no clear prescribed value). On a conceptual level it seems to me that for a data-generating distribution corresponding to a low-dimensional manifold embedded in a high-dimensional space the complementary distribution will essentially be uniform random noise, and in that case it’s unclear to me how it’s supposed to "simulate anomalies". - The way the proposed method is presented makes it look ad-hoc: several moving parts (InfoGAN, generator loss term encouraging it to learn the "complementary distribution", feature matching regularization term, pull-away loss term, discriminator entropy term) are connected together and their individual inclusion in the final loss is loosely justified. In practice, looking at the results it’s impossible for me to tell which term is necessary and which is not. - The word "modal" is used throughout as a noun. I’m not sure if the authors mean "model", "mode", or "modality", but based on the context I assume they mean "mode" as in "mode of the distribution". - The use of an InfoGAN architecture and loss is not credited clearly enough to Chen et al. and may give the impression to a casual reader that the idea is novel to this paper. The paper also does not make it clear how the number of categories or modes for the latent variable should be chosen, and what was the value used for the experiments. - The paper is legible, but there are several grammatical errors and typos throughout that make it harder to read than necessary.