Few-Shot One-Class Classification via Meta-Learning
Although few-shot learning and one-class classification have been separately well studied, their intersection remains rather unexplored. Our work addresses the few-shot one-class classification problem and presents a meta-learning approach that requires only few data examples from only one class to adapt to unseen tasks. The proposed method builds upon the model-agnostic meta-learning (MAML) algorithm (Finn et al., 2017) and explicitly trains for few-shot class-imbalance learning, aiming to learn a model initialization that is particularly suited for learning one-class classification tasks after observing only a few examples of one class. Experimental results on datasets from the image domain and the time-series domain show that our model substantially outperforms the baselines, including MAML, and demonstrate the ability to learn new tasks from only few majority class samples. Moreover, we successfully learn anomaly detectors for a real world application involving sensor readings recorded during industrial manufacturing of workpieces with a CNC milling machine using only few examples from the normal class.
This paper tackles an interesting problem, one-class classification or anomaly detection, using a meta-learning approach. The main contribution is to introduce a parameter such that the inner-loop of the meta-learning algorithm better reflects the imbalance which occurs during meta-testing. Results are shown comparing a few simple baselines to both MAML and the modified variant, on a few datasets such as image-based ones (MNIST, miniImageNet), a synthetic dataset, and a real-world time-series example from CNC milling machines. Overall, the paper presents an interesting problem and awareness that meta-learning might be general enough to solve it well, but provides no real novelty in the approach. The datasets and comparison to other state of art methods (including both other anomaly detection methods and out of distribution methods) is lacking. I suggest the authors perform more rigorous experimentation and focus the paper to be a paper about an understudied problem with rigorous experiments/findings, or improve their method beond the small modification made. Due to these weaknesses, I vote for rejection at this time. Detailed comments are below. Strengths - The problem is interesting and under-studied in the context of deep learning and transferable methods from similar ML problems (e.g. few-shot learning) - The method is simple and adapts a state of art in few-shot learning (meta-learning, and specifically MAML) Weaknesses - While I enjoyed reading the paper since it tackles an under-explored problem, it is hard to justify publishing the method/approach at a top machine learning conference. Changing the balance in meta-learning is a relatively obvious modification that one would do to better reflect the problem; I don*t think it results in general scientific/ML principles that can be used elsewhere. - The relationship to out-of-distribution detection (which some of the experiments, e.g. Multi-task MNIST and miniImagenet essentially test) is not discussed or compared to. How are anomalies defined and is it really different than just being out-of-distribution? - The datasets are limited. The MNIST dataset seems to choose a fixed two specific categories for meta-validation and meta-testing, as opposed to doing cross-validation. Results on just one meta-testing seems limited in this case with just one class. In terms of time-series, anomaly detection has been studied for a long time; is there a reason that the authors create a new synthetic dataset? For the milling example, how were anomalies provoked? - The baselines do not represent any state of art anomaly detection (e.g. density based, isolation forests, etc.) nor out of distribution detection; the latter especially would likely do extremely well for the simple image examples. - There is no analysis of what the difference is in representation (initialization) learning due to the differences between the OCC and FS setup. What are the characteristics of the improved initialization? One minor comment not reflecting the decision: - Exposition: Define the one-class classification problem; it*s not common so it would be good to define in the abstract, or mention anomaly detection which is a better-known term.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Few-Shot One-Class Classification via Meta-Learning
Although few-shot learning and one-class classification have been separately well studied, their intersection remains rather unexplored. Our work addresses the few-shot one-class classification problem and presents a meta-learning approach that requires only few data examples from only one class to adapt to unseen tasks. The proposed method builds upon the model-agnostic meta-learning (MAML) algorithm (Finn et al., 2017) and explicitly trains for few-shot class-imbalance learning, aiming to learn a model initialization that is particularly suited for learning one-class classification tasks after observing only a few examples of one class. Experimental results on datasets from the image domain and the time-series domain show that our model substantially outperforms the baselines, including MAML, and demonstrate the ability to learn new tasks from only few majority class samples. Moreover, we successfully learn anomaly detectors for a real world application involving sensor readings recorded during industrial manufacturing of workpieces with a CNC milling machine using only few examples from the normal class.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper tackles an interesting problem, one-class classification or anomaly detection, using a meta-learning approach. The main contribution is to introduce a parameter such that the inner-loop of the meta-learning algorithm better reflects the imbalance which occurs during meta-testing. Results are shown comparing a few simple baselines to both MAML and the modified variant, on a few datasets such as image-based ones (MNIST, miniImageNet), a synthetic dataset, and a real-world time-series example from CNC milling machines. Overall, the paper presents an interesting problem and awareness that meta-learning might be general enough to solve it well, but provides no real novelty in the approach. The datasets and comparison to other state of art methods (including both other anomaly detection methods and out of distribution methods) is lacking. I suggest the authors perform more rigorous experimentation and focus the paper to be a paper about an understudied problem with rigorous experiments/findings, or improve their method beond the small modification made. Due to these weaknesses, I vote for rejection at this time. Detailed comments are below. Strengths - The problem is interesting and under-studied in the context of deep learning and transferable methods from similar ML problems (e.g. few-shot learning) - The method is simple and adapts a state of art in few-shot learning (meta-learning, and specifically MAML) Weaknesses - While I enjoyed reading the paper since it tackles an under-explored problem, it is hard to justify publishing the method/approach at a top machine learning conference. Changing the balance in meta-learning is a relatively obvious modification that one would do to better reflect the problem; I don*t think it results in general scientific/ML principles that can be used elsewhere. - The relationship to out-of-distribution detection (which some of the experiments, e.g. Multi-task MNIST and miniImagenet essentially test) is not discussed or compared to. How are anomalies defined and is it really different than just being out-of-distribution? - The datasets are limited. The MNIST dataset seems to choose a fixed two specific categories for meta-validation and meta-testing, as opposed to doing cross-validation. Results on just one meta-testing seems limited in this case with just one class. In terms of time-series, anomaly detection has been studied for a long time; is there a reason that the authors create a new synthetic dataset? For the milling example, how were anomalies provoked? - The baselines do not represent any state of art anomaly detection (e.g. density based, isolation forests, etc.) nor out of distribution detection; the latter especially would likely do extremely well for the simple image examples. - There is no analysis of what the difference is in representation (initialization) learning due to the differences between the OCC and FS setup. What are the characteristics of the improved initialization? One minor comment not reflecting the decision: - Exposition: Define the one-class classification problem; it*s not common so it would be good to define in the abstract, or mention anomaly detection which is a better-known term.
assistant
This paper tackles an interesting problem, one-class classification or anomaly detection, using a meta-learning approach. The main contribution is to introduce a parameter such that the inner-loop of the meta-learning algorithm better reflects the imbalance which occurs during meta-testing. Results are shown comparing a few simple baselines to both MAML and the modified variant, on a few datasets such as image-based ones (MNIST, miniImageNet), a synthetic dataset, and a real-world time-series example from CNC milling machines. Overall, the paper presents an interesting problem and awareness that meta-learning might be general enough to solve it well, but provides no real novelty in the approach. The datasets and comparison to other state of art methods (including both other anomaly detection methods and out of distribution methods) is lacking. I suggest the authors perform more rigorous experimentation and focus the paper to be a paper about an understudied problem with rigorous experiments/findings, or improve their method beond the small modification made. Due to these weaknesses, I vote for rejection at this time. Detailed comments are below. Strengths - The problem is interesting and under-studied in the context of deep learning and transferable methods from similar ML problems (e.g. few-shot learning) - The method is simple and adapts a state of art in few-shot learning (meta-learning, and specifically MAML) Weaknesses - While I enjoyed reading the paper since it tackles an under-explored problem, it is hard to justify publishing the method/approach at a top machine learning conference. Changing the balance in meta-learning is a relatively obvious modification that one would do to better reflect the problem; I don*t think it results in general scientific/ML principles that can be used elsewhere. - The relationship to out-of-distribution detection (which some of the experiments, e.g. Multi-task MNIST and miniImagenet essentially test) is not discussed or compared to. How are anomalies defined and is it really different than just being out-of-distribution? - The datasets are limited. The MNIST dataset seems to choose a fixed two specific categories for meta-validation and meta-testing, as opposed to doing cross-validation. Results on just one meta-testing seems limited in this case with just one class. In terms of time-series, anomaly detection has been studied for a long time; is there a reason that the authors create a new synthetic dataset? For the milling example, how were anomalies provoked? - The baselines do not represent any state of art anomaly detection (e.g. density based, isolation forests, etc.) nor out of distribution detection; the latter especially would likely do extremely well for the simple image examples. - There is no analysis of what the difference is in representation (initialization) learning due to the differences between the OCC and FS setup. What are the characteristics of the improved initialization? One minor comment not reflecting the decision: - Exposition: Define the one-class classification problem; it*s not common so it would be good to define in the abstract, or mention anomaly detection which is a better-known term.