Semi-Supervised Learning with Normalizing Flows
We propose Flow Gaussian Mixture Model (FlowGMM), a general-purpose method for semi-supervised learning based on a simple and principled probabilistic framework. We approximate the joint distribution of the labeled and unlabeled data with a flexible mixture model implemented as a Gaussian mixture transformed by a normalizing flow. We train the model by maximizing the exact joint likelihood of the labeled and unlabeled data. We evaluate FlowGMM on a wide range of semi-supervised classification problems across different data types: AG-News and Yahoo Answers text data, MNIST, SVHN and CIFAR-10 image classification problems as well as tabular UCI datasets. FlowGMM achieves promising results on image classification problems and outperforms the competing methods on other types of data. FlowGMM learns an interpretable latent repesentation space and allows hyper-parameter free feature visualization at real time rates. Finally, we show that FlowGMM can be calibrated to produce meaningful uncertainty estimates for its predictions.
The paper describes how to use normalising flows for Semi Supervised Learning (SSL). Briefly, the method consists in finding a (bijective) map for transforming a mixture of Gaussians into a density approximating the empirical data-distribution -- as usual for flow methods, the parameters are found through likelihood maximisation. This is a elegant approach that naturally exploits the standard so-called cluster assumption in SSL. The papers also shows how to incorporate a consistency-based regularisation within the method. Although it is an elegant and simple approach, and the article is relatively well written, I think that the paper should be rejected because (1) on image classification tasks (and even with consistency regularisation), the performances are well-below the straightforward-to-implement Pi-model. (2) for tabular/NLP data, although the performances seems to be good, the comparison with standard methods could have been much better done -- I am still not convinced by the method. I agree with the authors that there are many situations where it is not possible to find good perturbation (eg. NLP / tabular / genomics / etc...). If the authors could demonstrate more carefully that their approach does lead to state-of-the-art performances in this type of situations, I do believe that the approach would be of great interest. Given that the methods does not work well at all for image classification, I think that he authors should have been much more careful with the comparisons with the standard methods when investigating the performances on NLP/tabular tasks. (1) basic k-NN benchmark? (2) basic dimension reduction (PCA / autoencoder / extract lower representation from a NN) associated with either k-NN or label-propagation? (3) it is *not* difficult at all to implement label propagation with fast nearest-neighbours (eg. FAISS library) and sparse linear algebra on the full datasets. In the current submission, it has not been done for the NLP datasets. (4) There are indeed several ways to compute distance / affinity within label-propagation-type approaches (5) Brief description of parameter tuning for label-prop should be added I think that the method has a lot of potential and the fact that it is not competitive for computer vision task is not important. I encourage the authors to carry out more convincing numerical comparisons ing tabular/NLP/etc.. settings in order to strengthen the message of the paper. If convincing results can be obtained, I believe that the method has a lot of potential. [Edit after rebuttal] I would like to thank the authors to have provided additional label propagation experiments and details -- the proposed method appears to be quite much better than this baseline approach, which is very reassuring and proves that it is worth exploring further this line of work.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Semi-Supervised Learning with Normalizing Flows
We propose Flow Gaussian Mixture Model (FlowGMM), a general-purpose method for semi-supervised learning based on a simple and principled probabilistic framework. We approximate the joint distribution of the labeled and unlabeled data with a flexible mixture model implemented as a Gaussian mixture transformed by a normalizing flow. We train the model by maximizing the exact joint likelihood of the labeled and unlabeled data. We evaluate FlowGMM on a wide range of semi-supervised classification problems across different data types: AG-News and Yahoo Answers text data, MNIST, SVHN and CIFAR-10 image classification problems as well as tabular UCI datasets. FlowGMM achieves promising results on image classification problems and outperforms the competing methods on other types of data. FlowGMM learns an interpretable latent repesentation space and allows hyper-parameter free feature visualization at real time rates. Finally, we show that FlowGMM can be calibrated to produce meaningful uncertainty estimates for its predictions.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper describes how to use normalising flows for Semi Supervised Learning (SSL). Briefly, the method consists in finding a (bijective) map for transforming a mixture of Gaussians into a density approximating the empirical data-distribution -- as usual for flow methods, the parameters are found through likelihood maximisation. This is a elegant approach that naturally exploits the standard so-called cluster assumption in SSL. The papers also shows how to incorporate a consistency-based regularisation within the method. Although it is an elegant and simple approach, and the article is relatively well written, I think that the paper should be rejected because (1) on image classification tasks (and even with consistency regularisation), the performances are well-below the straightforward-to-implement Pi-model. (2) for tabular/NLP data, although the performances seems to be good, the comparison with standard methods could have been much better done -- I am still not convinced by the method. I agree with the authors that there are many situations where it is not possible to find good perturbation (eg. NLP / tabular / genomics / etc...). If the authors could demonstrate more carefully that their approach does lead to state-of-the-art performances in this type of situations, I do believe that the approach would be of great interest. Given that the methods does not work well at all for image classification, I think that he authors should have been much more careful with the comparisons with the standard methods when investigating the performances on NLP/tabular tasks. (1) basic k-NN benchmark? (2) basic dimension reduction (PCA / autoencoder / extract lower representation from a NN) associated with either k-NN or label-propagation? (3) it is *not* difficult at all to implement label propagation with fast nearest-neighbours (eg. FAISS library) and sparse linear algebra on the full datasets. In the current submission, it has not been done for the NLP datasets. (4) There are indeed several ways to compute distance / affinity within label-propagation-type approaches (5) Brief description of parameter tuning for label-prop should be added I think that the method has a lot of potential and the fact that it is not competitive for computer vision task is not important. I encourage the authors to carry out more convincing numerical comparisons ing tabular/NLP/etc.. settings in order to strengthen the message of the paper. If convincing results can be obtained, I believe that the method has a lot of potential. [Edit after rebuttal] I would like to thank the authors to have provided additional label propagation experiments and details -- the proposed method appears to be quite much better than this baseline approach, which is very reassuring and proves that it is worth exploring further this line of work.
assistant
The paper describes how to use normalising flows for Semi Supervised Learning (SSL). Briefly, the method consists in finding a (bijective) map for transforming a mixture of Gaussians into a density approximating the empirical data-distribution -- as usual for flow methods, the parameters are found through likelihood maximisation. This is a elegant approach that naturally exploits the standard so-called cluster assumption in SSL. The papers also shows how to incorporate a consistency-based regularisation within the method. Although it is an elegant and simple approach, and the article is relatively well written, I think that the paper should be rejected because (1) on image classification tasks (and even with consistency regularisation), the performances are well-below the straightforward-to-implement Pi-model. (2) for tabular/NLP data, although the performances seems to be good, the comparison with standard methods could have been much better done -- I am still not convinced by the method. I agree with the authors that there are many situations where it is not possible to find good perturbation (eg. NLP / tabular / genomics / etc...). If the authors could demonstrate more carefully that their approach does lead to state-of-the-art performances in this type of situations, I do believe that the approach would be of great interest. Given that the methods does not work well at all for image classification, I think that he authors should have been much more careful with the comparisons with the standard methods when investigating the performances on NLP/tabular tasks. (1) basic k-NN benchmark? (2) basic dimension reduction (PCA / autoencoder / extract lower representation from a NN) associated with either k-NN or label-propagation? (3) it is *not* difficult at all to implement label propagation with fast nearest-neighbours (eg. FAISS library) and sparse linear algebra on the full datasets. In the current submission, it has not been done for the NLP datasets. (4) There are indeed several ways to compute distance / affinity within label-propagation-type approaches (5) Brief description of parameter tuning for label-prop should be added I think that the method has a lot of potential and the fact that it is not competitive for computer vision task is not important. I encourage the authors to carry out more convincing numerical comparisons ing tabular/NLP/etc.. settings in order to strengthen the message of the paper. If convincing results can be obtained, I believe that the method has a lot of potential. [Edit after rebuttal] I would like to thank the authors to have provided additional label propagation experiments and details -- the proposed method appears to be quite much better than this baseline approach, which is very reassuring and proves that it is worth exploring further this line of work.