Selective Self-Training for semi-supervised Learning
Semi-supervised learning (SSL) is a study that efficiently exploits a large amount of unlabeled data to improve performance in conditions of limited labeled data. Most of the conventional SSL methods assume that the classes of unlabeled data are included in the set of classes of labeled data. In addition, these methods do not sort out useless unlabeled samples and use all the unlabeled data for learning, which is not suitable for realistic situations. In this paper, we propose an SSL method called selective self-training (SST), which selectively decides whether to include each unlabeled sample in the training process. It is also designed to be applied to a more real situation where classes of unlabeled data are different from the ones of the labeled data. For the conventional SSL problems which deal with data where both the labeled and unlabeled samples share the same class categories, the proposed method not only performs comparable to other conventional SSL algorithms but also can be combined with other SSL algorithms. While the conventional methods cannot be applied to the new SSL problems where the separated data do not share the classes, our method does not show any performance degradation even if the classes of unlabeled data are different from those of the labeled data.
 This is an novel, interesting paper on an important topic: semi-supervised learning. Even though the proposed approach seems to have significant potential, the experimental is somewhat disorganized, and it also includes some weak claims that should be removed. For example, the number of labeled examples in Table 1 is fairly large and inconsistent (4K, 1K, 10K for the 3 organic datasets). In this reviewer*s opinion, it would be a lot more reasonable to have instead a learning curve showing the results for, say, 100, 500, 1K, 5K, and 10K labeled examples for all three domains. In 4.1, you are using different epsilon policies for synthetic vs organic datasets; why? The explanation for underperforming on SVHM (page 7) may be valid, but you could easily prove it right or wrong by adding an option to SST for "stratified SSL." Without this extra work, your claim is just a conjecture. You should also show the performance of regular SSL methods in the setup on Table 4. Last but not least, you have repeatedly made the claim combining SST and other SSL may further improve the performance; however, you do not provide any evidence for it, so you should avoid making such claims. Other comments: - on page 2, the two terms classification & selection network appear "out of the blue;" it would be quite helpful to make it clear from the abstract that the proposed implementation is for neural networks. - figures 2 & 3 should be a lot larger in order to be readable - 4.1.2 top of page 7: claims such as "SST could have obtained better performance" have no place in such a paper; you could instead make a note about the method being "prohibitively CPU intensive for the time being" - lower on the same page you say: "SST may get better performance" - see above
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Selective Self-Training for semi-supervised Learning
Semi-supervised learning (SSL) is a study that efficiently exploits a large amount of unlabeled data to improve performance in conditions of limited labeled data. Most of the conventional SSL methods assume that the classes of unlabeled data are included in the set of classes of labeled data. In addition, these methods do not sort out useless unlabeled samples and use all the unlabeled data for learning, which is not suitable for realistic situations. In this paper, we propose an SSL method called selective self-training (SST), which selectively decides whether to include each unlabeled sample in the training process. It is also designed to be applied to a more real situation where classes of unlabeled data are different from the ones of the labeled data. For the conventional SSL problems which deal with data where both the labeled and unlabeled samples share the same class categories, the proposed method not only performs comparable to other conventional SSL algorithms but also can be combined with other SSL algorithms. While the conventional methods cannot be applied to the new SSL problems where the separated data do not share the classes, our method does not show any performance degradation even if the classes of unlabeled data are different from those of the labeled data.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This is an novel, interesting paper on an important topic: semi-supervised learning. Even though the proposed approach seems to have significant potential, the experimental is somewhat disorganized, and it also includes some weak claims that should be removed. For example, the number of labeled examples in Table 1 is fairly large and inconsistent (4K, 1K, 10K for the 3 organic datasets). In this reviewer*s opinion, it would be a lot more reasonable to have instead a learning curve showing the results for, say, 100, 500, 1K, 5K, and 10K labeled examples for all three domains. In 4.1, you are using different epsilon policies for synthetic vs organic datasets; why? The explanation for underperforming on SVHM (page 7) may be valid, but you could easily prove it right or wrong by adding an option to SST for "stratified SSL." Without this extra work, your claim is just a conjecture. You should also show the performance of regular SSL methods in the setup on Table 4. Last but not least, you have repeatedly made the claim combining SST and other SSL may further improve the performance; however, you do not provide any evidence for it, so you should avoid making such claims. Other comments: - on page 2, the two terms classification & selection network appear "out of the blue;" it would be quite helpful to make it clear from the abstract that the proposed implementation is for neural networks. - figures 2 & 3 should be a lot larger in order to be readable - 4.1.2 top of page 7: claims such as "SST could have obtained better performance" have no place in such a paper; you could instead make a note about the method being "prohibitively CPU intensive for the time being" - lower on the same page you say: "SST may get better performance" - see above
assistant
 This is an novel, interesting paper on an important topic: semi-supervised learning. Even though the proposed approach seems to have significant potential, the experimental is somewhat disorganized, and it also includes some weak claims that should be removed. For example, the number of labeled examples in Table 1 is fairly large and inconsistent (4K, 1K, 10K for the 3 organic datasets). In this reviewer*s opinion, it would be a lot more reasonable to have instead a learning curve showing the results for, say, 100, 500, 1K, 5K, and 10K labeled examples for all three domains. In 4.1, you are using different epsilon policies for synthetic vs organic datasets; why? The explanation for underperforming on SVHM (page 7) may be valid, but you could easily prove it right or wrong by adding an option to SST for "stratified SSL." Without this extra work, your claim is just a conjecture. You should also show the performance of regular SSL methods in the setup on Table 4. Last but not least, you have repeatedly made the claim combining SST and other SSL may further improve the performance; however, you do not provide any evidence for it, so you should avoid making such claims. Other comments: - on page 2, the two terms classification & selection network appear "out of the blue;" it would be quite helpful to make it clear from the abstract that the proposed implementation is for neural networks. - figures 2 & 3 should be a lot larger in order to be readable - 4.1.2 top of page 7: claims such as "SST could have obtained better performance" have no place in such a paper; you could instead make a note about the method being "prohibitively CPU intensive for the time being" - lower on the same page you say: "SST may get better performance" - see above