Semi-supervised deep learning by metric embedding
Deep networks are successfully used as classification models yielding state-of-the-art results when trained on a large number of labeled samples. These models, however, are usually much less suited for semi-supervised problems because of their tendency to overfit easily when trained on small amounts of data. In this work we will explore a new training objective that is targeting a semi-supervised regime with only a small subset of labeled data. This criterion is based on a deep metric embedding over distance relations within the set of labeled samples, together with constraints over the embeddings of the unlabeled set. The final learned representations are discriminative in euclidean space, and hence can be used with subsequent nearest-neighbor classification using the labeled samples.
The authors propose a semi-supervised technique for neural networks which includes two objectives: (1) the neural net embeddings of two samples with identical labels is constrained to be closer than the embeddings of samples with different labels (2) the embedding of an unlabeled example is constrained to be close to the embeddings of the closest labeled sample (and far away from the other ones). While the authors list a number of previous works, they do not relate them very well with their approach, and actual differences appear unclear. In particular, the approach seems rather incremental with respect to (Hadsell et al. 2006), the way the neighbors are chosen being the main difference; in that respect, (Weston et al, 2008 or 2012) (which could be viewed as an application of DrLim from Hadsell et al, applied to semi-supervised learning) is even closer to the approach proposed here. There is also no balancing constraint in the proposed approach, which was known to be crucial in all these models from the 2000s. Concerning the experimental results, reported performance are very good; given the approach is very close to previous work, it is hard to know if good performance come from better neural net architectures or something else. This should be clarified. In summary, the novelty is not clearly defined in this paper; differences with existing literature should be highlighted. Experimental results are very good, but it is hard to know where comes the difference in performance with previous (very related) work.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Semi-supervised deep learning by metric embedding
Deep networks are successfully used as classification models yielding state-of-the-art results when trained on a large number of labeled samples. These models, however, are usually much less suited for semi-supervised problems because of their tendency to overfit easily when trained on small amounts of data. In this work we will explore a new training objective that is targeting a semi-supervised regime with only a small subset of labeled data. This criterion is based on a deep metric embedding over distance relations within the set of labeled samples, together with constraints over the embeddings of the unlabeled set. The final learned representations are discriminative in euclidean space, and hence can be used with subsequent nearest-neighbor classification using the labeled samples.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors propose a semi-supervised technique for neural networks which includes two objectives: (1) the neural net embeddings of two samples with identical labels is constrained to be closer than the embeddings of samples with different labels (2) the embedding of an unlabeled example is constrained to be close to the embeddings of the closest labeled sample (and far away from the other ones). While the authors list a number of previous works, they do not relate them very well with their approach, and actual differences appear unclear. In particular, the approach seems rather incremental with respect to (Hadsell et al. 2006), the way the neighbors are chosen being the main difference; in that respect, (Weston et al, 2008 or 2012) (which could be viewed as an application of DrLim from Hadsell et al, applied to semi-supervised learning) is even closer to the approach proposed here. There is also no balancing constraint in the proposed approach, which was known to be crucial in all these models from the 2000s. Concerning the experimental results, reported performance are very good; given the approach is very close to previous work, it is hard to know if good performance come from better neural net architectures or something else. This should be clarified. In summary, the novelty is not clearly defined in this paper; differences with existing literature should be highlighted. Experimental results are very good, but it is hard to know where comes the difference in performance with previous (very related) work.
assistant
The authors propose a semi-supervised technique for neural networks which includes two objectives: (1) the neural net embeddings of two samples with identical labels is constrained to be closer than the embeddings of samples with different labels (2) the embedding of an unlabeled example is constrained to be close to the embeddings of the closest labeled sample (and far away from the other ones). While the authors list a number of previous works, they do not relate them very well with their approach, and actual differences appear unclear. In particular, the approach seems rather incremental with respect to (Hadsell et al. 2006), the way the neighbors are chosen being the main difference; in that respect, (Weston et al, 2008 or 2012) (which could be viewed as an application of DrLim from Hadsell et al, applied to semi-supervised learning) is even closer to the approach proposed here. There is also no balancing constraint in the proposed approach, which was known to be crucial in all these models from the 2000s. Concerning the experimental results, reported performance are very good; given the approach is very close to previous work, it is hard to know if good performance come from better neural net architectures or something else. This should be clarified. In summary, the novelty is not clearly defined in this paper; differences with existing literature should be highlighted. Experimental results are very good, but it is hard to know where comes the difference in performance with previous (very related) work.