Learning from a few examples is a key characteristic of human intelligence that inspired machine learning researchers to build data efficient AI models. Recent progress has shown that few-shot learning can be improved with access to unlabelled data, known as semi-supervised few-shot learning(SS-FSL). We introduce an SS-FSL approach, dubbed as Prototypical Random Walk Networks(PRWN), built on top of Prototypical Networks (PN) (Ren et al., 2018). We develop a random walk semi-supervised loss that enables the network to learn representations that are compact and well-separated. Our work is related to the very recent development on graph-based approaches for few-shot learning. However, we show that compact and well-separated class representations can be achieved by modeling our prototypical random walk notion without needing additional graph-NN parameters or requiring a transductive setting where collective test set is provided (e.g., Kim et al. (2019)). Our model outperforms prior art in most benchmarks with significant improvements in some cases. For example, in a mini-Imagenet 5-shot classification task, we obtain 69.65% accuracy to the 64.59% state-of-theart. Our model, trained with 40% of the data as labelled, compares competitively against fully supervised prototypical networks, trained on 100% of the labels, even outperforming it in the 1-shot mini-Imagenet case with 50.89% to 49.4% accuracy. We also show that our model is resistant to distractors, unlabeled data that does not belong to any of the training classes, and hence reflecting robustness to labelled/unlabelled class distribution mismatch. We also performed a challenging discriminative power test, showing a relative improvement on top of the baseline of ≈14% on 20 classes on mini-Imagenet and ≈60% on 800 classes on Omniglot. Our code will be released upon acceptance.
This paper develops a random walk-based method on top of prototypical networks to address the semi-supervised few-shot learning, i.e. when each classification task can access only a few-shot labeled data but many unlabeled data. This paper defines a specific random walk: it walks from a prototype to an unlabeled sample, then walks for several steps between unlabeled samples, and then walk back to some prototype. They then propose two loss terms: one aims to maximize the probability of returning to the same prototype, and the other tries to make each unlabeled data having equal probability to be visited during the random walk. These two terms are added to the original prototypical network training loss and the resulted training procedure involves both the few-shot labeled data and the unlabeled data. Experiments on mini-ImageNet show that the proposed method outperforms the other baselines.
Intuitively, the main idea makes some sense to me. The writing is clear to understand the main idea but can be improved in various places. My major concerns are the motivation and analysis of the proposed random walk and the training algorithm, whose details cannot be found in the paper. The paper uses a lot of spaces to review the existing works and problem settings, but only spends one page (Section 2.2) on the description of their contribution, and the description lacks any in-depth discussion about “why it is designed in this way but not others” or “how this objective helps the few-shot learning”: it simply lists the procedures without convincing explanation.
Detailed comments:
1. It is called “random walks” but is different from the random walks used in classical methods in many key points and thus might not have all the existing properties of random walks. For example, it is not clear whether the negative of the square of Euclidean distance is a valid similarity for random walks. It is also not clear why the random walk probability matrix is computed by softmax instead of normalized Laplacian. Moreover, it fixes the starting nodes and end nodes, and also fixes the propagation steps: this cannot give any guarantee of any convergence (or asymptotic properties, mixing rate, etc) as the original random walks hold. This could be a serious problem because the random walks can stop at any non-stationary state after tau steps, which cannot be used for training. 
2. It is not clear why the two additional loss terms (L_walker and L_visit) can help to improve the prototype training. The claimed motivation of minimizing these two terms cannot be soundly related to the final few-shot classification goal, though each of them independently makes some sense if applied to a different goal. Why does maximizing the probability of a prototype transit to itself give new information of improving few-shot classification? If prototypes are involved in the random walk nodes, why do we need to walk back to itself through many unlabeled nodes rather than simply walk to itself (which has the largest probability)? Why do we need to visit as many unlabeled data as possible? Isn’t the case that we hope to only visit the unlabeled data mostly related to the prototype and its class and rule out other unrelated ones since they introduce noises?
Here is a counterexample, which is a trivial solution that can minimize the two terms but does make any sense here: it is easy to define a model that produces the same prototype for different classes (so L_walker is minimized), and achieve equal distance between all the unlabeled samples (so L_visit is minimized), but this does not help the prototype training at all. 
3. Another primary flaw here is that the training objective and the computational graph during training always change with the random walks, since the trajectory of the random walks keeps changing due to the randomness. Hence, the trained model does not have any guarantee and the convergence in the training is not the real convergence (it is not well-defined in the first place). If the random walks guarantee to converge to some stationary distribution, it might be possible to treat the varying computational graph as a stochastic one which has been analyzed in previous works. Unfortunately, due to the fixed number of steps and other issues mentioned above, this cannot hold. In Figure 2(a), we can see that even the largest tau they have tried cannot make the landing probability converge (not talking about the stationary distribution over all the nodes), which verifies my worries above.
4. Is the final performance sensitive to the hyperparameters lambda and tau? How did you choose their values in the experiments? 
5. The methodology behind most existing semi-supervised few-shot learning methods, as also indicated in this paper’s introduction, is to apply or extend existing semi-supervised learning techniques on top of existing few-shot learning models. In fact, it is quite straightforward to do so and test all the semi-supervised learning techniques on few-shot learning tasks. Hence, I think it is necessary to compare to baselines that directly use, for example, consistency loss from mean teacher method and progressive training on pseudo labeling, or other SSL techniques mentioned in this paper: https://papers.nips.cc/paper/7585-realistic-evaluation-of-deep-semi-supervised-learning-algorithms. 
-------------
Update:
Thanks for the authors' reply! After reading the rebuttal, I still think there are several main issues needed to be clarified or needing further study. Hence, I will keep my rating.