In Transfer Learning tutorial 16, the author shows a way to use a pre-trained model as a feature extractor. I have a few problems wrt this tutorial.


Could we call .eval() on the extractor so that dropout (although not used in the tutorial) and BN performs consistently? More generally, would this be helpful for the performance (Acc. something)?


Could we make this procedure async, aka, feature extractor deals with the (n+i)-th iteration via one GPU while the model is being trained in the n-th iteration in another GPU? When we have a HUGE model to train on top of the feature extractor, this will be quite useful.


======
I’ve just tried to solve 2. by threading 6. Might not be the best solution