In case of transfer learning, we usually train only the last fc layer. The number of epochs required is huge (300 odd) to achieve benchmark performance. I find it wasteful to every time  compute the activations  upto fc layer as they would be same for a particular image in the training set(as all earlier layers are frozen). Hence huge amount of time can be saved if the activations prior to fc layer for both train and test sets are precomputed and used. Fast.ai has such an approach. Can we implement such a functionality in any way using standard Pytorch?