Jimmie Munyi’s Research Blog – 17 Oct 21
  

  
    

Making Transfer Learning work in PyTorch 2

  Tips to effectively use Transfer Learning in the Vision Domain.


  

  
    
    
  

  


Hey good people. Here is a post about tips and tweaks you can employ to make transfer Learning work just right.
Most blog posts I have read about this topic just suggest changing the last linear layer and freezing all other parameters but there is more tweaks you can try
E.g
Discriminative Learning Rates, Don’t Freeze Batch Norm layers, Unfreezing the model after a few epochs, using a custom and better head (classifier)