ptrblck,
Thank you for your response.
I read this article about transfer learning and I understood how it is working. My question was how different is case 2 and and case 3.
Maybe I expressed my question in incomprehensible way. In other words, how it is possible to use pre-trained model and optimise parameters of all layers setting “set param.requires_grad = True” ? If you optimise params of all layers you train your model from scratch so you set “pretrained=False”.