Solved by RicCu in post #2 
              
              
                That looks good. Although I would also pass to the optimizer only the parameters of the last layer, i.e. 
optimizer = optim.SGD(model.conv11d.parameters(), lr=0.01, momentum=0.5)

You can verify if its working by comparing the values of the weights and/or biases of some frozen layers and the last la…