Interesting - I read into the PyTorch documentation and it mentions the same remedy for freezing parts of the network.
So to get this straight: even though requires_grad=False is set, it will still compute the gradients for that part of the network, but simply not update the weights? The name requires_grad seems a bit misleading, as the gradients are indeed calculated… ?