Solved by tumble-weed in post #4 
              
              
                In the forward hook, you are returning output.data, i usually return output. I did a run to check the type of a layer and layer.data for AlexNet, and both of them are tensor which is why you can return either, but when i checked the .requires_grad property, it seems for layer.data.requires_grad is F…