Hi,
I am trying to manually update weights, while using gradients from autograd (I need this as a base to implement my custom weight update algorithm later). So, based on the discussion above, is the following a good practice in general : using .grad.data and .grad.data.zero_() instead of .grad and .grad.zero_() ?
Note : I am using named_parameters() for debugging purposes only , the name is not used for compuatations inside the loop.
    with torch.no_grad():
        for name, p in model.named_parameters():
            p.data = p.data - learning_rate*p.grad.data 
            # Manually zero the gradients after updating weights
            p.grad.data.zero_()

Regards, Sumit