Thank you for your response.
But in my experiments I observe that for data.grad the value is reset to 0, and I think that is because for every iteration within the dataloader loop, data is a new tensor (since we have different datapoints for every new iteration), and consequently data.grad is set to none, and after loss.backward() data.grad gets a new unaccumulated gradient. Is this correct ?