The amp usage from your code snippet matches the AMP - Gradient accumulation example 2 so I think it’s correct from this point of view.
However, from a general perspective, I think you should remove the first zero_grad() operation, as it would remove the accumulated gradients.
Also, Variables are deprecated since PyTorch 0.4, so you can use tensors now.