Regarding first issue, as per your suggestion, I modified my code as:
if self.args.fp16:
    with amp.scale_loss(loss, optimizer) as scaled_loss:
        scaled_grad_params = torch.autograd.grad(self.scaler.scale(scaled_loss), params, allow_unused=True)

Before I can multiply params by inv_scale, it throws an error because scaled_grad_params is None.
I assume that amp.scale_loss is not supposed to be used like this. I searched in the documentation, this method is used with loss.backward() which I’m not calling. Can you please suggest what am I missing ?