scaler.unscale_(optimizer) unscales the .grad attributes of all params owned by optimizer, after those .grads have been fully accumulated for those parameters this iteration and are about to be applied.  If you intend to accumulate more gradients into .grads later in the iteration, scaler.unscale_ is premature.
Also, the unscale+inf/nan check kernel used by scaler.unscale_ is not autograd-aware.
In use cases I’ve seen, creating out-of-place gradients via torch.autograd.grad with create_graph=True is typically a setup for a double-backward that will accumulate gradients into the param.grad attributes.  For these cases, manually unscaling the out-of-place grads before using them to set up the double-backward is expected:  see the gradient penalty example 4.
In your snippet the control flow looks strange to me:

first you run backward(create_graph=True) that accumulates gradients into param.grads.
then you collect gradsH.append(0. if p.grad is None else p.grad + 0.), presumably the p.grad+0. ensures gradsH are separate tensors from param.grads (but if so why not also use an out-of-place torch.autograd.grad to create gradsH?)
Then you use those .grads as torch.autograd.grad graph roots to create hvs out of place.

Your optimizer is vanilla SGD and you don’t appear to use hvs, so it’s hard for me to recommend a way to create it.
FYI GradScaler also has an enabled argument, ie
      if fp16_enabled: 
        scaler.scale(loss).backward(create_graph=True)
      else: 
        loss.backward(create_graph=True)

is equivalent to
  scaler = GradScaler(init_scale=init_scale)
  ...
  scaler.scale(loss).backward(create_graph=True)