I think you could either use two different scalers for each optimizer or alternatively use scaler.step() on the constraint_optimizer, if the inner block wasn’t executed.
After the inner code was already executed, the gradients should be already unscaled, which are used in the first optimizer.
However, I’m currently unsure about your use case and don’t know, if both optimizers are using a different subset of all parameters, are partially overlapping, or reuse the same parameters.
Could you explain the use case a bit more?
Also, I think you have a typo in the posted code snippet, as you are calling scaler.step(loss), while an optimizer would be needed.