Based on your description it seems you are calculating a specific loss while caring about the PSNR, which is not represented in your loss.
I don’t know if slowing down the convergence would increase the PSNR, so could you explain this assumption a bit more and why this would be the case?
Without knowing details about your use case I would guess the better approach would be to track some kind of information of the PSNR in the loss.