Hello again @ptrblck, thanks for your reply! 
A working snippet will be difficult, because the training is done using multiple files and using a custom loss (https://github.com/victorca25/BasicSR/blob/3fb5851af3a411df153f0b0e5873f3b2de324a3d/codes/models/modules/loss.py#L767), but I think there’s nothing out of the ordinary, this is how the code looks like:
#In the init:
self.amp_scaler =  GradScaler()

#in the training loop:
        ### Network forward, generate SR
        with autocast():
            self.fake_H = self.netG(self.var_L)
        l_g_total = 0
        with autocast(): #this is separated, because there are some conditionals in the middle
            loss_results = self.generatorlosses(self.fake_H, self.var_H, self.log_dict, self.f_low) # multiple losses can be used here, but I'm testing with just the one loss in the link above
            l_g_total += sum(loss_results)/self.accumulations
        self.amp_scaler.scale(l_g_total).backward()
        if (step + 1) % self.accumulations == 0:
            self.amp_scaler.step(self.optimizer_G)
            self.amp_scaler.update() 
            self.optimizer_G.zero_grad()

I’m checking the isnan() and isinf() for the loss and output right after the loss. This might be difficult to try to debug as-is, but any pointers at what I should look at or how to catch the potential overflow would be awesome. It’s the only loss with this behaviour ATM.