Hi @colesbury,
I converted my batch-norm layers back to floats with this code, which works:
def batchnorm_to_fp32(module):
    if isinstance(module, nn.modules.batchnorm._BatchNorm):
        module.float()
    for child in module.children():
        batchnorm_to_fp32(child)
    return module

But autograd doesn’t like me using a mixture of float & half tensors, running loss.backward() leads to an error like this:

error.png1422×761 115 KB

Do you think you can help me?
Thank you if yes!