nn.BCELoss is most likely unsafe to be used in mixed-precision training and you should get a warning.
Besides that, you shouldn’t call model.half() and push the model to the device before initializing amp.
The docs 5 should give you an example usage.
That being said, we recommend to try out torch.cuda.amp 6 using the nightly binaries or by installing from source, as it can be used directly in PyTorch now without the need to build apex.