We have some code that is designed to run in mixed-precision. It contains a custom CUDA kernel call. Currently we are placing a with torch.cuda.amp.autocast(enabled=False) guard around the custom CUDA kernel call, and then cast all tensors to the same type. Is the autocast(enabled=False) really necessary? What might happen without this block? (context: this pull request, that proposes to move the casts from out of the autocast guard block, to enable compatibility with APEX, Enable both Pytorch native AMP and Nvidia APEX AMP for SRU by visionscaper · Pull Request #158 · asappresearch/sru · GitHub 4 )