Thanks for the autocast code.
Indeed, the scaler doesn’t seem trivial. I guess I’ll have to wait for a C++ implementation of it, as the tensor core speedup is specially interesting for training more than inference. Hopefully PyTorch 1.7 ?