I wouldn’t think you will see a significant performance benefit assuming your code would work with int32. Usually integer tensors are used e.g. as the target values for nn.CrossEntropyLoss, for indexing etc. Since these targets or inputs are usually tiny compared to the model parameters as well as intermediate activations, the memory saving would also be in the same ballpark. To accelerate your model I would recommend to check mixed-precision training as well as the performance guide 1.