TheeNinja:

In other words, is there a way for the half() to be done on the CPU tensor for the entire data batch, and have the tensor copied to GPU one-time after?


You could transform the data on the CPU to half, push them to the device, and transform the batch back to float before calling the forward function.
Note that your training also needs to store intermediate tensors to be able to calculate the gradients, but I assume you’ve already checked it.