Yes, calling .half() on the model as well as the inputs would give you an estimate of how much memory would be saved in the extreme case of using float16 for all operations. Make sure to check the memory usage via torch.cuda.memory_summary(), since nvidia-smi will also show the cache not only the allocated memory.