This can reduce the inference time, if TensorCores are used for the workload.
However, note that FP16 can easily overflow such that your model might return NaNs, if you are not careful (especially during training).
We thus recommend to use automatic mixed precision training 181, which can be used in the nightly binaries or from a master build.
Nevertheless, you could try to directly call half() and see, if the inference workload suffers from these issues.