wlike:

But we do NOT get significant improvement as expected
to be 2~4x faster by using HalfTensor.


Hi wlike,
Did you figure out why you didn’t get a speedup? Which Nvidia GPU cards did you use? Did they support float16 like the V100 or P100 or was it the 1080(ti) or Titan which does not support fast float16?
How was your GPU memory consumption when you changed to HalfTensor?
Thanks in advance