ptrblck:

The are not packed to 32bit, as it would completely get rid of the advantage of the reduced memory bandwidth.
Depending on the operations you are running, you might get some additional operations e.g. for permutations. Some FP16 convolutions use TensorCores in the channels-last memory format, so that a permutation kernel will be called inside the cudnn kernel. However, the number of additional operations should be low compared to the actual workflow, and you should still see a speedup in FP16, if TensorCores are used.


Another thing I just did a little more research on is that the device I’m using is the Tegra X1, and it looks like it doesn’t support TensorCore technology (although other versions do https://en.wikipedia.org/wiki/Tegra)
If there are no TensorCores, what benefits does FP16 have?