Thanks @ptrblck for your prompt response.  I am actually using FP16 mode for inference because it is more time-critical. I can see memory footprint reduction (not 2x as expected, but less), but inference time is actually longer (by around 2x). Is this related to the GPU support for mixed-precision?