@melgor
Here is the output of your script on Titan V:
cuda version= 9.0.176
cudnn version= 7005
densenet121 FP 32 avg over 100 runs: 86.58529281616211 ms
densenet121 FP 16 avg over 10 runs: 71.90406322479248 ms
vgg16 FP 32 avg over 100 runs: 111.51945352554321 ms
vgg16 FP 16 avg over 10 runs: 71.55098915100098 ms
resnet152 FP 32 avg over 100 runs: 174.67097759246826 ms
resnet152 FP 16 avg over 10 runs: 109.48643684387207 ms

1080 Ti:
cuda version= 9.0.176
cudnn version= 7005
densenet121 FP 32 avg over 100 runs: 96.0409951210022 ms
densenet121 FP 16 avg over 10 runs: 84.02540683746338 ms
vgg16 FP 32 avg over 100 runs: 139.82041120529175 ms
vgg16 FP 16 avg over 10 runs: 120.87807655334473 ms
resnet152 FP 32 avg over 100 runs: 205.37179708480835 ms
resnet152 FP 16 avg over 10 runs: 193.45839023590088 ms