I don&#x27;t know much about this (it&#x27;s not my area of expertise), but I thought this G+ post was interesting: <a href="https:&#x2F;&#x2F;plus.google.com&#x2F;u&#x2F;0&#x2F;+VincentVanhoucke&#x2F;posts&#x2F;6RQmgqcmx2d" rel="nofollow">https:&#x2F;&#x2F;plus.google.com&#x2F;u&#x2F;0&#x2F;+VincentVanhoucke&#x2F;posts&#x2F;6RQmgqcm...</a><p>It says that a lot of the reason TensorFlow initially lagged in performance is because a lot of those performance issues only manifested under NVCC, whereas they had been using GPUCC internally.