Performing inference with deep learning modes is computationally costly as well. Its a reason why NVidia has two streams of business, Tesla &#x2F; Titan &#x2F; upcoming Volta GPUs to train models and Tegra boards to enable near real time inference.