<i>But that&#x27;s the state of things, we have impressive results but from huge neural nets that we really cannot run in realtime on a mobile device.</i><p>This is absolutely incorrect.<p>A mobile device can execute a pretrained model fine. See, for example [1][2][3]. Google&#x27;s new TensorFlow NN system is explicitly designed to be able to run on mobile devices and comes with a pre-trained image classification NN that workd fine on mobile devices.<p>This doesn&#x27;t mean that energy saving is unimportant on a mobile device of course. But there are very widely deployed production systems (eg all of Android) that use them now with no special GPU acceleration.<p>[1] <a href="http:&#x2F;&#x2F;googleresearch.blogspot.com.au&#x2F;2015&#x2F;07&#x2F;how-google-translate-squeezes-deep.html" rel="nofollow">http:&#x2F;&#x2F;googleresearch.blogspot.com.au&#x2F;2015&#x2F;07&#x2F;how-google-tra...</a><p>[2] <a href="http:&#x2F;&#x2F;googleresearch.blogspot.com.au&#x2F;2015&#x2F;09&#x2F;google-voice-search-faster-and-more.html" rel="nofollow">http:&#x2F;&#x2F;googleresearch.blogspot.com.au&#x2F;2015&#x2F;09&#x2F;google-voice-s...</a><p>[3] <a href="http:&#x2F;&#x2F;static.googleusercontent.com&#x2F;media&#x2F;research.google.com&#x2F;en&#x2F;&#x2F;pubs&#x2F;archive&#x2F;41176.pdf" rel="nofollow">http:&#x2F;&#x2F;static.googleusercontent.com&#x2F;media&#x2F;research.google.co...</a>