Inplace is a nice late phase optimization, but the root problem here appears to be that Google&#x27;s convolution kernels are <i>crap</i> compared to those in cuDNN3 and Neon (I asked Scott Gray about this directly and I trust his wisdom).<p>No surprises there whatsoever.  The tensorflow engine is the most gold-plated POS I&#x27;ve seen in a long time.  if it were running on 1000+ servers, I&#x27;d get the level of overengineering they&#x27;ve applied here.  But single server pthreads?  WTF?<p>Also parameter server is dumb unless you sweat the implementation of the gather&#x2F;reduction ops, but I digress.