<p dir="auto">Not sure if this is a great idea since I solved this problem for myself by buying a GPU with a lot of memory, but...</p>
<p dir="auto">One of the ways that is suggested to get models running on smaller GPUs is by reducing the batch size, however given the fact that the batch size is meaningful for the optimization procedure as well as the actual execution, it may be useful to decouple the batches that gradients are computed with vs the batches that are used to do optimization steps, i.e. run several smaller physical batches where gradients are accumulated across batches before a gradient step is taken.</p>
<p dir="auto">Most of keras is compatible with numpy / scipy.<br>
It would be awesome ( and probably competitively fast at inference on cpu) if keras was available with a numpy/scipy backend.</p>
<p dir="auto">This would help simplify inference significantly on a variety of machines ( in my case a non-raspbery pi arm computer). It would also be very fast.</p>
<p dir="auto">Alternatively a simplified keras inference engine that does not require tensorflow/cntk would great as well.</p>
<p dir="auto">Here is a link to something similar that unfortunately lost maintenance: <a href="https://github.com/riga/tfdeploy">https://github.com/riga/tfdeploy</a></p>