markdaoust:

Given your requirements TFLite may be your best bet. It does have GPU support, and is designed for space-limited applications.


This It was requested also in another thread but  I suppose that currently we don’t have an Nvidia GPU delegate for this solution:

  
    
    
    Error when using TFLite interpreter in Flask General Discussion
  
  
    Yes TFlite has not a Nvidia/CUDA GPU delegate currently. 
On that GPU you need to use regular TF. See: