Someone on twitter suggested the onnxruntime might be worth looking into

  
      
      GitHub
  
  
    

microsoft/onnxruntime 6

ONNX Runtime: cross-platform, high performance scoring engine for ML models - microsoft/onnxruntime


  
  
    
    
  
  


including here for ppl stumbling upon this.