I shall upload the code to this repo once I am done:

  
      
      GitHub
  
  
    

QuantScientist/DarkTorch 43

A low-latency deep learning inference server using pytorch C++ frontend & NVIDIA GPUs.  - QuantScientist/DarkTorch