It sounds like you may have already seen this, but we often refer questions on PyTorch model serving/deployment to this post:

  
      
      pytorch.org
  
  
    

PyTorch 6

An open source deep learning platform that provides a seamless path from research prototyping to production deployment.


  
  
    
    
  
  


A few open source model servers support PyTorch directly. Some users roll their own, as in the flask example you referenced. Some take an ONNX export of a trained model and load that into another model server.
Note that for the ongoing work you referenced:

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      [FR][RFC] Build a serving framework to host and serve trained PyTorch models 4
    

    
      
        opened 
        
          
        
        Oct 9, 2019
      
      


      
        
          
          vdantu
        
      
    
  



  🚀 Feature
Document: https://torchserve-docs.s3-us-west-2.amazonaws.com/docs/torchserve_architecture_v0.pdf
NOTE: The above document is a draft and subject to change
Request to build a Serving framework to host and...



    feature
    triaged


  
  
    
    
  
  


This would be a separate application, not part of the PyTorch code itself, so wouldn’t necessarily be tied to PyTorch version numbers. Right now we’re ironing out requirements and basic design, but we’d be interested in any feature requests or feedback.