I’m in need of productionizing some models (written in PyTorch) through a rest api.
Torchscript is a good candidate for this, however for many production settings serving a model involves many moving parts, i.e. batching, logging, monitoring, rolling out new models etc.
Tensorflow has tensorflow serving which solves many of these issues.
Is there anything on the roadmap for this kind of functionality anytime soon (e.g. Pytorch serving)?
I have seen the PR: https://github.com/pytorch/pytorch/issues/27610 6, and states that the target is Q4 for experimental release; will this be released with 1.4 / 1.5 or is it delay / cancelled / don’t know?
I have seen multiple blogs that talks about this topic, however most of them are simple example of flask application.
Thanks.