I’d like to deploy four of my models with a total size of ~100mb when the state saved on disk. It would be great if the docker could take as small space as possible, no more than 700 mb.
Now I’m creating docker and install a few dependencies. Suddenly it takes 2.8 GB on disk, where PyTorch and related libraries take at least 800 MB in conda.
Therefore I’m looking for a simple way to deploy these models. As far as I understand I could use jit and e able to run models with small library libtorch. However, I’d like to run the model in Python, not C++. Is it possible?
And the second thing I don’t understand is whether I should use onnx or jit? What is the difference?
I’m looking for a simple guide with the steps to do that: export with jit and load with some lib I don’t know in Python? Or export to ONNX and then to TensorFlow? Or maybe get rid of conda and just install some PyTorch version and it should be no more than 80 MB on disk?
Any help appreciated!