I think it depends on your use case.
As far as I understand your current PyTorch model runs in a Flask app and works.
Do you see any bottlenecks regarding the performance, i.e. do you expect a huge workload which your current approach cannot handle?
I would look for the current bottlenecks and optimize according to these.