I found this thread really interesting because I am working on scaling my inference server (which uses uWSGI + flask + PyTorch on AWS with Elastic Inference), and when I increased the # of processes recently I came across some intermittent issues:
terminate called after throwing an instance of ‘c10::Error’
what():  [enforce fail at inline_container.cc:316] . PytorchStreamWriter failed writing central directory: file write failed
I couldn’t find any other references to this sort of error, but I am guessing it could be some kind of concurrency issue as I increased the number of processes.