Thanks @tom,
The problem wasn’t related to multiprocessing, I even removed the LSTM in feedforward. and I tried both Conda and pip installations.
Recent updates solved this problem for us - https://github.com/ikostrikov/pytorch-a3c/issues/11 51