Hi,
You will have to use the distributed autograd engine to be able to run autograd across multiple processes. But it is experimental right now.
But for your problem, if your tasks are very small, you might want to make sure that the mutltiprocess overhead won’t be greater than the gain.