Have you tried in with a larger workload or just a single forward pass?
I think the startup might take much longer in the multi-processing case, so it’s maybe still faster in the long run.
But as I said, it’s a lot of speculation, since I haven’t used this approach yet.