Thanks, @LeviViana
I have written some MPI codes years ago, if I remember right, this is for distributing the parallel code, and then open many process to run. It seems much more like Distributed Parallel (not sure)? And I have only one computer with 4 cards, I want to parallel the forward function within the main process, but using 4 GPU for acceleration. I wonder if this could work too?