Hello,
I have 4 GPUs available to me, and I’m trying to run inference utilizing all of them.  I’m confused by so many of the multiprocessing methods out there (e.g. Multiprocessing.pool, torch.multiprocessing, multiprocessing.spawn, launch utility).
I have a model that I trained.  However, I have several hundred thousand crops I need to run on the model so it is only practical if I run processes simultaneously on each GPU.  I have 4 GPUs available to me.  I would like to assign one model to each GPU and run 1/4 the data on each.  How can I do this?
Thank you in advance.