Sorry, my bad. I didn’t read the link properly.
Yes, I do have 1 GPU per model. I’m still confused about how to use multiprocessing with GPUs though. The links I’ve seen use something like
model = torch.nn.DataParallel(model, device_ids=[0,1]).cuda()

or
replicas = nn.parallel.replicate(module, device_ids)
outputs = nn.parallel.parallel_apply(replicas, inputs)

However what I want to do is run training functions in parallel and change the hyper parameters. Could you tell me how we can assign GPUs to functions and run them in parallel with multiprocess?