Hi,
If net1 and net2 do not depend on each other you can just spawn 2 processes and train each model in its own process. You can do
import torch.multiprocessing as mp

def your_func(int rank):
   if rank == 0: 
      train net1
   else:
      train net2
...
    mp.spawn(
        your_func,
        args=(),
        nprocs=2,
        join=True)

Would that work for you?