Hi, what you can do is to use dataparallel twice inside the nn.Module like this:
class model(Module):
def init():
   self.subnet1 = DataParallel(net1).to(device)
   self.subnet2 = DataParallel(net2).to(device)
def forward():
  x = self.subnet1(x)
  # Here x is gathered from all gpus
  x = self.subnet2(x) # DataParallel again
  return x