One approach would be to store the model copies from each GPU in e.g. a list, push the data chunks to the appropriate device, and execute the training using each model:
models = [modelA.to('cuda:0'), modelB.to('cuda:1), ...]
dataA = dataA.to('cuda:0')
dataB = dataB.to('cuda:1')
...
data = [dataA, dataB, ...]

for model, input in zip(models, data):
    output = model(input)
    ...

As long as the workload is not tiny and the CPU can schedule the kernel launches and run ahead, you should see overlapping computation.