I’m not sure, if nn.DataParallel works with lists as inputs and I would recommend to check it with a very simple model.


nn.DataParallel will split the input batch in dim0 and send each chunk to the corresponding device. The result will be gathered on the default device again. Passing this tensor into the next nn.DataParallel module will perform the same splitting. The general workflow is explained here 14 in more detail. If you need more control how the data is split etc. you could try to use a manual approach by cloning some logic from the nn.DataParallel implementation.