can you provide a short example on how to do that? this 10 does not help on how to do use DistributedDataParallel.
Using DataParallel alone is easy but then it has the problem that others mentioned. i.e if one gpu can run 256 batches, using Dataparallel for 2 gpus does not let us use 2*256.