What are you doing with these child modules inside the forward?
If you are calling model.module(input), you would be using a single GPU again, as nn.DataParallel would scatter the input in its forward method (called via model(input)).
Note that nn.DataParallel will create model replica on each device, so that manipulations in these replica won’t be visible in the main model.