I did not understand you at all. Dataprallel is not compatible with cpu training. You can manually assign each submodule to one gpu or dataparallel.
Using dataparallel in parallel with cpu is not worth as you will bottleneck the whole process. It will be faster if you allocate as much as possible into gpus and the least possible in cpu but sequentially.