In nn.DataParallel your model will be replicated onto the specified GPUs.
The data will then be split in the batch dimension and pushed to each replica.
What would your use case be, if A shouldn’t be split? The replicas won’t be able to use it in any calculation, since A would still be on your “master” device.