Dear @mrshenli,
thank you for the kind and fast answer  Yeah, you’re right.
The following in your reply makes me drop my jaw:

it will do model replication […] in each forward pass

It was not clear for me from the beginning.
Somewhere I read that at least the scattering operation is done automatically segmenting the input tensor on the first axis, is it true?
Finally, just to know, in your opinion, is the code is at least correct for the DataParallel construct?