whoab:

What do you mean two examples?


two examples refer to mean.to(v.device) and torch.zeros(...,device=v.device). They are solution to your  first 2 questions.
It’s not guaranteed to be distributed uniformly to all GPUs. how data_parallel distribute input is :

Slices tensors into approximately equal chunks and
distributes them across given GPUs. Duplicates
references to objects that are not tensors.