blade:

I didn’t quite get your point on this one.


I might have misunderstood the original question.

After fixing the second part, my output for
print(model.module.first_term(input_)[0])

is always on the first worker:
tensor([0.0020], device='cuda:0', grad_fn=<SelectBackward>)


Is the question about why the output of print(model.module.first_term(input_)[0]) always on cuda:0?