No, I use the usual __call__ interface (ie: treat the module instance as a function). It actually calls my forward_() method, but the self in that call is the original module instance, not the replica created by DataParallel. That means it’s parameters are not on  the same gpu as the inputs passed to forward.
I suspect it’s something related to the way python dynamically populates the self variable - when I do self.forward = self.forward_, the current self (ie: the original one that executes __init__) is associated with forward_, and this association isn’t changed when DataParallel clones the module. I looked at it with a debugger, and id(self) within the forward_ method is different to id(self) in Module.__call__() immediately beforehand (at least for threads working with a different gpu).
I put this here partly for others to find, should they encounter this behaviour, and partly because I don’t understand how that can happen, how the mechanism works to cause this behaviour.