vpolflie:

I was wondering if there is an easy way to write something like this which still allows for data parallelisation:


This should work, as DataParallel simply replicates model and scatters input. (assuming self.conv is a customized conv layer that replaces weight) So if you wrap that with DataParallel, different thread/replica should see samples/weights on a different device. Did you encounter any issue when doing this?