Ah right. The problem is that we’re numbering GPUs starting from 0. So all the modules and data is on GPU0, but you’re telling the DataParallel to run on GPU1 and GPU2 (i.e. 2nd and 3rd GPU). Can you change that and see if it helps? If that’s it, then we need to improve the error message.
Also, unless this is a helper used in forward that’s defined somewhere else, you need to call it like that. @fmassa wrote a nice explanation why.