Solved by ngimel in post #7 
              
              
                Try modifying your forward to also return the attribute you are interested in, e.g.: 
y = self.conv1(x)
x = self.conv2(y)
return x,y

and inspect returned values of your net. Or use distributed instead of DataParallel.