u_i = self.conv_units[i](x)

Is this a Pytorch list or a python list?
It seems whether some layers aren’t being sent to gpu or you aren’t properly sending the input. Can you post the model and how are you allocating inputs?