Solved by ptrblck in post #2 
              
              
                Based on the error message I guess that either Net or Normie_net is raising this error, since these are using batchnorm layers. 
In your custom conv layer you are hardcoding the device of the tensors to be CUDATensors, so I assume the output will also be on the GPU. Based on your code, conv_model sh…