Solved by ptrblck in post #2 
              
              
                The necessity of calling criterion.to(device) would depend on the used criterion and in particular if it’s stateful, i.e. if it contains internal tensors etc. 
In the latter case, you should get a device mismatch error in case this line is missing and needed, so in case you are not getting any error…