I encounter a similar issue today but the weird part is that when using l1_loss everything works well but after I switch to mse_loss, the RuntimeError happens.
  
    
    
    Switch loss function causes "RuntimeError: Found dtype Double but expected Float" quantization
  
  
    I tried to fine tune the ResNe18 model on GPU. The training code runs OK. However, I decided to further train the produced model using a different loss function (switched from L1 loss to L2 loss) but got the the error below. No other code or data changed. 
RuntimeError: Found dtype Double but expected Float

Any idea what might have caused the issue and how to fix it?