Solved by ptrblck in post #4 
              
              
                I would not expect a perfect 0 as the final loss in any case alone due to the limited floating point precision (but a small value instead). 
By “standard” criterion I meant the loss functions included in torch.nn assuming you are not inverting them or add any constant etc. 
E.g. depending on your us…