Thank you @albanD so much! Yes indeed, I now find that argmax is not differentiable, and I can use CrossEntropyLoss instead. But how could one ucknowledge that from the error message in this case, how is it related? Also, maybe that would be helpful for a user to issue a warning that the differentiation flow has stopped there? Or it would not be possible to do?