I believe I found the error: I did =+ rather than += when computing the average loss. Now I get:

Screenshot from 2019-09-14 18-37-54.png1175×426 38.8 KB

Nonetheless, they are now both very large! The scores are always the same, obviously. So now the question is: does it matter if the loss is high (I’m using cross-entropy)?