You would have to explicitly set it in the creation of the criterion via reduction='sum'.
Weird behavior, indeed.
Could you add a F.relu between the last two linear layers?
Also, how large are your images?