I think you have a typo in the network definition (F.losoftmax). Apart from that it looks ok I think. I haven’t compared that to the original architecture, but it looks good to me. Maybe you have too large lr? Did you investigate why the loss becomes NaN?