Solved by ptrblck in post #2 
              
              
                I would recommend to remove the last ReLU and dropout layers and directly return the logits. 
Also, what’s the reason you are implementing the loss manually instead of using nn.CrossEntropyLoss?