Solved by flauted in post #2 
              
              
                Your loss_function, CrossEntropyLoss expects raw logits and the end of your forward function applies a F.softmax. You should either not do the softmax and just return tag_space, or you should change it to a log_softmax and change your loss_function to NLLLoss.