Could you post some information about the last non-linearity your model is using and also with criterion you are applying?
Note that nn.CrossentropyLoss expects raw logits, while nn.NLLLoss needs log probabilities calculated by F.log_softmax.
Also, could you check the input and output of your model for the high loss values?