I removed softmax from the code, and now the code looks like -
.....
return out

instead of
return F.softmax(out, dim = 1)

I trained it for 200 epochs, without pretrained embeddings. This is the loss vs epoch plot i got. 
Screenshot_2019-10-06 Google Colaboratory(1).png1177×594 73.3 KB

I dont understand, why is the loss increasing now? even without the the pretrained embeddings it should decrease albeit slowly.