I was reading through this tutorial 2 on Text Classification and I can’t really see what’s wrong with your model.
One thing that could be a problem is with the cost function BCEWithLogitsLoss , after looking at the docs they pass a pos_weight variable. (which defaults to None). Perhaps this could be the issue? Maybe you’re multipling all your losses by None which would give a near 0 gradient?