Ok, I analyzed the output predictions from epoch 0, and it starts predicting class 1 for some examples… But 1 or 2 epochs later it stops predicting class 1 and starts predicting almost all examples as class 0 from epoch 1 and beyond. Also, investigating my dataset, both training and validation are very unbalanced for both classes, with 77% of examples in class 0 and 23% in class 1. Maybe the LSTM is overfitting and predicting only class 0 ?
I gonna try the pos_weight of the BCEWithLogitsLoss and see what happens.