I have found that the problem was caused by two things:


WeightedRandomSampler.

Dropout.

If I use either of them, or both, it is very difficult, even impossible, to fit the data. I would appreciate a lot if somebody could refer me to some references that discuss this kind of problems. Thanks.