Oh interesting, when you weighted the smaller class by a factor of 10, it just struggled to converge in general right? (while in the original case without weights, it converges to a point where it always predicts the more populated class).
Is data augmentation or self-supervised pretraining an option, or perhaps just need more data instead of overweighting the smaller class?