In your first case, the loss might decrease fast, if your model simply overfits the majority class(es), thus ignoring the minority class.
You could calculate the confusion matrix for both use cases and calculate the per-class accuracies to rate both approaches.
Note that a simple accuracy calculation might be misleading for an imbalanced use case as explained by the Accuracy Paradox 2.