Maybe these two questions can make things clearer:

In your application, what is the most important metric ?
In production, what is the distribution you expect to encounter (balanced or imbalanced) ?

My guess is that you want to apply early stopping for the metric the most matters under the distribution you expect to find in production.