If balancing through the multinomial sampler works to hopefully balance the batches then that’s good news, but wouldn’t there be an optimal num_samples in that case to make this happen? For example if you sample 2 weighted images from 6 million… then realistically your low weighted images would never get picked but choosing 5.9 million for a batch would cause the weighting to become obsolete as the sampler has to boost the excess space with more low weighted images to the point at which they unbalance the batch again.
I should note that I am using sampling with replacement here.