This is actually exactly related to my problem; working in a continual learning setting, the number of classes changes through time; If I want to use weight balancing, but the weights vector changes through time, what do I do here? Do I have to reinitialize my loss for every new task to use weight balancing?