Solved by ptrblck in post #14 
              
              
                Balancing the dataset should trace the accuracy of the majority class for an increase in the accuracy of the minority classes. 
I’ve created a tutorial a while ago here, which is quite old by now, but might give you some insight.