Solved by ptrblck in post #8 
              
              
                This would explain, why your model predicts random classes for these pixel locations. 
Basically, your model will not get any information for these locations, as they are ignored, and can just output any random class. In your case it seems the vegetation class was picked. 
You could transform all 25…