Correct, in the label generation process, I assign a label to every patch for every class (including background). This is possible because of the dataset I am working with which comes with a segmentation mask that is used during the labelling of every region.