The data outside of these two labels is in my dataset and I want the classifier be neutral to those labels, i.e., gives zero probability for labels 0 and 1 for images which have different labels than 0 and 1. So, the targets can not be the span of 0 and 1.