In the past I have managed to do multi-label classification with MultiLabelSoftMarginLoss, which worked well, but that was with training data that had complete set of labels (i.e. for a single image there were no missing labels).
I am now wondering what loss function to use for the case where each training image has only one label. So for example, I have the labels blue, red, green, square-shaped, curve-shaped, car, bike etc (there are about 200 labels). For the label blue I have a whole set of training images with blue items. For the label car I have another set of images with cars.
This problem is kind of an extreme case of “multi-label with missing labels” (MLML). In MLML papers the images often still have multiple labels attached. In my case there is really only 1 label per image.
I have tried to simply use BCEWithLogitsLoss. With ImageFolder I simply convert each sample to a 1-hot label vector and train with that. It works OK, but the accuracy is not very high. I think this is because if I train with a blue image that shows a blue bike I actually tell the network the image is blue (correct) but NOT a bike (wrong!).
Which loss function in pytorch is suitable for this problem or should I try another approach?