Hello everyone,
I’m currently trying to build a classifier for a multilabel classification where multiple annotators annotated one label each.
Let’s say the label vectors for annotators 1, 2, and 3 look like this:
A1: [1, 0, 0, 0]
A2: [1, 0, 0, 0]
A3: [0, 0, 0, 1]
I could only use the majority label and use [1, 0, 0, 0] as my label vector - or I could make it a multi-label classification problem with [1, 0, 0, 1] as my labels.
But is there a way to weight the loss based on the distribution of labels?
For instance, a loss function that could use [0.66, 0, 0, 0.33] as the label vector?
Thanks!