Hi Sam!



 shartzog:

technically, that last label is “Recovered at Home”


As a side note, for the “None” case to actually be “Recovered at Home,”
there must be some unstated assumptions in play, namely, that
pneumonia always leads to hospitalization, and that the person was
indeed sick and either died or recovered.  (Otherwise, cases like
“Never Sick,” “Still Sick at Home (but not with Pneumonia)” would be
consistent with the “None” case, and “Recovered at Home (from
Pneumonia)” would not be consistent with “None.”


label interdependencies might influence model results.


My general view is that if you have uniformly-structured “label
interdependencies,” you should look for a way to build them into
your model.  A clear example is that if you know that exactly one
of your labels will be active for each sample, then you recognize
that you are actually working with a single-label, multi-class problem,
and you model it as such (rather than as a multi-label problem).
This way you make it easier for your training because you have
“told” your model that there will be exactly one label, rather than
having your model “learn” this fact.
But if your label interdependencies are, so to speak, “a little of this,
a little of that,” then you’re probably better off letting your model
learn those interdependencies through the training process, rather
than developing some complicated scheme to build them into your
model.


there’s another obvious inter-label relation present in my use case, namely you won’t be intubated if you’re not hospitalized.


Assuming that all the other labels can be present or not, independent
of one another – that is, that this interdependency is not part of a larger
uniform structure – I would consider it appropriate for your model to
learn this.  (And I would assume that your training data contains many
samples with “Not Hospitalized and Not Intubated,” “Hospitalized and
Not Intubated,” and “Hospitalized and Intubated,” but no samples with
“Not Hospitalized and Intubated,” so that your model can, in fact, learn
it.)


In the end, the likelihoods themselves are the critical output, not a hard label prediction.  …  the ultimate goal is to have the ability to predict “risk of an outcome” rather than the outcome itself.


If your training data is labelled with probabilities, then you can sensibly
train your model to predict probabilities.  But – to my mind, at least – if
your labels are hard labels (e.g., 0 or 1, but not 0.625), then I view the
probabilistic predictions (e.g., logits from a final Linear layer) made
by your model to be more an uncertainty about what the model has
successfully learned, rather than a prediction of a probability.  But
perhaps such a distinction doesn’t matter in practice, or perhaps it’s
purely semantic.
Best.
K. Frank