I don’t know, if there is a good implementation of this approach for this loss function, but instead of using a new “missing” label, you could also ignore the missing label.
To do so, you could calculate the unreduced loss first with any label (including a missing label) via reduction='none' while creating the loss function. Afterwards you could multiply this unreduced loss with a mask to set the missing losses to zero, and reduce it e.g. via mean().