You could create the unreduced loss via reduction='none', create a mask with 1s for all valid outputs/targets and 0s for the invalid ones, multiply the loss with it, reduce it (e.g. via mean), and calculate the gradients via backward().