You could create a mask using all black pixels to zero out the losses corresponding to these pixels.
After you’ve created the mask, you could calculate the unreduced loss, multiply it with the mask, average and finally call backward.