For example, how to define a suitable “zero loss” so that it has no effect in net’s backward/step when I have not any box to regress.
...
if positive_count > 0:
    positive_indexes = (labels != 0).nonzero()[:, 0]
    boxes = boxes[positive_indexes]
    box_target = box_target[positive_indexes]
    box_loss = F.smooth_l1_loss(boxes, box_target)
else:
    box_loss = ???
box_loss.backward()
net.step()
...