@ptrblck  The code is changed from https://github.com/open-mmlab/mmdetection/blob/master/mmdet/models/anchor_heads/free_anchor_retina_head.py.
match_matrix = bboxoverlaps(gt_bboxes, bbox_preds)
# match_matrix: torch.float32, shapes: [num_gts, num_anchors]
matched_iou, matched = torch.topk(match_matrix, 50, dim=1, sorted=False)

# cls_prob:  torch.float32, shapes: [num_anchors, 80]
temp = cls_prob[matched]

I have some difficulty reproducing this error, because this error happened very accidentally. Sometimes, it will appear immediately after training, sometimes, a few epochs it will appear, and sometimes, I can complete the training process. I am very distressed.