Same problem happened when install nightly version by pip wheel, 1.9.0.dev20210428+cu111, python 3.8.8
  File "/opt/conda/envs/fpn/lib/python3.8/site-packages/torchvision/models/detection/rpn.py", line 363, in forward
    loss_objectness, loss_rpn_box_reg = self.compute_loss(
  File "/opt/conda/envs/fpn/lib/python3.8/site-packages/torchvision/models/detection/rpn.py", line 295, in compute_loss
    sampled_pos_inds, sampled_neg_inds = self.fg_bg_sampler(labels)
  File "/opt/conda/envs/fpn/lib/python3.8/site-packages/torchvision/models/detection/_utils.py", line 45, in __call__
    positive = torch.where(matched_idxs_per_image >= 1)[0]
RuntimeError: CUDA error: device-side assert triggered

/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:97: operator(): block: [0,0,0], thread: [2,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.

Same code, and problem solved by  pytorch 1.7.1 + cuda 11.0,  I think this may be the problem of cuda 11.1