Very late response, but I fixed this with a bit of a hack:
from torchvision.models.detection import rpn

old_onnx_get_num_anchors_and_pre_nms_top_n = rpn._onnx_get_num_anchors_and_pre_nms_top_n  # pylint: disable = protected-access

def _onnx_get_num_anchors_and_pre_nms_top_n_fixed(*args, **kwargs):
    num_anchors, pre_nms_top_n = old_onnx_get_num_anchors_and_pre_nms_top_n(*args, **kwargs)
    if not isinstance(num_anchors, int):
        num_anchors = num_anchors.item()
    return num_anchors, pre_nms_top_n

rpn._onnx_get_num_anchors_and_pre_nms_top_n = _onnx_get_num_anchors_and_pre_nms_top_n_fixed

Put this in the top of your file. It seems num_anchors is incorrectly returned as a cpu tensor (should just be an int) during onnx.export.
Edit:
On further consideration, I think the reason why it is converted to a tensor is to be sure the trace works, as it could end up converting the int to a constant when converting to onnx (it seemed to work fine for me with my posted solution, but could be a possibility). Perhaps the safer move is to replace num_anchors = num_anchors.item() with num_anchors = num_anchors.to(DEVICE) where DEVICE is given outside the scope of the function. Or for simplicity: num_anchors = num_anchors.cuda() if you know you will run it on cuda.