Could you try to pass the target as a list of dicts, where each dict in the list corresponds to the boxes and labels for the element in the data list.
EDIT: This code should work:
model = models.detection.fasterrcnn_resnet50_fpn()

x = [torch.rand(3, 300, 400), torch.rand(3, 500, 400)]
y = [{'boxes': torch.tensor([[0.6547, 1.0000, 0.4703, 0.4938],
        [0.7802, 0.9403, 0.0472, 0.0796],
        [0.9897, 0.9823, 0.0000, 0.4879],
        [1.0000, 0.4943, 0.8626, 0.3522],
        [1.0000, 1.0000, 0.0000, 0.0000],
        [1.0000, 0.2750, 0.8687, 0.0000],
        [0.9956, 0.7544, 0.8100, 0.5774],
        [1.0000, 0.9531, 0.0000, 0.1137]]),
     'labels': torch.tensor([227, 445, 432, 307, 390, 281, 419, 333])},
    {'boxes': torch.tensor([[0.6547, 1.0000, 0.4703, 0.4938],
        [0.7802, 0.9403, 0.0472, 0.0796],
        [0.9897, 0.9823, 0.0000, 0.4879],
        [1.0000, 0.4943, 0.8626, 0.3522],
        [1.0000, 1.0000, 0.0000, 0.0000],
        [1.0000, 0.2750, 0.8687, 0.0000],
        [0.9956, 0.7544, 0.8100, 0.5774],
        [1.0000, 0.9531, 0.0000, 0.1137]]),
     'labels': torch.tensor([227, 445, 432, 307, 390, 281, 419, 333])}
]

output = model(x, y)