Actually You need not create any tensor for labels and boxes. The collate_fn will create the tensor for those labels and targets. Remove that torch.from_numpy() and try again.
I got the same issue today morning, the problem was I created tensor for labels.

image1920×1080 144 KB