The error seems to be:
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:605: indexSelectSmallIndex: block: [0,0,0], thread: [0,0,0] Assertion `srcIndex < srcSelectDimSize` failed.

most likely caused in:
x1, x2 = sanitize_coordinates(boxes[:, 0], boxes[:, 2], w, padding, cast=False)

so I assume the indexing in boxes fails.
You could rerun the script with CUDA_LAUNCH_BLOCKING=1 python script.py args and verify the line of code raising this error.