No problem!
Try replacing the numpy arrays with torch and see if that makes a difference. Note that if you are doing this in a loop, you can just create the tensor for indices once and reuse that in the loop.