Did you wrap the code in a with torch.no_grad() block, since you are dealing with an inference use case?
This will make sure to avoid storing the intermediate tensors, which would be needed to calculate the gradients during the backward call.
If you’ve already done that, could you check how close you are to the memory limit on your device using dim1=2000? If you are close to an OOM, creating multiple instances of smaller tensors, could yield the OOM e.g. due to memory fragmentation.