Hi there,
_, outputs = self.model(batch) is not within the context manager as per your snippet.
Put a tab on the line to push it within the torch.no_grad() context manager.
Since you asked the other factors, one could be the batch size, in your case this might mot be the solution, but increasing the batch size means computing and keeping more gradients on your device, eventually eating up device memory. This is keeping in mind that you had a pretrained model and did not trained on your device, you are only inferencing.
Thanks