How large are your inputs?
Freezing the weights by setting their requires_grad attribute to False won’t avoid storing the intermediate activations and you should also wrap the code part in a with torch.no_grad() block.