Depending on the used operations the intermediate tensors would be needed to calculate the gradients and you won’t be able to delete them unless you disable the gradient calculation e.g. via wrapping the code in a with torch.no_grad() block.