Sorry to cause the confusion. What I expect is that after I call model.forward with no_grad, only the outputs, i.e. o and h, occupy the memory and all the local variables inside the forward call is automatically freed. But as the example shows, I need to manually call torch.cuda.empty_cache() to free them.