No, optimizer.zero_grad() will set the .grad attributes of all passed parameters to zeros (or to None if set_to_none=True is used). The computation graph (and thus the intermediate tensors) will be freed during the backward() operation in the default setup (i.e. if retain_graph=False is used).