I was reading here 135 that:

In the next iteration, a fresh new graph is created and ready for back-propagation.

I am wondering. When exactly is the fresh new graph created? Is it when we call:

optimizer.zero_grad()
output.backward()
optimizer.step()

or at some other time?