This overhead is mainly the discovery of what needs to be done to compute gradients. So it needs to traverse all the graph of computation, which takes a bit of time.
Note that if you’re simply experimenting, this overhead won’t kill you. But it won’t be 0.