Thanks for the code. It seems you are using timeit instead of toch.utils.benchmark.
Adding this utility to your code I get a ~2x slowdown, which might be expected given you are calculating the gradients manually for a single parameter and not in a larger model.
Autograd would thus add some overhead, which would be visible for tiny workloads (and if that’s your use case, I don’t see an issue manually calculating the gradients if needed).
I would assume to see a benefit using a larger model and specifying some internal parameters, as you would have to manually write the backward pass otherwise.