Autograd will create a tree with one branch for each operation (each “+” in your example) applied to a variable. That’s the only way to automatically compute the derivative. If you want to save memory, you can indeed create your own backward, so that a call of your whole function will count as a single operation in the tree.