That’s what I’m doing now, but it still doesn’t help me much when the two calls are in different parts of my code. Call A needs output 1, and call B needs output 2; I don’t want to have to pass around stuff in order to do both calls. I just want to have the intermediate computations cached by default, until e.g. .backward() is called.
If there’s no way to do this currently, might it be worth adding as a feature? I think it would bring great efficiency gains to many workflows, and enable much more elegant code. Tensorflow 1.x was able to do this natively, and I used that ability all the time (which is how I developed this coding style).