That’s an interesting idea, but it really is a gamble. You shouldn’t assume anything about the state the library functions retain for backward, so your code could work just fine in one version, and be silently broken in another one. I think it’s safer to implement such things as a “fused” autograd function yourself. You don’t want to waste weeks of experimentation to discover bugs like these only later.