This will work for now, but may break in the future. We’re still actively working on autograd internals, and there are two possible ways we can take now, but we’re still thinking which one is the best. The only caveat right now is that instead of BatchNorm you may find BatchNormBackward in the graph. Are you on slack? I can keep you posted about the currently used data structures if you want.