Ah, now that explains a lot. Thanks for this insight.
Although it strikes as odd that backward() doesn’t understand that I’m still using the intermediate tensors because the values on which  I call backward() depend on those intermediate tensors.
For instance, I would have expected backward() to recognise that penalty_value depends on g which is the grad since I’m passing it explicitly in the call of backward().
BTW, which is better recommended from these two options? Thanks!

.backward(retain_graph=True)
grad(output, input, create_graph=True)