kirk86:

For instance, I would have expected backward() to recognise that penalty_value depends on g which is the grad since I’m passing it explicitly in the call of backward() .


g = grad(mean_error, dummy_w)[0] will already remove the intermediates, so that backward cannot do anything about it anymore.



 kirk86:

BTW, which is better recommended from these two options?


It depends highly on your use case and I can’t give a general answer.