I want to do similar things, basically like in meta learning, you have a function y = f_{\theta}(x,t), and you update your variable t such that f(x,t) approximates the target y_target. The mathematics is like this: t’ = t-\nabla_t |f(x,t)-y_target|^2, then you update \theta using \theta = \theta-\nabla_{\theta}|f(x,t’)-y_target|^2. here you need to first update your t, and then once t is updated, you want t to still carry the gradient and then you update the theta variables. I don’t think the proposed solution can do this? But in tensorflow this is very easy and you can update t for multiple time, which means it can not only take the secondary derivative but also the third derivative and more. Is pytorch able to achieve this?