You can do it with pytorch (that’s what higher does  ).
Be careful that the nn module is not built for that unfortunately.
keep in mind in particular:

nn.Parameter can only contains leafs (gradient with no history).
optim.step() is NOT differentiable and won’t propagate gradients.

Good luck!
Don’t hesitate to post here if you need inputs.