Hi all,
I’m working on a implementation of MAML (paper link 20). There’s a few implementations out there but from what can see they all rely on the functional form of a model. I have a requirement to make this as general as possible so it can be used with a variety of underlying models. To that end I would like to be able to copy a nn.Module instance in the same way one can copy a variable or parameter. The key is that the gradients need to be able to get back to the copied nn.Module.
I’ve tried to hack something together via state-dicts and hooks but I keep running into cases where I need to copy a nn.Module without removing it from the computational graph.
Any thoughts on the problem would be appreciated!
Thanks!