After playing around a little more I think it comes down to being able to do the following:
Let f and g both be instances of the same nn.Module. Throw some input values through g when g is in training mode. Now the parameters in g have gradients and are part of a computation graph. Let A be a parameter in f and B be the corresponding parameter in g. Now set C=B.clone(). Here’s the bit I don’t know how to do, reassign A to be C.
This seems like a use-case that should be possible with PyTorch without being too hacky. Anyone have a path forward?
Thanks!