Feed to forward new parameters as computed manually by update rule
Hi,

I would like to update the parameters of BertModel manually while training without using automatic optimizer to do it. I am trying meta-learning on BertModel and if I use automatic optimization on both inner and outer loops I get in-place operation errors. I also tried deepcopying but I cannot use deepcopy with DataParallel. So, I have opted for configuring my own manual optimization mechanism. how can I feed new parameters I computed from autograd of loss with respect to old parameters and update rule into the forward pass of BertModel manually? Is it a feature that exists? Otherwise is there a workaround?

Thanks,
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
