If you go through the neat hack of using MetaModule instead of the nn.Module provided by PyTorch as mentioned in my first response to your question, you will be able to write and run your meta-learning algorithms.
However, it was not clear until recently why it works only that way. Keeping in mind that nn.Parameters don’t record history, please go through my most recent response here: https://discuss.pytorch.org/t/higher-order-derivatives-meta-learning/93051 16
Hopefully, this will help you understand how to find a work-around to implement meta-learning algorithms without having to use packages such as higher 4, pytorch-meta 7, etc.