gunnxx:

Hi, those equations are an example of how you do MAML with two steps of gradient descent in one task. f is the loss function. The first two lines are the two steps of gradient descent for the task objective. The third line is the gradient descent for the meta objective.


Ah I see so it was part of the inner loop. That makes sense.



 gunnxx:

Ha, I found this article! We can do data parallelism + centarlized synchronous update.


Unfortunately I can’t read that article because I don’t subscribe to medium.
I’m using nn.DataParallel() currently. I was just asking if it works when you aren’t using backwards()? I guess maybe you’d use this? torch.distributed.autograd
P.S. do you know how many subtasks you need for meta-learning to be viable?