I met a strange problem that the same code executed on 3080 is much slower than on 2080.
Detailed environments on 3080:
CUDA 11.1
Pytorch 1.8
learn2learn 0.1.5
Detailed environments on 2080:
CUDA 10.2
Pytorch 1.1
learn2learn 0.1.3
My code clip can be summarized as:
optimizer.zero_grad()
learner = model.clone()
loss1 = torch.nn.MSE(learner(x), y)
learner.adapt(loss1)

loss2 = torch.nn.MSE(learner(x), y)
loss2.backward()
optimizer.step()

And I found most of the time is spent on the loss2.backward() step. Can anyone provide hints about this problem? Maybe the reason is pytorch 1.8 has changed some features related to clone()?