Hi,
Thanks for the quick response. I start with the list of 10 networks, and move them to CPU with
model[i].to(‘cpu’)
and I put them in eval mode. Then, when I want to train one of them for an epoch, I say
model[i].to(‘cuda’)
model[i].train()
optimizer[i].zero_grad()
model[i].zero_grad() – I know this is redundant, just thought I’d try it
…
loss.backward()
optimizer[i].step()
…
models[i].eval()
models[i].to(‘cpu’)
Running a small number of models (2) seems to work fine.