Hi,
At the moment I am trying to implement a meta-learning algorithms and the size of the model is quite large so I am also trying  to use DataParallel. However, I am currently encountering an issue with one GPU taking the brunt of the load and running out of memory. This is since I generate weights for each sample in my batch, which means I have to loop over these weights and apply a functional conv and thus this operation cant be data paralleled and it ends up on the same GPU.
Is there any easy way to feed a batch of weights to a functional conv or are there any plans to implement this in pytorch in the near future?
Cheers,
Vincent