@JuanFMontesinos Thank you very much for the reply. I have already tried both dataparallel and distributeddataparallel. I thought the former would replicate the model in multiple gpus so all of them would have the embedding layer, making no difference in this case, am I right?
For the distributed one, I had to lower the batch size from 8 to 2 for it to run without memory, offsetting its increased speed.
P.S. I am using 4 GPUs. Have been stuck on this issue for days…