Hi,
I am training a transformer and I see that the training is only using 1GB memory out of 8, which is obviously training very slowly. I want to know if there’s a way I can parallelize the training on the same GPU. I read posts like these but they seem to talk about multiple GPUs. I also thought of increasing the batch size but that might lead to an efficient model.
So is there a way I can increase the speed by parallelising?