It seems that it I don’t use my dataloader then even without parallelizing the Linear layers, the GPU utilization is full. In case of my dataloader I am loading the data from a pickle file in the constructor and then doing random sampling using torch.multinomial to generate triplet samples. Do you have any suggestions on it?
I have also enabled pin_memory.