I have written code using all PyTorch functions like torch.mul, torch.sub etc.
With the increase in batch size (for my example 32—>64) the time for the code increases (I am running my code on Titan XP GPUs). I thought PyTorch does computation in parallel.
Is this because I am bottlenecked by the pipelining in GPU?