I’m new to semantic segmentation and distribute dataparallel of PyTorch.
And I run this pspnet 1 code on my machine (2 x P100).
To my knowledge, larger batch size usually brings shorter training time. But it didn’t happen this time.
On one GPU, the training time with batch_size=8 is actually the same as (even a little longer) than batch_size=2. Specifically, the training time for 10 iterations with bs=8 is 4 times of that bs=2.
Did I make some mistake here?