I’ve seen similar results when working with 3D ResNext models where group convolution is used extensively (in group size of 32). In 3D, the ResNext model actually only has around 50% of parameters (and thus flops) compared to the Resnet counterpart but the forward/backward pass is 25%/60% slower with batch size 1 and 3% and 10% slower with batch size 128 split to 4 GPUs.