Thanks for posting the question. Your analysis is correct, it should be equal to link bandwidth for N=2 (provided the input dimensions are large enough for the runtime to be dominated by the bandwidth). The implementation in Gloo ensures there is always 1 chunk in flight and 1 chunk being reduced, so it is possible there is something wrong there for the N=2 case.
I’ll investigate and post back here.
edit: I added https://github.com/facebookincubator/gloo/issues/169 13 to not lose track of it.