One would imagine that on a Dyson sphere sized computer, intercommunication (limited by speed of light) will become major bottleneck. At some point amassing components will stop winning you performance due to physical dimensions.