You can figure it out from your gradient size (could just take size of your model checkpoint) + step time. For instance Resnet50 160ms per batch, 50MB checkpoint, therefore each worker needs to send and receive 50/.16 = 312 MB per second, means you need >=2.5 Gbps
What matters here is the ratio of compute time to parameter size. If you double computation + double parameter size, the network requirement is unaffected. Conv nets have good ratio of compute/bandwidth, transformers will need more bandwidth because of matmuls