text: Title: Multi-node multi-GPUs training is slower than single-node multi-GPUs training[BUG]
  Body: **Describe the bug**
  I utilized zero-stage3 of Deepspeed for fine-tuning the Qianwen2-1.5b model and observed that the **training speed of 2 nodes with total 4 A10 GPUs is one time slower than that of single node with total 2 A10 GPUs**.  Here are some details.
  
  The training speed of 2 nodes with total 4 A10 GPUs：
  <img width="1521" alt="image (1)" src="https://github.com/user-attachments/assets/7b569170-4cdd-4851-8224-1ba404c83213">
  <img width="671" alt="image" src="https://github.com/user-attachments/assets/edb69c6f-a2be-4c5a-acb0-7071fbd44ef8">
  The training speed is about 8.68s/iter, and  the forward  latency, backward latency is 1.6s, 2.51s.
  
  However, the training speed of single node with total 2 A10 GPUs：
  ![image (2)](https://github.com/user-attachments/assets/2aeea642-7158-4763-986b-7a3ecb77004c)
  <img width="660" alt="image (3)" src="https://github.com/user-attachments/assets/463e50e5-8b67-45e5-9049-4a17557e8ef5">
  The training speed is about 2.46s/iter, and  the forward  latency, backward latency is 357ms, 673ms.
  
  The above results show that the training speed of 2-node 4GPUs is much slower than that of single-node GPUs in **feedforward and feedback processes**. I thought it was a bandwidth problem of network, but my calculations showed it wasn't as follws：
  
  The average receiving and sending bandwidths during the training were 8.74Gbit/s and 9.28Gbit/s, respectively.
  model weights size: 1.5\*(10^8)\*16bit, gradient size: 1.5\*(10^8)\*16bit, the communication consume is：
  3\*1.5\*(10^8)\*16bit/2/(8.74\*(10^9))=0.41s.
  
  So I want to know what's wrong with the results?  I'd like to ask for people's help.
  
  Thanks!