text: Title: when I finetune the model use deepspeed on 2 4*A800s，log only contain worker1
  Body: when I finetune the model use deepspeed on 2 A800，log only contain worker1，no worker2.
  Is there any way to print the loss of Worker2? The GPUs on both machines are running normally, and the GPU memory is floating normally。
  The script I use
  
  ![image](https://github.com/user-attachments/assets/10278c41-6799-4ca8-81d5-1732c15b7e1d)
  
  
  The zero2.json I use
  
  ![image](https://github.com/user-attachments/assets/40b0652a-b042-4e04-b1e1-2faf9a08f27f)
     
  
  
  The log
  ![image](https://github.com/user-attachments/assets/e2a3b73a-9abf-4d28-9c19-5896603c25d2)
  