Solved by wish in post #2 
              
              
                I found it is my misunderstood. 
It is Broadcast::backward()  rather than Scatter::backward() gathers the grads from all of the devices.