github.com/ikostrikov/pytorch-a3c
  
  
      
    
  


  A cleaner solution to grad sharing problem



  by apaszke
  on 09:36PM - 14 Mar 17 UTC



  2 commits
  changed 3 files
  with 12 additions
  and 22 deletions.


  
  
    
    
  
  


In ensure_shared_grads, wouldn’t shared_param.grad NOT be None after the first time the parameters are updated? In my testing it seems that the function returns without syncing the gradient in every step after the first. Related discussion here.

  
    
    
    Problem on Variable.grad.data? 
  
  
    Thank you~ For the safe issue, i noticed that DM’s paper explicitly said they don’t put a lock on the shared weights. 
For your solution to the shared grad to pytorch-a3c. 
def ensure_shared_grads(model, shared_model):
    for param, shared_param in zip(model.parameters(), shared_model.parameters()):
        if shared_param.grad is not None:
            return
        shared_param._grad = param.grad

Will this code restrain the shard_model grad only being bounded with one local_model? 
Cause sha…
  


def ensure_shared_grads(model, shared_model):
    for param, shared_param in zip(model.parameters(), shared_model.parameters()):
        if shared_param.grad is not None:
            return
        shared_param._grad = param.grad

In DM’s paper they say they perform async updates without locks. What does it mean?
what will happen if shared_param.grad is changed by another thread after returning from ensure_shared_grads?
Will the gradient of the current thread be discarded?