Hi guys, I implement the Decoupled Neural Interfaces using Synthetic Gradients in pytorch. The paper uses synthetic gradient to decouple the layers among the network, which is pretty interesting since we won’t suffer from update lock anymore. I test my model in mnist and almost achieve result as the paper claimed.

  
      
      GitHub
  
  
    

andrewliao11/dni.pytorch 74

Implement Decoupled Neural Interfaces using Synthetic Gradients in Pytorch - andrewliao11/dni.pytorch


  
  
    
    
  
  


reference:

Decoupled Neural Interfaces using Synthetic Gradients 3
Understanding Synthetic Gradients and Decoupled Neural Interfaces 8