I asked this question on stack a while ago:

  
      stackoverflow.com
  
  
      
    
  

  Numerical differences between manually computed gradient and PyTorch chain rule 45



  python, deep-learning, pytorch, backpropagation



  asked by
  
  
    jean
  
  on 12:16PM - 22 Mar 19 UTC


  
  
    
    
  
  


Does someone have an explanation ? Are there some operations that are more numerically stable, that I should use ?
What are the order of magnitude in the relative differences that should be expected ? Thanks again.