Solved by harsha_g in post #10 
              
              
                @Sam_Lerman I can assure you my knowledge of Autograd is atrocious . However, just in case, did you take a look at this post  Exact meaning of grad_input and grad_output? 
On a side note, the two posts I was referring to were on either side of my first reply. Your original post and your subsequent p…