Solved by markdaoust in post #8 
              
              
                There’s a lot going on in this question. 
But you shouldn’t need to go that low level to do this. 
Given Alex’s response I hope we can get that issue fixed eventually. 
3 things that may help here in the mean time: 


GradientTape.gradient accepts an output_gradients argument. To run your gradient i…