Hi Alban,
Thanks so much for breaking that down for me; I’m glad its expected and not a bug. My overall goal is for the graph to continue through the update of the weights/biases in my network by the gradient, because I want to use future losses to update my matrix M.
What I am doing is passing data through the model, getting gradient, applying ops to it using M, updating parameters with that transformed gradient, and then repeating. I am hoping that I can differentiate the loss calculated using parameters updated through gradients modified by M in order to update M. Will what you suggest enable me to do that?