So given a deep model f=(f_3 . M . f_2 . M. f_1), when we evaluate f(x), M is applied twice. Once to f_1(x) and once to f_2(M(f_1(x))), the output at each point will of course be a new tensor.
If T is not constant, and if we needed to compute the gradient w.r.t. T and the input to M, we would need to save the both T and the inputs to M in the forward pass to be able to compute the backward pass. But since T is constant, and we only need to compute the gradient w.r.t. x, the backward pass of M is itself constant (it’s T).
The question is whether I can safely assume that autograds computation graph is constructed in such a way that it does not store unnecessary copies of T. (This might very well be a naive question, with an obvious answer, but I’m not super knowledgeable about the inner workings of autograd  )