Actually I was interested in the loss itself and not the gradient.
To be specific about my use case I wanted to use two models in sequence and propagate the loss from the 2nd model to the 1st.
Any suggestions on how this can be accomplished?