someAdjectiveNoun:

I was confused about using if score.requires_grad: on the score variable as it is not a leaf-node variable, but an intermediate-node variable.


The intermediate Tensor will have have their requires_grad=True in python. So you can use that as a hint if they are used in the computation of something that needs gradient computed.