Yeah, I know the rule you talked about,it would update the parameters in dictionary by it’s corresponding learning late.But I still wonder how exactly it tells which parameters you have gradient and which return None,I guess it has something to do with grad_fn? @albanD would you please give USA details information? Thanks