@ptrblck ，
Thank you！
I will try your ideas.
As loss.backward() will calculate gradient according to Chain Rule,  I am considering where the gradient is calculated.  that is what I am asking in this post:
I register all parameters in top Module and use each parameter in each forward function of sub-module. I am considering whether the gradient of each parameter will be calculated according to the definition Module or the Module where the parameter is used?