Solved by ayalaa2 in post #2 
              
              
                So in general, if we perform a loss function on the output of a model, we would expect that any module (parameters) that the input came across would receive a gradient. If we’re finding that some of your parameters are not being given a gradient (thus not changing after a step), then they must not b…