Solved by ptrblck in post #2 
              
              
                The trainable parameters will be updated by the optimizer, once gradients were calculated in the backward() pass and optimizer.step() was called. 
In your current code snippet you are printing the value of self.gamma before and after using it in the forward, so it’s expected that these values weren’…