piapple:

When I try to print out the values of self.model.time_factors.weight, the values do not change over time. However, then I try to print the values of self.model.time_factors.weight.grad, the values change.


You don’t show how you instantiate the optimizer and which, maybe it’s not in there for some reason or maybe the gradients are too small? SGD runs into trouble when the things you are optimizing work at different scales or there is a scale mismatch between the optimized quantity and the gradients (Adam, LARS/LAMB less so).
Best regards
Thomas