Are you sure that you send this param to the optimizer? Would be better to help if you provide more code, especially for the optimizer creation and forward() function to see where you use the self.alpha