if len(state) == 0:
                    state['step'] = 0
                    # print(p.data)
                    # Exponential moving average of gradient values
                    state['exp_avg'] = torch.zeros_like(p.data.to_dense())
                    # Exponential moving average of squared gradient values
                    state['exp_avg_sq'] = torch.zeros_like(p.data.to_dense())


I have to change the code in sparseadam, now it is working. But I am not sure if it is working correctly.