Hi, I was wondering if it makes sense to just reset relevant gradients between loss.backward() and optimizer.step() to achieve this goal. So basically do the following: word_embed.weight.grad[1000:] = 0.