As long as these embeddings are part of the computation graph on which backward was called, they will get their gradient assigned (check embeddings.weight.grad after the backward call). To modify these weights, either you have to manually update them or you can pass them to the optimizer that will do the update for you (embeddings.weight is an instance of torch.nn.Parameter so model.parameters() returns them as well)