Hi there,
i am experiencing a similar problem in that the embeddings in my model don’t seem to change at all. As an example to find out what is going on I actually used the pytorch code from the word embeddings tutorial 53. This is what I am running:
import torch
import torch.nn.functional as F
from torch import nn, optim, autograd
import numpy as np

CONTEXT_SIZE = 2
EMBEDDING_DIM = 10

test_sentence = """When forty winters shall besiege thy brow,
And dig deep trenches in thy beauty's field,
Thy youth's proud livery so gazed on now,
Will be a totter'd weed of small worth held:
Then being asked, where all thy beauty lies,
Where all the treasure of thy lusty days;
To say, within thine own deep sunken eyes,
Were an all-eating shame, and thriftless praise.
How much more praise deserv'd thy beauty's use,
If thou couldst answer 'This fair child of mine
Shall sum my count, and make my old excuse,'
Proving his beauty by succession thine!
This were to be new made when thou art old,
And see thy blood warm when thou feel'st it cold.""".split()

trigrams = [([test_sentence[i], test_sentence[i + 1]], test_sentence[i + 2])
        for i in range(len(test_sentence) - 2)]

print(trigrams[:3])

vocab = set(test_sentence)
word_to_ix = {word: i for i, word in enumerate(vocab)}


class NGramLanguageModeler(nn.Module):

    def __init__(self, vocab_size, embedding_dim, context_size):
        super(NGramLanguageModeler, self).__init__()
        self.embeddings = nn.Embedding(vocab_size, embedding_dim)
        **self.embeddings.weight.requires_grad = True**
        self.linear1 = nn.Linear(context_size * embedding_dim, 128)
        self.linear2 = nn.Linear(128, vocab_size)

    def forward(self, inputs):
        embeds = self.embeddings(inputs).view((1, -1))
        out = F.relu(self.linear1(embeds))
        out = self.linear2(out)
        log_probs = F.log_softmax(out)
        return log_probs


losses = []
criterion = nn.NLLLoss()
model = NGramLanguageModeler(len(vocab), EMBEDDING_DIM, CONTEXT_SIZE)
optimizer = optim.SGD(model.parameters(), lr=0.001)

for epoch in range(100):
    total_loss = torch.Tensor([0])
    for context, target in trigrams:
        **print('----------------------------------------')**
        **print('Before update')**
        **before = model.embeddings.weight.data.numpy()**

        context_var = autograd.Variable(torch.LongTensor(context_idxs))

        model.zero_grad()

        log_probs = model(context_var)

        loss = criterion(log_probs, autograd.Variable(torch.LongTensor([word_to_ix[target]])))

        loss.backward()
        optimizer.step()

        **print('After update')**
        **after = model.embeddings.weight.data.numpy()**

        **# calculate the diff between the weights before an update and after the update**
        **print(np.nonzero(before-after))  # show if something has changed**

        total_loss += loss.data
    losses.append(total_loss)
print(losses)

This is pretty much the exact example code from the tutorial. To show the change in the weights I added a few lines of code. I highlighted the lines (the lines beginning and ending with **) that I added in order to check if the embedding weights actually get updated after taking one (or at least multiple) optimizer step(s). I explicitly set the self.embeddings.weight.requires_grad = True but that didn’t change anything (still no visible updates).
I guess it simply is some stupid mistake that I make in the code, or I am trying to do it the wrong way. It would be great if someone could take a look and help me out on this.
Thanks