Hello,
I defined a sentiment analysis model using lstm i usually do it with torchtext this this time i wanted to try it without. I used glove as embeddings and a 2 layers LSTM connected with a fc layer.
My problem is even if my gradients are changing the loss doesn’t change and I don’t understand why?
Model below:


class ReccurentModel(nn.Module):
def init(self,glove_matrix, n_layers, hidden_dim, bidirectional, dropout,output_dim):
super().init()
self.emb = nn.Embedding.from_pretrained(glove_matrix)
self.lstm = nn.LSTM(300,hidden_dim, n_layers, batch_first=True)
self.fc1 = nn.Linear(hidden_dim * 2 if bidirectional else hidden_dim, output_dim)
self.dropout = nn.Dropout(dropout)
def forward(self, x):
x_emb = self.emb(x)
output, (hidden, cell) = self.lstm(x_emb)
hidden = self.dropout(hidden[-1,:,:])
x_res = self.fc1(hidden)
return x_res


glove_mat is a matrix with size vocabulary*300(size of an embedding)


loss = nn.BCEWithLogitsLoss()
optimizer = optim.SGD(model.parameters(), lr=1e-4)


image885×634 13 KB
If you have any idea to change this ?