How does you training loop for an epoch/batch look like, i.e. where you have to call something like model.zero_grad() and optimizer.step().
Also, in the definition of your LSTM:
self.lstm = nn.LSTM(300,hidden_dim, n_layers, batch_first=True)

you probably want to add user bidirectional parameter, Otherwise you LSTM is always unidirectional and might not match self.fc1 if bidirectional=True.