Unless your target size is the same as your lstm’s hidden dim, you need to apply a linear layer after the lstm to transform it to the right size and also get rid of the tanh activation coming out of the lstm.
nn.linear(512, TARGET_SIZE)
I also recommend removing the dropout if this is at the very end of your network.