It is because you are only optimizing the lstm not the linear layer. Because of this the linear layer is not learning so your model cannot overfit. To optimize both you can do this
params = list(linear.parameters()) + list(lstm.parameters())
optimizer = torch.optim.Adam(params, lr=2e-3)

I tested that and my model overfit the data.