In the most basic for of lstm, IIRC, the hidden layer value for the first iteration should be full of 0.
Since the module you use performs all the iterations in one call. You only ever want to provide a Tensor full of 0s for it.
If you want to train the initial hidden layer value, you can declare them as nn.Parameter and given them as output to the lstm (but still ignore the returned value).