Hi Jason,
just wanted to let you know that you are not alone: I am encountering your same difficulties in understanding how to use LSTM networks properly.
Moreover, I could not even find the source code for them: reading the docs at https://pytorch.org/docs/stable/_modules/torch/nn/modules/rnn.html#LSTM 578, the  ´forward´ method uses the ´torch._C._VariableFunctions.lstm_cell´ implementation, that I could not find anywhere.
I think it might be some C++ integration among those in https://github.com/pytorch/pytorch/tree/master/torch/csrc 45, but still, I really don’t know where to look at.
Having at least the source code to inspect could be a starting point.