In this case, the author is treating the initial state as a learned value (see this block 22 of code). In that case, it makes sense to use a randomly initialized vector to break symmetry, just like any other parameter.
Learned initial states are atypical – most architectures I’ve come across use a zero initial state. In PyTorch, you would just omit the second argument to the LSTM object.