As pytorch is a library for efficient creation of models and so they provide built in classes for lstm and gru for you. No need to create yourself but as always you can always create your own custom class for them. The code and formulas  for lstm and gru classes can be found here:  https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/rnn.py 59