Random number generator is needed for layers such as Dropout as well. If you checkpoint module which contains Dropout you want you dropout mask to be the same when doing forward pass and when re-computing tensors in backward pass.