Yes, thats a bit confusing too, gradcheck tests to see if requires_grad=True on each input, if it is False, it doesn’t run the check on that input.
Normally during training grad=True this is only required if you are intending to propagate the gradient outside the model, say into embeddings or something like that.  That’s the confusing bit 
However, it is helpful, because sometimes there are inputs that you don’t actually want to grad-check.  So in that case, you can have gradcheck ignore them by setting requires_grad=False.