Yes gradcheck only checks gradient for inputs to the function that are Tensor that require gradients.
Note that in this case, you can “cheat” by doing: gradcheck(lambda inp, *ignore: model(inp), (inp, *model.parameters()).
This will make all the parameters look like inputs for gradcheck. And all inplace changes to them must properly update the forward of the model (this will work with pytorch vanilla Module at least).