Gradcheck checks a single function (or a composition) for correctness, eg when you are implementing new functions and derivatives.
For your application, which sounds more like “I have a network, where does funny business occur”, Adam Paszke’s script to find bad gradients in the computational graph might be a better starting point. Check out the thread below.
Best regards
Thomas

  
    
    
    How to check for vanishing/exploding gradients autograd
  
  
    I think the canonical reference for finding bad gradients is this snippet by Adam Paszke: 


It checks for NaN (by using x!=x if and only if x is NaN) and very large gradients, but you could easily adapt is_bad_grad to best fit your purpose. 
Best regards 
Thomas