If the function is not differentiable, you’ll get an error.
If gradients cannot be computed (usually due to inplace ops), yes you will get an error.
In general, if a gradient is computed, it is correct.