What order of magnitude are the gradients? A general rule of thumb is to set eps to 1e-6 times the maximum magnitude of the gradient because floating-point precision is usually accurate to 6 decimal places.