How does the approximate Hessian update in LBFGS work? Looking at the wikipedia page for BFGS... Wikipedia
It looks like a rearranging of Newton's method, but I can't really explain why the update to the approximate Hessian would be given by the following line...
$$
B_{k+1}(\mathbf{x}_{k+1} - \mathbf{x}_k) = \nabla f(\mathbf{x}_{k+1}) - \nabla f(\mathbf{x}_{k})
$$
I don't see why simply subtracting the derivative from the derivative at the updated function evaluation would give an approximation of the Hessian. Shouldn't there be a denominator to normalize the difference by something in order to be precise, such as in the simple and traditional derivative calculation which dvides by $h$?

The Wiki page goes on:



*Set $\mathbf{s}_{k}=\alpha_{k} \mathbf{p}_{k}$ and update $\mathbf{x}_{k+1}=\mathbf{x}_{k}+\mathbf{s}_{k}$

*$\mathbf{y}_{k}=\nabla f\left(\mathbf{x}_{k+1}\right)-\nabla f\left(\mathbf{x}_{k}\right)$

*$B_{k+1}=B_{k}+\frac{\mathbf{y}_{k} \mathbf{y}_{k}^{\mathrm{T}}}{\mathbf{y}_{k}^{\mathrm{T}} \mathbf{s}_{k}}-\frac{B_{k} \mathbf{s}_{k} \mathbf{s}_{k}^{\mathrm{T}} B_{k}^{\mathrm{T}}}{\mathbf{s}_{k}^{\mathrm{T}} B_{k} \mathbf{s}_{k}}$

where $\alpha_k$ is a step size.  So both $y_k$ and $s_k$ include a step size. The two update bits of step 5 both have two $\alpha$s in the numerator and two in the denominator (via $y_k$ or $s_k$)
