Why use transpose of nabla in gradient descent For gradient descent we have the formula:
$f(x_{k}+d_{k})\approx f(x_{k}) + \nabla f(x_{k})^T  d_{k}  $
What I don't understand is, why we use the transpose of nabla and not just nabla.

Fundamentally this is just a notational issue. Assuming $x_k$ and $d_k$ are multidimensional "vectors" (say $n$ dimensional), the standard is often that all vectors are "columns", i.e. $n \times 1$ matrices. $\nabla f(x_k)$ will also be a $n \times 1$ matrix. To do a dot product via matrix multiplication, you transpose $\nabla$. Then you're multiplying a $1 \times n$ matrix by an $n \times 1$ matrix, so you get the scalar you want.
