Computing $D_v^2(f)(x_0)$, if $f(x,y)=x^3+x+y^2-2y+1$, $v=\frac{1}{5}(3,4), x_0=(\frac{1}{3},1)$ 
Compute $D_v^2(f)(x_0)$, if $f(x,y)=x^3+x+y^2-2y+1$, $v=\frac{1}{5}(3,4), x_0=(\frac{1}{3},1)$ using the formula $D_v^2(f)(x_0)=v^TH_f(x_0)v$.

So, $v^T= \left[
\begin{array}{cc}
  \frac{3}{5}\\
  \frac{4}{5}
\end{array}
\right] $
and I found also Hessian matrix to be: $ \left[
\begin{array}{cc}
  2&0\\
  0&6
\end{array}
\right] $ and $v=(\frac{3}{5}, \frac{4}{5})$ but matrices cannot be multiplied in this way. Would appreciate if somebody could help.

You're almost there, but are misunderstanding the convention. Actually, $$v=\begin{bmatrix} \frac{3}{5} \\ \frac{4}{5}\end{bmatrix}$$ and
$$v^T=\begin{bmatrix} \frac{3}{5} & \frac{4}{5}\end{bmatrix}.$$
