Why mean squared error surface takes bowl shape I was trying to understand geometric interpretation of regularization and came across following statement here:

$$\text{Mean Square Error}\; E(y,\hat{y})=\frac{1}{n}\lVert\hat{y}-y\rVert^2$$
$$=\frac{1}{n}(b^TX^TXb-2b^TX^Ty+y^Ty)$$
Since $X^TX$ is positive semidefinite, we know that $b^TX^TXb\geq0$. Furthermore, we know that (from vector calculus) it will be a paraboloid (bowl-shaped surface) in the $(E,b_1,b_2)$ space. The following diagram depicts this situation.


I am not able to get the sentence above. Specifically I didnt get "positive semidefinite" and "we know from vector calculas that it will be a paraboloid". How / why the mean square error surface is bowl shaped? Is their intuition (visual or geometrical if possible) behind it?

Suppose $b$ is 1 dimensional. Then you have a loss function of the shape
$$ a_1 \cdot b^2 + a_2 \cdot b + a_3
$$
Which is a parabola. If $a_1 > 0$ then it will be a "smiling" / "convex" / "upward facing" parabola.
This generalizes to bigger dimensions:
$$b^T A_1b + b^T a_2 + a_3 
$$
Only now, for it to be "smiling" we require $A_1 \succ 0$ to be positive-definite.
