what is the expectation value of a norm of a random variable from the standard normal distribution? Let's suppose $\zeta_k \in \mathbb{R}^p$ is a random variable from a normal distribution with mean zero and 1 standard deviation. why $E[|| \zeta_k ||^2]=p$?

Since the components are independent, it means that the squared $L_2$ norm of $\zeta$ has a $\chi^2_k$ distribution, which is the sum of squared independent standard normal variables.
The expected value of a $\chi^2_k$ variable is $k$.
In more generality, we do not even require the independence.
$$
\mathbb E\bigg[\vert\vert\zeta\vert\vert^2\bigg] = \mathbb E\bigg[
\sum_{i=1}^k \zeta_i^2
\bigg] = \sum_{i=1}^k\mathbb E\bigg[
\zeta_i^2
\bigg] = \sum_{i=1}^k 1 = k
$$
