What does orthogonality mean in function space? The functions $x$ and $x^2 - {1\over2}$ are orthogonal with respect to their inner product on the interval [0, 1]. However, when you graph the two functions, they do not look orthogonal at all. So what does it truly mean for two functions to be orthogonal?


Functions can be added together, scaled by constants, and taken in linear combination--just like the traditional Euclidean vectors. 
$$\vec{u} = a_1\vec{v}_1 + a_2\vec{v}_2 + \cdots a_n\vec{v}_n$$
$$g(x) = a_1f_1(x) + a_2f_2(x) + \cdots + a_nf_n(x) $$
Just as $\left( \begin{array}{c} 5\\ 2\\ \end{array} \right) = 
         5\left( \begin{array}{c} 1\\ 0\\ \end{array} \right) +
         2\left( \begin{array}{c} 0\\ 1\\ \end{array} \right) $, I can  say $5x + 2x^2 - 1 = 5(x) + 2(x^2 - \frac{1}{2})$
"Orthogonality" is a measure of how much two vectors have in common. In an orthogonal basis, the vectors have nothing in common. If this is the case, I can get a given vector's components in this basis easily because the inner product with one basis vector makes all other basis vectors in the linear combination go to zero.
If I know $\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right)$, 
$\left( \begin{array}{c} 0\\ 1\\ 0\\ \end{array} \right)$ and 
$\left( \begin{array}{c} 2\\ 0\\ -4\\ \end{array} \right)$ are orthogonal. I can quickly get the components of any vector in that basis:
$$\left( \begin{array}{c} 8\\ -2\\ 5\\\end{array} \right) = 
a\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) + 
b\left( \begin{array}{c} 0\\ 1\\ 0\\ \end{array} \right) + 
c\left( \begin{array}{c} 2\\ 0\\ -4\\ \end{array} \right)$$
$$\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) \cdot \left( \begin{array}{c} 8\\ -2\\ 5\\\end{array} \right) = 
\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) \cdot
\big[
a\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) + 
b\left( \begin{array}{c} 0\\ 1\\ 0\\ \end{array} \right) + 
c\left( \begin{array}{c} 2\\ 0\\ -4\\ \end{array} \right)
\big]$$
I know the $b$ and $c$ term disappear due to orthogonality, so I set them to zero and forget all about them.
$$8.25 = 1.25 a$$
I can also get function basis components easily this way. Take the fourier series on $[0,T]$ for example, which is just a (infinitely long) linear combination of vectors/functions:
$$
f(x) = a_0 + \sum_n^\infty a_ncos(\frac{2\pi n}{T}x) + \sum_n^\infty b_nsin(\frac{2\pi n}{T}x)
$$
I know that all the $cos$ and $sin$ basis functions are orthogonal, so I can take the inner product with $cos(\frac{2\pi 5}{T}x)$ and easily get a formula for the $a_5$ coefficient because all the other terms vanish when I do an inner product.
$$\int cos(\frac{2\pi 5}{T}x) f(x) dx = a_5 \int cos^2(\frac{2\pi 5}{T}x) dx$$
Of course I could do this in general with $cos(\frac{2\pi q}{T}x)$ to get any of the $a_q$ components.
