Question: What exactly is the matrix of a quadratic form? I have seen this notation occuring in a few papers (e.g. Siegel's unreadable German papers), with particular reference to the trace of a quadratic form. I'm at a loss as to what this means, and as a bonus question in passing I'd be interested if the trace of a quadratic form was interesting for an "obvious" reason (echoes of character theory, maybe?) 

For example what is the matrix of the quadratic form $x^2+y^2+z^2$? Or $x^2+xy+y^2$?

Answer: 
The matrix of a quadratic form $Q$ is the symmetric matrix $A$ such that

$$Q(\vec{x}) = \vec{x}^T A \vec{x}$$

For example,

$$x^2 + xy + y^2 = \left(\begin{matrix}x & y \end{matrix}\right) 
 \left(\begin{matrix}1 & \frac{1}{2} \\ \frac{1}{2} & 1 \end{matrix}\right) 
 \left(\begin{matrix}x \\ y \end{matrix}\right) 
$$
A quadratic form $q \colon V \to K$ on a vector space $V$ arises as "diagonal" of a symmetric bilinear form $\beta\colon V \times V \to K$ via $q(v) = \beta(v,v)$. Given $V$ has a basis $(v\_i)$, we can associate a matrix $B = (b\_{ij})$ with $\beta$, where $b\_{ij} = \beta(v\_i, v\_j)$. For $v = \sum \lambda\_i v\_i$, $w = \sum\mu\_i v\_i \in V$ we have then 
\[ \beta(v,w) = \sum\_{i,j} \lambda\_i b\_{ij} \mu\_j = (\lambda\_i)\_i^T B (\mu\_j)\_j, \]
so for $q$ we have
\[ q(v,v) = (\lambda\_i)^T B (\lambda\_i) \]
For your examples: 
\[
\begin{pmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\end{pmatrix}\text{ and } 
\begin{pmatrix} 1 & \frac 12 \\ \frac 12 & 1 \end{pmatrix}.
\]