Question: If we have a rewrite system for primitive recursive functions, which simplifies each term according to how the function was defined, then what is the computational complexity of this calculation? That is, what is the coplexity of the normalization procedure? I have heard a claim that for a closed term calculating the value of the function requires transfinte induction up to $\epsilon\_0$. Is this true and where can I find a proof of this? 

For example in (Schwichtenberg & Wainer 2012) there is a lemma which says that a primitive recursive function is computable in $F\_{\alpha}$-bounded time, for some $\alpha<\omega$, where the set of all $F\_{\alpha}$ for $\alpha<\epsilon\_0$ is the Fast Growing Hierarchy. 

Is the measure of transfinite induction related to this way of bounding the complexity?

Answer: 
I find some aspects of this question difficult to understand. For example, the part about "calculating the value of the function requires transfinite induction up to $\varepsilon\_0$" seems to conflate methods of proof (like transfinite induction) with methods of computation. Also, much (if not all) of the question appears to be about primitive recursive functionals of higher type (as in Gödel's Dialectica interpretation) rather than mere primitive recursive functions. So, the following may not be what the OP was looking for, but here goes anyway:

Not much can be said about the time complexity of computing primitive recursive functions (from natural numbers to natural numbers) except that it's primitive recursive and thus bounded by a finite level $F\_n$ of the Grzegorczyk hierarchy (essentially equivalent to the fast-growing hierarchy, I believe). 

On the other hand, for primitive recursive functionals of finite type, you can consider the problem of evaluating terms of type 0 (i.e., terms that denote natural numbers). Here, the only bound I can see for the time-complexity is level $\varepsilon\_0$ of the Grzegorczyk hierarchy. For any fixed primitive recursive functional, evaluation of its numerical values would take time bounded by $F\_\alpha$ for some $\alpha<\varepsilon\_0$, but different functionals would need different $\alpha$'s, with no uniform bound below $\varepsilon\_0$. 

The relevance of the proof principle of transfinite induction up to $\varepsilon\_0$ (for open formulas) is that this is what is needed, on top of primitive recursive arithmetic, to prove that every closed term of type 0, built from primitive recursive functionals of higher type, can be reduced to a numerical value.
Dear Andreas,

yes indeed. We have the follwing (not too difficult) fact: Let S^m(0) be the numeral for $m$.
For any PRA term $t(x\_1,\ldots,x\_n)$ The derivation length
of $t(S^{m\_1}(0),\ldots,S^{m\_n}(0))$ will be bounded by a primitive recursive
function (depending on $t$) with arguments $m\_1,\ldots,m\_n$. (Here I assume
standard rewrite systems for modelling primitive recursion.)

For terms from Gödel's T the derivation lengths become more complex depending
on the typelevel of the terms in question. 

My general conjecture is that 
for typical (not too small) subrecursive classes C the derivation lengths functions
for terms in C will not leave C.

The result for primitive recursion is not difficult whereas the result for Gödel's T is
somewhat involved.

For PRA there is also a tradeoff in terms of termination orderings.
Termination for rewrite systems for prim rec functions can be shown 
by the multiset path ordering. And termation proofs for the multiset
path ordering lead to prim rec derivation lengths (result by Dieter 
Hofbauer). The corresponding result for multiple recursion is one
of my earliest results in term rewriting theory. Wilfried Buchholz 
gave a nice prooftheoretic proof for these results in APAL.

There is also some nice application of term rewriting to more involved schemes of primitive
recursive functions. For example it can be used to show that prim rec functions
are closed under parameter recursion, or simple nested recursion, or even
unnested multiple recursion.

Best, 
Andreas