\documentclass{memoir}
\usepackage{notestemplate}

%\logo{~/School-Work/Auxiliary-Files/resources/png/logo.png}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

%\makeindex

\begin{document}

% \maketitle

% Notes taken on 06/06/21

Because a field is clearly Noetherian, we get an interesting result:
\begin{cor}
	Every ideal in the polynomial ring \(F[x_1,x_2,\ldots,x_n]\) is finitely generated.
\end{cor}
The colection of leading coefficients of polynomials in \(I\triangleleft R[x]\) form a useful ideal in \(R\) that characterizes \(I\). We can utilize this to study \(F[x_1,x_2,\ldots,x_n]\), but we need to exercise caution-- we need an ordering on the monomials in order to determine the leading term of a polynomial.

\begin{defn}[Monomial Ordering]
	A \textbf{monomial ordering} is a well ordering on the set of polynomials that satisfies \(mm_1\geq mm_2\) whenever \(m_1\geq m_2\) for monomials \(m,m_1,m_2\).
\end{defn}

\begin{defn}[Leading Terms]
	Fix a monomial ordering on the polynomial ring \(F[x_1,x_2,\ldots,x_n]\). The \textbf{leading term} of a nonzero polynomial \(f \in F[x_1,x_2,\ldots,x_n]\), denoted \(LT(f)\), is the monomial term of maximal order in \(f\). The \textbf{multidegree of \(f\)}, denoted \(\partial(f)\), is the multidegree of the leading term of \(f\).\\

	If \(I\) is an ideal in \(F[x_1,x_2,\ldots,x_n]\), the \textbf{ideal of leading terms}, denoted \(LT(I)\), is the ideal generated by the leading terms of all the elements in the ideal:
	\begin{align*}
		LT(I) = (LT(f) \mid f \in I).
	\end{align*}
\end{defn}
It is clear that the choice of ordering affects the leading term and hence the multidegree of a polynomial. Notice that \(\partial(fg) = \partial f + \partial g\) when \(f\) and \(g\) are nonzero, and hence \(LT(fg) = LT(f) + LT(g)\).\\

By construction, \(LT(I)\) is generated by monomials. We refer to these kinds of ideals as \textbf{monomial ideals}.

\begin{hw}
	Show that a polynomial \(p(x)\) is contained in the monomial ideal \((f_1(x),f_2(x),\ldots,f_n(x))\) (where \(f_i(x)\) is a monomial) if and only if each of the monomial terms of \(p(x)\) is a multiple of one of the generators \(f_i(x)\).
\end{hw}
If \(I = (f_1,\ldots,f_m)\), notice that \(LT(I)\) contains the ideals of each of the leading terms:
\begin{align*}
	(LT(f_1),\ldots,LT(f_m)) \subset LT(I).
\end{align*}
This inequality can in fact be strict.

\begin{exmp}

\end{exmp}
The intuition behind this strange fact is that while a smaller polynomial may not be in \((LT(f_1),LT(f_2),\ldots,LT(f_n))\), one could potentially generate polynomials by canceling out the leading terms of \(f_1,f_2\). Hence it is natural to look for a basis that generates these smaller polynomials as well.

\begin{defn}[Grobner Basis]
	A \textbf{Grobner basis} for an ideal \(I\) in \(F[x_1,\ldots,x_n]\) is a finite set of generators \(\left\{ g_1,\ldots,g_m \right\} \) for \(I\) whose leading terms generate the ideal of all leading terms in \(I\):
	\begin{align*}
		I = (g_1,\ldots,g_m) \quad LT(I) = (LT(g_1),\ldots,LT(g_m)).
	\end{align*}
\end{defn}
Notice that it is a basis in the sense that it is a set of generators-- it is not a basis in the sense of vector spaces.\\

A Grobner basis allows for every polynomial to be written uniquely as a sum of elements in \(I\) and remainder \(r\). Hence, it is a tool that allows us to give back more structure to multivariable polynomial rings.

\begin{general}[General Polynomial Division]
	Let \(F[x_1,\ldots,x_n]\) be a monomial ordering and suppose \(g_1,\ldots,g_m \in F[x_1,\ldots,x_n]\) is a set of nonzero polynomials. If \(f \in F[x_1,\ldots,x_n]\) then we construct a set of quotients \(q_1,\ldots,q_m\) and a remainder \(r\) (initially all zero) and test if \(LT(f)\) is divisible by \(LT(g_i)\) in order.\\

	If \(LT(g_i)\mid LT(f)\), i.e \(LT\left(  = a_iLT(g_i) \right) \), then \(q_i' = q_i+a_i\) and \(f' = f-a_ig_i\), and reiterate.\\

	Once the leading term  \(LT(f)\) is not divisible by any \(LT(g_1),\ldots,LT(g_m)\), set \(r' = r+LT(f)\) and \(f' = f-LT(f)\) and reiterate.\\

	The process terminates when \(f = 0\) and results in a set of quotients and remainder such that:
	\begin{align*}
		f = q_1g_1 + \ldots + q_mg_m + r.
	\end{align*}
	Notice that \(q_ig_i\) has multidegree less than or equal to the multidegree of \(f\). Furthermore, no nonzero term in \(r\) is divisible by any \(LT(g_i)\).
\end{general}
Unfortunately, the remainder is dependent on the choices of \(g_i\), and is nonunique. In fact, sometimes the remainder is zero for the right choice of \(g_i\), but not others, making it difficult to utilize the factorization. Of course, our Grobner basis is the right tool to fix this.

\begin{thm}
	Fix a monomial ordering on \(R = F[x_1,\ldots,x_n]\) and suppose \(g_1,\ldots,g_m\) is a Grobner basis for the nonzero ideal \(I\triangleleft R\). Then every polynomial \(f\) can be written uniquely as
	\begin{align*}
		f = f_I + 4
	\end{align*}
	where \(f_I \in I\) and no nonzero monomial term of \(r\) is divisible by \(LT(g_i)\).\\

	Both \(f_I\) and \(r\) are computed by general polynomial division by \(g_1,\ldots,g_m\) and are independent of the order of \(g_i\).\\

	Finally, the remainder \(r\) provides a unique representative for the coset of \(f\) in \(F[x_1,\ldots,x_n] / I\). This also tells us that \(f \in I\) if and only if \(r=0\).
\end{thm}
\begin{proof}

\end{proof}
Now we will see that if we have a well-defined division algorithm for some set of polynomials, that they must be a Grobner basis.

\begin{prop}
	Fix a monomial ordering on \(R = F[x_1,\ldots,x_n]\) and let \(I \triangleleft R\) be nonzero. If \(g_1,\ldots,g_m \in I\) such that
	\begin{align*}
		LT(I) = (LT(g_1),\ldots,LT(g_m))
	\end{align*}
	then \(g_1,\ldots,g_m \in I\) is a Grobner basis for \(I\).\\

	For any ideal \(I\triangleleft R\) nonzero, the ideal has a Grobner basis.
\end{prop}

Now our goal is to form some criterion to determine whether a set of generators forms a Grobner basis. Recall that our main difficulty in forming a nice basis is that it is possible to cancel leading terms of two polynomials. One way this is done is via the equation below:
\begin{align*}
	S(f_1,f_2) := \frac{M}{LT(f_1)}f_1 - \frac{M}{LT(f_2)}f_2.
\end{align*}
where \(M\) is the monic lcm of \(LT(f_1),LT(f_2)\). One can see that this in fact causes the leading terms of \(f_1,f_2\) to cancel.

\begin{lemma}
	Suppose \(f_1,\ldots,f_m \in F[x_1,\ldots,x_n]\) are polynomials with multidegree \(\alpha \) and determined so that
	\begin{align*}
		h = a_1f_1 + \ldots + a_mf_m
	\end{align*}
	with constants \(a_i \in F\) has strictly smaller multidegree. Then
	\begin{align*}
		h = \sum_{i=2}^{m} b_i S(f_{i-1},f_i)
	\end{align*}
	for some constants \(b_i \in F\).
\end{lemma}
We can use this to show that a set of generators is Grobner if there are no new leading terms among \(S(g_i,g_j)\).

\begin{rmrk}
	Let \(R = F[x_1,\ldots,x_n]\) and \(G = \left\{ g_1,\ldots,g_m \right\} \). We write \(f \equiv r \pmod{G}\) if \(r\) is the remainder obtained by general polynomial division of \(f\) by polynomials \(g1,\ldots,g_m\).
\end{rmrk}

\begin{prop}[Buchberger's Criterion]
	Let \(R = F[x_1,\ldots,x_n]\) and fix a monomial ordering on \(R\). If \(I = (g_1,\ldots,g_m)\) is a nonzero ideal in \(R\), then \(G = \left\{ g_1,\ldots,g)m \right\} \) is a Grobner basis for \(I\) if and only if \(S(g_i,g_j) \equiv 0 \pmod{G}\) for \(1\leq i<j\leq m\).
\end{prop}
\begin{proof}

\end{proof}

We can use this to form a new algorithm to find a Grobner basis.

\begin{general}[Buchberger's Algorithm]
	Let \(I = (g_1,\ldots,g_m)\). If \(S(g_i,g_j)\) leaves a remainder of \(0\) when divided by  \(G = \left\{ g_1,\ldots,g_m \right\} \) using general polynomial division, then \(G\) is a Grobner basis. Otherwise, we proceed with the algorithm.\\

	If \(G\) is not a Grobner basis, then \(S(g_i,g_j)\) has a nonzero remainder \(r\). We increase \(G\) by appending the polynomial \(g_{m+1} =r\) to form
	 \begin{align*}
		G' = \left\{g_1,\ldots,g_m,g_{m+1}  \right\}
	\end{align*}
	and repeat general polynomial division.\\

	This procedure terminates after a finite number of steps, and will result in a generating set \(G\) that satisfies Buchberger's Criterion.\\
\end{general}
If \(\left\{ g_1,\ldots,g_m \right\} \) is a Grobner basis for the ideal \(I\), and \(LT(g_i)\mid LT(g_j)\) for some \(i\neq j\), then \(LT(g_j)\) can be dropped from the list without affecting the basis.\\

We can assume without loss of generality that the leading term of each \(g_i\) is monic. A Grobner basis \(G = \left\{g_1,\ldots,g_m \right\} \) for \(I\) where \(LT(g_i)\) is monic and no leading temr divides the other is called a \textbf{minimal Grobner basis}.\\

A minimal Grobner basis is not unique, but the number of elements and their leading terms are unique. We can impose additional restrictions to make a basis unique.

\begin{defn}[Reduced Grobner Basis]
	Fix a monomial ordering on \(R = F[x_1,\ldots,x_n]\). A Grobner basis \(\left\{ g_1,\ldots,g_m \right\} \) for a nonzero ideal \(I\triangleleft R\) is called a \textbf{reduced Grobner basis} if each \(LT(g_i)\) is monic and \(LT(g_i)\not\mid g_j\) for \(i\neq j\).
\end{defn}
This is of course a stronger type of minimal Grobner basis. We can turn a minimal Grobner basis into a reduced Grobner basis by replacing each \(g_i\) by its remainder after division by \(g_j\).

\begin{thm}
	Fix a monomial ordering on \(R = F[x_1,\ldots,x_n]\). Then there is a unique reduced Grobner basis for every nonzero ideal \(I \triangleleft R\).
\end{thm}
This is really useful to distinguish ideals in a polynomial ring.

\begin{cor}
	Let \(I,J \triangleleft F[x_1,\ldots,x_n]\). Then \(I=J\) if and only if \(I,J\) have the same reduced Grobner basis with respect to any fixed monomial ordering on \(F[x_1,\ldots,x_n]\).
\end{cor}

\begin{exmp}

\end{exmp}

\subsection{Solving Algebraic Equations}
\label{sub:solving_algebraic_equations}

We can use Grobner bases to help solve systems of algebraic equations. Suppose \(S = \left\{ f_1,\ldots,f_m \right\} \) is a collection of polynomials in \(n\) variables \(x_1,\ldots,x_n\), and we are trying to find a solution to the system of equations \(f_i = 0\) for all \(i \in \left\{ 1,\ldots,m \right\} \). Notice that if \((a_1,\ldots,a_n)\) is a solution to this system, then every element \(f \in I\) where \(I\) is generated by \(S\) also satisfies \(f_1(a_1,\ldots,a_n) = 0\).\\

The ideas of Grobner bases allow us to expand the theory of linear polynomial equations to nonlinear polynomial equations. The process of finding elements of the ideal \(I\) independent of variables (so as to reduce the equation similar to Gauss-Jordan elimination) is part of \textit{elimination theory}.

\begin{defn}[Elimination Ideal]
	If \(I\triangleleft F[x_1,\ldots,x_n]\), then
	\begin{align*}
		I_i = I \cap F[x_{i+1},\ldots,x_n]
	\end{align*}
	is called the \(i\)-th \textbf{elimination ideal} of \(I\) with respect to the ordering \(x_1>\ldots>x_n\).
\end{defn}

\begin{prop}[Elimination]
	Suppose \(G = \left\{ g_1,\ldots,g_m \right\} \) is a Grobner basis for the nonzero ideal \(I \triangleleft F[x_1,\ldots,x_n]\) with respect to the lexicographic monomial ordering \(x_1>\ldots>x_n\). Then
	\begin{align*}
		G \cap F[x_{i+1},\ldots,x_n]
	\end{align*}
	is a Grobner basis of the \(i\)-th elimination ideal
	\begin{align*}
		I_i = I \cap F[x_{i+1},\ldots,x_n] \subset I.
	\end{align*}
	In particular, \(I_i = 0\) if and only if \(G \cap F[x_{i+1},\ldots,x_n] = \emptyset\).
\end{prop}
\begin{proof}

\end{proof}

\begin{exmp}

\end{exmp}

\begin{prop}
	If \(I,J \triangleleft F[x_1,\ldots,x_n]\), then
	\begin{align*}
		tI + (1-t)J \triangleleft F[t,x_1,\ldots,x_n]
	\end{align*}
	and
	\begin{align*}
		I \cap J = (tI + (1-t)J) \cap F[x_1,\ldots,x_n].
	\end{align*}
	Hence \(I\cap J\) is the first elimination ideal of \(tI + (1-t)J\) with respect to the ordering \(t > x_1 > \ldots > x_n\).
\end{prop}

% \printindex
\end{document}


%!TEX root=../main.tex
\appendix
\chapter{Griffith Revisited} \label{chap:griffithrevis}

In this chapter we revisit the Griffith's theory of dynamic fracture and provide a variational formulation of Griffith's law \eqref{eq:gtgc}. The basic problem setting is similar to that assumed in \cref{sec:linkDF}. Here the symbol $\Gamma_t$ refers to a sharp-interface crack inside a two-dimensional linear elastic body.

\section*{Reformulation Based on a Space-Time Action Integral}
The sharp-interface dynamic fracture problem will be formulated in line with the dynamic gradient damage model outlined in \cref{def:dynagraddama}. We will construct a space-time action integral similar to that introduced in \cite{Adda-BediaAriasAmarLund:1999} and then calculate directly the action variation corresponding to arbitrary virtual displacement variation and crack advance. The basic energetic ingredients of the dynamic fracture problem are defined as follows. By virtue of \eqref{eq:grad} and \eqref{eq:v}, they can be directly formulated in the initial configuration.
\begin{itemize}
\item The elastic energy is given by
\begin{equation} \label{eq:elastici}
\mathcal{E}(\vec{u}_t,l_t)=\mathcal{E}^*(\vec{u}_t^*,l_t)=\int_\domaini\psi\bigl({\textstyle\frac{1}{2}}\nabla\vec{u}_t^*\nabla\philt^{-1}+{\textstyle\frac{1}{2}}\nabla\philt^{-\mathsf{T}}(\nabla\vec{u}_t^*)^\mT\bigr)\det\nabla\philt\dxx.
\end{equation}
where $\psi$ refers to the classical (damage-independent) linear elastic energy density. The stress tensor conjugate to the strain variable is thus given by $\sig_t=\tens{A}_0\eps(\vec{u}_t)$.

\item The kinetic energy is formally given by \eqref{eq:kinetici} as long as we interpret $\Gamma_t$ as a sharp-interface strong discontinuity in the body.

\item The Griffith surface energy \eqref{eq:StGriffith} reads in our case
\begin{equation} \label{eq:griffith}
\mathcal{S}(l_t)=\gc\cdot(l_t-l_0).
\end{equation}
It is the sharp-interface counterpart of the damage dissipation energy \eqref{eq:surfacei}.

\item The external work potential $\mathcal{W}_t$ is formally given by \eqref{eq:externalworki} as long as we interpret $\Gamma_t$ as a sharp-interface strong discontinuity in the body.
\end{itemize}

The admissible function spaces for the displacement $\vec{u}^*$ and for the crack evolution $l$ are still formally given by respectively \eqref{eq:Cu} and \eqref{eq:Dl}. We can now form the space-time action integral given by
\begin{equation} \label{eq:action}
\mathcal{A}(\vec{u}^*,l)=\int_I\mathcal{L}_t(\vec{u}^*_t,\dot{\vec{u}}^*_t,l_t,\dot{l}_t)\D{t}=\int_I\bigl(\mathcal{E}^*(\vec{u}_t^*,l_t)+\mathcal{S}(l_t)-\mathcal{K}^*(\vec{u}_t^*,\dot{\vec{u}}_t^*,l_t,\dot{l}_t)-\mathcal{W}_t^*(\vec{u}_t^*,l_t)\bigr)\D{t}
\end{equation}
which involves a generalized Lagrangian $\mathcal{L}_t(\vec{u}^*_t,\dot{\vec{u}}^*_t,l_t,\dot{l}_t)$. The coupled evolution described by the couple $(\vec{u}^*,l)\in\mathcal{C}(\vec{u}^*)\times\mathcal{Z}(l)$ will then be governed by
\begin{definition}[Variational Formulation of the Griffith's Theory of Dynamic Fracture] \label{def:griffith} \noindent
\begin{enumerate}
\item \textbf{Irreversibility}: the crack length is a non-decreasing function of time $\dot{l}_t\geq 0$.
\item \textbf{First-order stability}: the first-order action variation is non-negative with respect to arbitrary admissible displacement and crack evolutions
\begin{equation} \label{eq:stability}
\mathcal{A}'(\vec{u}^*,l)(\vec{v}^*-\vec{u}^*,s-l)\geq 0\text{ for all $\vec{v}^*\in\mathcal{C}(\vec{u}^*)$ and all $s\in\mathcal{Z}(l)$}.
\end{equation}
\item \textbf{Energy balance}: the only energy dissipation is due to crack propagation such that we have the following energy balance
\begin{equation} \label{eq:eb}
\mathcal{H}_t=\mathcal{H}_0+\int_0^t\left(\int_{\Omega\setminus\Gamma_s}\bigl(\sig_s\cdot\eps(\dot{\vec{U}}_s)+\rho\ddot{\vec{u}}_s\cdot\dot{\vec{U}}_s\bigr)\dx-\mathcal{W}_s(\dot{\vec{U}}_s)-\dot{\mathcal{W}}_s(\vec{u}_s)\right)\D{s}
\end{equation}
where the total energy is defined by
\begin{equation}
\mathcal{H}_t=\mathcal{E}^*(\vec{u}_t^*,l_t)+\mathcal{S}(l_t)+\mathcal{K}^*(\vec{u}_t^*,\dot{\vec{u}}_t^*,l_t,\dot{l}_t)-\mathcal{W}^*_t(\vec{u}_t^*,l_t).
\end{equation}
\end{enumerate}
\end{definition}

In the first-order stability principle \eqref{eq:stability}, the notation $\mathcal{A}'(\vec{u}^*,l)(\vec{v}^*-\vec{u}^*,s-l)$ denotes the Gâteaux derivative of the action functional with respect to the displacement variation $\vec{w}^*=\vec{v}^*-\vec{u}^*$ and crack advance $\delta l=s-l$. Recall that the transported displacement $\vec{u}_t^*$ is defined on the initial configuration $\Omega\setminus\Gamma_0$ which is fixed during the (virtual) crack increment, thanks to the introduction of the diffeomorphism $\philt$. The displacement variation $\vec{w}^*$ is thus independent from that of the crack $\delta l$, and induces automatically a variation $\vec{w}$ in the current material configuration via a pushforward operation $\vec{w}\circ\philt=\vec{w}^*$.

\section*{Equivalence with the Classical Formulations}
We will show in this section that the variational approach to dynamic fracture summarized by \cref{def:griffith} is equivalent to the usual elastodynamic equation in the uncracked bulk and Griffith's law of crack evolution \eqref{eq:gtgc}. However, it should be noted that the variational formulation is more general. To achieve this goal, we will carefully evaluate the derivative of the action functional with respect to arbitrary displacement variation $\vec{w}^*=\vec{v}^*-\vec{u}^*$ and crack advance $\delta l=s-l$. Lengthy calculations are detailed in \cref{chap:detailedcal}, and only the main results are presented here.

By firstly evaluating the action variation corresponding to zero virtual crack advance $\delta l=s-l=0$ and using the fact that $\vec{v}_t^*-\vec{u}_t^*=\vec{w}_t^*\in\mathcal{C}_0$ is a vector space, one obtains by virtue of the regularity hypotheses
\begin{equation} \label{eq:actionvariationzerocrackadvance}
\begin{aligned}
\mathcal{A}'(\vec{u}^*,l)(\vec{w}^*,0) &= \int_I\left(\int_\domaint\bigl(\rho\ddot{\vec{u}}_t-\div\sig_t-\vec{f}_t\bigr)\cdot\vec{w}_t\dx+\int_{\partial\Omega_F}(\sig_t\vec{n}-\vec{F}_t)\cdot\vec{w}_t\D{\vec{s}}\right. \\
&\pushright{\left.+\int_{\Gamma_t}\sig_t\vec{n}\cdot\vec{w}_t\D{\vec{s}}\right)\D{t}} \\
&= 0\text{ for all $\vec{w}_t^*\in\mathcal{C}_0$}
\end{aligned}
\end{equation}
from which the classical wave equation is deduced
\begin{equation} \label{eq:classicalwave}
\rho\ddot{\vec{u}}_t-\div\sig_t=\vec{f}_t\quad\text{in }\domaint\,,\quad\sig_t\vec{n}=\vec{F}_t\quad\text{on }\partial\Omega_F\quad\text{and}\quad\sig_t\vec{n}=\vec{0}\quad\text{on }\Gamma_t.
\end{equation}
We then evaluate the action derivative with zero virtual displacement variation $\vec{w}^*=\vec{0}$, leading to
\begin{equation} \label{eq:actionvariationzerodisplacement}
\mathcal{A}'(\vec{u}^*,l)(\vec{0},\delta l)=\int_I(\gc-G_t)\delta l_t\D{t}\geq 0\text{ for all $\delta l_t\geq 0$ with $t\in(0,T)$}
\end{equation}
where the dynamic energy release rate $G_t$ to be compared with the fracture toughness $\gc$ reads
\begin{equation} \label{eq:Gt}
G_t=\int_\domaint\Bigl(\bigl(\kappa(\dot{\vec{u}}_t)-\psi\bigl(\eps(\vec{u}_t)\bigr)\bigr)\div\vtheta_t+\sig_t\cdot(\nabla\vec{u}_t\nabla\vtheta_t)+\div(\vec{f}_t\otimes\vtheta_t)\cdot\vec{u}_t+\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t\Bigr)\dx.
\end{equation}
From \eqref{eq:actionvariationzerodisplacement}, we retrieve the desired crack stability condition which states that the dynamic energy release rate must be smaller or equal to the material fracture toughness. The consistency condition
\[
(G_t-\gc)\dot{l}_t=0
\]
can then be derived thanks to the energy balance principle \eqref{eq:eb} and detailed calculations in \cref{chap:detailedcal}. The combination of these two conditions along with the irreversibility condition leads to the desired Griffith's law of crack propagation \eqref{eq:gtgc}.

Note that we retrieve the static energy release rate \cite{Destuynder:1981} by setting the velocity $\dot{\vec{u}}_t$ and the acceleration $\ddot{\vec{u}}_t$ in \eqref{eq:Gt} to zero. A similar formula for $G_t$ is obtained in \cite{AttiguiPetit:1996} by constructing an \emph{ad-hoc} field $0\leq\norm{\vtheta_t}\leq 1$ which transforms surface (line) integrals to volume (surface) integrals. Here the dynamic energy release rate $G_t$ is identified by calculating the variation of the space-time action integral \eqref{eq:action} with respect to crack increment \emph{evolution}. Using the Euler-Lagrange equation
\[
\mathcal{A}'(\vec{u}^*,l)(\vec{0},\delta l)=\int_I\left(\frac{\partial\mathcal{L}_t}{\partial l_t}-\frac{\md}{\md t}\frac{\partial\mathcal{L}_t}{\partial\dot{l}_t}\right)\cdot\delta l_t\D{t}
\]
and the fact that the Lagrangian depends on the crack velocity $\dot{l}_t$ solely via the kinetic energy $\mathcal{K}^*$, we find the same expression for the dynamic energy release rate $G_t$ as indicated in \cite[p.~423]{Freund:1990}
\[
G_t=\frac{\partial(\mathcal{K}^*+\mathcal{W}_t^*-\mathcal{E}^*)}{\partial l_t}-\frac{\md}{\md t}\frac{\partial\mathcal{K}^*}{\partial\dot{l}_t}.
\]
Contrary to the quasi-static regime, this quantity $G_t$ doesn't possess the physical meaning of the derivative of the Lagrangian with respect to crack extension due to the presence of the term $(\md/\md t)(\partial\mathcal{K}^*/\partial\dot{l}_t)$, as has been already noted in \cite{NishiokaAtluri:1983}.

Although $\vtheta_t$ enters into the definition of $G_t$ in \eqref{eq:Gt}, the dynamic energy release rate is independent of the exact virtual perturbation used to establish the bijection \eqref{eq:philt}, owing to the following
\begin{proposition} \label{prop:J}
The dynamic energy release rate $G_t$ is equivalent to the classical dynamic $J$-integral in the form of a path integral \eqref{eq:Jdyn}. As a corollary, the dynamic energy release rate $\eqref{eq:Gt}$ is independent of the virtual perturbation.
\end{proposition}

\begin{proof}
For theoretic analysis purpose a specific integration path $C_r$ is used in \eqref{eq:Jdyn}. The symbol $\vec{n}$ is assumed to be the normal pointing out of the ball $B_r(\vec{P}_t)$ with $C_r=\partial B_r(\vec{P}_t)$ its boundary. To removing any singularities near the crack tip $\vec{P}_t$, we will partition the cracked domain $\domaint$ into the part $\tilde{B}_r=B_r(\vec{P}_t)\setminus\Gamma_t$ included in the ball $B_r(\vec{P}_t)$, and the part $\Omega_r=\Omega\setminus\bigl(\Gamma_t\cup B_r(\vec{P}_t)\bigr)$ outside the ball, see \cref{fig:partition}.
\begin{figure}[htbp]
\centering
\includegraphics[width=0.45\textwidth]{remove_singularity.pdf}
\caption{Partition of the cracked domain $\domaint$ using a $\vec{P}_t$-centered ball $B_r(\vec{P}_t)$ of radius $r$} \label{fig:partition}
\end{figure}
Using the following identity in $\Omega_r$
\begin{multline} \label{eq:div}
\div\Bigl(\bigl(\kappa(\dot{\vec{u}}_t)+\vec{f}_t\cdot\vec{u}_t-\psi\bigl(\eps(\vec{u}_t)\bigr)\bigr)\vtheta_t\Bigr)=\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t+\nabla\vec{f}_t\vtheta_t\cdot\vec{u}_t+\vec{f}_t\cdot\nabla\vec{u}_t\vtheta_t \\
-\sig_t\cdot\eps(\nabla\vec{u}_t)\vtheta_t+\bigl(\kappa(\dot{\vec{u}}_t)+\vec{f}_t\cdot\vec{u}_t-\psi\bigl(\eps(\vec{u}_t)\bigr)\bigr)\div\vtheta_t
\end{multline}
and performing an integration by parts
\begin{equation} \label{eq:signunt}
\int_{\Omega_r}\sig_t\cdot(\nabla\vec{u}_t\nabla\vtheta_t)\dx=-\int_{C_r}(\nabla\vec{u}_t^\mT\sig_t)\vec{n}\cdot\vtheta_t\dx-\int_{\Omega_r}(\div\sig_t\cdot\nabla\vec{u}_t\vtheta_t+\sig_t\cdot\nabla^2\vec{u}_t\vtheta_t)\dx\,,
\end{equation}
the dynamic energy release rate $G_t$ reads
\begin{align*}
G_t &= \int_{\tilde{B}_r}(\ldots)\dx+\int_{\Omega_r}\Bigl(\div\Bigl(\bigl(\kappa(\dot{\vec{u}}_t)+\vec{f}_t\cdot\vec{u}_t-\psi\bigl(\eps(\vec{u}_t)\bigr)\bigr)\vtheta_t\Bigr)-(\div\sig_t+\vec{f}_t-\rho\ddot{\vec{u}}_t)\cdot\nabla\vec{u}_t\vtheta_t\Bigr)\dx \\
&\pushright{-\int_{C_r}(\nabla\vec{u}_t^\mT\sig_t)\vec{n}\cdot\vtheta_t\ds} \\
&= \int_{\tilde{B}_r}(\ldots)\dx-\int_{C_r}\left(\Bigl(\kappa(\dot{\vec{u}}_t)+\vec{f}_t\cdot\vec{u}_t-\psi\bigl(\eps(\vec{u}_t)\bigr)\Bigr)(\vtheta_t\cdot\vec{n})+(\nabla\vec{u}_t^\mT\sig_t)\vec{n}\cdot\vtheta_t\right)\ds \\
&= \int_{\tilde{B}_r}(\ldots)\dx+\int_{C_r}\vec{E}_t\vec{n}\cdot\vtheta_t\ds-\int_{C_r}(\vec{f}_t\cdot\vec{u}_t)(\vtheta_t\cdot\vec{n})\ds
\end{align*}
where the second equality follows from dynamic equilibrium \eqref{eq:classicalwave}. On the last line $\vec{E}_t$ denotes the dynamic Eshelby tensor \cite{Maugin:1994}
\begin{equation} \label{eq:Eshelby}
\vec{E}_t=\Bigl(\psi\bigl(\eps(\vec{u}_t)\bigr)-\kappa(\dot{\vec{u}}_t)\Bigr)\mathbb{I}-\nabla\vec{u}_t^\mT\sig_t.
\end{equation}
The last term involving the body force density $\vec{f}_t$ will have a vanishing contribution as $r\to 0$, since $\vec{f}_t$ is supposed to be regular and asymptotically $\vec{u}_t$ is of order $\mathcal{O}(r^{1/2})$ in linear elastic fracture mechanics.

To solve the contradiction of having the Lagrangian density in \eqref{eq:Eshelby} and the Hamiltonian density in \eqref{eq:Jdyn}, contributions from the integral on $\tilde{B}_r$ must be considered, see \cite{Maugin:1994}. By classical singularity analysis and the steady state condition $\dot{\vec{q}}_t\approx-\nabla\vec{q}_t\dot{l}_t\vtau_t$ verified for all (tensorial) fields $\vec{q}$ near the crack tip \cite{Freund:1990}, the first two terms of $G_t$ in \eqref{eq:Gt} are of order $\mathcal{O}(r^{-1})$ and hence have a vanishing contribution when integrated with the area element $r\D{r}\D{\theta}$ on $\tilde{B}_r$ as $r$ tends to zero. Similarly the term involving the body force density $\vec{f}_t$ is not singular enough to contribute. However the last two terms $\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t$ are integrable \cite{NishiokaAtluri:1983} and will yield a finite value in the limit. Using the real velocity field $\dot{\vec{u}}_t$ in the steady state condition and the fact that $\vtheta_t\to\vtau_t$ when $r$ becomes small due to continuity, we have
\[
\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t=\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t\quad\text{as}\quad r\to 0.
\]
Then an integration by parts in $\tilde{B}_r$ gives (noting that $\vtheta_t\cdot\vec{n}=0$ on $\Gamma_t$)
\begin{equation} \label{eq:magicformula}
\int_{\tilde{B}_r}\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t\dx=\int_{C_r}\rho(\dot{\vec{u}}_t\cdot\dot{\vec{u}}_t)(\vtheta_t\cdot\vec{n})\ds-\int_{\tilde{B}_r}\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t\dx-\int_{\tilde{B}_r}\rho\dot{\vec{u}}_t\cdot\dot{\vec{u}}_t\div\vtheta_t\dx
\end{equation}
from which the contribution from the last two terms in \eqref{eq:Gt} can be deduced
\begin{multline*}
\lim_{r\to 0}\int_{\tilde{B}_r}(\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t)\dx=\lim_{r\to 0}\int_{\tilde{B}_r}2\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t\dx=\lim_{r\to 0}\int_{C_r}\rho(\dot{\vec{u}}_t\cdot\dot{\vec{u}}_t)(\vtheta_t\cdot\vec{n})\ds \\
=\lim_{r\to 0}\int_{C_r}2\kappa(\dot{\vec{u}}_t)\vtheta_t\cdot\vec{n}\ds
\end{multline*}
where the last term in \eqref{eq:magicformula} vanishes in the limit $r\to 0$. We obtain hence
\begin{equation}
G_t=\lim_{r\to 0}\int_{C_r}(\vec{E}_t+2\kappa(\dot{\vec{u}}_t)\mathbb{I})\vec{n}\cdot\vtheta_t\ds=\lim_{r\to 0}\int_{C_r}\vec{J}_t\vec{n}\cdot\vtau_t\ds.
\end{equation}
which completes the proof.
\end{proof}

\begin{remark}
Compared to the classical $J$-integral, the advantage of the dynamic energy release rate in the form of \eqref{eq:Gt} resides in its direct usage for numerical computations with finite elements, since it involves an integral in the cells.
\end{remark}

\begin{remark}[Generalization to curved or kinked crack paths]
Let us recall that the crack $l\mapsto\vec{\gamma}(l)$ is supposed to follow a pre-defined straight path in this contribution. It can be generalized to arbitrary but smooth enough \emph{pre-defined} curved paths without much technical difficulties. It suffices to carefully construct a virtual perturbation $\vtheta^*$ verifying \cref{def:velocityfield}, and define the bijection $\philt_t(\vec{x})$ as a solution to a particular Cauchy evolution problem
\[
\left\{
\begin{aligned}
& \philt_0(\vec{x})=\vec{x}\,, \\
& \frac{\md}{\md t}{\philt}_t(\vec{x})=\vtheta^*\bigl(\philt_t(\vec{x})\bigr)\text{ for $t>0$}\,,
\end{aligned}
\right.
\]
see \cite{KhludnevSokolowskiSzulc:2010} for a mathematical analysis of this problem.. The obtained \emph{scalar} crack equation of motion will be formally the same as \eqref{eq:gtgc}, which predicts the crack length $l_t$ as a function of time along the this path. Note however that crack propagation direction should be at least continuous in time (curved path) so that the shape derivative method embodied by the diffeomorphism $\philt$ makes sense. In the presence of a crack kinking associated with a temporal discontinuity of the crack tangent (cf. \cref{fig:kink}), the shape derivative methods should be adapted to capture the topology change due to the kinking \cite{Hintermuller:2011}.

When the crack path is unknown, an interesting attempt is to include the crack tangent angle into the action integral \eqref{eq:action} and evaluate the variation induced by arbitrary crack direction change. Remark that the propagation criterion derived in \cite{Oleaga:2001,Adda-BediaAriasAmarLund:1999} corresponds in fact to a vectorial extension of the scalar propagation law \eqref{eq:gtgc}
\[
\lim_{r\to 0}\int_{C_r}\vec{J}_t\vec{n}\ds=\gc\vtau_t
\]
and the component perpendicular to the crack propagation direction $\vtau_t$ determines the crack path.
\end{remark}

\chapter{Detailed Calculations} \label{chap:detailedcal}

\section*{Calculation of the First-Order Action Variation}
We will carefully explore the first-order stability principle \eqref{eq:stability} by calculating the action variation with respect to arbitrary displacement and crack variations. The following easily established identities
\begin{align*}
\frac{\md}{\md l_t}\det\nabla\philt(\vec{x}^*) &= \det\nabla\philt(\vec{x}^*)\tr(\nabla\vtheta^*(\vec{x}^*)\nabla\philt(\vec{x}^*)^{-1})=\div\vtheta_t(\vec{x})\det\nabla\philt(\vec{x}^*), \\
\frac{\md}{\md l_t}\nabla\philt(\vec{x}^*)^{-1} &= -\nabla\philt(\vec{x}^*)^{-1}\nabla\vtheta^*(\vec{x}^*)\nabla\philt(\vec{x}^*)^{-1}=-\nabla\philt(\vec{x}^*)^{-1}\nabla\vtheta_t(\vec{x}).
\end{align*}
will be used for all subsequent calculations.

The classical wave equation can be obtained by calculating the action variation with zero crack advance $\delta l=0$
\begin{multline*}
\mathcal{A}'(\vec{u}^*,l)(\vec{w}^*,0) \\
=\int_I\D{t}\int_\domaini\Big(\tens{A}_0\bigl({\textstyle\frac{1}{2}}\nabla\vec{u}_t^*\nabla\philt^{-1}+{\textstyle\frac{1}{2}}\nabla\philt^{-\mathsf{T}}(\nabla\vec{u}_t^*)^\mT\bigr)\cdot\bigl({\textstyle\frac{1}{2}}\nabla\vec{w}_t^*\nabla\philt^{-1}+{\textstyle\frac{1}{2}}\nabla\philt^{-\mathsf{T}}(\nabla\vec{w}_t^*)^\mT\bigr)\det\nabla\philt \\
-\rho(\dot{\vec{u}}_t^*-\dot{l}_t\nabla\vec{u}_t^*\nabla\philt^{-1}\vtheta^*)\cdot(\dot{\vec{w}}_t^*-\dot{l}_t\nabla\vec{w}_t^*\nabla\philt^{-1}\vtheta^*)\det\nabla\philt\Big)\dxx-\mathcal{W}_t^*(\vec{w}^*_t)\,,
\end{multline*}
which gives
\begin{multline} \label{eq:aprime1}
\mathcal{A}'(\vec{u}^*,l)(\vec{w}^*,0)=\int_I\D{t}\int_\domaint\bigl(\sig_t\cdot\eps(\vec{w}_t)+\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{w}_t\vtheta_t\bigr)\dx-\mathcal{W}_t(\vec{w}_t) \\
+\underbrace{\int_I\D{t}\int_\domaini\rho\frac{\md}{\md t}\bigl((\dot{\vec{u}}_t^*-\nabla\vec{u}_t^*\nabla\philt^{-1}\,\dot{l}_t\vtheta^*)\det\nabla\philt\bigr)\cdot\vec{w}_t^*\dxx}_R
\end{multline}
where $\vec{w}$ denotes the pushforward of $\vec{w}^*$ to the current cracked configuration via \eqref{eq:transportofu}.

To proceed, we observe that the real acceleration $\ddot{\vec{u}}_t$ can be obtained by differentiating \eqref{eq:v}
\begin{equation} \label{eq:acc}
\ddot{\vec{u}}_t(\vec{x})=-\nabla\dot{\vec{u}}_t(\vec{x})\dot{l}_t\vtheta^*(\vec{x}^*)+\frac{\md}{\md t}\left(\dot{\vec{u}}_t^*(\vec{x}^*)-\nabla\vec{u}_t^*(\vec{x}^*)\nabla\philt(\vec{x}^*)^{-1}\dot{l}_t\vtheta^*(\vec{x}^*)\right)
\end{equation}
where $\nabla\dot{\vec{u}}_t$ is the (Eulerian) velocity gradient. Using \eqref{eq:acc}, the last term above can be written
\begin{multline*}
R=\int_I\D{t}\int_\domaini\bigl(\rho(\ddot{\vec{u}}_t\circ\philt+(\nabla\dot{\vec{u}}_t\circ\philt)\dot{l}_t\vtheta^*)\cdot\vec{w}_t^*\det\nabla\philt \\
+\rho\dot{l}_t(\dot{\vec{u}}_t^*-\nabla\vec{u}_t^*\nabla\philt^{-1}\,\dot{l}_t\vtheta^*)\cdot\vec{w}_t^*\tr(\nabla\philt^{-1}\nabla\vtheta^*)\det\nabla\philt\bigr)\dxx\,,
\end{multline*}
which leads to
\begin{equation} \label{eq:aprime2}
\begin{aligned}
R &= \int_I\D{t}\int_\domaint(\rho\ddot{\vec{u}}_t\cdot\vec{w}_t+\rho\dot{l}_t\nabla\dot{\vec{u}}_t\vtheta_t\cdot\vec{w}_t+\rho\dot{l}_t\dot{\vec{u}}_t\cdot\vec{w}_t\div\vtheta_t)\dx \\
&= \int_I\D{t}\int_\domaint(\rho\ddot{\vec{u}}_t\cdot\vec{w}_t-\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{w}_t\vtheta_t)\dx\,,
\end{aligned}
\end{equation}
where an integration by parts in $\Omega\setminus\Gamma_t$ has been used on establishing the last equality. Regrouping \eqref{eq:aprime1} and \eqref{eq:aprime2}, we obtain thus the spatially weak dynamic equilibrium
\begin{equation} \label{eq:spatiallyweakdyneq}
\mathcal{A}'(\vec{u}^*,l)(\vec{w}^*,0)=\int_I\D{t}\int_\domaint\bigl(\sig_t\cdot\eps(\vec{w}_t)+\rho\ddot{\vec{u}}_t\cdot\vec{w}_t\bigr)\dx-\mathcal{W}_t(\vec{w}_t).
\end{equation}
An integration by parts then gives the desired wave equation \eqref{eq:actionvariationzerocrackadvance} for the displacement.

We then evaluate the action variation with respect to arbitrary crack increment $\delta l$ but zero displacement variation
\begin{multline} \label{eq:aprime3}
\mathcal{A}'(\vec{u}^*,l)(\vec{0},\delta l)=\int_I\gc\cdot\delta l_t\D{t} \\
+\int_I\delta l_t\D{t}\int_\domaint\Bigl(\bigl(\psi\bigl(\eps(\vec{u}_t)\bigr)-\kappa(\dot{\vec{u}}_t)\bigr)\div\vtheta_t-\sig_t\cdot(\nabla\vec{u}_t\nabla\vtheta_t)-\div(\vec{f}_t\otimes\vtheta_t)\cdot\vec{u}_t\Bigr)\dx \\
-\underbrace{\int_I\D{t}\int_\domaini\bigl(\rho(\dot{\vec{u}}^*_t-\dot{l}_t\nabla\vec{u}^*_t\nabla\philt^{-1}\vtheta^*)\cdot(-\nabla\vec{u}_t^*\nabla\philt^{-1}\vtheta^*\cdot\dot{\overline{\delta l}}_t+\dot{l}_t\nabla\vec{u}_t^*\nabla\philt^{-1}\nabla\vtheta^*\nabla\philt^{-1}\vtheta^*\cdot\delta l_t)\det\nabla\philt\bigr)\dxx}_R.
\end{multline}
The last term can be written using integration by parts in the time domain
\begin{multline*}
R=\int_I\delta l_t\D{t}\int_\domaini\rho\frac{\md}{\md t}\bigl((\dot{\vec{u}}^*_t-\dot{l}_t\nabla\vec{u}^*_t\nabla\philt^{-1}\vtheta^*)\cdot(\nabla\vec{u}_t^*\nabla\philt^{-1}\vtheta^*)\det\nabla\philt\bigr)\dxx \\
+\int_I\delta l_t\D{t}\int_\domaint\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\nabla\vtheta_t\vtheta_t\dx\,,
\end{multline*}
which gives
\begin{multline*}
R=\int_I\delta l_t\D{t}\int_\domaini\big(\rho(\ddot{\vec{u}}_t\circ\philt+(\nabla\dot{\vec{u}}_t\circ\philt)\dot{l}_t\vtheta^*)\cdot(\nabla\vec{u}_t^*\nabla\philt^{-1}\vtheta^*)\det\nabla\philt \\
+\rho(\dot{\vec{u}}^*_t-\dot{l}_t\nabla\vec{u}^*_t\nabla\philt^{-1}\vtheta^*)\cdot(\nabla\dot{\vec{u}}^*_t\nabla\philt^{-1}\vtheta^*-\dot{l}_t\nabla\vec{u}^*_t\nabla\philt^{-1}\nabla\vtheta^*\nabla\philt^{-1}\vtheta^*)\det\nabla\philt\big)\dxx \\
+\int_I\delta l_t\D{t}\int_\domaint(\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t\div\vtheta_t+\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\nabla\vtheta_t\vtheta_t)\dx.
\end{multline*}
We obtain thus
\begin{multline*}
R=\int_I\delta l_t\D{t}\int_\domaint(\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{l}_t\nabla\dot{\vec{u}}_t\vtheta_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t\div\vtheta_t)\dx \\
+\int_I\delta l_t\D{t}\int_\domaini\rho(\dot{\vec{u}}^*_t-\dot{l}_t\nabla\vec{u}^*_t\nabla\philt^{-1}\vtheta^*)\cdot(\nabla\dot{\vec{u}}^*_t\nabla\philt^{-1}\vtheta^*)\det\nabla\philt\dxx.
\end{multline*}
Differentiating \eqref{eq:grad} to obtain the material time derivative of the deformation gradient $(\md/\md t)(\nabla\vec{u}_t)$
\[
\frac{\md}{\md t}(\nabla\vec{u}_t(\vec{x}))=\nabla\dot{\vec{u}}_t^*(\vec{x}^*)\nabla\philt(\vec{x}^*)^{-1}-\dot{l}_t\nabla\vec{u}_t(\vec{x})\nabla\vtheta_t(\vec{x}),
\]
and with its definition
\[
\frac{\md}{\md t}(\nabla\vec{u}_t(\vec{x}))=\nabla\dot{\vec{u}}_t(\vec{x})+\nabla^2\vec{u}_t(\vec{x})\dot{l}_t\vtheta^*(\vec{x}^*)
\]
where $\nabla^2\vec{u}_t$ is the second gradient of the displacement field (a third-order tensor), we obtain
\begin{multline*}
R=\int_I\delta l_t\D{t}\int_\domaint\big(\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{l}_t\nabla\dot{\vec{u}}_t\vtheta_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t+\rho\dot{l}_t\dot{\vec{u}}_t\cdot(\nabla^2\vec{u}_t\vtheta_t)\vtheta_t  \\
+\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\nabla\vtheta_t\vtheta_t+\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t\div\vtheta_t\big)\dx.
\end{multline*}
Using an integration by parts in the domain $\Omega\setminus\Gamma_t$ knowing that $\vtheta_t=\vec{0}$ on $\partial\Omega$ and $\vtheta_t\cdot\vec{n}=0$ on $\Gamma_t$ by definition
\[
\int_\domaint\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\nabla\vtheta_t\vtheta_t\dx=-\int_\domaint\bigl(\rho\dot{l}_t\dot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t\div\vtheta_t+\rho\dot{l}_t\nabla\dot{\vec{u}}_t\vtheta_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{l}_t\dot{\vec{u}}_t\cdot(\nabla^2\vec{u}_t\vtheta_t)\vtheta_t\bigr)\dx\,,
\]
we get finally
\[
R=\int_I\delta l_t\D{t}\int_\domaint(\rho\ddot{\vec{u}}_t\cdot\nabla\vec{u}_t\vtheta_t+\rho\dot{\vec{u}}_t\cdot\nabla\dot{\vec{u}}_t\vtheta_t)\dx
\]
which permits with \eqref{eq:aprime3} to deduce the desired equations \eqref{eq:actionvariationzerodisplacement} and \eqref{eq:Gt}.

\section*{Local Energy Balance Condition}
In this section we will derive the equivalent local condition of the global energy balance \eqref{eq:eb}, which gives the desired Griffith's law of motion \eqref{eq:gtgc} when combined with the local stability condition \eqref{eq:actionvariationzerodisplacement}. The Lagrangian defined in \eqref{eq:action} is explicitly dependent on time solely through the external work potential \eqref{eq:externalworki}. Its total derivative can thus be given by
\begin{equation} \label{eq:lagrangianderivativetime}
\frac{\md\mathcal{L}}{\md t}=\frac{\partial\mathcal{L}}{\partial\vec{u}_t^*}\dot{\vec{u}}_t^*+\frac{\partial\mathcal{L}}{\partial\dot{\vec{u}}_t^*}\ddot{\vec{u}}_t^*+\frac{\partial\mathcal{L}}{\partial l_t}\dot{l}_t+\frac{\partial\mathcal{L}}{\partial\dot{l}_t}\ddot{l}_t+\frac{\partial\mathcal{L}}{\partial t}.
\end{equation}
Using the weak dynamic equilibrium \eqref{eq:spatiallyweakdyneq} and the fact that $\dot{\vec{u}}_t^*-\dot{\vec{U}}_t\in\mathcal{C}_0$, we have
\begin{equation} \label{eq:eulerlagrangeu}
\frac{\partial\mathcal{L}}{\partial\vec{u}_t^*}(\dot{\vec{u}}_t^*-\dot{\vec{U}}_t)-\frac{\md}{\md t}\frac{\partial\mathcal{L}}{\partial\dot{\vec{u}}_t^*}(\dot{\vec{u}}_t^*-\dot{\vec{U}}_t)=\vec{0}.
\end{equation}
Plugging \eqref{eq:eulerlagrangeu} into \eqref{eq:lagrangianderivativetime}, we obtain
\begin{equation} \label{eq:eblocal1}
\frac{\md\mathcal{L}}{\md t}=\frac{\md}{\md t}\left(\frac{\partial\mathcal{L}}{\partial\dot{\vec{u}}_t^*}\dot{\vec{u}}_t^*\right)+\frac{\partial\mathcal{L}}{\partial\vec{u}_t^*}\dot{\vec{U}}_t-\frac{\md}{\md t}\frac{\partial\mathcal{L}}{\partial\dot{\vec{u}}_t^*}\dot{\vec{U}}_t+\frac{\partial\mathcal{L}}{\partial l_t}\dot{l}_t+\frac{\partial\mathcal{L}}{\partial\dot{l}_t}\ddot{l}_t+\frac{\partial\mathcal{L}}{\partial t}.
\end{equation}

With all necessary temporal regularity, we note that the energy balance condition \eqref{eq:eb} can be equivalently written as
\begin{equation} \label{eq:eblocal2}
\frac{\md\mathcal{H}}{\md t}=\frac{\md}{\md t}\left(\mathcal{L}+2\mathcal{K}\right)=\frac{\md}{\md t}\left(\mathcal{L}-\frac{\partial\mathcal{L}}{\partial\dot{\vec{u}}_t^*}\dot{\vec{u}}_t^*-\frac{\partial\mathcal{L}}{\partial\dot{l}_t}\dot{l}_t\right)=\frac{\partial\mathcal{L}}{\partial\vec{u}_t^*}\dot{\vec{U}}_t-\frac{\md}{\md t}\frac{\partial\mathcal{L}}{\partial\dot{\vec{u}}_t^*}\dot{\vec{U}}_t+\frac{\partial\mathcal{L}}{\partial t}.
\end{equation}
Comparing \eqref{eq:eblocal1} and \eqref{eq:eblocal2}, we obtain the desired local energy balance condition
\[
\left(\frac{\partial\mathcal{L}}{\partial l_t}-\frac{\md}{\md t}\frac{\partial\mathcal{L}}{\partial\dot{l}_t}\right)\cdot\dot{l}_t=0.
\]


%!TEX root = ../main.tex
	This section outlines the modifications that must be done to the argument [Stovall] to extend it to the complex case. The paper reduces the analytic result (whether the operator is bounded from a certain $L^p$ to a certain $L^q$) to a geometric result, previously proven by Dendrinos and Wright in the real case, proven in section \ref{sec:the_geometry} for the complex scenario.

	\subsection{Uniform Local Restriction} % (fold)
	\label{sub:uniform_local_restriction}

	The first step in the proof is the local result:

	\begin{thm}[Theorem 2.1 in {[Stovall]}]\label{thm:uniform_local_restriction}
	 	Fix $d\ge 2$, $N$, and $(p,q)$ satisfying ZZZZ. Then, for every ball $B\subseteq \mathbb R^d$ and every degree $N$ polynomial $\gamma: \mathbb C \to \mathbb C^d$ satisfying
	 	\begin{equation}
	 		0<C_1 \le J_\gamma(z) \le C_2, \,\,\, z \in B
	 	\end{equation}
	 	we have the extension estimate
	 	\begin{equation}
	 		\|\mathcal E_\gamma(\chi_B f)\|_q \le C_{d,N,\log {\frac {C_2}{C_1}}} \|f\|_p
	 	\end{equation}
	 \end{thm}

	An important preliminary fact, proven in [Stovall, lemma XXX] is that whenever $J_\gamma \lesssim 1$ on a compact, convex set, there is a change of co-ordinates $L \in SU(d;\mathbb C)$ so that all the coefficients in $L\circ J_\gamma$ are $O_{N,d}(1)$. The proof is done through a compactness argument, and transfers without any significant modification to the complex case for convex domains. The exact formulation that we will use is:

	\begin{lemma}
		Fix $N,d \ge 2$, $\epsilon>0$. Then for any polynomial curve $\gamma$ and triangle $T$ with $B_\epsilon \subset T\subset  B(0,1)$  obeying that $|J_\gamma(T)|\subseteq [1/2, 2]$ there exists a transformation $A \in SU(n;\mathbb C)$ so that $\|A\gamma\|_{C^N(K)} \lesssim_\epsilon 1$.
	\end{lemma}

	\begin{proof}
		Define $\gamma_\epsilon = \epsilon^{\frac{d^2+d}{2}}\gamma(\epsilon^{-1} z)$. Now $\gamma_\epsilon$ has the property that $|J_{ \gamma_\epsilon}(B(0,1))|\subseteq [1/2, 2]$. This reduces the problem to the situation in Lemma XXXX in [Sto]
	\end{proof}

	The second preliminary is a statement about offspring curves. Given $\mbf h:= (\mbf h_1, \dots, \mbf h_k)$, the offspring curve $\gamma_h(z)$ is defined as $\gamma_h := \frac 1 K \sum_{i=1}^K \gamma(z+h_i)$. Lemma 2.3 in [Stovall] states:

	\begin{lemma}
	Fix $N,d \ge 2$ and $\epsilon>0$. There exists a constant $c_d > 0$ and a radius $\delta:=\delta(\epsilon,N,d)$ so that for any triangle $B_\epsilon(0) \subseteq T \subseteq B_1$ and so that the following conditions hold for any polynomial curve $\gamma$ satisfying

	\begin{equation}
	\label{eq:stovall_offspring_hybound}
		|J_\gamma(T)|\subseteq [1/2, 2]
	\end{equation}

	For any ball $B$ of radius $\delta$ centered at a point in T, and any $\mbf h:= (\mbf h_1, \dots, \mbf h_k) \in \mathbb C^k$, the curve $\gamma_h$ satifies the following inequalities $\tilde B = \bigcap_{i=1}^k (B - h_j)$:

	\begin{equation}
		\label{eq:stovall_ineq_1}
		|J_{\gamma_h}(\mbf z)| \gtrsim_{N,d,\epsilon} |v( \mbf z)| \prod_{i=1}^d |\Lambda^{(d)}_{\gamma_h}( \mbf z_i)|, \text{ for any }\mbf z \in \tilde B^d
	\end{equation}

	\begin{equation}
		\label{eq:stovall_ineq_2}
		|L_{\gamma_h}(z)| \approx_{N,d,\epsilon} \text{ for any } 1 \in \tilde B
	\end{equation}
	in particular, one can cover any such $T$ by $O_{N,d,\epsilon}(1)$ open sets so that the conclusions \eqref{eq:stovall_ineq_1} and \eqref{eq:stovall_ineq_2} hold for any polynomial that follows \eqref{eq:stovall_offspring_hybound}.
	\end{lemma}

	\begin{proof}
		First note, that by choosing $\delta$ small enough and the Lemma [THE PREVIOUS LEMMA], we have that $$ |J_\gamma(T + B_\delta)|\subseteq [1/4, 4]$$. This fact prevents any issues arising from the different triangle shapes. From here on, the result follows as the proof in [STOVALL, Lemma 2.3].
	\end{proof}

	Using those preliminaries, we can proceed to the proof of Theorem \ref{thm:uniform_local_restriction}. The proof follows exactly as in [xxxx], which is in itself a sketch of the proof in [That old paper that's super nice]. The following lemma is a simple computation, that takes the role of the equivalent necessary result in the real case.

	\begin{lemma}
		The function $|v(0, z_1, \dots z_{d-1})|^{-2(a-1)}$ belongs to $L^{\frac{d}{2(a-1)}-\epsilon}_{B_1^d(0)}$ for $\epsilon > 0$ small.
	\end{lemma}

	other than this difference (and the factors of $2$ that appear at the exponent everywhere where the Jacobian appears, that lead to the factor of $2$ in Lemma [XXX the lemma above]), the proof of Lemma 2.4 in [Sto] applies mutatis mutandis to the complex case.

	% subsection uniform_local_restriction (end)

	\subsection{Almost orthogonality} % (fold)
	\label{sub:almost_orthogonality}

	The main result in this section is:

	\begin{lemma}
		Let $\gamma:\mathbb C\to \mathbb C^d$ be a complex polynomial curve, and let $T_j$ be one of the sets in Lemma \ref{lem:geometric_lemma}. For $n\in \mathbb  Z$ define the dyadic partition
		\begin{equation}
			T_{j,n} = \{z \in T_j, |z_j - b_j| \sim 2^n\}
		\end{equation}
		Then for each $(p,q)$ satisfying $q= \frac{d(d+1)}{2}p'$ and $\infty > q > \frac{d^2+d+2}{2}$ and $f \in L^p(d\lambda_\gamma)$ we have:
		\begin{equation}
			\|\mathcal E_\gamma (\chi {T_j} f)\|_{L^q(\mathbb R^{2d})} \lesssim
			\left\|
			\left(
				\sum_n |\mathcal E_\gamma (\chi {T_{j,n}} f)|^2
			\right)^{1/2}
			\right\|_{L^q(\mathbb R^{2d})}
		\end{equation}
	\end{lemma}

	\begin{proof}
		The proof is a standard Littlewood-Paley argument, using property ZZZ for the support of the Littlewood-Paley blocks, see Stovall again.
	\end{proof}

	The goal is now to show that we can sum the pieces in the Littlewood-Paley decomposition above. The main proposition to do so (essentially Lemma 4.1 in [Stovall]) is:

	\begin{lemma}
		There exists $\epsilon(N,d,p)>0$ such that, if $n_1 \le \dots \le n_d$, and $f_i$ is Schwartz and supported in $T_{j,n_i}$ we have:

		\begin{equation}
			\left \|\prod_{i=1}^d \mathcal E_\gamma [f_i]\right\|_{L^{d+1}} \lesssim 2^{n_D-n_1} \prod_{i=1}^d \|f_i\|_{L^2(d\lambda)}
		\end{equation}
	\end{lemma}

	before proving Lemma XXX, we will prove an auxiliary result (the complex version of a lemma by Christ in [STOXXX12]) that will be needed during the interpolation. We provide an outline here:

	\begin{lemma}

	\begin{equation}
		\int_{\mathbb C^l \sim \mathbb R^{2l}} \prod_{1\le i\le l}f_i(z_i) \prod_{1\le i<j\le l} g_{i,j}(z_i-z_j) dz \lesssim \prod_{i=1}^l \|f_i\|_p \prod_{1\le i<j\le l} \|g_{i,j}\|_{q,\infty}
	\end{equation}
	whenever $ 2 l =  2 l p^{-1} + \frac{2 l (l-1)}{2} q^{-1}$

	\end{lemma}
	\begin{proof}
	The proof starts by studying the set of powers $p_i, q_{i,j}$ with $\sum_{i} p_i^{-1} + \sum_{i,j} q_i^{-1} = l$ for which the estimate

	\begin{equation}
		\int_{\mathbb C^l \sim \mathbb R^{2l}} \prod_{1\le i\le l}f_i(z_i) \prod_{1\le i<j\le l} g_{i,j}(z_i-z_j) dz \lesssim \prod_{i=1}^l \|f_i\|_{p_i} \prod_{1\le i<j\le l} \|g_{i,j}\|_{q_{i,j}}
	\end{equation}
	holds. We will denote by capital letters the vectors $(p_1, \dots p_l, p_{1,2}, \dots p_{l-1,l})$, which we will think of as elements of the affine subspace $H:=\{\sum_{i} p_i^{-1} + \sum_{i,j} q_i^{-1} = l\}$.

	The base cases are $A:=(p_{i}^{-1}=1, q_{i,j}^{-1} = 0)$ and $B:=(p_{i}^{-1}=\delta_{i,1}, q_{i,j}^{-1} = \delta_{i+1,j})$, which are Fubini's theorem. Now, the result is invariant over permutations over all the indices $(i,j)$. This allows us to extend the second base $B$ case to all the cases $B_\sigma$ permutations obtained from those permutations.

	By Riesz-Torin, the result is then true for $A':=\frac 1 {n!} \sum_{\sigma \in S_n} B_\sigma = (p_i = \frac 1 n, q_i = \frac 1 {2n})$. By Riesz-Torin again, the result is true for all the points interpolating $A$ and $A'$. This proves the strong version of the theorem.

	To get the strong case, it suffices to show that all the points joining $A$ and $A'$ lie on the interior of the interpolation polytope (interior with the affine topology on $H$). By convexity again, it suffices to show that $A'$ does. The geometric argument can be seen in Christ (it is exactly the same as in dimension 2).
	\end{proof}
	% subsection almost_orthogonality (end)


\section{Simulation Results}
Here we are going to evaluate the forecast performance of CV model averaging through controlled numerical simulation. Specifically, we are going to use three different designs of the true data generating process: (i). an AR(2) process plus five exogenous predictors with ARCH(1) errors,
\begin{subequations}
            \begin{align}
                y_{t} & = \mu + \rho_{1}y_{t-1} + \rho_{2}y_{t-2} + \sum_{i=1}^{5}\theta_{i}x_{i} + e_{t} \\
                e_{t} & = v_{t}\sqrt{h_{t}}\\
                h_{t} & = \alpha_{0} + \alpha_{1}e_{t-1}^2
            \end{align}
\end{subequations}
(ii). an AR(2) process plus two exogenous predictors with heteroscedastic errors drawing from this distribution $N(0,y_{t-1}^2)$.
\begin{equation}
                y_{t} = \mu + \rho_{1}y_{t-1} + \rho_{2}y_{t-2} + \sum_{i=1}^{2}\theta_{i}x_{i} + e_{t}
\end{equation}
(iii). an AR(2) process with a break in the variance of the error term. We consider this design to study the forecasting performance of CV model averaging in the Great Moderation type environment. The break date of the error term variance is not identical to that of the conditional mean\footnote{In the simulation we set the break fraction of the error term variance at $0.5$ relative to the training sample. But the break fraction for the conditional mean is set at $0.3$ relative to the training sample.}. We allow for this break date difference hoping to better approximate the environment forecasters face in practice. Mathematically, the data generating process for design (iii) is the following:
\begin{equation}
            y_{t} = \mu + \rho_{1}y_{t-1} + \rho_{2}y_{t-2} + e_{t}
\end{equation}
where
\[ e_{t} \sim \left\{
  \begin{array}{l l}
    N(0,\sigma^2) & \quad t \in [1,\tau_{v}]\\
    N(0,\frac{1}{4}\sigma^2) & \quad t \in [\tau_{v}+1, R]
\end{array} \right.\]
In all three designs there is a one-time structural break in all coefficients of the conditional mean happening at the $30\%th$ observation of the training sample $R$, that is, $\tau = 0.3$. We let the structural break of the parameters take the multiplicative form, that is, if the pre-break coefficient is $\beta$, the post-break parameter becomes $\delta\beta$, where $\delta$ controls the break size. For the ARCH process, $v_{t}$s are drawn independently and identically from the standard normal distribution. Other predictors are drawn i.i.d as the following: $x_{1} \sim \mathrm{N}(0,4)$, $x_{2} \sim \mathrm{U}[-2,2]$, $x_{3} \sim \mathrm{N}(0,16)$, $x_{4} \sim \mathrm{t}(5)$ and $x_{5} \sim \mathrm{Binomial}(1,0.02)$. The parameter values for all data generating processes are $\mu = 2, \rho_{1} = 0.4, \rho_{2} = 0.2$, $\theta_{1} = 0.8,\theta_{2} = -0.4,\theta_{3} = 2,\theta_{4} = -3.5,\theta_{5} = 10$, $\alpha_{0} = 1, \alpha_{1} = 0.4$. These values are chosen to satisfy the stationarity and ARCH error regularity assumptions. Note that in our simulation, the post-break coefficient values become smaller than their pre-break counterparts. This choice of break direction provides us with more freedom in controlling the break size. For example, if the true data generating process is an intercept-free AR(1) model with pre-break parameter value $0.9$, to ensure regime-wise stationarity, $\delta$ should not take values greater than $1.1$ if we prefer larger post-break parameter value\footnote{Bai and Perron \cite{bai_perron98} assume that the break size is large enough in order to be identified and estimated. Though we have not found any leading metric measuring the break size, break size of $1.1$ as in the example is not large enough to identify especially when the data is highly volatile as shown in our simulation work.}.

To make our simulation design close to relevant empirical studies and to capture the difficulty many researchers face in finding the best approximating models, the model used to forecast differs from the true data generating process\footnote{The difference of the AR order between the DGP and the forecasting model captures the fact that in practice, it is hard to fully capture the dynamics by selecting the true order. By the principle of parsimony, researchers or practitioners tend to select a model of small order.}: in case (i) the model to forecast is based those five exogenous predictors in the DGP, $y_{t} = \mu + \sum_{i=1}^{5}\theta_{i}x_{i} + e_{t}$; in case (ii), again the model to forecast does not involve the AR component, $y_{t} = \mu + \sum_{i=1}^{2}\theta_{i}x_{i} + e_{t}$; in case (iii), the model to forecast is the AR(1) with intercept, $y_{t} = \mu + \rho_{1}y_{t-1} + e_{t}$.

For each case, we evaluate the out-of-sample (OOS) forecasting performance by comparing the root mean squared forecast error divided by that of the equal weights method. Recursive window is used to generate OOS forecasts as it mimics the practice that forecasters update their forecast when new data become available. Out-of-sample forecast is constructed by the following steps. First, we split the time series sample into two parts: the prediction or training sample of size $R$ and the evaluation sample of size $P$. Under the recursive window, at each point in time, the estimated parameter is updated by adding one more observation starting with sample size $R$. For example, $\beta_{t} = (\sum_{s=1}^{t-1} x_{s} x_{s}')^{-1}\sum_{s=1}^{t-1}x_{s}y_{s+1},\beta_{t+1} = (\sum_{s=1}^{t} x_{s} x_{s}')^{-1}\sum_{s=1}^{t}x_{s}y_{s+1}$. By this procedure, we estimate parameters recursively and then generate a sequence of forecasts of size $P$ based on these estimated parameters. We can compare this sequence of forecasts with those reserved data in the evaluation sample, and evaluate our forecasts according to some loss function, for example, RMSFE or MSFE. See Calhoun \cite{calhoun_bootstrap_2013} \cite{calhoun_overfit_2014}, McCracken \cite{mccracken_JE2000} \cite{mccracken_JE2007}, Rossi \cite{rossi_HANDBOOK2013}, Clark and McCracken \cite{clark_mccracken_JE2001} \cite{clark_mccracken_JE2005} \cite{clark_mccracken_HANDBOOK2013}, Clark and West \cite{clark_west_JE2007} and West \cite{west_HANDBOOK2006} for more details on out-of-sample forecasting.

The total sample size, $T$, is $200$\footnote{This sample size is chosen to be relevant to most macroeconomic time series.}. In our pseudo one-step ahead out-of-sample forecasting simulation, we reserve the first $170$ or $150$ ($R = 170$ or $R = 150$) observations as the training sample and the rest as the prediction sample ($P = 30$ or $P = 50$). For the break model, we use the post-break window method to forecast out-of-sample. Other techniques, such as the optimal window method proposed by Pesaran and Timmermann \cite{pesaran_timmermann_JE2007} or the robust weight method proposed by Pesaran, Pick and Pranovich \cite{pesaran_pick_pranovich_2011} could also be considered. For simplicity, we only apply the post-break window method in this paper \footnote{Currently, researchers are sill working on developing theory and methods related to forecasting with breaks, and we are not aware of any dominant method that performs well in most situations faced by practitioners. The simulation conducted by Pesaran and Timmermann suggests that there is little gain from complicated methods. The simple rule, to forecast using the data after the detected break, seems to work as well as anything else.}.

In each case, to evaluate and compare performance, we produce forecasts using six methods \footnote{Methods such as Bates-Granger combination, Granger-Ramanathan combination and common factor combination are not considered in our simulation. In a related paper, Clark and McCracken \cite{clark_mccracken_averagingVAR_2008} conclude that ``\emph{...it is clear that the simplest forms of model averaging---such as those that use equal weights across all models---consistently perform among the best methods...forecasts based on OLS-type combination and factor-based combination rank among the worst}". So we only compare our method with either closely related or empirically proven effective methods.}:
\begin{inparaenum}[(i)]
\item Mallows' model averaging (\textbf{Cp});
\item CV model averaging (\textbf{CV});
\item Bayesian model averaging\footnote{We call this method ``Bayesian'' not in a strict sense: the Bayesian weight for each model is calculated based on the value of the Schwarz-Bayesian information criterion, i.e. the weight for the beak model is $w_{b} = \exp{(SIC^{b})}/(\exp{(SIC^{b})} + \exp{(SIC^{s})})$} (\textbf{SIC});
\item stable model (\textbf{Stable});
\item break model (\textbf{Break}); and
\item equal weights\footnote{Each model receives weight of $0.5$.} (\textbf{Equal}).
\end{inparaenum}
We evaluate their forecast performance by root mean-square forecast error (\textbf{RMSFE}). For ease of comparison, we pick the equal weight method as the benchmark\footnote{The reason to pick equal weights as the benchmark is because of the aforementioned forecast combination puzzle: equally weighted forecasts tend to perform better than other complicated methods in many applications. The puzzle is generally discussed in the forecasting literature without allowing breaks. In this study we try to examine whether it dominates our method when facing structural breaks.} and compute the relative performance (\textbf{Ratio}) for each method, for example, RMSFE\textsuperscript{CV}/RMSFE\textsuperscript{Equal}. If the ratio is less than one, it indicates better performance than equal weights. The smaller the ratio is, the better the forecasting performance is for given sample split.
\subsection{Design I}
\begin{table}
    \caption{Monte Carlo Simulation: Design I} \label{ntb:1}
    \centering
    \begin{adjustbox}{width=\textwidth,totalheight=\textheight,keepaspectratio}
    \begin{threeparttable}
    \begin{tabular}{ccccccccccc}
    \toprule
     & \multicolumn{5}{c}{$P = 30$} & \multicolumn{5}{c}{$P = 50$} \\
    \cmidrule(r){2-6}
    \cmidrule(r){7-11} \\
      Break Size   & Cp     &   CV   & SIC    & Stable & Break  & Cp     & CV     & SIC    & Stable & Break \\
             $100$ & 0.6312 &0.6298  &1.2987  & 1.6557 &0.6297  & 0.6599 &0.6585  &1.2849  & 1.6220 &0.6584 \\
             $10$  & 0.6644 &0.6627  &1.2563  & 1.6148 &0.6627  & 0.6871 &0.6854  &1.2473  & 1.5874 &0.6853 \\
             $5$   & 0.7085 &0.7066  &1.2063  & 1.5605 &0.7065  & 0.7289 &0.7271  &1.2005  & 1.5335 &0.7270 \\
             $3$   & 0.7658 &0.7636  &1.1517  & 1.4782 &0.7636  & 0.7869 &0.7850  &1.1454  & 1.4489 &0.7850 \\
             $2$   & 0.8330 &0.8308  &1.0974  & 1.3734 &0.8308  & 0.8500 &0.8483  &1.0925  & 1.3471 &0.8483 \\
    \bottomrule
    \end{tabular}
    \begin{tablenotes}[para, flushleft] \footnotesize
    Notes: The DGP is $y_{t} = \mu + \rho_{1}y_{t-1} + \rho_{2}y_{t-2} + \sum_{i=1}^{5}\theta_{i}x_{i} + e_{t}, e_{t} = v_{t}\sqrt{h_{t}}, h_{t} = \alpha_{0} + \alpha_{1}e_{t-1}^2$ and the forecasting model is $y_{t} = \mu + \sum_{i=1}^{5}\theta_{i}x_{i} + e_{t}$. $\mathrm{P}$ is the evaluation sample size, total sample size is $200$, break fraction relative to the training sample is $\tau = 0.3$, OOS forecasts are generated by the recursive window, 5000 times replication. Equal weight is chosen as the benchmark and the numbers in the table represent the RMSFE ratio between each individual method and equal weight. Cp: Mallows' weights. CV: cross-validation weights. SIC: Schwarz-Bayesian weights. Stable: model without structural break. Break: model with a full structural break.
    \end{tablenotes}
    \end{threeparttable}
    \end{adjustbox}
\end{table}
Simulation results for the ARCH\footnote{Our results also hold in the GARCH error case.} error design are presented in table \ref{ntb:1}. We can see from the table that CV forecasts better than Cp across all considered break sizes and prediction sample sizes. Both of CV and Cp's relative RMSFE decreases monotonically as the break size increases, but CV decreases at a slightly faster speed. Bayesian weighting does slightly worse than the equal weight method, but its performance deteriorates when the break size becomes large as it fails to capture the fact that the evidence supporting break is becoming stronger. It should not be a surprise that the break model does well since structural break indeed happens in the DGP, but it performs slightly worse than CV because it does not take the volatility into account.

Our results indicate that when there is ARCH type conditional heteroscedasticity in the data and when the break is not strictly dominated by the volatility, CV forecasts better than Mallows' model averaging. Additionally, CV forecasts better than equal weight so the forecast combination puzzle does not apply in this design. Bayesian model averaging is approximately equivalent to equal weight, but it is less sensitive to the size of break. Compared with CV, Bayesian weighting method does not put more weight on the proper model even the break size increases significantly.

It is worth mentioning that in our design, the post-break coefficients become smaller than their pre-break counterparts by various degrees. This procedure is adopted to ensure that piece-wise stationarity is maintained under structural break. \footnote{The simulation results stay the same if we reverse the break size direction by starting with smaller pre-break parameter values.}
\subsection{Design II}
\begin{table}
    \caption{Monte Carlo Simulation: Design II} \label{ntb:2}
    \centering
    \begin{adjustbox}{width=\textwidth,totalheight=\textheight,keepaspectratio}
    \begin{threeparttable}
    \begin{tabular}{ccccccccccc}
    \toprule
     & \multicolumn{5}{c}{$P = 30$} & \multicolumn{5}{c}{$P = 50$} \\
    \cmidrule(r){2-6}
    \cmidrule(r){7-11} \\
    Break Size      & Cp     & CV     & SIC    & Stable & Break  & Cp     & CV     & SIC    & Stable & Break  \\
    $100$           & 0.4610 &0.2586  &1.0717  & 1.9477 &0.2586  & 0.5706 &0.3415  &1.0649  & 1.8951 &0.3415 \\
    $10$            & 0.7007 &0.5681  &1.0393  & 1.6830 &0.5683  & 0.6945 &0.5419  &1.0392  & 1.6930 &0.5421 \\
    $5$             & 0.8422 &0.7700  &1.0194  & 1.4212 &0.7701  & 0.8699 &0.7946  &1.0191  & 1.3916 &0.7948 \\
    $3$             & 0.8978 &0.8541  &1.0111  & 1.2809 &0.8543  & 0.9135 &0.8800  &1.0126  & 1.2551 &0.8803 \\
    $2$             & 0.9188 &0.8778  &1.0082  & 1.2352 &0.8781  & 0.9417 &0.9320  &1.0074  & 1.1578 &0.9323 \\
    \bottomrule
    \end{tabular}
    \begin{tablenotes}[para, flushleft] \footnotesize
    Notes: The DGP is $y_{t} = \mu + \rho_{1}y_{t-1} + \rho_{2}y_{t-2} + \sum_{i=1}^{2}\theta_{i}x_{i} + e_{t}, e_{t} \sim \mathrm{N}(0, y_{t-1}^{2})$ and the forecasting model is $y_{t} = \mu + \sum_{i=1}^{2}\theta_{i}x_{i} + e_{t}$. $\mathrm{P}$ is the evaluation sample size, total sample size is $200$, break fraction relative to the training sample is $\tau = 0.3$, OOS forecasts are generated by the recursive window, 5000 times replication. Equal weight is chosen as the benchmark and the numbers in the table represent the RMSFE ratio between each individual method and equal weight. Cp: Mallows' weights. CV: cross-validation weights. SIC: Schwarz-Bayesian weights. Stable: model without structural break. Break: model with a full structural break.
    \end{tablenotes}
    \end{threeparttable}
    \end{adjustbox}
\end{table}
Simulation results for the second design are shown in table \ref{ntb:2}. In this case we can see from the table that CV forecasts much better than Cp across all break sizes and prediction sample sizes. Both of their relative RMSFE decreases monotonically as the break size increases, but in this design the RMSFE of CV decreases at a much faster speed. Bayesian weighting does almost the same as equal weight, but its performance deteriorates when the break size becomes large as shown in the previous design. The pattern of our results holds for both cases of the out-of-sample size.

Our results indicate that when there is this ``wild'' type heteroscedasticity in the data as modeled in the DGP and when the break is not strictly dominated by the volatility, CV forecasts significantly better than Mallows' model averaging, especially when the break size is large. Additionally, CV forecasts better than equal weight so the forecast combination puzzle does not apply in this design. Bayesian model averaging is approximately equivalent to equal weight. Again, compared with CV, Bayesian weighting method does not put more weight on the proper model when the break size increases.
\subsection{Design III}
\begin{table}
    \caption{Monte Carlo Simulation: Design III} \label{ntb:4}
    \centering
    \begin{adjustbox}{width=\textwidth,totalheight=\textheight,keepaspectratio}
    \begin{threeparttable}
    \begin{tabular}{ccccccccccc}
    \toprule
     & \multicolumn{5}{c}{$P = 30$} & \multicolumn{5}{c}{$P = 50$} \\
    \cmidrule(r){2-6}
    \cmidrule(r){7-11} \\
    Break Size      & Cp     & CV     & SIC    & Stable & Break  & Cp     & CV     & SIC    & Stable & Break  \\
    $100$           & 0.9810 &0.9759  &1.0011  & 1.0839 &0.9760  & 0.9825 &0.9769  &1.0011  & 1.0789 &0.9770 \\
    $10$            & 0.9860 &0.9789  &1.0006  & 1.0716 &0.9790  & 0.9880 &0.9822  &1.0006  & 1.0656 &0.9823 \\
    $5$             & 0.9919 &0.9850  &1.0003  & 1.0583 &0.9852  & 0.9933 &0.9868  &1.0003  & 1.0534 &0.9870 \\
    $3$             & 0.9977 &0.9903  &1.0000  & 1.0455 &0.9906  & 0.9975 &0.9905  &1.0001  & 1.0428 &0.9908 \\
    $2$             & 1.0009 &0.9940  &0.9999  & 1.0347 &0.9944  & 1.0013 &0.9952  &0.9999  & 1.0316 &0.9958 \\
    \bottomrule
    \end{tabular}
    \begin{tablenotes}[para, flushleft] \footnotesize
    Notes: The DGP is $y_{t} = \mu + \rho_{1}y_{t-1} + \rho_{2}y_{t-2} + e_{t}$, $e_{t} \sim N(0,\sigma^2) \quad t \in [1,\tau_{v}]$ and $e_{t} \sim N(0,\frac{1}{4}\sigma^2) \quad t \in [\tau_{v}+1, R]$, $\tau_{v} = 0.5R$, the forecasting model is $y_{t} = \mu + \rho_{1}y_{t-1} + e_{t}$. $\mathrm{P}$ is the evaluation sample size, total sample size is $200$, break fraction relative to the training sample is $\tau = 0.3$, OOS forecasts are generated by the recursive window, 5000 times replication. Equal weight is chosen as the benchmark and the numbers in the table represent the RMSFE ratio between each individual method and equal weight. Cp: Mallows' weights. CV: cross-validation weights. SIC: Schwarz-Bayesian weights. Stable: model without structural break. Break: model with a full structural break.
    \end{tablenotes}
    \end{threeparttable}
    \end{adjustbox}
\end{table}
Simulation results for this Great Moderation type design are shown in table \ref{ntb:4}. The general pattern shown in the previous two designs remains in this case. CV forecasts better than Cp across all listed break sizes and prediction sample sizes. Both of their relative RMSFE decreases monotonically as the break size increases, but the relative RMSFE of CV decreases at a slightly faster speed. Bayesian weighting does almost the same as equal weight, but its performance is less sensitive to the break size in this case. The pattern of our results holds for both cases of the out-of-sample size.

In this design our results indicate that when the data has the Great Moderation appearance as shown in the U.S. GDP growth data, and when the break is not strictly dominated by the volatility, CV forecasts better than Mallows' model averaging. Additionally, the forecast combination puzzle does not apply here. This motivates us to apply our method to forecasting real U.S. GDP growth rate and to comparing its performance with other related methods.


\chapter{Introduction to Boosting \label{chapter:boosting}}

Random forests (Chapter~\ref{chapter:randomforests}) are one approach to ensemble learning. Today we will examine another approach, \textbf{boosting}, that relies on a completely different set of ideas. The first practical boosting algorithm, AdaBoost, was invented in 1995 by Freund and Schapire. However, boosting is a general approach to choosing a set of weak learners that, together, create a strong learner. Since the same idea has spawned many different approaches, boosting is referred to as a \textbf{meta-algorithm}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{AdaBoost}

Assume we are trying to solve a supervised learning problem. As usual, our data look like this:
$$\{ (x^{(1)}, y^{(1)}),  \dots, (x^{(n)}, y^{(n)}) \}$$

\noindent where the $x^{(i)}$ are feature vectors of length $p$. For now, we will assume that the outcome, $Y$, is binary (so this is a binary classification problem -- see Chapter~\ref{chapter:classification}). We'll make one small notational change from earlier chapters. Instead of $Y \in \{0, 1\}$, we'll say $Y \in \{-1, 1\}$. The fact that the positive and negative training examples have opposite sign will help us write the algorithm more concisely.

The basic strategy behind AdaBoost is to build classifiers in series while maintaining a set of weights for the different training examples. The weights change with each subsequent classifier, increasing if previous classifiers have made incorrect predictions and decreasing if previous classifiers were correct. In this way, the importance of difficult-to-classify training examples increases, leading us to prefer future classifiers that successfully predict these examples.

Here is the algorithm:

\begin{enumerate}
\item Initialize the observation weights to $w_i^{(1)} = \frac{1}{N}$ for $i = 1, \dots, N$.
\item For $m = 1, \dots, M$:
    \begin{enumerate}
    \item[(a)] Select a classifier, $G_m(x)$, that minimizes the weighted training error according to the current set of weights, $w_i^{(m)}$. Depending on the algorithm, it may be possible to train a single classifier on the weighted training set; in other cases, one may need to select the best-performing classifier from among a predefined set.
    \item[(b)] Compute
    $$ \text{err}_m = \frac{\sum_{i=1}^N w_i^{(m)} \cdot \mathcal{I}(y^{(i)} \neq G_m (x^{(i)}))}{\sum_{i=1}^N w_i^{(m)}} $$
    \item[(c)] Compute voting weight for classifier $m$:
    $$ \alpha_m = \log \left( \frac{1 - \text{err}_m}{\text{err}_m} \right) $$
    \item[(d)] Set
    $$ w_i^{(m+1)} \coloneqq w_i^{(m)} \cdot \text{exp} \left[ \alpha_m \cdot \mathcal{I}(y^{(i)} \neq G_m (x^{(i)})) \right] $$
    for $i = 1, \dots, N$.
    \end{enumerate}
\item Output
$$G(x) = \text{sign} \left[ \sum_{m=1}^M \alpha_m G_m(x) \right]$$
\end{enumerate}

The output definition in step~$3$ is really the key to the whole thing. Your goal is to construct a set of classifiers whose votes will be added together to produce an overall decision. AdaBoost provides one possible set of instructions for how to choose/build the individual classifiers, $G_m(x)$, and how to weight their votes.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Visualizing the Steps of AdaBoost}

We will now train AdaBoost on the happiness dataset from Section~\ref{section:happiness}. Here is the dataset, using the new notation for $Y$. The astute observer will notice that we've added a fourth covariate, $X_4$ (whether or not the person has a pet). The reason is that subjects $5$ and $10$ are exactly the same otherwise, so we can never get perfect separation using the original dataset.

{\small
\begin{center}
\begin{tabular}{cccccc}
\toprule
Subject ID & friends $(X_1)$ & money $(X_2)$ & free time $(X_3)$ & pet $(X_4)$ & happy $(Y)$ \\
\midrule
1 & 1 & 1 & 0 & 0 & -1 \\
2 & 1 & 1 & 1 & 0 & -1 \\
3 & 0 & 1 & 1 & 0 & -1 \\
4 & 0 & 0 & 0 & 0 & -1 \\
5 & 1 & 0 & 0 & 0 & -1 \\
6 & 0 & 0 & 0 & 0 & -1 \\
\midrule
7 & 1 & 2 & 1 & 0 & 1 \\
8 & 1 & 0 & 1 & 0 & 1 \\
9 & 0 & 0 & 1 & 1 & 1 \\
10 & 1 & 0 & 0 & 1 & 1 \\
\bottomrule
\end{tabular}

\begin{align*}
	X_1 &= \left\{ \begin{array}{cl} 0 & \text{no friends} \\ 1 & \text{friends} \end{array} \right.
	&
	X_2 &= \left\{ \begin{array}{cl} 0 & \text{poor} \\ 1 & \text{enough money} \\ 2 & \text{rich} \end{array} \right. \\
	X_3 &= \left\{ \begin{array}{cl} 0 & \text{no free time} \\ 1 & \text{some free time} \end{array} \right.
	&
	X_4 &= \left\{ \begin{array}{cl} 0 & \text{no pet} \\ 1 & \text{has a pet} \end{array} \right.
\end{align*}
\end{center}
}
\vspace{3mm}

\noindent Now, let's go through the process of applying AdaBoost to this dataset, step by step.

\begin{enumerate}
\item[(a)] Initialize the observation weights for the training data. (Make them uniform.)
\begin{center}
\begin{tabular}{p{0.45\textwidth}p{0.45\textwidth}}
$w_1^{(1)} = $ \hfill & $w_6^{(1)} = $ \hfill \\
$w_2^{(1)} = $ \hfill & $w_7^{(1)} = $ \hfill \\
$w_3^{(1)} = $ \hfill & $w_8^{(1)} = $ \hfill \\
$w_4^{(1)} = $ \hfill & $w_9^{(1)} = $ \hfill \\
$w_5^{(1)} = $ \hfill & $w_{10}^{(1)} = $ \hfill \\
\end{tabular}
\end{center}

\item[(b)] We will now grow a decision tree on this dataset, $G_1(x)$, using these weights. We'll use the \texttt{rpart} package in R, just as in Section~\ref{section:happiness}. Here is the first tree.

\begin{center}
\includegraphics[width=0.6\textwidth]{img/happiness-boosting-tree-1.png}
\end{center}

\noindent Here are the predictions for this tree, along with the current weights.

{\small
\begin{center}
\begin{tabular}{cccc}
\toprule
Datapoint ID & $w_i^{(1)}$ & happy $(Y)$ & $G_1(x)$ \\
\midrule
1 & 0.1 & -1 & -1 \\
2 & 0.1 & -1 & -1 \\
3 & 0.1 & -1 & -1 \\
4 & 0.1 & -1 & -1 \\
5 & 0.1 & -1 & -1 \\
6 & 0.1 & -1 & -1 \\
\midrule
7 & 0.1 & 1 & -1 \\
8 & 0.1 & 1 & -1 \\
9 & 0.1 & 1 & 1 \\
10 & 0.1 & 1 & 1 \\
\bottomrule
\end{tabular}
\end{center}
}

\noindent Compute the misclassification error of this tree, $\text{err}_1$. Compare this tree to the one we constructed by hand in Chapter~\ref{chapter:decisiontrees}.

$$ \text{err}_1 = \qquad \qquad \qquad $$
\vspace{3mm}

\item[(c)] Based on how $G_1(x)$ performs, calculate $\alpha_1$, its voting weight.
$$ \alpha_1 = \qquad \qquad \qquad $$

\item[(d)] Re-weight the observation weights for the training examples.
\begin{center}
\begin{tabular}{p{0.45\textwidth}p{0.45\textwidth}}
$w_1^{(2)} = $ \hfill & $w_6^{(2)} = $ \hfill \\
$w_2^{(2)} = $ \hfill & $w_7^{(2)} = $ \hfill \\
$w_3^{(2)} = $ \hfill & $w_8^{(2)} = $ \hfill \\
$w_4^{(2)} = $ \hfill & $w_9^{(2)} = $ \hfill \\
$w_5^{(2)} = $ \hfill & $w_{10}^{(2)} = $ \hfill \\
\end{tabular}
\end{center}

\item[(e)] Now we grow another decision tree, $G_2(x)$, using these new weights as inputs to the \texttt{rpart} package.

\begin{center}
\includegraphics[width=0.6\textwidth]{img/happiness-boosting-tree-2.png}
\end{center}

Here are the predictions for this tree, along with the current weights.

{\small
\begin{center}
\begin{tabular}{cccc}
\toprule
Datapoint ID & $w_i^{(2)}$ & happy $(Y)$ & $G_2(x)$ \\
\midrule
1 & 0.1 & -1 & -1 \\
2 & 0.1 & -1 & -1 \\
3 & 0.1 & -1 & -1 \\
4 & 0.1 & -1 & -1 \\
5 & 0.1 & -1 & -1 \\
6 & 0.1 & -1 & -1 \\
\midrule
7 & 0.4 & 1 & 1 \\
8 & 0.4 & 1 & 1 \\
9 & 0.1 & 1 & 1 \\
10 & 0.1 & 1 & -1 \\
\bottomrule
\end{tabular}
\end{center}
}

\noindent Compute the misclassification error of this tree, $\text{err}_2$.

$$ \text{err}_2 = \qquad \qquad \qquad $$
\vspace{3mm}

\item[(f)] Based on how $G_2(x)$ performs, calculate $\alpha_2$, its voting weight.
$$ \alpha_2 = \qquad \qquad \qquad $$

\item[(g)] Re-weight the observation weights for the training examples.
\begin{center}
\begin{tabular}{p{0.45\textwidth}p{0.45\textwidth}}
$w_1^{(3)} = $ \hfill & $w_6^{(3)} = $ \hfill \\
$w_2^{(3)} = $ \hfill & $w_7^{(3)} = $ \hfill \\
$w_3^{(3)} = $ \hfill & $w_8^{(3)} = $ \hfill \\
$w_4^{(3)} = $ \hfill & $w_9^{(3)} = $ \hfill \\
$w_5^{(3)} = $ \hfill & $w_{10}^{(3)} = $ \hfill \\
\end{tabular}
\end{center}

\item[(h)] Now we grow another decision tree, $G_3(x)$, using these new weights as inputs to the \texttt{rpart} package.

\begin{center}
\includegraphics[width=0.6\textwidth]{img/happiness-boosting-tree-3.png}
\end{center}

Here are the predictions for this tree, along with the current weights.

{\small
\begin{center}
\begin{tabular}{cccc}
\toprule
Datapoint ID & $w_i^{(3)}$ & happy $(Y)$ & $G_3(x)$ \\
\midrule
1 & 0.1 & -1 & -1 \\
2 & 0.1 & -1 & -1 \\
3 & 0.1 & -1 & -1 \\
4 & 0.1 & -1 & -1 \\
5 & 0.1 & -1 & 1 \\
6 & 0.1 & -1 & -1 \\
\midrule
7 & 0.4 & 1 & 1 \\
8 & 0.4 & 1 & 1 \\
9 & 0.1 & 1 & -1 \\
10 & 1.5 & 1 & 1 \\
\bottomrule
\end{tabular}
\end{center}
}

\item[(i)] Compute the misclassification error of this tree, $\text{err}_3$.

$$ \text{err}_3 = \qquad \qquad \qquad $$
\vspace{3mm}

\item[(j)] Based on how $G_3(x)$ performs, calculate $\alpha_3$, its voting weight.
$$ \alpha_3 = \qquad \qquad \qquad $$

\item[(k)] Re-weight the observation weights for the training examples.
\begin{center}
\begin{tabular}{p{0.45\textwidth}p{0.45\textwidth}}
$w_1^{(4)} = $ \hfill & $w_6^{(4)} = $ \hfill \\
$w_2^{(4)} = $ \hfill & $w_7^{(4)} = $ \hfill \\
$w_3^{(4)} = $ \hfill & $w_8^{(4)} = $ \hfill \\
$w_4^{(4)} = $ \hfill & $w_9^{(4)} = $ \hfill \\
$w_5^{(4)} = $ \hfill & $w_{10}^{(4)} = $ \hfill \\
\end{tabular}
\end{center}

\item[(l)] Now we grow another decision tree, $G_4(x)$, using these new weights as inputs to the \texttt{rpart} package.

\begin{center}
\includegraphics[width=0.6\textwidth]{img/happiness-boosting-tree-4.png}
\end{center}

Here are the predictions for this tree, along with the current weights.

{\small
\begin{center}
\begin{tabular}{cccc}
\toprule
Datapoint ID & $w_i^{(4)}$ & happy $(Y)$ & $G_4(x)$ \\
\midrule
1 & 0.1 & -1 & -1 \\
2 & 0.1 & -1 & 1 \\
3 & 0.1 & -1 & 1 \\
4 & 0.1 & -1 & -1 \\
5 & 1.4 & -1 & -1 \\
6 & 0.1 & -1 & -1 \\
\midrule
7 & 0.4 & 1 & 1 \\
8 & 0.4 & 1 & 1 \\
9 & 1.4 & 1 & 1 \\
10 & 1.5 & 1 & 1 \\
\bottomrule
\end{tabular}
\end{center}
}

\item[(m)] Compute the misclassification error of this tree, $\text{err}_4$.

$$ \text{err}_4 = \qquad \qquad \qquad $$
\vspace{3mm}

\item[(n)] Based on how $G_4(x)$ performs, calculate $\alpha_4$, its voting weight.
$$ \alpha_4 = \qquad \qquad \qquad $$

\item[(o)] Output the weighted average of the four classifiers' votes for each training example:
$$ G(x) = \text{sign}\left[ \alpha_1 G_1(x) + \alpha_2 G_2(x) + \alpha_3 G_3(x) + \alpha_4 G_4(x) \right] $$
What is the final training error?
\vspace{3mm}
{\small
\begin{center}
\begin{tabular}{cccccc|c}
\toprule
Datapoint ID & happy $(Y)$ & $G_1(x)$ & $G_2(x)$ & $G_3(x)$ & $G_4(x)$ & $G(x)$ \\
\midrule
1 & -1 & -1 & -1 & -1 & -1 & \\
2 & -1 & -1 & -1 & -1 & 1 & \\
3 & -1 & -1 & -1 & -1 & 1 & \\
4 & -1 & -1 & -1 & -1 & -1 & \\
5 & -1 & -1 & -1 & 1 & -1 & \\
6 & -1 & -1 & -1 & -1 & -1 & \\
\midrule
7 & 1 & -1 & 1 & 1 & 1 & \\
8 & 1 & -1 & 1 & 1 & 1 & \\
9 & 1 & 1 & 1 & -1 & 1 & \\
10 & 1 & 1 & -1 & 1 & 1 & \\
\bottomrule
\end{tabular}
\end{center}
}
\end{enumerate}

\vspace{3mm}

\begin{question}{}
The most difficult thing to understand in all of this is how the updated weights play into the construction of subsequent trees. A clue comes from the dataset percentages shown in the nodes of each tree. For example, trees $1$ and $4$ actually have the same structure, but the impurities at each node are different due to the different weights. Discuss how the weights could inform the variables the splitting algorithm chooses to split on (revisit Chapter~\ref{chapter:decisiontrees}, if necessary, to see the math).
\end{question}

\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Revisiting Breast Cancer Classification}

Let's once again revisit the classification example for which we built a single decision tree in Chapter~\ref{chapter:decisiontrees} and a random forest in Chapter~\ref{chapter:randomforests}. The Wisconsin Breast Cancer Dataset contains information about $30$ different imaging features of fine needle aspirate (FNA) samples from breast masses in $569$ study participants.

Here is a plot showing the evolution of the training and test error as a function of the number of trees:
\begin{center}
\includegraphics[width=0.7\textwidth]{img/wisconsin-boosting-error.png}
\end{center}
where $450$ samples are used for training and the remaining $119$ for testing. Here are tree numbers 1, 3, 6, and 8:
\begin{center}
\includegraphics[width=0.45\textwidth]{img/wisconsin-boosting-tree-1.png}
\includegraphics[width=0.45\textwidth]{img/wisconsin-boosting-tree-3.png}
\includegraphics[width=0.45\textwidth]{img/wisconsin-boosting-tree-6.png}
\includegraphics[width=0.45\textwidth]{img/wisconsin-boosting-tree-8.png}
\end{center}

\vspace{3mm}

\begin{question}{}
Compare and contrast boosting and random forests on the basis of:
\begin{itemize}
\item Whether each tree uses all or part of the dataset
\item Whether they consider a subset of the predictors at each split or all the predictors
\item Whether you can parallelize the construction of different trees (i.e., build them at the same time on different processors)
\item Whether the votes of different classifiers are independent
\item Ease of use and interpretability
\end{itemize}
\end{question}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\begmath 15.4 Cumulative Distribution Function for Poisson
\hbox{Probability Distribution}

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

The procedure described in this chapter computes the Cumulative Distribution
Function (CDF) of the Poisson probability distribution. The CDF is sometimes
called the lower tail. The lower tail,
or CDF, $Q(n|\lambda )$, and the upper tail, $P(n|\lambda )$ for the Poisson
probability distribution with parameter $\lambda $ and argument $n$ are
defined by%
\begin{gather*}
\Pr \{x<n|\lambda \}=Q(n|\lambda )=e^{-\lambda }\sum_{j=0}^{
n-1}\frac{\lambda ^j}{j!},\phantom{=1-Q(n|\lambda )}\\
\Pr \{x\geq n|\lambda \}=P(n|\lambda )=e^{-\lambda }
\sum_{j=n}^\infty \frac{\lambda ^j}{j!}=1-Q(n|\lambda ).
\end{gather*}
\subsection{Usage}

\subsubsection{Program Prototype, Single Precision}

\begin{description}
\item[REAL]  \ {\bf LAMDA, P, Q}

\item[INTEGER]  \ {\bf N, IERR}
\end{description}

Assign values $n$ to N and $\lambda $ to LAMDA, and obtain P $= P(n|\lambda
) $ and Q $= Q(n|\lambda )$ by using
$$
\fbox{{\bf CALL SCDPOI (N, LAMDA, P, Q, IERR)}}
$$

\subsubsection{Argument Definitions}

\begin{description}
\item[N]  \ [in] Argument $n$ of the functions $P(n|\lambda )$ and $%
Q(n|\lambda ).$ N must be nonnegative.  Must be positive if LAMBDA =
0.

\item[LAMDA]  \ [in] Parameter $\lambda $ of the functions $P(n|\lambda )$
and $Q(n|\lambda ).$ LAMBDA must be nonnegative.  Must be positive if
N = 0.

\item[P]  \ [out] The value of the function $P(n|\lambda ).$

\item[Q]  \ [out] The value of the function $Q(n|\lambda ).$

\item[IERR]  \ [out] A flag that normally is zero to indicate successful
computation. See Section E below for discussion of non-zero values.
\end{description}

\subsubsection{Modifications for Double Precision}

For double precision computation, change the REAL type statement to DOUBLE
PRECISION and change the initial letter of the procedure name to D.

\subsection{Example and Remarks}

See DRDCDPOI and ODDCDPOI for an example of the usage of this subprogram.

The procedures SGAMIK and SGAMIE, described in Chapter~2.19, are used to
control the procedure SGAMI and determine the error estimate it returns.
SGAMI is used as described in Section D below.

\subsection{Functional Description}

\subsubsection{Method}

The identities $P(n|\lambda )=P(a,x)$ and $Q(n|\lambda )=Q(a,x)$, with $a=n$
and $x=\lambda $, where $P(a,x)$ and $Q(a,x)$ are incomplete gamma function
ratios, are used. The procedure SGAMI described in Chapter~2.19 is used to
evaluate $P(a,x)$ and $Q(a,x).$

\subsubsection{Accuracy Tests}

See Section~2.19.D.

\subsection{Error Procedures and Restrictions}

The procedure SGAMI issues error messages, under several conditions, at
level 2 + MSGOFF, where MSGOFF is zero unless specified by a call to SGAMIK
(see Chapter~2.19) at some time before calling SCDPOI. If error termination
is suppressed by setting MSGOFF $< 0$, or by calling ERMSET (see Chapter
19.2), IERR will be set to a non-zero value.

If the desired tolerance could not be achieved, IERR is set to~2.

If SCDPOI is called with both LAMDA and N zero, IERR is set to~3 and P is
set to~3.0.

If SCDPOI is called with one or both of LAMDA and N negative, IERR is set to
4 and P is set to~4.0.

\subsection{Supporting Information}

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.35in} {\bf Required Files}\vspace{2pt} \\
DCDPOI & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DCDPOI, DCSEVL, DERF, DERM1, DERV1, DGAM1, DGAMMA, DINITS, DRCOMP,
DREXP, DRLOG, DXPARG, ERFIN, ERMSG, IERM1, IERV1\rule[-5pt]{0pt}{8pt}}\\
SCDPOI & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMSG, IERM1, IERV1, SCDPOI, SCSEVL, SERF, SERM1, SERV1, SGAM1,
SGAMMA, SINITS, SRCOMP, SREXP, SRLOG, SXPARG}\\\end{tabular}

Designed and programmed by W. V. Snyder, JPL, 1993.

\begcodenp
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRDCDPOI}\vspace{10pt}
\lstinputlisting{\codeloc{dcdpoi}}

\vspace{30pt}\centerline{\bf \large ODDCDPOI}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{dcdpoi}}
\end{document}