\documentclass[main.tex]{subfiles}
\begin{document}

\subsection{The Klein paradox}

\marginpar{Thursday\\ 2020-3-19, \\ compiled \\ \today}

The fact that it was not possible to derive a conserved charge for the KG equation is not just a mathematical inconvenience: we will use a gedankenexperiment to show the physical consequences of this.

Consider the scattering of a particle, which is described by a pure plane wave, on a an electromagnetic potential step.
Suppose we are in a frame in which \(qA^{\mu } = (V(x), \vec{0})\); if we are not in such a frame we can always perform a boost so that we are.
Also suppose that the potential looks like a step: \(V(x) = V_0 [x \geq 0]\). Here ``\(x\)'' refers to a 1d spatial coordinate.

Now, suppose that we have an incoming wave with energy \(\omega \) and momentum \(k_{x}\) in the \(x<0\) region.
Upon impact, there will in general be a reflected wave in the \(x<0\) region and a transmitted wave in the \(x>0\) region. We will call the former \(\varphi_{1}\) and the latter \(\varphi_{2}\).

We can decompose both of these into a time and space exponential: \(\varphi_{i} (t, x) = e^{-i \omega t} \chi_{i}(x)\), for \(i = 1, 2\).

Now, the incoming wave contributes to the global wavefunction with an exponential \(e^{ik_x x}\) in the \(x<0\) region.
In the same region we can find the reflected wave, which has the opposite momentum; also, its amplitude is less than that of the incoming wave. We write its contribution to \(\chi_{1}\) as \(r e^{-i k_x x}\), where \(r\) is the reflection coefficient.

By a similar reasoning, in the \(x>0\) region we will have a contribution \(t e^{i k_x^{\prime } x}\): the direction of propagation is the same as the incoming wave, the momentum might be different. In the end, our wavefunction looks like
%
\begin{subequations}
\begin{align}
\varphi (t, x) &= \varphi_1 [x\leq 0] + \varphi_2 [x \geq 0] \\
&= e^{-i \omega t} \qty[\chi_1 [x\leq 0] + \chi_2 [x \geq 0]] \\
&=
 e^{-i \omega t} \qty[ \qty(e^{i k_x x} + r e^{-i k_x x})[x\leq 0] + t e^{i k_x^{\prime }x } [x\geq 0]]
\,.
\end{align}
\end{subequations}

This wavefunction will need to satisfy the KG equation everywhere, however the equation has a different form in the two regions: if \(\DD^{\mu } = \partial^{\mu } + iqA^{\mu }\), then we can say that the two equations are
%
\begin{subequations}
\begin{align}
  \qty[\partial^{\mu } \partial_{\mu } + M^2] \varphi_1 &= 0 \\
  \qty[\DD^{\mu } \DD_{\mu } + M^2] \varphi_2 &= 0
\,,
\end{align}
\end{subequations}
%
so we can insert the solutions into the equations: for the region without potential we get
%
\begin{subequations}
\begin{align}
\qty[\partial^{\mu } \partial_{\mu } + M^2] \varphi_1 &= \qty[+ \partial_0^2 - \partial_{x}^2 + M^2 ] \qty[e^{-i \omega t} \qty(e^{ik_{x}x} + r e^{-ik_x x})]   \\
&= (-i\omega)^2 \varphi_1 + M^2 \varphi_1 - e^{-i \omega t}\partial_{x}^2  \qty(e^{ik_{x}x} + r e^{-ik_x x}) \\
&= -\omega^2 \varphi_1 + M^2 \varphi_1 - e^{-i \omega t}  \qty((ik_x)^2 e^{ik_{x}x} + r (-ik_x)^2 e^{-ik_x x}) \\
&= \qty[-\omega^2 + M^2 + k_x^2 ] \varphi_1
\,,
\end{align}
\end{subequations}
%
notice that even though the function has components with momentum in either direction it still is an eigenfunction of the operator \(\square\)!
For the other region we can use the fact we discussed earlier: with the minimal substitution, the momentum becomes \(p^{\mu } = i \partial^{\mu } \rightarrow i \DD^{\mu } = p^{\mu } - qA^{\mu }\), so in our case we will have \(p^{0} \rightarrow \omega_{k} - q A^{0} = \omega_{k} - V_0 \), while \(p^{i} \rightarrow \vec{k}'\), since \(\vec{A} = 0\). We can substitute the energy directly since the potential is constant, so there is no \(\partial_0 V_0 \) term.
Then, the rest of the calculation is perfectly analogous, with \(k_{x}'\) instead of \(k_x\).

So, we have the two relations
%
\begin{align} \label{eq:dispersion-relation-general}
- \omega_k^2 + k_x^2 + M^2 = 0
\qquad \text{and} \qquad
- \qty(\omega_{k} - V_0 )^2 + k_x^{\prime 2}  + M^2 = 0
\,.
\end{align}

We choose the solutions in which the wave is propagating towards increasing \(x\): so we get
%
\begin{align}
k_x = \sqrt{\omega_{k}^2  - M^2}
\qquad \text{and} \qquad
k_x^{\prime} = \sqrt{\qty(\omega_{k} - V_0)^2 - M^2}
\,.
\end{align}

Now we must make these consistent with each other, by imposing that at the border the function be \(\mathcal{C}^{1}\).

Why do we fix only the first derivative? Probably something to do with the fact that the KG eq is second order, so we would need \(\square = \DD^{\mu } \DD_{\mu }\) at the boundary, which cannot be the case.

We can work with the spatial components \(\chi_{i}\), since the time evolution factors. So, for continuity we get that \(\chi_1 (0) = \chi_2 (0)\) implies:
%
\begin{align}
1+r = t
\,,
\end{align}
%
while for differentiability we get that \(\partial_{x} \chi_1(0) = \partial_{x} \chi_2 (0)\) implies:
%
\begin{align}
i k_x  - r i k_x = i t k_x'
\implies
\frac{k_x'}{k_x} = \frac{1- r}{t}
\,.
\end{align}

Now, what do we know of these variables? Well, from the dispersion relations if we fix the mass \(M\), the energy \(\omega \) and the potential \(V_0 \) we have \(k_x\) and  \(k_x^{\prime }\).
Then, from the two equations we have found we can calculate \(r\) and \(t\); their expressions are exactly those found in the nonrelativistic case. Since \(1-r = 2-t\) we have
%
\begin{align}
\frac{k_{x}'}{k_x} = \frac{2}{t} - 1 \implies
\frac{k_x^{\prime } + k_x}{k_x} = \frac{2}{t}
\implies t = \frac{2 k_x}{k_x' + k_x}
\qquad \text{and} \qquad
r = t-1 = \frac{k_x - k_x'}{k_x' + k_x}
\,.
\end{align}

Now, we shall discuss the probability currents for the two regions: the formula is always
%
\begin{align}
j^{\mu } = \frac{i}{2} \varphi^{*} \partial^{\mu } \varphi  + \frac{q}{2} A^{\mu } \varphi^{*} \varphi  + \text{c. c.}
\,,
\end{align}
%6
but in the first region the potential is identically zero.

One non-obvious thing to consider is that, in general, we could have \(k_x'\) be either real or imaginary, since while \( \omega^2 - M^2\) must be \(>0\) there is no such constraint on \((\omega- V_0 )^2 - M^2\).
If we conjugate an exponential as \(\qty(e^{iz})^{*}\), the result is \(e^{-i z^{*}}\) for a general complex \(z\).
The calculation is as follows:
%
\begin{subequations}
\begin{align}
\rho_1 \overset{\text{def}}{=}
j^{0}_{(1)} &= \frac{i}{2} e^{i \omega t} \partial^{0} e^{-i \omega t} \abs{\chi_1 }^2  + \text{c. c.} = \frac{2}{2} \omega \abs{\chi_1 }^2
\marginnote{The \(\chi_1 \) is not time-dependent, we get two terms by adding the conjugate. We get a minus sign from lowering the index}  \\
j_1 \overset{\text{def}}{=} j^{x}_{(1)} &= - \frac{i}{2} \qty(e^{ik_x x} + r e^{-ik_x x}) \partial_{x} \qty(e^{ik_x x} + r e^{-ik_x x}) + \text{c. c. } \\
&= - \frac{2}{2} i (i k_x ) \qty(e^{ik_x x} + r e^{-ik_x x}) \qty(e^{ik_x x} - r e^{-ik_x x}) = k_x \qty(1 - \abs{r}^2)  \\
\rho_2 \overset{\text{def}}{=}  j^{0}_{(2)} &= \frac{i}{2} t^{*} e^{i \omega t} e^{-i k_x^{\prime *}x} t (-i \omega ) e^{-i \omega t} e^{i k_x^{\prime } x} + \frac{1}{2} V_0 \varphi^{*} \varphi  + \text{c. c.}  \\
&= (\omega - V_0 ) \abs{t}^2 e^{i \qty(k_x^{\prime }- k_x^{\prime *})x}  \\
j_2 \overset{\text{def}}{=} j^{x}_{(2)} &= -\frac{i}{2} t^{*} e^{i \omega t} e^{-i k_x^{\prime *} x}
(i k_x^{\prime }) e^{-i \omega t} e^{i k_x^{\prime }x } + \text{c. c. }  \\
&= \frac{1}{2} \abs{t}^2 e^{i \qty(k_x^{\prime } - k_x^{\prime *})x} k_x^{\prime } + \text{c. c. }  \\
&= \frac{k_x^{\prime } + k_x^{\prime *}}{2} \abs{t}^2 e^{i \qty(k_x^{\prime } - k_x^{\prime *})x}
\,.
\end{align}
\end{subequations}

We can define the reflection and transmission coefficients, which are probabilities:
%
\begin{align}
\mathcal{R} = \frac{j _{\text{in}} - j_1}{j _{\text{in}}} = \abs{r}^2
\qquad \text{and} \qquad
\mathcal{T} = \frac{j_2}{j _{\text{in}}}
= \frac{k_x^{\prime } + k_x^{\prime *}}{2 k_x} \abs{t}^2 e^{i (k_x^{\prime } -k_x^{\prime *})x}
\,,
\end{align}
%
where \(j _{\text{in}} = k_x\) is the current density corresponding to the incoming wave alone: it is calculated as the current density we would measure in region 1 if we did not have the reflected wave component:
%
\begin{subequations}
\begin{align}
j_{\text{in}} \overset{\text{def}}{=} \eval{j^{x}_{(1)}}_{\chi_1 = e^{ik_x x}} &= - \frac{i}{2} \qty(e^{ik_x x}) \partial_{x} \qty(e^{ik_x x} ) + \text{c. c. } \\
&= - \frac{2}{2} i (i k_x ) e^{ik_x x} e^{ik_x x} = k_x
\,.
\end{align}
\end{subequations}

% \todo[inline]{What is the motivation behind these definitions? Can they be shown formally to represent probabilities, or is that interpretation something we add ``after the fact''?}

Now, we have three possible cases for the behavior of \(k_x^{\prime }\): it is
\begin{enumerate}
  \item real if \(\omega_{k} - M > V_0 \): this is the nonrelativistic case;
  \item imaginary if \(\abs{\omega_{k} - V_0 } < M\): this is the intermediate case;
  \item real again if \(\omega_{k} + M < V_0 \): this is the fully relativistic case.
\end{enumerate}

\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{figures/primed_momentum_potential.pdf}
\caption{An illustration of how \(k_x^{\prime }\) changes as \(V_0 \) increases. We have fixed the mass \(M\) and the the energy \(\omega_{k}\) to \(\num{.5}\) and \(\num{.7}\) respectively. The regions are separated by the critical points of \(\abs{k_x'}\), at \(V_0 = \num{.2}\) and \(V_0 = \num{1.2}\) respectively. Do note that this is a plot of the \emph{absolute value} of \(k_x^{\prime }\): in the intermediate region it is purely imaginary.}
\label{fig:primed_momentum_potential}
\end{figure}

This is shown in figure \ref{fig:primed_momentum_potential}.
The NR approximation can only be applied in the first case, and in the second case only as long as \(V_0 < \omega_{k}\): the case \(\omega_{k} - M < V_0 \ll \omega_{k}\) is nonrelativistic quantum tunneling.

\subsubsection{Nonrelativistic case}

Now, \(k_x\) and \(k_x'\) are both real, so \(k_x^{\prime } = k_x^{\prime *}\), which means that the probability currents read:
%
\begin{subequations}
\begin{align}
\rho_1 &= \omega \abs{\chi_1 }^2
&
j_1 &= k_x \qty(1 - \abs{r}^2) \\
\rho_2 &= (\omega - V_0 ) \abs{t}^2
&
j_2 &=  k_x^{\prime } \abs{t}^2
\,,
\end{align}
\end{subequations}
%
and, since \(r\) and \(r\) are both (real and) \(<1\) these quantities are all positive. Also, we must have \(k_x' < k_x\), so the quantities
%
\begin{align}
\mathcal{R} &= \abs{r}^2
\qquad \text{and} \qquad
\mathcal{T} = \frac{k_x'}{k_x} \abs{t}^2
\,
\end{align}
%
are both \(<1\). So, it will be consistent if we impose (?) a probabilistic Born-like interpretation in which these coefficients are reflection and transmission probabilities, with \(\mathcal{R} + \mathcal{T} = 1\).

\subsubsection{Intermediate case}

Now, we have \(\abs{\omega_{k} - V_0 } < M\), so, while \(k_x>0\),
%
\begin{align}
k_x' = \pm i \sqrt{M^2 - \qty(\omega_{k} - V_0 )^2}
\,.
\end{align}

In principle we could have both \(+\) or \(-\), but we select the positive imaginary because it is the physical in which the wave travels rightward.

Now, in our computation of the probability density \(\rho_2 \) we will have a factor
%
\begin{align}
\exp(i (k_x' - k_x^{\prime *})x) = \exp(\pm 2 i^2 \abs{k_x'}x) =  \exp(\mp 2 \abs{k_x^{\prime }} x)
\,,
\end{align}
%
while for the probability current \(j_2 \) we will get a factor \(k_x^{\prime } + k_x^{\prime *} = 0 \) multiplying everything.

So, in the end we get
%
\begin{subequations}
\begin{align}
\rho_1 &= \omega \abs{\chi_1 }^2
&
j_1 &= k_x \qty(1 - \abs{r}^2) \\
\rho_2 &= (\omega - V_0 ) \abs{t}^2 e^{-2 \abs{k_x^{\prime }} x}
&
j_2 &= 0
\,.
\end{align}
\end{subequations}

Notice that in the intermediate region we can have both \(V_0 < \omega_{k}\) and \(V_0 > \omega_{k}\).

In this case, we get for the reflection and transmission probabilities
%
\begin{align}
\mathcal{R} = \abs{r}^2 = \abs{\frac{k_x - i k_x^{\prime }}{k_x + i k_x^{\prime }}} = \abs{\frac{z}{z^{*}}}^2 = 1
\marginnote{The ratio of \(z\) and \(z^{*}\) is only a phase}
\,,
\end{align}
%
while \(\mathcal{T} \propto j_2 = 0\).

So, in this case the particle is certainly reflected; this is compatible with the probabilistic interpretation \(\mathcal{R} + \mathcal{T} = 1\).
However, \(\rho_2  \) is not positive defined so it cannot be interpreted as a probability density in general.

If \(V_0 < \omega_{k} \) we still get the vanilla-QM result of \(\rho_2 \sim e^{-2 \abs{k_x^{\prime }}x}\): the particle penetrates the classically forbidden region.
This effect is suppressed as \(V_0 \) increases.

\subsubsection{Fully relativistic case}

This is the situation which is completely outside the realm of classical QM description, in which we expect to see relativistic effects.
We have \(V_0 > \omega_{k} + M \).

Now
%
\begin{align}
k_x^{\prime } = \pm \sqrt{(\omega_{k} - V_0 )^2 - M^2} \in \mathbb{R}
\,,
\end{align}
%
so the densities are
%
\begin{subequations}
\begin{align}
\rho_1 &= \omega \abs{\chi_1 }^2
&
j_1 &= k_x \qty(1 - \abs{r}^2) \\
\rho_2 &= (\omega - V_0 ) \abs{t}^2 < 0
&
j_2 &= k_{x}^{\prime } \abs{t}^2
\,,
\end{align}
\end{subequations}
%
where the sign of \(j_2 \) is determined by that of \(k_x^{\prime }\), which is not necessarily positive.
Actually, it must be negative: to see this, we can calculate the group velocity, which would be the global velocity of the wavepacket we are approximating. We can calculate \(\omega_{k}\) from the dispersion relation \eqref{eq:dispersion-relation-general} to get
%
\begin{align}
v_G = \pdv{\omega_{k} }{k_x^{\prime }}
= \pdv{k_x^{\prime }} \qty(V_0 + \sqrt{k_x^{\prime 2} + M^2})
= \frac{k_x^{\prime }}{\sqrt{k_x^{\prime 2} + M^2}}
= \frac{k_x^{\prime }}{\omega_{k} - V_0 }
\,,
\end{align}
%
where we used the dispersion relation again in the last step.
Now, for the packet to propagate forward we need \(v_G > 0\), but since the denominator is negative this means \(k_x^{\prime } < 0\).

Then, we will have
%
\begin{align}
r = \frac{k_x - k_x^{\prime }}{k_x + k_x^{\prime }}
\implies
\mathcal{R} = \abs{\frac{k_x - k_x^{\prime }}{k_x + k_x^{\prime }}}^2 > 1
\,,
\end{align}
%
while the other coefficient is
%
\begin{align}
\mathcal{T} = \frac{k_x^{\prime }}{k_x} \abs{t}^2
= \frac{k_x^{\prime }}{k_x} \abs{\frac{2 k_x}{k_x^{\prime } + k_x}}^2
= k_x^{\prime } \frac{4 k_x}{(k_x^{\prime } + k_x)^2}
< 0
\,.
\end{align}
%

So, we can still maintain the condition \(\mathcal{R} + \mathcal{T} =1\), but reflection is ``more than certain''? Surely this is not the correct physical interpretation.

This is the heart of the Klein-Gordon paradox.

What is actually happening is that in our description we are fixing the number of particles to be one; a proper (``grand-canonical'' instead of ``canonical'', in statistical mechanics terms) description of the situation would allow us to see that the energy in the system is enough to create real particle-antiparticle pairs.

The antiparticles gain energy under the potential (since they perceive it as \(-V_0 \), having a negative charge), so they are transmitted, while the particles are reflected. On average, then, more than one particle is reflected.

In order to properly study these extreme conditions, we need the new paradigm of \emph{Quantum Field Theory}.

\end{document}


\chapter{Lecture 17 Jun 08th 2018}%
\label{chp:lecture_17_jun_08th_2018}
% chapter lecture_17_jun_08th_2018

\section{Group Action (Continued 2)}%
\label{sec:group_action_continued_2}
% section group_action_continued_2

\subsection{Group Action (Continued 2)}%
\label{sub:group_action_continued_2}
% subsection group_action_continued_2

\begin{note}[Recall \cref{thm:orbit_decomposition_theorem}]
  Let $G$ act on a finite set $X \neq \emptyset$. Let\sidenote{$X_f$ is also called the set of elements of $X$ that are fixed by the action of $G$.}
  \begin{equation*}
    X_f = \{x \in X : a \cdot x = x, \, a \in G \}
  \end{equation*}
  Let $G \cdot x_1, G \cdot x_2, ..., G \cdot x_n$ be distinct nonsingleton orbits (ie. $\abs{G \cdot x_i} > 1$). Then
  \begin{equation*}
    \abs{X} = \abs{X_f} + \sum_{i=1}^{n} [ G : S(x_i) ].
  \end{equation*}
\end{note}

\begin{eg}[Conjugacy Class \& Centralizer]
  Let $G$ be a finite group acting on itself by \hlnoteb{conjugation}. In the context of \cref{thm:orbit_decomposition_theorem}, we have that
  \begin{align*}
    X &= G \\
    G_f &= \{x \in G : gxg^{-1} = x, \, g \in G \} \\
        &= \{x \in G : gx = xg, \, g \in G \} = Z(G),
  \end{align*}
  where we recall that $Z(G)$ is the \hyperref[defn:center_of_a_group]{center of $G$}. Now for any $x \in G$, we have
  \begin{equation*}
    G \cdot x = \{ gxg^{-1} : g \in G \},
  \end{equation*}
  which is known as the \hldefn{conjugacy class} of $x$. We also have
  \begin{equation*}
    S(x) = \{g \in G : gxg^{-1} = x \} = \{g \in G : gx = xg \} = C_G(x),
  \end{equation*}
  which is called the \hldefn{centralizer} of $x$.
\end{eg}

Putting the above example with \cref{thm:orbit_decomposition_theorem}, we have the following corollary.

\begin{crly}[Class Equation]
\index{Class Equation}
\label{crly:class_equation}
  Let $G$ be a finite group and $\{gx_1 g^{-1} : g \in G \}, \, ..., \, \{g x_n g^{-1} : g \in G \}$ denote the distinct nonsingleton conjugacy classes. Then
  \begin{equation*}
    \abs{G} = \abs{Z(G)} + \sum_{i=1}^{n} [ G : C_G(x_i) ].
  \end{equation*}
\end{crly}

\begin{lemma}
\label{lemma:abs_x_equiv_abs_x_f}
  Let $G$ be a group of order $p^m$, where $p$ prime and $m \in \mathbb{N}$, which acts on a finite set $X$. Let
  \begin{equation*}
    X_f = \{ x \in X : a \cdot x = x, \, a \in G \}.
  \end{equation*}
  Then we have
  \begin{equation*}
    \abs{X} \equiv \abs{X_f} \mod p
  \end{equation*}
\end{lemma}

\begin{proof}
  By the \hyperref[thm:orbit_decomposition_theorem]{Orbit Decomposition Theorem}, we have that
  \begin{equation*}
    \abs{X} = \abs{X_f} + \sum_{i=1}^{n} [ G : S(x_i) ],
  \end{equation*}
  where $[ G : S(x_i) ] > 1$ for $1 \leq i \leq n$. For any $x_i$, by \hyperref[thm:lagrange_s_theorem]{Lagrange's Theorem}, $[ G : S(x_i) ] \big| \abs{G} = p^m$. Since $[ G : S(x_i) ] > 1$, we have, by the \hlnotea{Fundamental Theorem of Arithmetic}, that $[ G : S(x_i) ]$ must be a multiple of $p$, i.e. $p$ divides $[ G : S(x_i) ]$, for all $i$. Therefore, $p \, | \, \left( \abs{X} - \abs{X_f} \right)$, i.e.
  \begin{equation*}
    \abs{X} \equiv \abs{X_f} \mod p,
  \end{equation*}
  as required. \qed
\end{proof}

\newthought{Recall} \hyperref[thm:lagrange_s_theorem]{Lagrange's Theorem}: If $G$ is finite and $g \in G$, then
\begin{equation*}
  o(g) \, \big| \, \abs{G}.
\end{equation*}

An interesting question to ask here is: Is the converse true? I.e., given a group $G$ with an integer $m$ such that $m \, \big| \, \abs{G}$, does $G$ contain an element of order $m$?

Consider $K_4$, the Klein $4$-group. Note that all elements of $K_4$ have order at most $2$, but $4 | \abs{K_4} = 4$.

Now if $m$ is some prime, is the converse still true?

\begin{thm}[Cauchy's Theorem]
\index{Cauchy's Theorem}
\label{thm:cauchy}
  Let $p$ be a prime, $G$ be a finite group. If $p \, \big| \abs{G}$, then $G$ contains an element of order $p$.
\end{thm}

\begin{proof}[McKay]
  Let $\abs{G} = n$. Suppose $p \, | \, n$. Let
  \begin{equation*}
    X = \{(a_1, ..., a_p) : a_i \in G, \, a_1 \hdots a_p = 1 \}.
  \end{equation*}
  Note that $X \neq \emptyset$, since $(1, ..., 1) \in X$ (so the proof is not vacuous). Take any $a_1, ..., a_{p - 1} \in G$, then $a_p$ is uniquely determined, i.e.
  \begin{equation*}
    a_p = (a_1 \hdots a_{p - 1})^{-1}.
  \end{equation*}
  Now for each $a_i$, we have $n$ choices, thus $\abs{X} = n^{p - 1}$.\sidenote{Convince yourself why this is true.}

  Let $\mathbb{Z}_p = ( \mathbb{Z}_p, + )$ act on $X$ by ``cycling'', i.e. $\forall k \in \mathbb{Z}_p$,
  \begin{equation*}
    k \cdot (a_1, a_2, ..., a_p) = (a_{k + 1}, a_{k + 2}, ..., a_p, a_1, ..., a_k).
  \end{equation*}
  \sidenote{We want to use \cref{thm:orbit_decomposition_theorem} from here.} Note that \\
  \begin{align*}
    (a_1, ..., a_p) \in X_f &\iff \text{ every cycled shift of } (a_1, ..., a_p) \text{ is itself } \\
      &\iff a_1 = a_2 = \hdots = a_p \text{ and } a_1 a_2 ... a_p = 1
  \end{align*}
  i.e. all of the components of the $p$-tuple are the same. Now if $(a_1, ..., a_p)$ has at least 2 distinct components, then its orbits must have $p$ elements. In other words, for some $r \in \mathbb{N}$, for each $1 \leq i \leq r$, we have that $[ G : S(x_i) ] = p$. Then, by the \hyperref[thm:orbit_decomposition_theorem]{Orbit Decomposition Theorem},
  \begin{gather*}
    n^{p - 1} = \abs{X} = \abs{X_f} + \sum_{i=1}^{r} [ G : S(x_i) ] \\
    \abs{X_f} = n^{p - 1} - rp.
  \end{gather*}
  We observe that $\abs{X_f}$ is indeed divisible by $p$ and is non-zero, since $(1, ..., 1) \in X_f$. Therefore, there exists some $a \neq 1 \in G$, such that $(a, ..., a) \in X_f$, i.e. $a^p = 1$. We know that $p$ is the smallest power by construction, and therefore $o(a) = p$ as required. \qed
\end{proof}

% subsection group_action_continued_2 (end)

% section group_action_continued_2 (end)

% chapter lecture_17_jun_08th_2018 (end)


% Created 2021-08-19 Thu 17:47
% Intended LaTeX compiler: pdflatex
\documentclass[presentation,aspectratio=169]{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{khpreamble}
\usepackage{amssymb}
\usepgfplotslibrary{groupplots}
\newcommand*{\shift}{\operatorname{q}}
\usetheme{default}
\author{Kjartan Halvorsen}
\date{\today}
\title{Process Automation Laboratory - Modeling first-order systems}
\hypersetup{
 pdfauthor={Kjartan Halvorsen},
 pdftitle={Process Automation Laboratory - Modeling first-order systems},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.3 (Org mode 9.4.6)},
 pdflang={English}}
\begin{document}

\maketitle

\section{Fitting first-order model}
\label{sec:orgca8c9f6}
\begin{frame}[label={sec:org6022261}]{First-order system example: A tank}
\begin{center}
\includegraphics[width=0.7\linewidth]{../../figures/tank-with-hole-no-variables}
\end{center}

\begin{enumerate}
\item What is the \alert{state} of the system?
\item What is the \alert{input signal} and \alert{output signal}?
\end{enumerate}
\end{frame}

\begin{frame}[label={sec:orgde4504a}]{First-order system example: A tank}
\begin{center}
\includegraphics[width=0.7\linewidth]{../../figures/tank-with-hole-simple}
\end{center}

\begin{align*}
\frac{d}{dt} (Ah) &=  z(t) - x(t) = z(t) - a \sqrt{2gh}\quad \Rightarrow\\
\frac{d}{dt} h(t) &= - \frac{a\sqrt{2g}}{A} \sqrt{h(t)} + \frac{1}{A} z(t)
\end{align*}
\end{frame}

\begin{frame}[label={sec:org42acba5}]{Intuition}
\includegraphics[width=0.2\linewidth]{../../figures/tank-with-hole-no-variables}

\alert{Individual activity} A constant inflow has been present since forever, but at time \(t_1\) the flow in is suddenly shut off. Which of the responses of the water level \(h(t)\) below is correct?

\begin{tikzpicture}
\small

\begin{axis}[
width=7cm,
height=2.5cm,
xlabel={$t$},
ylabel={$h(t)$},
xmin=-3.5,
xmax=10.5,
ytick = {0},
xtick = {0},
xticklabels = {$t_1$},
]
\addplot+[black, no marks, domain=-4:10, samples=400,variable=k] { (k < 0) + (k>0)*(1+exp(-4))/(1+exp(4*(0.5*k-1)))};

\node[black!40!red] at (axis cs: 5, 0.5) {\huge 1};
\end{axis}

\begin{axis}[
xshift=7cm,
width=7cm,
height=2.5cm,
xlabel={$t$},
ylabel={$h(t)$},
xmin=-3.5,
xmax=10.5,
ytick = {0},
xtick = {0},
xticklabels = {$t_1$},
]
\addplot+[black, no marks, domain=-4:10, samples=400,variable=k] { (k<0) + ((k>=0) - (k>4))*(1/4*(4-k)) };
\node[black!40!red] at (axis cs: 5, 0.5) {\huge 2};
\end{axis}

\begin{axis}[
xshift=0cm,
yshift=-2.5cm,
width=7cm,
height=2.5cm,
xlabel={$t$},
ylabel={$h(t)$},
xmin=-3.5,
xmax=10.5,
ytick = {0},
xtick = {0},
xticklabels = {$t_1$},
]
\addplot+[black, no marks, domain=-4:10, samples=400,variable=k] { (k<0) + (k>0)*exp(-0.9*k)};
\node[black!40!red] at (axis cs: 5, 0.5) {\huge 3};
\end{axis}

\begin{axis}[
xshift=7cm,
yshift=-2.5cm,
width=7cm,
height=2.5cm,
xlabel={$t$},
ylabel={$h(t)$},
xmin=-3.5,
xmax=10.5,
ytick = {0},
xtick = {0},
xticklabels = {$t_1$},
]
\addplot+[black, no marks, domain=-4:10, samples=400,variable=k] { (k<0) + ((k>=0) - (k>4))*(1-1/16*pow(-k,2)) };
\node[black!40!red] at (axis cs: 5, 0.5) {\huge 4};
\end{axis}

\end{tikzpicture}
\end{frame}

\begin{frame}[label={sec:orge1d1a40}]{Deviation variables}
\begin{center}
\includegraphics[width=0.7\linewidth]{../../figures/tank-with-hole}
\end{center}

Flow in: \(z(t) = z_0 + w(t)\). Level of water: \(h(t) = h_0 + y(t)\). The constants \(h_0\) and \(z_0\) define an \emph{operating point}.

\begin{align*}
\frac{d}{dt} h(t) &= - \frac{a\sqrt{2g}}{A} \sqrt{h(t)} + \frac{1}{A} z(t)
\end{align*}

\alert{Individual activity} Given \(h_0\) determine the operating point for the inflow, \(z_0\), such that the system is in equilibrium at the operating point.
\end{frame}

\begin{frame}[label={sec:orgc3faefa}]{Intuition}
Which change \(y(t)\) in the water level corresponds to a step change \(w(t)\) in the inflow?

\begin{center}
\includegraphics[width=0.7\linewidth]{../../figures/dc-response-exercise}
\end{center}
\end{frame}

\begin{frame}[label={sec:org237f19c}]{Fitting a first-order model}
Assuming a plant model of first-order with time-constant \(T\)
\[  \quad \textcolor{green!50!black}{Y(s)} = \frac{K}{sT + 1}\textcolor{blue!80!black}{U(s)} \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad \textcolor{green!50!black}{y(t)} = u_f K\big( 1 - \mathrm{e}^{-\frac{t}{T}}\big)u_H(t)\]
\def\Tcnst{3}
\def\tdelay{0.0}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=14cm,
    height=4.5cm,
    grid = both,
    xtick = {0,  \two},
    xticklabels = {0, $T$},
    ytick = {0, \ytwo, \uampl, \yfinal},
    yticklabels = {0,  $ $, $u_f$, $y_f$},
    xmin = -0.2,
    %minor y tick num=9,
    %minor x tick num=9,
    %every major grid/.style={red, opacity=0.5},
    xlabel = {$t$},
    ]
      \addplot [thick, green!50!black, no marks, domain=0:10, samples=100] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
    \end{axis}
  \end{tikzpicture}
\end{center}

\alert{Individual activity} Evaluate the response \(y(t)\) at the time instant \(t=T\) and for \(t\to\infty\)!
\end{frame}

\begin{frame}[label={sec:orgbb67976}]{Fitting a first-order model}
Assuming a plant model of first-order with time-constant \(T\)
\[  \quad \textcolor{green!50!black}{Y(s)} = \frac{K}{sT + 1}\textcolor{blue!80!black}{U(s)} \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad \textcolor{green!50!black}{y(t)} = u_f K\big( 1 - \mathrm{e}^{-\frac{t}{T}}\big)u_H(t)\]
\def\Tcnst{3}
\def\tdelay{0.0}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \small
  \begin{tikzpicture}
    \begin{axis}[
    width=14cm,
    height=3.5cm,
    grid = both,
    xtick = {0,  \two},
    xticklabels = {0, $T$},
    ytick = {0, \ytwo, \uampl, \yfinal},
    yticklabels = {0,  $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    %minor y tick num=9,
    %minor x tick num=9,
    %every major grid/.style={red, opacity=0.5},
    xlabel = {$t$},
    ]
      \addplot [thick, green!50!black, no marks, domain=0:10, samples=100] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
    \end{axis}
  \end{tikzpicture}
\end{center}

\alert{Time-constant:} Find the time \(t=T\) at which the response has reached 63.2\% of its final value

\alert{Gain:} \(y_f = \lim_{t\to\infty}y(t) = Ku_f \quad \Rightarrow \quad K = \frac{y_f}{u_f}\)
\end{frame}

\section{CSTR}
\label{sec:org63c92d8}

\begin{frame}[label={sec:org86f683c}]{A Continuous Stirred Tank Reactor}
\begin{center}
\includegraphics[height=0.8\textheight]{../../figures/stirred_tank_heat_exchange}
\end{center}
\end{frame}

\begin{frame}[label={sec:org315c48d}]{A Continuous Stirred Tank Reactor}
\begin{columns}
\begin{column}{0.6\columnwidth}
\begin{center}
\includegraphics[height=0.6\textheight]{../../figures/stirred_tank_heat_exchange}
\end{center}
\end{column}

\begin{column}{0.4\columnwidth}
Assume:
\begin{enumerate}
\item Constant flow \(f\) through the tank reactor
\item Constant temperatures \(T_i\) and \(T_s\)
\item Perfect mixing in the tank reactor
\item Perfect mixing in the heating jacket
\item Isothermic reaction
\end{enumerate}
\end{column}
\end{columns}
\end{frame}

\begin{frame}[label={sec:orgca7623b}]{A Continuous Stirred Tank Reactor}
\begin{center}
\includegraphics[height=0.4\textheight]{../../figures/stirred_tank_heat_exchange}
\end{center}

Energy balance:
\begin{align*}
\frac{dT(t)}{dt} &= k_1\big( T_i - T(t) \big) + k_2 \big( T_h(t) - T(t)\big)\\
\frac{dT_h(t)}{dt} &= k_3f_h(t)\big( T_s - T_h(t) \big) - k_4 \big( T_h(t) - T(t)\big)
\end{align*}
\end{frame}

\begin{frame}[label={sec:orgcb60eff}]{Intuition}
\begin{center}
\includegraphics[height=0.3\textheight]{../../figures/stirred_tank_heat_exchange}
\end{center}

\begin{center}
\begin{tikzpicture}
    \footnotesize

    \pgfmathsetmacro{\fnull}{0.6}
    \pgfmathsetmacro{\fstep}{1.5}
    \pgfmathsetmacro{\Tnull}{1}
    \pgfmathsetmacro{\Ti}{0}

    \begin{groupplot}[group style={group size=1 by 2, vertical sep=0.3cm, horizontal sep=1.3cm},
    width=15cm,
    height=3.5cm,
    xlabel={$t$},
    xmin=-1.5,
    xmax=10.5,
    ytick = \empty,
    xtick = {0},
    ymin=-0.2, ymax=2,
    ]
    \nextgroupplot[ytick={ 0, \fnull},
    yticklabels={0, $f_0$,}, ylabel={$f_h(t)$}, xlabel={},]
    \addplot[orange!60!red, thick,] coordinates { (-1, \fnull) (0,\fnull) (0,\fstep) (5,\fstep) (5,0) (10,0)};
    \nextgroupplot[ytick={ \Ti, \Tnull},
    yticklabels={$T_i$, $T_0$,}, ylabel={$T(t)$}]
    \addplot[red!60!black, thick, ] coordinates { (-1, \Tnull) (0,\Tnull)};

    \end{groupplot}
  \end{tikzpicture}
\end{center}
\end{frame}

\section{Fitting first-order model with delay}
\label{sec:org4da89d4}
\begin{frame}[label={sec:orgd4c7560}]{Fitting first-order model with delay}
Assuming a plant model of first-order with time constant \(T\) and delay \(\tau\)
\[  \quad \textcolor{green!50!black}{Y(s)} = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}\textcolor{blue!80!black}{U(s)} \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad \textcolor{green!50!black}{y(t)} = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_H(t-\tau)\]
\def\Tcnst{3}
\def\tdelay{0.6}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=14cm,
    height=4.5cm,
    grid = both,
    xtick = {0, \tdelay, \tone, \two},
    xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    yticklabels = {0, $ $, $ $, $u_f$, $y_f$},
    xmin = -0.2,
    %minor y tick num=9,
    %minor x tick num=9,
    %every major grid/.style={red, opacity=0.5},
    xlabel = {$t$},
    ]
      \addplot [thick, green!50!black, no marks, domain=0:10, samples=100] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
    \end{axis}
  \end{tikzpicture}
\end{center}

\alert{Individual activity} Evaluate the response \(y(t)\) at the two time instants \(t=\tau + \frac{T}{3}\) and \(t=\tau + T\)!
\end{frame}

\begin{frame}[label={sec:orgb4c2701}]{Fitting first-order model with delay}
Assuming a plant model of first-order with time constant \(T\) and delay \(\tau\)
\[  \quad \textcolor{green!50!black}{Y(s)} = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}\textcolor{blue!80!black}{U(s)} \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad \textcolor{green!50!black}{y(t)} = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_H(t-\tau)\]
\def\Tcnst{3}
\def\tdelay{0.6}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=14cm,
    height=4.5cm,
    grid = both,
    xtick = {0, \tdelay, \tone, \two},
    xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    yticklabels = {0, $0.283y_{f}$, $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    %minor y tick num=9,
    %minor x tick num=9,
    %every major grid/.style={red, opacity=0.5},
    xlabel = {$t$},
    ]
      \addplot [thick, green!50!black, no marks, domain=0:10, samples=100] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
    \end{axis}
  \end{tikzpicture}
\end{center}

\[ y_f = \lim_{t\to\infty} y(t) = u_f K \quad \Rightarrow \quad K = \frac{y_f}{u_f}. \]
\end{frame}

\begin{frame}[label={sec:orgd2f9e32}]{First-order model with delay - example}
\[  \quad Y(s) = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}U(s) \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad y(t) = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_s(t-\tau)\]
\def\Tcnst{2.1}
\def\tdelay{1}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=12cm,
    height=4cm,
    grid = both,
    %xtick = {0, \tdelay, \tone, \two},
    %xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    %ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    %yticklabels = {0, $0.283y_{f}$, $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    minor y tick num=9,
    minor x tick num=9,
    every major grid/.style={red, opacity=0.5},
    %xlabel = {$t$},
    clip = false,
    ]
      \addplot [thick, green!50!black, smooth, no marks, domain=0:10, samples=16] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
      \draw[thick, green!70!black, dashed] (axis cs: 10, \yfinal) -- (axis cs: -1, \yfinal, -0.9) node[left, anchor=east] {$y_f = \yfinal$};
      \draw[blue!70!black, dashed] (axis cs: 0, \uampl) -- (axis cs: -1, \uampl, -0.9) node[left, anchor=east] {$u_f = \uampl$};
    \end{axis}
  \end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}[label={sec:org01be687}]{First-order model with delay - example}
\[  \quad Y(s) = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}U(s) \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad y(t) = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_s(t-\tau)\]
\def\Tcnst{2.1}
\def\tdelay{1}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=12cm,
    height=4cm,
    grid = both,
    %xtick = {0, \tdelay, \tone, \two},
    %xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    %ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    %yticklabels = {0, $0.283y_{f}$, $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    minor y tick num=9,
    minor x tick num=9,
    every major grid/.style={red, opacity=0.5},
    %xlabel = {$t$},
    clip = false,
    ]
      \addplot [thick, green!50!black, smooth, no marks, domain=0:10, samples=16] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: \two, -0.9) node[below] {$t_2 = \two = \tau + T$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: -1, \ytwo, -0.9) node[left, anchor=east] {$0.632y_f = \ytwo$};
      \draw[thick, green!70!black, dashed] (axis cs: 10, \yfinal) -- (axis cs: -1, \yfinal, -0.9) node[left, anchor=east] {$y_f = \yfinal$};
      \draw[blue!70!black, dashed] (axis cs: 0, \uampl) -- (axis cs: -1, \uampl, -0.9) node[left, anchor=east] {$u_f = \uampl$};
    \end{axis}
  \end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}[label={sec:org9d59a42}]{First-order model with delay - example}
\[  \quad Y(s) = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}U(s) \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad y(t) = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_s(t-\tau)\]
\def\Tcnst{2.1}
\def\tdelay{1}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=12cm,
    height=4cm,
    grid = both,
    %xtick = {0, \tdelay, \tone, \two},
    %xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    %ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    %yticklabels = {0, $0.283y_{f}$, $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    minor y tick num=9,
    minor x tick num=9,
    every major grid/.style={red, opacity=0.5},
    %xlabel = {$t$},
    clip = false,
    ]
      \addplot [thick, green!50!black, smooth, no marks, domain=0:10, samples=16] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
      \draw[thick, red, dashed] (axis cs: \tone, \yone) -- (axis cs: \tone, -0.45) node[below] {$t_1 = \tone = \tau + \frac{T}{3}$};
      \draw[thick, red, dashed] (axis cs: \tone, \yone) -- (axis cs: -1,\yone) node[left, anchor=east] {$0.283y_f = \yone$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: \two, -0.9) node[below] {$t_2 = \two = \tau + T$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: -1, \ytwo, -0.9) node[left, anchor=east] {$0.632y_f = \ytwo$};
      \draw[thick, green!70!black, dashed] (axis cs: 10, \yfinal) -- (axis cs: -1, \yfinal, -0.9) node[left, anchor=east] {$y_f = \yfinal$};
      \draw[blue!70!black, dashed] (axis cs: 0, \uampl) -- (axis cs: -1, \uampl, -0.9) node[left, anchor=east] {$u_f = \uampl$};
    \end{axis}
  \end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}[label={sec:orgd6024cf}]{First-order model with delay - example}
\[  \quad Y(s) = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}U(s) \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad y(t) = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_s(t-\tau)\]
\def\Tcnst{2.1}
\def\tdelay{1}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=12cm,
    height=4cm,
    grid = both,
    %xtick = {0, \tdelay, \tone, \two},
    %xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    %ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    %yticklabels = {0, $0.283y_{f}$, $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    minor y tick num=9,
    minor x tick num=9,
    every major grid/.style={red, opacity=0.5},
    %xlabel = {$t$},
    clip = false,
    ]
      \addplot [thick, green!50!black, smooth, no marks, domain=0:10, samples=16] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
      \draw[thick, red, dashed] (axis cs: \tone, \yone) -- (axis cs: \tone, -0.45) node[below] {$t_1 = \tone = \tau + \frac{T}{3}$};
      \draw[thick, red, dashed] (axis cs: \tone, \yone) -- (axis cs: -1,\yone) node[left, anchor=east] {$0.283y_f = \yone$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: \two, -0.9) node[below] {$t_2 = \two = \tau + T$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: -1, \ytwo, -0.9) node[left, anchor=east] {$0.632y_f = \ytwo$};
      \draw[thick, green!70!black, dashed] (axis cs: 10, \yfinal) -- (axis cs: -1, \yfinal, -0.9) node[left, anchor=east] {$y_f = \yfinal$};
      \draw[blue!70!black, dashed] (axis cs: 0, \uampl) -- (axis cs: -1, \uampl, -0.9) node[left, anchor=east] {$u_f = \uampl$};
    \end{axis}
  \end{tikzpicture}
\end{center}
\[ \begin{cases} \tone = \tau + \frac{T}{3}\\ \two = \tau + T \end{cases} \quad \Rightarrow \quad \begin{cases} \tau = \tdelay \\ T = \Tcnst \end{cases}, \qquad  K = \frac{y_f}{u_f} = \frac{\yfinal}{\uampl} = \ggain \]
\end{frame}

\begin{frame}[label={sec:org5ee71dd}]{First-order model with delay - example}
\[  \quad Y(s) = \frac{K\mathrm{e}^{-s\tau}}{sT + 1}U(s) \quad \overset{U(s) = \frac{u_f}{s}}{\Longrightarrow} \quad y(t) = u_f K\big( 1 - \mathrm{e}^{-\frac{t-\tau}{T}}\big)u_s(t-\tau)\]
\def\Tcnst{2.1}
\def\tdelay{1}
\def\ggain{2}
\def\uampl{0.8}
\pgfmathsetmacro{\yfinal}{\uampl*\ggain}
\pgfmathsetmacro{\yone}{0.283*\yfinal}
\pgfmathsetmacro{\ytwo}{0.632*\yfinal}
\pgfmathsetmacro{\tone}{\tdelay + \Tcnst/3}
\pgfmathsetmacro{\two}{\tdelay + \Tcnst}

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[
    width=12cm,
    height=4cm,
    grid = both,
    %xtick = {0, \tdelay, \tone, \two},
    %xticklabels = {0, $\tau$, $\tau+\frac{T}{3}$, $\tau + T$},
    %ytick = {0, \yone, \ytwo, \uampl, \yfinal},
    %yticklabels = {0, $0.283y_{f}$, $0.632y_f$, $u_f$, $y_f$},
    xmin = -0.2,
    minor y tick num=9,
    minor x tick num=9,
    every major grid/.style={red, opacity=0.5},
    %xlabel = {$t$},
    clip = false,
    ]
      \addplot [thick, green!50!black, smooth, no marks, domain=0:10, samples=16] {\uampl*\ggain*(x>\tdelay)*(1 - exp(-(x-\tdelay)/\Tcnst)} node [coordinate, pos=0.9, pin=-90:{$y(t)$}] {};
      \addplot [const plot, thick, blue!80!black, no marks, domain=-1:10, samples=100] coordinates {(-1,0) (0,0) (0,\uampl) (10,\uampl)} node [coordinate, pos=0.9, pin=-90:{$u(t)$}] {};
      \draw[thick, red, dashed] (axis cs: \tone, \yone) -- (axis cs: \tone, -0.45) node[below] {$t_1 = \tone = \tau + \frac{T}{3}$};
      \draw[thick, red, dashed] (axis cs: \tone, \yone) -- (axis cs: -1,\yone) node[left, anchor=east] {$0.283y_f = \yone$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: \two, -0.9) node[below] {$t_2 = \two = \tau + T$};
      \draw[thick, orange, dashed] (axis cs: \two, \ytwo) -- (axis cs: -1, \ytwo, -0.9) node[left, anchor=east] {$0.632y_f = \ytwo$};
      \draw[thick, green!70!black, dashed] (axis cs: 10, \yfinal) -- (axis cs: -1, \yfinal, -0.9) node[left, anchor=east] {$y_f = \yfinal$};
      \draw[blue!70!black, dashed] (axis cs: 0, \uampl) -- (axis cs: -1, \uampl, -0.9) node[left, anchor=east] {$u_f = \uampl$};
    \end{axis}
  \end{tikzpicture}
\end{center}
\[ \begin{cases} \tone = \tau + \frac{T}{3}\\ \two = \tau + T \end{cases} \quad \Rightarrow \quad \begin{cases} \tau = \tdelay \\ T = \Tcnst \end{cases}, \qquad  K = \frac{y_f}{u_f} = \frac{\yfinal}{\uampl} = \ggain \]
\end{frame}

\section{Second-order model}
\label{sec:orgfaccc8e}
\end{document}


\documentclass[12pt,letterpaper,twoside]{article}

 \newif\ifsolution\solutiontrue   % Include the solutions
% \newif\ifsolution\solutionfalse  % Exclude the solutions

\usepackage{cme212}

\begin{document}

{\centering \textbf{Paper Exercise 1 \\ Due
    Tuesday, January 28th at 4:30 P.M. (on Gradescope)} \par}
\vspace*{-8pt}\noindent\rule{\linewidth}{1pt}
\vspace{-3em}

\paragraph{Question 0: Lecture 3}
Are the preconditions for the following functions correct? If not, where does it go wrong, and how would you rewrite them?

\textbf{(a)}
\begin{cpp}
/* Normalize a vector in place
 *@param[in] _v_ stl vector of doubles to normalize
 *
 *@post the sum of all the elements of _v_ is 1
*/
void normalize(std::vector<double>& v){
	double sum = 0.0;
	for(auto it = v.begin(); it != v.end(); it++){
		sum += *it;
	}
	for(auto it = v.begin(); it != v.end(); it++){
		*it/=sum;
	}

}
\end{cpp}

\begin{solution}

\end{solution}

\textbf{(b)}
\begin{cpp}
	/* Compute the cosine similarity between two vectors
	* @param[in] _v_ stl vector of doubles
	* @param[in] _w_ stl vector of doubles
	* @return the cosine of the angle between _v_ and _w_
	*
	* @pre _v_.size() == _w_.size()
	* @pre for all 0<i<_v_.size() _v_[i]!=0 and _w_[i]!=0
	*/

	double cosine_sim(const std::vector<double>& v, const std::vector<double>& w){
		double num = 0.0;
		double norm_v = 0.0;
		double norm_w = 0.0;

		if(v.size() != w.size){
			std::cout<<"vectors are not the same length"<<std::endl;
			return 2.0;
		}
		for(unsigned int i=0; i<v.size(); i++){
			num += v[i]*w[i];
			norm_w += w[i] * w[i];
			norm_v += v[i] * v[i];
		}

		return num/(sqrt(norm_w)*sqrt(norm_v));
	}
\end{cpp}

\begin{solution}

\end{solution}

\paragraph{Question 1: Lecture 2}
\quad \vspace{-20pt}\\
\begin{cpp}
/** Find the first element in a sorted array that does not
 *  satisfy a predicate.
 * @param[in] _a_         Array to search.
 * @param[in] _low_,_high_  Search in the index range [_low_, _high_).
 * @param[in] _pred_      Predicate to consider
 * @return  An index into array _a_, or _high_.
 *
 * @tparam T     Type of the elements.
 * @tparam Pred Type of the predicate with signature:
 *                bool operator()(const T&)
 *
 * @pre  0 <= _low_ <= _high_ <= Size of _a_.
 * @pre  There exists k in [_low_, _high_] such that
 *   for all i with _low_ <= i < k, _pred_(_a_[i]), and
 *   for all j with k <= j < _high_, !_pred_(_a_[j]).
 *
 * @post For all i,j with _low_ <= i < result <= j < _high_,
 *           _pred_(_a_[i]) and !_pred_(_a_[j]).
 *
 * Performs O(log(_high_ - _low_)) operations.
 */
template <typename T, typename Pred>
int lower_bound(const T* a, int low, int high, const Pred& pred);
\end{cpp}

Using the \texttt{lower\_bound} function with the above specification, implement a \texttt{lower\_bound} function with the following specification; \textbf{(a)} first, with a functor, and \textbf{(b)} second, with a lambda function.
\begin{cpp}
/** Find the first element in a sorted array that is not less than a value.
 * @param[in] a         Sorted array to search.
 * @param[in] low,high  Search in the index range [_low_, _high_).
 * @param[in] v         Value to search for.
 * @return  An index into array _a_, or _high_.
 *
 * @tparam T Type of the elements.
 *   T is comparable with 'bool operator<(T,T)'.
 *   Operator< is irreflexive, i.e. operator<(x, x) returns false
 *
 * @pre  0 <= _low_ <= _high_ <= Size of _a_.
 * @pre  For all i,j with _low_ <= i < j < _high_,
 *          !(_a_[j] < _a_[i])
 *
 * @post For all i,j with _low_ <= i < result <= j < _high_,
 *          _a_[i] < _v_ and !(_a_[j] < _v_).
 *
 * Performs O(log(_high_ - _low_)) operations.
 */
template <typename T>
int lower_bound(const T* a, int low, int high, const T& v);
\end{cpp}
\begin{solution}

\end{solution}

\paragraph{Question 2: Lecture 4}
\quad \\
\textbf{(a)} What are the 4 operators and the 2 functions an iterator needs to implement?

\begin{solution}

\end{solution}

\textbf{(b)} As long as an iterator class has the above methods defined, it can be used with any of the \texttt{stl} methods that use iterators. We will write an iterator class for a book that is represented as a \texttt{std::vector<std::vector<string>>}. In this case, each inner \texttt{std::vector<string>} represents a sentence of the book, split into individual words. Write a wrapper iterator that iterates over all the words in a \texttt{Book}. The starter code is given below as well as in \href{https://github.com/cme212/CME212-2020/blob/master/EX1/q2_starter.cpp}{\texttt{q2\_starter.cpp}}

\begin{cpp}
#include <string>
#include <iostream>
#include <vector>
#include <fstream>
#include <algorithm>

//alias pointer for convenience
using book_ptr = std::vector<std::vector<string>>*

//Helper function to split string into a vector based on a splitting character.
//This function is used to load the book.
const std::vector<std::string> split(const std::string& s, const char& c)
{
	std::string buff{""};
	std::vector<std::string> v;

	//iterate character by character and push the buffer to the vector if the splitting char is reached.
	for(auto n:s)
	{
		if(n != c) buff+=n; else
		if(n == c && buff != "") { v.push_back(buff); buff = ""; }
	}
	if(buff != "") v.push_back(buff);

	return v;
}

//This class represents a book as a vector of vector of strings,
//where each vector of strings is sentence in the book.
class Book:
{
	std::vector<std::string> temp_book_;
	std::vector<std::vector<std::string>> book_;

	public:
	//Constructor for the book class.
	//Takes in a .txt file and splits it into a
	//std::vector<std::vector<string>>
	Book(std::string filename)
	{
		//Read in the book as a vector of strings
		//where each string is a sentence.
		std::string sent;
		std::ifstream book_file(filename);
		std::vector<std::string> temp;

		//Read the file line by line.
		while(std::getline(book_file, sent))
		{
			temp_book_.push_back(sent);
		}
		book_file.close();

		//Split each sentence by ' '
		for (unsigned int i = 0; i< temp_book_.size(); i++)
		{
			//Skip empty lines
			if (temp_book_[i].length()==0){
				continue;
			}
			else{
				temp = split(temp_book_[i], ' ');
				book_.push_back(temp);
			}

		}

	}

	//return an iterator pointing at the start of the book
	BookIter begin()
	{
		//Your code here
	}

	//return an iterator pointing at the end of the book
	BookIter end()
	{
		//Your code here
	}

	private:
	std::vector<std::vector<string>> book_;

}

class BookIter:
{
	public:

	//Increments to the next word in the book class.
	BookIter operator++()
	{
		//Your code here
	}

	//Defines equality between two iterators
	bool operator==(BookIter book_iter)
	{
		//Your code here
	}

	//Defines inequality between two iterators
	bool operator!=(BookIter book_iter)
	{
		//Your code here
	}

	//Dereference operator
	std::string operator*()
	{
		//Your code here
	}

	private:
	friend class Book;
	//Your code here

	//Private constructor that can be accessed by the Book class.
	BookIter(){
		//Your code here
	}
}

\end{cpp}

\begin{solution}

\end{solution}

\textbf{(c)} Write a functor that implements a comparator between two strings based on the length of the strings, returning \texttt{true} if and only if the first string is shorter than the second string.

\begin{solution}

\end{solution}

\textbf{(d)} The code below reads in Moby Dick into the book class. Use the iterator class, the functor and an stl algorithm from the algorithms package to find the longest word in Moby Dick.

\begin{cpp}
int main()
{
	Book moby_dick("moby_dick.txt");
	\\ Your code here.
	std::cout<< longest_word << std::endl;
}
\end{cpp}

\begin{solution}

\end{solution}
\end{document}


\chapter{Abbreviations}

\begin{longtable}{ll}
3D&Three dimensions\\
ABS&Acrylonitrile butadiene styrene\\
\SI{}{\adu}&Analogue to digital units\\
AIC&Akaike information criterion\\
AM&Additive manufacturing\\
ANOVA&Analysis of variance\\
Approx.&Approximate\\
AUC&Area under the receiver operating characteristic curve\\
BH&Benjamini and Hochberg\\
BIC&Bayesian information criterion\\
BW&Black/white\\
CAD&Computer-aided design\\
CC BY&Creative Commons Attribution\\
$\CPoisson$&Compound Poisson-gamma\\
EM&Expectation maximisation\\
Expec.&Expected\\
FDK&Feldkamp-Davis-Kress\\
FDR&False discovery rate\\
FWER&Family-wise error rate\\
Freq.&Frequency\\
GLM&Generalised linear models\\
GPU&Graphics processing unit\\
i.i.d.&Independent and identically distributed\\
IOP&Institute of Physics\\
IQR&Interquartile range\\
MADA-mode&Median absolute deviation around the mode\\
m.g.f.&Moment generating function\\
Obs.&Observed\\
PCER&Per-comparison error rate\\
p.d.f.&Probability density function\\
px&Pixels\\
Q-Q&Quantile-quantile\\
ROC&Receiver operating characteristic\\
ROI&Region of interest\\
Std&Standard deviation\\
STL&Sterolithography or Standard tessellation language\\
Var&Variance\\
XCT&X-ray computed tomography
\end{longtable}

\chapter{Compound Poisson}

\section{Moment Generating Function}
\label{chapter:appendix_compoundPoissonMgf}

Let $Y\sim\poisson(\lambda)$, $U_1,U_2,U_3,\ldots$ be i.i.d.~random variables and $X|Y=\sum_{i=1}^YU_i$. Let the m.g.f.~of $X$ be $M_X(\theta)=\expectation\left[\euler^{\theta X}\right]$. This can be computed using the result for conditional expectations $M_X(\theta)=\expectation\expectation\left[\euler^{X\theta}|Y\right]$. This results in
\begin{align}
  M_X(\theta)&=\expectation\expectation\left[\exp\left(\theta U_1 + \theta U_2 + \ldots + \theta U_Y\right)|Y\right]
  \nonumber \\
  &=\expectation\expectation\left[\euler^{\theta U_1}\cdot\euler^{\theta U_2}\cdot\ldots\cdot\euler^{\theta U_Y}|Y\right]
  \ .
\end{align}
Because $U_i$, for $i=1,2,3,\ldots$, are i.i.d.~and each $U_i$ has a m.g.f.~$M_U(\theta)=\expectation\left[\euler^{U\theta}\right]$, then
\begin{align}
  M_X(\theta)&=\expectation\left(
    \expectation\left[\euler^{\theta U_1}|Y\right]\cdot
    \expectation\left[\euler^{\theta U_2}|Y\right]\cdot
    \ldots \cdot
    \expectation\left[\euler^{\theta U_Y}|Y\right]
  \right)
  \nonumber\\
  &=\expectation\left[\left(M_U(\theta)\right)^Y\right]
  \nonumber\\
  &=\expectation\left[\euler^{Y\ln(M_U(\theta))}\right]
  \nonumber\\
  & = M_Y\left(\ln(M_U(\theta)\right)
\end{align}
where $M_Y(\theta)=\expectation\left[\euler^{Y\theta}\right]$ is the m.g.f.~of $Y$. It can be shown that the m.g.f.~of $Y$ is
$
  M_Y(\theta)=
  \exp
  \left[
    \lambda
    \left(
      \euler^\theta-1
    \right)
  \right]
$,
hence
\begin{equation}
  M_X(\theta)=
  \exp\left[
    \lambda
    \left(
      M_U(\theta)-1
    \right)
  \right]
  \ .
\end{equation}

Moments of $X$ can be obtained from the m.g.f.~by differentiating it and setting $\theta$ to zero
\begin{align}
  M_X'(\theta)&=\exp\left[\lambda\left(M_U(\theta)-1\right)\right]\cdot\lambda M_U'(\theta)
  \nonumber\\
  &=M_X(\theta)\lambda M_U'(\theta)
  \\
  M_X'(0)&=\lambda\expectation\left[U\right]
\end{align}
which results in
\begin{equation}
  \expectation\left[X\right]=\lambda\expectation\left[U\right]
  \ .
\end{equation}
Conducting the same procedure
\begin{align}
  M_X''(\theta)&=M_X'(\theta)\lambda M_U'(\theta)+M_X(\theta)\lambda M_U''(\theta)
  \nonumber\\
  &=\lambda M_X(\theta)\left[
    \lambda\left(M_U'(\theta)\right)^2+M_U''(\theta)
  \right]
  \\
  M_X''(0)&=\lambda\left[\lambda\left(\expectation[U]\right)^2+\expectation\left[U^2\right]\right]
\end{align}
the variance can be obtained
\begin{align}
  \variance\left[X\right]&=M_X''(0)-\left[M_X'(0)\right]^2
  \nonumber \\
  &=\lambda\left[\lambda\left(\expectation[U]\right)^2+\expectation\left[U^2\right]\right]-\left[\lambda\expectation\left[U\right]\right]^2
  \nonumber \\
  &=\lambda\expectation\left[U^2\right] \ .
\end{align}
Differentiating the m.g.f.~one more time
\begin{align}
  M_X'''(\theta) &=
  \begin{multlined}[t]
    \lambda M_X'(\theta)\left[
      \lambda\left(M_U'(\theta)\right)^2+M_U''(\theta)
    \right]
    \\+
    \lambda M_X(\theta)\left[
      2\lambda M_U'(\theta)M_U''(\theta)+M_U'''(\theta)
    \right]
  \end{multlined}
  \nonumber \\
  &=
  \begin{multlined}[t]
    \lambda^2 M_X(\theta)M_U'(\theta)\left[
      \lambda\left(M_U'(\theta)\right)^2+M_U''(\theta)
    \right]
    \\+
    \lambda M_X(\theta)\left[
      2\lambda M_U'(\theta)M_U''(\theta)+M_U'''(\theta)
    \right]
  \end{multlined}
  \nonumber \\
  &=
  \lambda M_X(\theta)\left[
    \lambda^2 \left(M_U'(\theta)\right)^3 + 3\lambda M_U'(\theta)M_U''(\theta) + M_U'''(\theta)
  \right]
  \\
  M_X'''(0) &=
  \lambda \left[
    \lambda^2 \left(\expectation[U]\right)^3 + 3\lambda \expectation[U]\expectation\left[U^2\right] + \expectation\left[U^3\right]
  \right]
\end{align}
and the third moment about the mean is
\begin{align}
  \expectation\left[\left(X-\expectation[X]\right)^3\right] & =
  M_X'''(0) - 3M_X''(0)M_X'(0) + 2\left(M_X'(0)\right)^3
  \nonumber\\
  &=
  \begin{multlined}[t]
    \lambda \left[
      \lambda^2 \left(\expectation[U]\right)^3 + 3\lambda \expectation[U]\expectation\left[U^2\right] + \expectation\left[U^3\right]
    \right]
    \\-3\lambda\left[\lambda\left(\expectation[U]\right)^2+\expectation\left[U^2\right]\right]\lambda\expectation[U] + 2\left(\lambda\expectation[U]\right)^3
  \end{multlined}
  \nonumber\\
  &=\lambda\expectation\left[U^3\right] \ .
\end{align}

\section{Tweedie Dispersion Exponential Family}
\label{chapter:appendix_tweedie}
Let $X\sim\CPoisson(\lambda,\alpha,\beta)$ and have p.d.f.
\begin{equation}
  p_X(x) =
  \begin{cases}
    \delta(x) \euler^{-\lambda} & \text{ for } x=0 \\
    \dfrac{\euler^{-x\beta-\lambda}}{x}
    \displaystyle\sum_{y=1}^{\infty}\dfrac{\beta^{y\alpha}}{\Gamma(y\alpha)}x^{y\alpha}\frac{\lambda^y}{y!} & \text{ for } x>0
  \end{cases}
  \ .
  \label{eq:appendix_compoundPoisson_pdf}
\end{equation}

\cite{jorgensen1987exponential} uses the following parametrisation: $p=\dfrac{2+\alpha}{1+\alpha}$, $\mu=\dfrac{\lambda\alpha}{\beta}$, $\phi = \dfrac{\alpha+1}{\beta^{2-p}(\lambda\alpha)^{p-1}}$. The parameters can be rearranged
\begin{equation}
  \lambda=\frac{\mu^{2-p}}{\phi(2-p)}
\end{equation}
\begin{equation}
  \alpha=\frac{2-p}{p-1}
\end{equation}
\begin{equation}
  \beta=\frac{1}{\phi(p-1)\mu^{p-1}} \ .
\end{equation}
Substituting these parameters into Equation \eqref{eq:appendix_compoundPoisson_pdf}, the p.m.f.~at zero is
\begin{equation}
  \prob(X=0) = \exp
  \left[
      -\frac{\mu^{2-p}}{\phi(2-p)}
  \right]
\end{equation}
and the p.d.f.~for $x>0$ is
\begin{align*}
  p_X(x) &=
  \begin{multlined}[t]
    \exp\left[
        \frac{-x}{\phi(p-1)\mu^{p-1}}
        -\frac{\mu^{2-p}}{\phi(2-p)}
    \right]
    \dfrac{1}{x}
    \\
    \sum_{y=1}^{\infty}
    \left[
      \frac{1}{\phi(p-1)\mu^{p-1}}
    \right]^{y\alpha}
    \frac{1}{\Gamma(y\alpha)}
    x^{y\alpha}
    \left[
      \frac{\mu^{2-p}}{\phi(2-p)}
    \right]^y
    \frac{1}{y!}
  \end{multlined}
  \\
  &=
  \begin{multlined}[t]
    \exp\left[
      \frac{1}{\phi}\left(x\frac{\mu^{1-p}}{1-p}-\frac{\mu^{2-p}}{2-p}\right)
    \right]
    \frac{1}{x}
    \\
    \sum_{y=1}^{\infty}\frac{x^{y\alpha}\mu^{y[2-p-\alpha(p-1)]}}{\phi^{y(1+\alpha)}(p-1)^{y\alpha}(2-p)^yy!\Gamma(y\alpha)}
    \ .
  \end{multlined}
\end{align*}
It should be noted that $2-p-\alpha(p-1) = 2-p - \frac{2-p}{p-1}(p-1) =0$ so that
\begin{equation}
  p_X(x) =
  \exp\left[
    \frac{1}{\phi}
    \left(
      x\frac{\mu^{1-p}}{1-p}-\frac{\mu^{2-p}}{2-p}
    \right)
  \right]
  \frac{1}{x}
  \sum_{y=1}^{\infty}W_y(x,p,\phi)
\end{equation}
where
\begin{equation}
  W_y(x,p,\phi)=\frac{x^{y\alpha}}{\phi^{y(1+\alpha)}(p-1)^{y\alpha}(2-p)^yy!\Gamma(y\alpha)}
  \ .
\end{equation}

This is in the form of a distribution in the exponential family
\begin{equation}
  p_X(x)=\dfrac{\exp\left(x\theta\right)g(x)}{Z(\theta)}
\end{equation}
where $Z(\theta)$ is the partition function and $\theta$ is the natural parameter. The partition function has some useful properties. It is a normalisation constant $Z=Z(\theta)=\int_{0}^{\infty}\exp(x\theta)g(x)\diff x$. The derivative of the log-partition function is
\begin{align*}
  \dfrac{\partial \ln Z}{\partial \theta} &= \dfrac{1}{Z}\dfrac{\partial Z}{\partial \theta}
  \\
  & = \dfrac{1}{Z}\dfrac{\partial}{\partial \theta} \int_0^\infty \exp(x\theta) g(x) \diff x
  \\
  & = \dfrac{1}{Z}\int_0^\infty x \exp(x\theta) g(x) \diff x
\end{align*}
which results in
\begin{equation}
\expectation[X] = \dfrac{\partial \ln Z}{\partial \theta} \ .
\end{equation}
The second-order derivative of the log partition function is
\begin{align*}
  \dfrac{\partial^2\ln Z}{\partial \theta^2} &=
  \dfrac{
    Z\frac{\partial^2 Z}{\partial\theta^2}-\left(\frac{\partial Z}{\partial\theta}\right)^2
  }{
    Z^2
  }
  \\
  &=\dfrac{1}{Z}\dfrac{\partial^2 Z}{\partial \theta^2} - \left(\dfrac{1}{Z}\dfrac{\partial Z}{\partial \theta}\right)^2
  \\
  &=\dfrac{1}{Z}\dfrac{\partial^2}{\partial \theta^2}\int_0^\infty \exp(x\theta) g(x) \diff x - \left(\expectation\left[X\right]\right)^2
  \\
  &=\dfrac{1}{Z}\int_0^\infty x^2\exp(x\theta) g(x) \diff x - \left(\expectation\left[X\right]\right)^2
  \\
  &=\expectation\left[X^2\right] - \left(\expectation\left[X\right]\right)^2
\end{align*}
thus
\begin{equation}
\variance\left[X\right] = \dfrac{\partial^2\ln Z}{\partial \theta^2} \ .
\label{eq:appendix_variancePartitionFunction}
\end{equation}

For the compound Poisson-gamma distribution, the natural parameter is
\begin{equation}
  \theta = \theta(\mu) = \dfrac{\mu^{1-p}}{\phi(1-p)}
  \label{eq:appendix_cp_naturalParameter}
\end{equation}
and the partition function is
\begin{equation*}
  Z(\theta) =
  \exp\left[
    \frac{\mu^{2-p}}{\phi(2-p)}
  \right]
  \ .
\end{equation*}
Rearranging the natural parameter $\mu=(\theta\phi(1-p))^{1/(1-p)}$, the partition function is
\begin{equation}
  Z = \exp\left[
    \phi^{\frac{1}{1-p}}
    \cdot
    \theta^{\frac{2-p}{1-p}}
    \cdot
    \dfrac{
      (1-p)^{\frac{2-p}{1-p}}
    }{
      2-p
    }
  \right]
  \ .
\end{equation}
The log-partition function is
\begin{equation}
  \ln Z = \phi^{\frac{1}{1-p}}
    \cdot
    \theta^{\frac{2-p}{1-p}}
    \cdot
    \dfrac{
      (1-p)^{\frac{2-p}{1-p}}
    }{
      2-p
    }
  \ .
\end{equation}
Taking the first-order derivative
\begin{align}
  \dfrac{\partial \ln Z}{\partial \theta} &=
  \phi^{\frac{1}{1-p}}
  \cdot
  \dfrac{2-p}{1-p}\cdot\theta^{\frac{2-p}{1-p}-1}
  \cdot
  \dfrac{(1-p)^\frac{2-p}{1-p}}{2-p}
  \nonumber \\
  &= \phi^{\frac{1}{1-p}} \cdot \theta^{\frac{1}{1-p}} \cdot (1-p)^\frac{1}{1-p} \ .
\end{align}
Taking the second-order derivative
\begin{align}
  \dfrac{\partial^2 \ln Z}{\partial \theta^2} &=
  \phi^{\frac{1}{1-p}}
  \cdot
  \dfrac{1}{1-p}\cdot\theta^{\frac{1}{1-p}-1}
  \cdot
  (1-p)^\frac{1}{1-p}
  \nonumber \\
  &=
  \phi^{\frac{1}{1-p}}
  \cdot
  \theta^{\frac{p}{1-p}}
  \cdot
  (1-p)^\frac{p}{1-p} \ .
\end{align}
Substitute in Equation \eqref{eq:appendix_cp_naturalParameter}
\begin{align*}
  \dfrac{\partial^2 \ln Z}{\partial \theta^2} &=
  \phi^{\frac{1}{1-p}}
  \left[\dfrac{\mu^{1-p}}{\phi(1-p)}\right]^{\frac{p}{1-p}}
  (1-p)^\frac{p}{1-p}
  \nonumber \\
  & = \phi \mu^p
\end{align*}
and using the result from Equation \eqref{eq:appendix_variancePartitionFunction}
\begin{equation}
 \variance[X] = \phi \mu^p
\end{equation}
shows that the compound Poisson-gamma distribution is in the Tweedie dispersion exponential family with $1<p<2$.

\section{Method of Moments}
\label{chapter:appendix_methodofmoments}
Let $X\sim\CPoisson(\lambda,\alpha,\beta)$. Suppose $\widehat{\mu}_1$ is an estimator of $\mu_1=\expectation[X]$ and $\widehat{\mu}_j$ is an estimator of $\mu_j=\expectation\left[\left(X-\expectation[X]\right)^j\right]$ for $j=2,3$. It is given (see Section \ref{chapter:compoundPoisson_compoundPoissonGamma}) that
\begin{align}
  \mu_1&=\frac{\alpha\lambda}{\beta}
  \label{eq:appendix_methodofmoments_mu1}
  \\
  \mu_2&=\frac{\alpha(\alpha+1)\lambda}{\beta^2}
  \label{eq:appendix_methodofmoments_mu2}
  \\
  \mu_3&= \frac{\alpha(\alpha+1)(\alpha+2)\lambda}{\beta^3}
  \label{eq:appendix_methodofmoments_mu3}
  \ .
\end{align}

The ratios $\mu_2/\mu_1$ and $\mu_3/\mu_2$ are
\begin{equation}
\dfrac{\mu_2}{\mu_1}= \dfrac{\alpha+1}{\beta}
\label{eq:appendix_methodofmoments_ratio1}
\end{equation}
and
\begin{equation}
\dfrac{\mu_3}{\mu_2} = \dfrac{\alpha+2}{\beta} \ .
\label{eq:appendix_methodofmoments_ratio2}
\end{equation}
Subtracting $\mu_3/\mu_2$ from $\mu_2/\mu_1$ obtains
\begin{equation*}
  \dfrac{\mu_3}{\mu_2}-\dfrac{\mu_2}{\mu_1}=
  \dfrac{1}{\beta}
\end{equation*}
and rearranging
\begin{align}
  \beta&=\dfrac{1}{\dfrac{\mu_3}{\mu_2}-\dfrac{\mu_2}{\mu_1}}
  \nonumber\\
  \beta&=\dfrac{\mu_1\mu_2}{\mu_1\mu_3-\mu_2^2}
  \label{eq:appendix_methodofmoments_beta}
\end{align}
gets a justification for the estimator
\begin{equation}
  \widehat{\beta}=\dfrac{\widehat{\mu}_1\widehat{\mu}_2}{\widehat{\mu}_1\widehat{\mu}_3-\widehat{\mu}_2^2} \ .
\end{equation}

Rearranging Equation \eqref{eq:appendix_methodofmoments_ratio1}
\begin{equation}
\alpha=\dfrac{\mu_2}{\mu_1}\beta - 1
\end{equation}
and substituting in Equation \eqref{eq:appendix_methodofmoments_beta}
\begin{align}
  \alpha &= \dfrac{\mu_2}{\mu_1}\times\dfrac{\mu_1\mu_2}{\mu_1\mu_3-\mu_2^2} - 1
  \nonumber \\
  & = \dfrac{\mu_2^2 - \mu_1\mu_3+\mu_2^2}{\mu_1\mu_3-\mu_2^2}
  \nonumber \\
  & = \dfrac{2\mu_2^2 - \mu_1\mu_3}{\mu_1\mu_3-\mu_2^2}
  \label{eq:appendix_methodofmoments_alpha}
\end{align}
obtains the estimator
\begin{equation}
  \widehat{\alpha} =
  \dfrac{2\widehat{\mu}_2^2 - \widehat{\mu}_1\widehat{\mu}_3}{\widehat{\mu}_1\widehat{\mu}_3-\widehat{\mu}_2^2} \ .
\end{equation}

Finally, rearranging Equation \eqref{eq:appendix_methodofmoments_mu1}
\begin{equation*}
\lambda = \dfrac{\beta\mu_1}{\alpha}
\end{equation*}
and substituting in Equations \eqref{eq:appendix_methodofmoments_beta} and \eqref{eq:appendix_methodofmoments_alpha}
\begin{align}
\lambda &=
  \dfrac{\mu_1\mu_2}{\mu_1\mu_3-\mu_2^2}
  \times
  \dfrac{\mu_1\mu_3-\mu_2^2}{2\mu_2^2 - \mu_1\mu_3}
  \times
  \mu_1
  \nonumber \\
  & = \dfrac{\mu_1^2\mu_2}{2\mu_2^2-\mu_1\mu_3}
\end{align}
which leads to
\begin{equation}
\widehat{\lambda} = \dfrac{\widehat{\mu}_1^2\widehat{\mu}_2}{2\widehat{\mu}_2^2-\widehat{\mu}_1\widehat{\mu}_3} \ .
\end{equation}

\section{Normal Approximation}
\label{chapter:appendix_normalApproximation}
Let $X\sim\CPoisson(\lambda,\alpha,\beta)$ with m.g.f.~$M_X(\theta)=\exp\left[\lambda\left(\left(\frac{\beta}{\beta-\theta}\right)^{\alpha}-1\right)\right]$. The m.g.f.~in this form is not useful because when considering $\lambda\rightarrow\infty$ or $\alpha\rightarrow\infty$ then $M_X(\theta)\rightarrow\infty$. Also for $\beta\rightarrow\infty$, then $M_X(\theta)\rightarrow 1$ which is not useful either.

The compound Poisson-gamma random variable $X$ can be standardised to obtain useful limiting results from the m.g.f. Let
\begin{equation}
  Z = \frac{X-\expectation[X]}{\sqrt{\variance[X]}}
\end{equation}
then in the form of $Z = bX+a$,
\begin{equation}
  b = \frac{\beta}{\sqrt{\alpha(\alpha+1)\lambda}}
\end{equation}
and
\begin{equation}
  a = -\sqrt{\frac{\alpha\lambda}{\alpha+1}}
  \ .
\end{equation}

The m.g.f.~of $Z$ is
\begin{align}
  M_Z(\theta)&=\expectation\left[\euler^{Z\theta}\right]
  \nonumber \\
  &=\expectation\left[\euler^{(bX+a)\theta}\right]
  \nonumber \\
  &=\euler^{a\theta}M_X(b\theta)
  \ .
\end{align}
Substituting in $a$ and $b$
\begin{align}
  M_Z(\theta)&=
  \exp\left(
      -\theta\sqrt{\frac{\alpha\lambda}{\alpha+1}}
    \right)
  \exp\left[
    \lambda
    \left(
      \left(
        \frac{\beta}{\beta-\frac{\beta\theta}{\sqrt{\alpha(\alpha+1)\lambda}}}
      \right)^\alpha
      -1
    \right)
  \right]
  \nonumber\\
  &=
  \exp\left(
      -\theta\sqrt{\frac{\alpha\lambda}{\alpha+1}}
    \right)
  \exp\left[
    \lambda
    \left(
      \left(
        \frac{\sqrt{\alpha(\alpha+1)\lambda}}{\sqrt{\alpha(\alpha+1)\lambda}-\theta}
      \right)^\alpha
      -1
    \right)
  \right]
  \nonumber\\
  &=
  \exp\left(
      -\theta\sqrt{\frac{\alpha\lambda}{\alpha+1}}
    \right)
  \exp\left[
    \lambda
    \left(
      \left(
        1-\frac{\theta}{\sqrt{\alpha(\alpha+1)\lambda}}
      \right)^{-\alpha}
      -1
    \right)
  \right]
  \ .
\end{align}

Using the binomial expansion
\begin{multline}
  \left(
    1-\frac{\theta}{\sqrt{\alpha(\alpha+1)\lambda}}
  \right)^{-\alpha}
  =
  1+
  \sum_{r=1}^\infty \frac{\theta^r\prod_{s=1}^r(\alpha+s-1)}{(\alpha(\alpha+1)\lambda)^{r/2}r!}
  \\
  \text{for }\frac{\|\theta\|}{\sqrt{\alpha(\alpha+1)\lambda}}<1 \ .
\end{multline}
obtains
\begin{equation}
  M_Z(\theta)=
  \exp\left(
      -\theta\sqrt{\frac{\alpha\lambda}{\alpha+1}}
    \right)
  \exp\left[
    \lambda
    \sum_{r=1}^\infty \frac{\theta^r\prod_{s=1}^r(\alpha+s-1)}{(\alpha(\alpha+1)\lambda)^{r/2}r!}
  \right]
  \ .
\end{equation}
Writing in full the $r=1,2$ terms
\begin{equation*}
  M_Z(\theta)=
  \exp\left(
      -\theta\sqrt{\frac{\alpha\lambda}{\alpha+1}}
    \right)
  \exp\left[
    \theta\sqrt{\frac{\alpha\lambda}{\alpha+1}}
    +\frac{\theta^2}{2}
    +\lambda\sum_{r=3}^\infty \frac{\theta^r\prod_{s=1}^r(\alpha+s-1)}{(\alpha(\alpha+1)\lambda)^{r/2}r!}
  \right]
\end{equation*}
and a term cancels out
\begin{equation}
  M_Z(\theta)=
  \exp\left[
    \frac{\theta^2}{2}
    +\sum_{r=3}^\infty
    \frac
      {\theta^r\prod_{s=1}^r(\alpha+s-1)}
      {(\alpha(\alpha+1))^{r/2}r!}
    \lambda^{1-r/2}
  \right]
  \ .
\end{equation}

For large $\lambda$
\begin{equation}
  \lim_{\lambda\rightarrow\infty}M_Z(\theta)=\exp\left[\frac{\theta^2}{2}\right]
\end{equation}
which is the same as the m.g.f.~of a standard Normal distribution, therefore
\begin{equation}
  \lim_{\lambda\rightarrow\infty}Z\sim\normal(0,1) \ .
\end{equation}
This should make sense as for high $\lambda$, the Poisson random variable has a high expectation, increasing the number of gamma random variables in a summation. Increasing the number of terms in a summation will trigger the central limit theorem.

For high $\alpha$
\begin{align}
  \lim_{\alpha\rightarrow\infty}
  \frac{
    \prod_{s=1}^r(\alpha+s-1)
  }
  {
    (\alpha(\alpha+1))^{r/2}
  }
  &=
  \lim_{\alpha\rightarrow\infty}
  \frac{
    \prod_{s=1}^r\alpha
  }
  {
    \alpha^{r}
  }
  \nonumber\\
  &= 1
  \ .
\end{align}
As a result
\begin{equation}
  \lim_{\alpha\rightarrow\infty}
  M_Z(\theta)=
  \exp\left[
    \frac{\theta^2}{2}
    +\sum_{r=3}^\infty
    \frac{\theta^r}{r!}
    \lambda^{1-r/2}
  \right]
  \ .
\end{equation}
This shows that taking the limit $\alpha\rightarrow\infty$ is not enough to get a Normal limiting distribution. The limit must be accompanied with the limit $\lambda\rightarrow\infty$ to obtain
\begin{equation}
  \lim_{\lambda\rightarrow\infty}\lim_{\alpha\rightarrow\infty}Z\sim\normal(0,1)
  \ .
\end{equation}

It should be noted that $M_Z(\theta)$ is independent of $\beta$. Thus $\beta$ does not affect the limiting distribution.

The above results justify the use of the approximation
\begin{equation}
  X\sim\normal\left(\frac{\lambda\alpha}{\beta},\frac{\lambda\alpha(\alpha+1)}{\beta^2}\right)
\end{equation}
for large $\lambda$. The limiting case where $\lambda\rightarrow 0$, $\alpha\rightarrow 0$ and $\beta\rightarrow 0$ will not be discussed here.

\section{Saddlepoint Approximation}
\label{chapter:appendix_saddlepoint}
Let $X\sim\CPoisson(\lambda,\alpha,\beta)$ with m.g.f.~$M_X(\theta)=\exp\left[\lambda\left(\left(\frac{\beta}{\beta-\theta}\right)^{\alpha}-1\right)\right]$. Let the cumulant generating function of $X$ be $K_X(\theta)=\ln M_X(\theta)$. The saddlepoint approximation is
\begin{equation}
  p_X(x)\approx\left(2\pi K_X''(s)\right)^{-1/2}\exp\left[K_X(s)-sx\right]
  \label{eq:appendix_saddlepoint}
\end{equation}
where $s=s(x)$ is the solution to the saddle point equation $K_X'(s)=x$.

The cumulant generating function is
\begin{equation}
  K_X(\theta) = \lambda
  \left[
    \left(\frac{\beta}{\beta-\theta}\right)^\alpha-1
  \right]
  \ .
  \label{eq:appendix_cgf}
\end{equation}
Taking the derivative with respect to $\theta$
\begin{equation}
  K_X'(\theta)=\frac{\lambda\alpha\beta^\alpha}{(\beta-\theta)^{\alpha+1}}
\end{equation}
and this is known as the saddlepoint equation. The quantity $s=s(x)$ is the solution to the equation $K_X'(s)=x$, that is
\begin{equation*}
  \frac{\lambda\alpha\beta^\alpha}{(\beta-s)^{\alpha+1}} = x
\end{equation*}
with solution
\begin{equation}
  s = \beta - \left(\frac{\lambda\alpha\beta^\alpha}{x}\right)^{\frac{1}{\alpha+1}}
  \ .
  \label{eq:appendix_saddlepointSolution}
\end{equation}
The second-order derivative of the cumulant generating function is
\begin{equation}
  K_X''(\theta)=\frac{\lambda\alpha(\alpha+1)\beta^\alpha}{(\beta-\theta)^{\alpha+2}} \ .
  \label{eq:appenfix_2nddiffcgf}
\end{equation}

Substituting Equations \eqref{eq:appendix_cgf} and \eqref{eq:appenfix_2nddiffcgf} into Equation \eqref{eq:appendix_saddlepoint}
\begin{equation*}
  p_X(x)\approx
  \frac{1}{\sqrt{2\pi}}
  \left[
    \frac{
      (\beta-s)^{\alpha+2}
    }
    {
      \lambda\alpha(\alpha+1)\beta^\alpha
    }
  \right]^{1/2}
  \exp\left[
    \lambda\left(\left(\frac{\beta}{\beta-s}\right)^\alpha-1\right)-sx
  \right]
  \ .
\end{equation*}
Substituting in Equation \eqref{eq:appendix_saddlepointSolution}
\begin{multline*}
  p_X(x)\approx
  \frac{1}{\sqrt{2\pi}}
  \left[
    \frac{\left(\beta-\beta+\left(\frac{\lambda\alpha\beta^\alpha}{x}\right)^{\frac{1}{\alpha+1}}\right)^{\alpha+2}}{\lambda\alpha(\alpha+1)\beta^\alpha}
  \right]^{1/2}
  \\
  \exp\left[
    \lambda
    \left(
      \left(
        \frac{\beta}{\beta-\beta+\left(\frac{\lambda\alpha\beta^\alpha}{x}\right)^{\frac{1}{\alpha+1}}}
      \right)^\alpha
      -1
    \right)
    -x\left(
      \beta-\left(\frac{\lambda\alpha\beta^\alpha}{x}\right)^{\frac{1}{\alpha+1}}
    \right)
  \right]
\end{multline*}
simplifying further
\begin{align*}
  p_X(x)&\approx
  \begin{multlined}[t]
    \frac{1}{\sqrt{2\pi}}
    \left[
      \frac{
        \left(\frac{\lambda\alpha\beta^\alpha}{x}\right)^{\frac{\alpha+2}{\alpha+1}}
      }
      {
        \lambda\alpha(\alpha+1)\beta^\alpha
      }
    \right]^{1/2}
    \\
    \exp\left[
      \lambda
      \left(
        \beta^\alpha
        \left(\frac{x}{\lambda\alpha\beta^\alpha}\right)^{\frac{\alpha}{\alpha+1}}
        -1
      \right)
      -x\beta
      +(\lambda\alpha\beta^\alpha)^{\frac{1}{\alpha+1}}x^{1-\frac{1}{\alpha+1}}
    \right]
  \end{multlined}
  \\
  &\approx
  \begin{multlined}[t]
    \frac{1}{\sqrt{2\pi(\alpha+1)}}x^{-\frac{\alpha+2}{2(\alpha+1)}}
    \left(\lambda\alpha\beta^\alpha\right)^{\left(\frac{\alpha+2}{\alpha+1}-1\right)/2}
    \\
    \exp\left[
      \lambda\left(
        \beta^\alpha
        \left(\frac{x}{\lambda\alpha\beta^\alpha}\right)^{\frac{\alpha}{\alpha+1}}
        -1
      \right)
      -x\beta
      +(\lambda\alpha\beta^\alpha)^{\frac{1}{\alpha+1}}x^{\frac{\alpha}{\alpha+1}}
    \right]
  \end{multlined}
  \\
  &\approx
  \begin{multlined}[t]
    \frac{
      \left(
        \lambda\alpha\beta^\alpha
      \right)^{\frac{1}{2(\alpha+1)}}
    }
    {
      \sqrt{2\pi(\alpha+1)}}x^{-\frac{\alpha+2}{2(\alpha+1)}
    }
    \\
    \exp\left[
      -\lambda
      -x\beta
      +x^{\frac{\alpha}{\alpha+1}}
      \left(
        \frac{
          \lambda\beta^\alpha
        }
        {
          \left(\lambda\alpha\beta^\alpha\right)^{\frac{\alpha}{\alpha+1}}
        }
        +
        \left(
          \lambda\alpha\beta^\alpha
        \right)^{\frac{1}{\alpha+1}}
      \right)
    \right]
  \end{multlined}
  \\
  &\approx
  \begin{multlined}[t]
    \frac{
      \left(
        \lambda\alpha\beta^\alpha
      \right)^{\frac{1}{2(\alpha+1)}}\euler^{-\lambda}
    }
    {
      \sqrt{2\pi(\alpha+1)}}x^{-\frac{\alpha+2}{2(\alpha+1)}
    }
    \euler^{-x\beta}
    \\
    \exp\left[
      x^{\frac{\alpha}{\alpha+1}}
      \left(
        (\lambda\beta^\alpha)^{1-\frac{\alpha}{\alpha+1}}\alpha^{-\frac{\alpha}{\alpha+1}}
        +
        \left(
          \lambda\alpha\beta^\alpha
        \right)^{\frac{1}{\alpha+1}}
      \right)
    \right]
  \end{multlined}
  \\
  &\approx
  \begin{multlined}[t]
    \frac{
      \left(\lambda\alpha\beta^\alpha\right)^{\frac{1}{2(\alpha+1)}}\euler^{-\lambda}
    }
    {
      \sqrt{2\pi(\alpha+1)}
    }
    x^{-\frac{\alpha+2}{2(\alpha+1)}}\euler^{-x\beta}
    \\
    \exp\left[
      x^{\frac{\alpha}{\alpha+1}}
      \left(
        (\lambda\beta^\alpha)^{\frac{1}{\alpha+1}}
        \alpha^{-\frac{\alpha}{\alpha+1}}
        +
        \left(\lambda\alpha\beta^\alpha\right)^{\frac{1}{\alpha+1}}
      \right)
    \right]
  \end{multlined}
  \\
  &\approx
  \begin{multlined}[t]
    \frac{
      \left(\lambda\alpha\beta^\alpha\right)^{\frac{1}{2(\alpha+1)}}\euler^{-\lambda}
    }
    {
      \sqrt{2\pi(\alpha+1)}
    }
    x^{-\frac{\alpha+2}{2(\alpha+1)}}\euler^{-x\beta}
    \\
    \exp\left[
      x^{\frac{\alpha}{\alpha+1}}(\lambda\beta^\alpha)^{\frac{1}{\alpha+1}}
      \left(
        \alpha^{-\frac{\alpha}{\alpha+1}}+\alpha^{\frac{1}{\alpha+1}}
      \right)
    \right]
    \ .
  \end{multlined}
\end{align*}

The expression $\alpha^{-\frac{\alpha}{\alpha+1}}+\alpha^{\frac{1}{\alpha+1}}$ can be simplified by putting the two terms over a common denominator
\begin{align}
  \alpha^{-\frac{\alpha}{\alpha+1}}+\alpha^{\frac{1}{\alpha+1}}
  & = \alpha^{\frac{1}{\alpha+1}}+\frac{1}{\alpha^{\frac{\alpha}{\alpha+1}}}
  \nonumber\\
  & = \frac{\alpha^{\frac{1}{\alpha+1}}\alpha^{\frac{\alpha}{\alpha+1}}+1}{\alpha^{\frac{\alpha}{\alpha+1}}}
  \nonumber\\
  & = \frac{\alpha+1}{\alpha^{\frac{\alpha}{\alpha+1}}}
\end{align}
so that the saddlepoint approximation is
\begin{equation}
p_X(x)\approx
  \frac{
    \left(\lambda\alpha\beta^\alpha\right)^{\frac{1}{2(\alpha+1)}}\euler^{-\lambda}
  }
  {
    \sqrt{2\pi(\alpha+1)}
  }
  x^{-\frac{\alpha+2}{2(\alpha+1)}}
  \euler^{-x\beta}
  \exp\left[
    x^{\frac{\alpha}{\alpha+1}}
    \frac{(\lambda\beta^\alpha)^{\frac{1}{\alpha+1}}(\alpha+1)}{\alpha^{\frac{\alpha}{\alpha+1}}}
  \right]
  \ .
\end{equation}

\section{Series Evaluation}
\label{chapter:appendix_compoundPoissonSeries}

Let $X\sim\CPoisson(\lambda,\alpha,\beta)$ with p.d.f.
\begin{equation*}
  p_X(x) =
  \begin{cases}
    \delta(x) \exp
      \left[
          -\dfrac{\mu^{2-p}}{\phi(2-p)}
      \right]
    &\text{ for } x=0
    \\
    \displaystyle
    \exp\left[
      \frac{1}{\phi}
      \left(
        x\frac{\mu^{1-p}}{1-p}-\frac{\mu^{2-p}}{2-p}
      \right)
    \right]
    \frac{1}{x}
    \sum_{y=1}^{\infty}W_y(x,p,\phi)
    &\text{ for } x>0
  \end{cases}
\end{equation*}
where $p=\dfrac{2+\alpha}{1+\alpha}$, $\mu=\dfrac{\lambda\alpha}{\beta}$, $\phi = \dfrac{\alpha+1}{\beta^{2-p}(\lambda\alpha)^{p-1}}$ and
\begin{equation}
  W_y = W_y(x,p,\phi)=\frac{x^{y\alpha}}{\phi^{y(1+\alpha)}(p-1)^{y\alpha}(2-p)^yy!\Gamma(y\alpha)}
  \ .
\end{equation}
\cite{dunn2005series} approximated the sum by truncation
\begin{equation}
  \sum_{y=1}^\infty W_y \approx \sum_{y=y_\text{l}}^{y_\text{u}}W_y
\end{equation}
where $y_\text{l}<y_{\text{max}}<y_\text{u}$ and $y_{\text{max}}$ is the value of $y$ which maximises $W_y$. \cite{dunn2005series} treated $W_y$ as a continuous and differentiable function of $y$.

It is easier to differentiate $\ln(W_y)$ where
\begin{align}
  \ln(W_y) &=
  \begin{multlined}[t]
    y\alpha\ln(x)-y(1+\alpha)\ln(\phi)-y\alpha\ln(p-1)\\-y\ln(2-p)-\ln(y!)-\ln\Gamma(y\alpha)
  \end{multlined}
  \nonumber\\
  &=
  y
  \ln\left(
    \frac{x^\alpha}{\phi^{1+\alpha}(p-1)^\alpha(2-p)}
  \right)
  -\ln(y!)-\ln\Gamma(y\alpha)
  \ .
\end{align}
Using Stirling's approximation $\ln(n!)\approx\ln\Gamma(n)\approx n\ln(n!)-n$
\begin{equation}
  \ln(W_y) \approx
  y\ln\left(
    \frac{x^\alpha}{\phi^{1+\alpha}(p-1)^\alpha(2-p)}
  \right)
  -y\ln(y)+y-y\alpha\ln(y\alpha) + y\alpha
  \ .
\end{equation}
Taking the derivative with respect to $y$
\begin{align}
  \frac{\partial \ln(W_y)}{\partial y} &\approx
  \begin{multlined}[t]
    \ln\left(
      \frac{x^\alpha}{\phi^{1+\alpha}(p-1)^\alpha(2-p)}
    \right)
    -\ln(y)-1+1
    \\
    -\alpha\ln(y\alpha)-\alpha+\alpha
  \end{multlined}
  \nonumber\\
  &\approx
  \ln\left(
    \frac{x^\alpha}{\phi^{1+\alpha}(p-1)^\alpha(2-p)}
  \right)
  -\ln(y)
  -\alpha\ln(y\alpha)
  \ .
\end{align}

Setting the derivative to zero
\begin{equation*}
  0 \approx \ln\left(
    \frac{
      x^\alpha
    }
    {
      \phi^{1+\alpha}(p-1)^\alpha(2-p)y_{\text{max}}^{1+\alpha}\alpha^\alpha
    }
  \right)
\end{equation*}
\begin{equation*}
  1 \approx
  \frac{x^\alpha}{\phi^{1+\alpha}(p-1)^\alpha(2-p)y_{\text{max}}^{1+\alpha}\alpha^\alpha}
\end{equation*}
\begin{equation*}
  y_{\text{max}}^{1+\alpha} \approx
  \frac{x^\alpha}{\phi^{1+\alpha}(p-1)^\alpha(2-p)\alpha^\alpha}
\end{equation*}
\begin{equation*}
  y_{\text{max}} \approx
  \frac{1}{\phi}
  \left(
    \frac{x}{(p-1)\alpha}
  \right)^{\frac{\alpha}{1+\alpha}}
  (2-p)^{\frac{-1}{1+\alpha}}
  \ .
\end{equation*}
This can be simplified using the fact that $\alpha=\frac{2-p}{p-1}$, $\frac{1}{1+\alpha} = p-1$ and $\frac{\alpha}{1+\alpha} = 2-p$ then
\begin{equation*}
  y_{\text{max}} \approx
  \frac{1}{\phi}
  \left(
    \frac{x}{2-p}
  \right)^{2-p}
  (2-p)^{1-p}
\end{equation*}
and finally
\begin{equation}
  y_{\text{max}} \approx \frac{x^{2-p}}{\phi(2-p)}
  \ .
\end{equation}

To verify that $y_\text{max}$ is a maximum, the second-order derivative can be investigated
\begin{equation}
  \frac{\partial^2\ln(W_y)}{\partial y^2}
  \approx
  -\frac{1}{y}(\alpha+1)
\end{equation}
to see that
\begin{equation}
  \frac{\partial^2\ln(W_y)}{\partial y^2} < 0 \quad \text{for }y=1,2,3,\ldots
\end{equation}
therefore $y_\text{max}$ is a maximum.

\chapter{Expectation of $\widehat{\sigma}^2_0$}
\label{chapter:appendix_expectationNullStdEstimator}

It can be shown that the null variance estimator $\widehat{\sigma}_0^2$ is approximately unbiased when estimating using $n$ samples of $\normal(\mu_0, \sigma_0^2)$ random variables. Suppose the random variables are $Z_1, Z_2, \ldots, Z_n$ and are i.i.d.~with probability density function $f(z)$. Recall that
\begin{equation}
  \widehat{\sigma}_0 = \left[
    \left.
      -\dfrac{\partial^2}{\partial z^2}\ln\widehat{p}(z)
    \right|_{z=\widehat{\mu}_0}
  \right]^{-1/2}
\end{equation}
where
\begin{multline}
  \dfrac{
    \partial^2
  }
  {
    \partial z^2
  }
  \ln\widehat{p}_Z(z)
  =
  \left[
    h\sum_{i=1}^n
    \phi\left(
      \dfrac{
        Z_i-z
      }
      {
        h
      }
    \right)
  \right]^{-2}
  \times
  \left\{
    \left[
      \sum_{i=1}^n
      \phi\left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)
    \right]
  \right.
  \\
  \left.
    \times
    \left[
      \sum_{i=1}^n
      \phi\left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)
      \left(
        \left(
          \dfrac{
            Z_i-z
          }
          {
            h
          }
        \right)^2
        -1
      \right)
    \right]
    -
    \left[
      \sum_{i=1}^n
      \phi\left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)
    \right.
  \right.
  \\
  \left.
    \left.
      \left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)
    \right]^2
  \right\}
  \ .
\end{multline}
Apply an approximation such that
\begin{multline}
  \expectation\left[-\widehat{\sigma}_0^{-2}\right]
  \approx
  \left[
    h\sum_{i=1}^n
    \expectation\left[\phi\left(
      \dfrac{
        Z_i-z
      }
      {
        h
      }
    \right)\right]
  \right]^{-2}
  \times
  \left\{
    \left[
      \sum_{i=1}^n
      \expectation\left[
      \phi\left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)\right]
    \right]
  \right.
  \\
  \left.
    \times
    \left[
      \sum_{i=1}^n
      \expectation\left[
      \phi\left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)
      \left(
        \left(
          \dfrac{
            Z_i-z
          }
          {
            h
          }
        \right)^2
        -1
      \right)
    \right]\right]
    -
    \left[
      \sum_{i=1}^n
      \expectation\left[\phi\left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)
    \right.\right.
  \right.
  \\
  \left.
    \left.\left.
      \left(
        \dfrac{
          Z_i-z
        }
        {
          h
        }
      \right)\right]
    \right]^2
  \right\}
  \ .
  \label{eq:appendix_expectationNullStd}
\end{multline}

The following function of $Z$ was studied
\begin{equation}
g_t(Z) = \phi\left(
  \dfrac{Z - z_0}{h}
\right)
\left(
  \dfrac{Z - z_0}{h}
\right)^t
\ .
\end{equation}
The expectation is
\begin{equation}
\expectation\left[g_t(Z)\right] =
\int_{-\infty}^{\infty}
\phi\left(
  \dfrac{z - z_0}{h}
\right)
\left(
  \dfrac{z - z_0}{h}
\right)^t
f(z) \diff z \ .
\end{equation}
By substituting $u=(z-z_0)/h$ then
\begin{equation}
\expectation\left[g_t(Z)\right] =
\int_{-\infty}^{\infty}
h\phi(u)u^tf(z_0+uh)\diff u
\end{equation}
which a Taylor series can be used to expand $f(z_0+uh)$
\begin{equation}
\expectation\left[g_t(Z)\right] =
\int_{-\infty}^{\infty}
h\phi(u)u^t
\sum_{r=0}^\infty \dfrac{
  f^{(r)}(z_0)
  }
  {
  r!
  }
(uh)^r
\diff u
\end{equation}
to get
\begin{equation}
\expectation\left[g_t(Z)\right] =
\sum_{r=0}^\infty
\dfrac{
  f^{(r)}(z_0)h^{r+1}
}
{
  r!
}
M^{(t+r)}(0)
\end{equation}
where $M(\theta)=\int_{-\infty}^{\infty}\euler^{\theta u}\phi(u)\diff u$ is the moment generating function of the standard Normal distribution. It is left as an exercise to show that
\begin{align}
M(0) &= 1
\\
M^{(1)}(0) & = 0
\\
M^{(2)}(0) & = 1
\\
M^{(3)}(0) & = 0
\\
M^{(4)}(0) & = 3
\\
M^{(5)}(0) & = 0
\\
M^{(6)}(0) & = 15 \ .
\end{align}
Then for $t=0,1,2$
\begin{align}
\expectation\left[g_0(Z)\right]
&=
f(z_0)h
+\dfrac{f^{(2)}(z_0)h^3}{2}
+\dfrac{3f^{(4)}(z_0)h^5}{4!}
+O(h^7)
\label{eq:appendix_expectation1}\\
\expectation\left[g_1(Z)\right]
&=
f^{(1)}(z_0)h^2
+\dfrac{3f^{(3)}(z_0)h^4}{3!}
+O(h^6)
\label{eq:appendix_expectation2}\\
\expectation\left[g_2(Z)\right]
&=
f(z_0)h
+\dfrac{3f^{(2)}(z_0)h^3}{2}
+\dfrac{15f^{(4)}(z_0)h^5}{4!}
+O(h^7)
\label{eq:appendix_expectation3}\ .
\end{align}
The derivatives of $f(z)$ are
\begin{align}
f^{(1)}(z) &= \dfrac{f(z)}{\sigma_0^2}\left[-(z-\mu_0)\right]
\\
f^{(2)}(z)&=\dfrac{f(z)}{\sigma_0^2}\left[
  \dfrac{(z-\mu_0)^2}{\sigma_0^2}-1
\right]
\\
f^{(3)}(z) &= \dfrac{f(z)}{\sigma_0^4}
\left[
3(z-\mu_0)-\dfrac{(z-\mu_0)^3}{\sigma_0^2}
\right]
\\
f^{(4)}(z) &= \dfrac{f(z)}{\sigma_0^4}
\left[
  3 - \dfrac{6(z-\mu_0)^2}{\sigma_0^2} + \dfrac{(z-\mu_0)^4}{\sigma_0^4}
\right] \ .
\end{align}
Then substituting these into Equations \eqref{eq:appendix_expectation1}, \eqref{eq:appendix_expectation2} and \eqref{eq:appendix_expectation3} obtains
\begin{multline}
\expectation\left[g_0(Z)\right]
=
f(z_0)h
+
\dfrac{h^3}{2} \dfrac{f(z_0)}{\sigma_0^2}\left[
  \dfrac{(z_0-\mu_0)^2}{\sigma_0^2}-1
\right]
\\ +
\dfrac{3h^5}{4!} \dfrac{f(z_0)}{\sigma_0^4}
\left[
  3 - \dfrac{6(z_0-\mu_0)^2}{\sigma_0^2} + \dfrac{(z_0-\mu_0)^4}{\sigma_0^4}
\right]
+O(h^7)
\label{eq:appendix_expectationExpand1}
\end{multline}
\begin{multline}
\expectation\left[g_1(Z)\right]
=
-h^2\dfrac{f(z_0)}{\sigma_0^2}(z_0-\mu_0)
+
\dfrac{h^4}{2} \dfrac{f(z_0)}{\sigma_0^4}
\left[
3(z_0-\mu_0)-\dfrac{(z_0-\mu_0)^3}{\sigma_0^2}
\right]
\\
+O(h^6)
\label{eq:appendix_expectationExpand2}
\end{multline}
\begin{multline}
\expectation\left[g_2(Z)\right]
=
f(z_0)h
+
\dfrac{3h^3}{2} \dfrac{f(z_0)}{\sigma_0^2}\left[
  \dfrac{(z_0-\mu_0)^2}{\sigma_0^2}-1
\right]
\\ +
\dfrac{15h^5}{4!} \dfrac{f(z_0)}{\sigma_0^4}
\left[
  3 - \dfrac{6(z_0-\mu_0)^2}{\sigma_0^2} + \dfrac{(z_0-\mu_0)^4}{\sigma_0^4}
\right]
+O(h^7) \ .
\label{eq:appendix_expectationExpand3}
\end{multline}

The definition for $\expectation\left[g_t(Z)\right]$ can be used to simplify Equation \eqref{eq:appendix_expectationNullStd} to
\begin{equation}
  \expectation\left[\widehat{\sigma}_0^{2}\right]
  =
  \dfrac{
    -\left(h\expectation\left[g_0(Z)\right]\right)^2
  }
  {
    \expectation\left[g_0(Z)\right]\left[
      \expectation\left[g_2(Z)\right] - \expectation\left[g_0(Z)\right]
    \right]
    - \left(\expectation\left[g_1(Z)\right]\right)^2
  }
  \ .
\end{equation}
Substituting in Equations \eqref{eq:appendix_expectationExpand1}, \eqref{eq:appendix_expectationExpand2} and \eqref{eq:appendix_expectationExpand3}  and setting $z_0 = \widehat{\mu}_0$ obtains
\begin{equation}
\expectation\left[\widehat{\sigma}_0^{2}\right]
=
\dfrac{
  \sigma_0^2+h^2\left[\dfrac{(\widehat{\mu}_0-\mu_0)^2}{\sigma_0^2}-1\right]
  +O(h^4)
}
{
  1 + h^2\left[\dfrac{(\widehat{\mu}_0-\mu_0)^2}{\sigma_0^4}-\dfrac{2}{\sigma_0^2}\right]
  +O(h^4)
}
\end{equation}
where the following may be useful
\begin{multline}
\expectation\left[g_0(Z)\right]\left[
    \expectation\left[g_2(Z)\right] - \expectation\left[g_0(Z)\right]
  \right]
=
h^4\dfrac{f^2(z_0)}{\sigma_0^2}\left[
  \dfrac{(z_0-\mu_0)^2}{\sigma_0^2}-1
\right]
\\
+h^6\dfrac{f^2(z_0)}{\sigma_0^4}\left[
  2 - \dfrac{4(z_0-\mu_0)^2}{\sigma_0^2} + \dfrac{(z_0-\mu_0)^4}{\sigma_0^4}
\right]
+O(h^8)
\ .
\end{multline}
The result implies that $\widehat{\sigma}_0^{2}$ is an unbiased estimator of $\sigma_0^{2}$ up to the first-order approximation. Higher-order terms consist of at least even-order polynomials of $(\widehat{\mu}_0-\mu_0)$ and $h$. As a result, any bias in $\widehat{\mu}_0$ would contribute to the bias of $\widehat{\sigma}_0^{2}$. It is not clear if $\widehat{\sigma}_0^{2}$ is a consistent estimator because $h$ depends on $n$ and the result for $\variance\left[\widehat{\sigma}_0^{2}\right]$ is difficult to obtain.


% -*- latex -*-
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%
%%%% This text file is part of the source of
%%%% `Introduction to High-Performance Scientific Computing'
%%%% by Victor Eijkhout, copyright 2012-7
%%%%
%%%% This book is distributed under a Creative Commons Attribution 3.0
%%%% Unported (CC BY 3.0) license and made possible by funding from
%%%% The Saylor Foundation \url{http://www.saylor.org}.
%%%%
%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

MapReduce~\cite{Google:mapreduce} is a programming model for certain
parallel operations. One of its distinguishing characteristics is that
it is implemented using \indexterm{functional programming}.
The MapReduce model handles computations of the following form:
\begin{itemize}
\item For all available data, select items that satisfy a certain
  criterion;
\item and emit a key-value pair for them. This is the mapping stage.
\item Optionally there can be a combine/sort stage where all pairs with the
  same key value are grouped together.
\item Then do a global reduction on the keys, yielding one or more of
  the corresponding values. This is the reduction stage.
\end{itemize}

We will now give a few examples of using MapReduce, and present the
functional programming model that underlies the MapReduce abstraction.

\Level 2 {Expressive power of the MapReduce model}

The reduce part of the MapReduce model makes it a prime candidate for
computing global statistics on a dataset.
One example would be to count how many times each of a set of words
appears in some set of documents. The function being mapped knows the
set of words, and outputs for each document a pair of document name
and a list with the occurrence counts of the words. The reduction then
does a componentwise sum of the occurrence counts.

The combine stage of MapReduce makes it possible to transform data.
An example is a `Reverse Web-Link Graph': the map function
outputs target-source pairs for each link to a target URL found in
a page named "source". The reduce function concatenates the list of
all source URLs associated with a given target URL and emits the pair
target-list(source).

A less obvious example is computing PageRank
(section~\ref{sec:pagerank}) with MapReduce. Here we use the fact that
the PageRank computation relies on a distributed sparse matrix-vector
product. Each web page corresponds to a column of the web matrix~$W$;
given a probability $p_j$ of being on page~$j$, that page can then
compute tuples $\langle i,w_{ij}p_j\rangle$. The combine stage of MapReduce
then sums together $(Wp)_i=\sum_j w_{ij}p_j$.

Database operations can be implemented with MapReduce but since it has
a relatively large latency, it is unlikely to be competitive with
standalone databases, which are optimized for fast processing of a
single query, rather than bulk statistics. For other applications see
\url{http://horicky.blogspot.com/2010/08/designing-algorithmis-for-map-reduce.html}.

\Level 2 {Mapreduce software}

The implementation of MapReduce by Google was released under the name \indexterm{Hadoop}.
While it suited the Google model of single-stage reading and processing of data,
it had considerable disadvantages for many other users:
\begin{itemize}
\item Hadoop would flush all its data back to disc after each MapReduce cycle, so
  for operations that take more than a single cycle the file system
  and bandwidth demands are too great.
\item In computing center environments, where a user's data is not continuously online,
  the time required for loading data into \indexac{HDFS} would like overwhelm the
  actual analysis.
\end{itemize}
For these reasons, further projects such as Apache \indexterm{Spark} (\url{https://spark.apache.org/})
offer caching of data.

\Level 2 {\bf Functional programming}
\index{functional programming|(}
%
The mapping and reduction operations are easily implemented on any
type of parallel architecture, using a combination of threading and
message passing. However, at Google where this model was developed
traditional parallelism was not attractive for two reasons. First of
all, processors could fail during the computation, so a traditional
model of parallelism would have to be enhanced with \indexterm{fault
  tolerance} mechanisms. Secondly, the computing hardware could
already have a load, so parts of the computation may need to be
migrated, and in general any type of synchronization between tasks
would be very hard.

MapReduce is one way to abstract from such details of parallel
computing, namely through adopting a functional programming model. In
such a model the only operation is the evaluation of a function,
applied to some arguments, where the arguments are themselves the
result of a function application, and the result of the computation is
again used as argument for another function application. In
particular, in a strict functional model there are no variables, so
there is no static data.

A function application, written in \indexterm{Lisp} style as \n{(f a
  b)} (meaning that the function \n{f} is applied to arguments \n{a}
and \n{b}) would then be executed by collecting the inputs from
whereven they are to the processor that evaluates the
function~\n{f}. The mapping stage of a MapReduce process is denoted
\begin{verbatim}
  (map f (some list of arguments))
\end{verbatim}
and the result is a list of the function results of applying \n{f} to
the input list. All details of parallelism and of guaranteeing that
the computation successfully finishes are handled by the \n{map}
function.

Now we are only missing the reduction stage, which is just as simple:
\begin{verbatim}
(reduce g (map f (the list of inputs)))
\end{verbatim}
The \n{reduce} function takes a list of inputs and performs a
reduction on it.

The attractiveness of this functional model lies in the fact that
functions can not have \indexterm{side effects}: because they can only
yield a single output result, they can not change
their environment, and hence there is no coordination problem of
multiple tasks accessing the same data.

Thus, MapReduce is a useful
abstraction for programmers dealing with large amounts of data.
Of course, on an implementation level the MapReduce software uses
familiar concepts such as decomposing the data space, keeping a work
list, assigning tasks to processors, retrying failed operations, et
cetera.

\index{functional programming|)}

\endinput

\paragraph{{\bf Performance characteristics}}

MapReduce is sometimes mentioned as an alternative to databases: the
map and reduce functions can be used to form certain queries in the
data. However, MapReduce has a much higher latency than database
queries, which are typically optimized for minimum response time.