\lab{Image Compression (SVD)}{SVD}

\objective{Learn how to compute the compact SVD and explore the SVD as a method of image compression}
\label{lab:SVD}

The theoretical use of the \emph{Singular Value Decomposition} or \emph{SVD} has long been appreciated.
In fact, the idea of a canonical way of decomposing a matrix was so alluring that the SVD was independently discovered by at least four people through use of both  integral equations and systems of linear equations.

However, it wasn't until Erhard Schmidt showed how the SVD could be a computational tool for providing low-rank approximations that the practical applications became apparent.
Since Schmidt's work, further devolpments have confirmed the importance of the SVD in both computational and theoretical applications.

\begin{comment}

The \emph{Singular Value Decomposition} is so important that it was discovered multiple times by different people in different ways.
The foundation work in systems of linear equations was laid by Gauss and Cauchy in the 1820's and later by Jacobi in 1846 with his work with the LU decomposition; however, the real work with the SVD started with Eugenio Beltrami.
In a paper he published in 1873 he was the first to work with the SVD; he was limited to real, square, nonsingular matrices with distinct singular values.
A year later, Camille Jordan published his own independent work that was more rigourus and avoided some of the pitfalls of Beltrami's research.
Together, Beltrami and Jordan are considered the codiscoverers of the singular value decomposition.

Later, James Joseph Sylvester independently presented an iterative algorithm and a rule for carrying out the reduction in 1889.
This rule was essentially Beltrami's work.
However Sylvester sent the note detailing his rule to the same  journal Jordan had published in, showing not just his ignorance of Jordan's and Beltrami's works but his perception of the SVD's importance.

The SVD's concurrent and independant derivation shows its importance as a theoretical tool, but it wasn't until Erhard Schmidt discovered a computational use for the SVD that the practical applications became apparent.
Schmidt used integral equations rather than linear equations to derive the SVD, but
his most important contribution was showing how the SVD could be a computational tool to obtain optimal, low-rank approximations.

Since Schmidt's work, further devolpments have confirmed the importance of the SVD in both practical and theoretical applications.
\end{comment}

\subsection*{Computing the SVD}

The Singular Value Decomposition separates a $m \times n$ matrix $A$ into two unitary matrices and a diagonal matrix.
This takes the form of
\begin{equation*}
A = U \Sigma V^H
\end{equation*}
where $U$ and $V$ are square and unitary of sizes $m$ and $n$ respectively, and $\Sigma$ is diagonal and of size $m \times n$.
The values of $\Sigma$ are the \emph{singular values} of $A$, and are the square root of the eigenvalues of $A^HA$.
% if $A$ is square, are equal to the absolute value of the eigenvalues of $A$.
Commonly the singular values are listed in decreasing order. Thus we get

\begin{equation*}
\Sigma = \mbox{diag}(\sigma_1,\sigma_2,\ldots,\sigma_n)
\end{equation*}
where $\sigma_1 \geq \sigma_2 \geq \ldots \geq \sigma_n \geq 0$ are the singular values of $A$.

The \emph{compact SVD} is where the $r$ column vectors of $U$ and the $r$ row vectors of $V$ are calculated, cooresponding to the $r$ nonzero singular values.
We lose the decomposition of the nullspace of $A$, but can still regain the full matrix $A$.
It takes the form $A= U_1 \Sigma_1 V_1^H$ where $U_1$ is $m\times r$, $\Sigma_1$ is $r\times r$ and diagonal, and $V_1^H$ is $r\times n$.

The \emph{truncated SVD} is similar to the compact SVD, but instead of taking all the nonzero singular values, we only take the $k$ largest.
While this saves space, it means that we cannot recover the whole matrix.
Instead we have an approximation that shares the largest $k$ singular values.
We end up with $\widehat A = U_k\Sigma_k V_k^H$ where $\widehat A$ is an approxiamation of $A$, $U_k$ is $m\times k$, $\Sigma$ is $k \times k$ and diagonal, and $V_k^H$ is $k \times n$.

The only difference in computing the compact SVD verses the truncated SVD is the number of singular values we keep.
If we keep all the nonzero singular values, it is the compact SVD.
If we only keep some of the nonzero singular values, then it is the truncated SVD.

To compute the compact or truncated SVD:

\begin{itemize}
\item The singular values of $A$ are the square root of the eigenvalues of $A^HA$ and are sorted in descending order. These are the diagonal of $\Sigma$. For the compact SVD, keep all nonzero singular values. For the truncated SVD, keep the first $k$.
\item The columns of $V$ are the eigenvectors of $A^HA$, where the $i$th column matches the $i$th singular value.
\item The columns of $U$ are $U_i = \frac{1}{\sigma_i} AV_i$.
\end{itemize}

\begin{comment}
To calculate the SVD:
\begin{itemize}

\item Find $A^HA$

\item Find the eigenvalues and eigenvectors of $A^HA$.

\item Sort the eigenvalues in decreasing order.

\item The $i^{th}$ column of $V$, $V_i$, is the eigenvector corresponding to the $i^{th}$ eigenvalue.

\item Find the singular values by taking the square root of the eigenvalues.

\item The $i^{th}$ column of $U$ is $U_i = \frac{1}{\sigma_i} AV_i$.

\end{itemize}
\end{comment}

\begin{comment}
\begin{algorithm}
\begin{algorithmic}[1]
\caption{SVD for Full Rank Square Matrices.}
\label{Alg:svd_full}
\Procedure{SVD}{$A$}
\State $n \gets \shape{A}$
\State $eigs \gets$ eigenvalues$(A^HA)$
\State sort$(eigs)$
\State $V \gets$ eigenvectors$(A^HA)$
\State $svals \gets \sqrt{eigs}$
\State $U \gets \allocate{n,n}$
\For{$i=0\ldots n-1$}
      \State $U_i \gets  \frac{1}{svals[i]} AV_i$
\EndFor
\State $S \gets$ diag$(svals)$
\State \pseudoli{return} $U, S, V^H$
\EndProcedure
\end{algorithmic}
\end{algorithm}
\end{comment}

\begin{problem}
Write a function to calculate either the truncated SVD or the compact SVD.
Since the process is the same, we only need to write one function.

\begin{lstlisting}
def truncated_svd(A,r=None,tol=10**-6):
	``''''Computes the truncated SVD of A. If r is None or equals the number of nonzero singular values, it is the compact SVD.
	Parameters:
		A: the matrix
		r: the number of singular values to use
		tol: the tolerance for zero
	Returns:
		U - the matrix U in the SVD
		s - the diagonals of Sigma in the SVD
		Vh - the matrix V^H in the SVD
	``''''
	m,n = A.shape

	# Find the eigenvalues and eigenvectors of A^H A

	# Find the singular values

	# Sort the singular values and only keep the greatest r

	# Calculate V

	# Calculate U

	return U, s, Vh
\end{lstlisting}

Check your function by calculating the compact SVD and seeing if $U_1\Sigma_1 V_1^H = A$ using \li{np.close()}.
\label{prob:calc_svd}
\end{problem}

If $A$ is full rank and square, it has $n$ nonzero singular values and this process gives us the full SVD.
If $A$ is not full rank or not square, we can compute the first $r$ columns of $U$ and $V$, where $r$ is the number of nonzero singular values, in this way.
To find the rest of the SVD, we can use Gram-Schmidt orthonormalization.

More specifically, let $r$ be the number of nonzero singular values.
If $r<n$ then we only have $r$ eigenvectors to fill $V$, which is a $n\times n$ matrix.
We find the remaining columns of $V$ by using Gram-Schmidt orthonormalization to finish the basis for $n$-space.
Similarly, if $r<m$ then we only have $r$ columns of $U$ so we use Gram-Schmidt orthonormalization to finish the basis for $m$-space.
The compact SVD does not have this problem because $U_1$ and $V_1$ only have $r$ columns, the remaining columns are the decomposition of the null spcae of the matrix, and are not included.
In this lab we calculate the compact SVD for simplicity.

\subsection*{Visualizing the SVD}

Recall that a matrix is one way to express a linear transformation.
A $m\times n$ matrix sends points from $\mathbb{R}^n$ to $\mathbb{R}^m$.
These transformations are a mix of rotations and rescalings.
The SVD decomposes these transformations into their indivual parts.
$V^H$ is a rotation, $\Sigma$ a rescaling along the principal axes, and $U$ is another rotation.

\begin{figure}
\centering
\begin{subfigure}[b]{.49\textwidth}
  \centering
  \includegraphics[width=\textwidth]{unit_circle.pdf}
  \caption{The unit circle, $S$, with two unit vectors.}
\end{subfigure}
\begin{subfigure}[b]{.49\textwidth}
  \centering
  \includegraphics[width=\textwidth]{vcircle.pdf}
  \caption{$V^HS$}
  %\label{fig:svals_plot}
\end{subfigure}
\begin{subfigure}[b]{.49\textwidth}
  \centering
  \includegraphics[width=\textwidth]{svcircle.pdf}
  \caption{$\Sigma V^H S$}
  %\label{fig:svals_plot}
\end{subfigure}
\begin{subfigure}[b]{.49\textwidth}
  \centering
  \includegraphics[width=\textwidth]{full_transformation.pdf}
  \caption{$U \Sigma V^H S$}
  %\label{fig:svals_plot}
\end{subfigure}
\caption{Each step in transforming the unit circle and two unit vectors using the matrix $A$.}
\label{fig:sol1}
\end{figure}

\begin{problem}
Using either your SVD from problem \ref{prob:calc_svd} or from scipy.linalg.svd, plot each part of the transformation of the matrix
\begin{equation}
A =  \begin{pmatrix}3 & 1\\1 & 3\end{pmatrix}.
\end{equation}

If $S$ are the points on the unit circle, your solution should show $S$, $V^HS$, $\Sigma V^HS$, and $ U\Sigma V^HS$.
Since $A$ is square and full rank, the compact SVD is equal to the full SVD.
You solution should look like Figure \ref{fig:sol1}
\end{problem}

\subsection*{Image Data Compression}

In this lab, we explore how the SVD can be used to compress image data.
Recall that an image is simply a matrix where each position is the color value for the pixel in that position.
The SVD lets us choose how much information to keep, and what information is most important.
Larger eigenvalues correspond to columns of $U$ and $V$ that contain more information, while smaller eigenvalues correspond to less important columns.
This idea is used in many areas of applied mathematics including signal processing, statistics, semantic indexing (search engines), and control theory.

\subsection*{Low rank data storage}
If the rank of a given matrix is significantly smaller than its dimensions, the compact SVD offers a way to store $A$ with less memory.
Without the SVD, an $m\times n$ matrix requires storing $mn$ values.
By decomposing the original matrix into the compact SVD, $U_1$, $\Sigma_1$ and $V_1$ together require $mr+r+nr$ values.
Thus if $r$ is much smaller than both $m$ and $n$, we can obtain considerable efficiency.
For example, suppose $m=100$, $n=200$ and $r=20$.
Then the original matrix would require storing $20,000$ values whereas the compact SVD only requires storing $6020$ values.

\subsection*{Low rank approximation}
The truncated SVD is useful in approxiamating a matrix with one of lower rank.
By only keeping the first $k$ singular values, we can create an approximation $\widehat A = U_k\Sigma_k V_k^H$.

\begin{comment}
We can also calculate $\widehat A$ by finding the full SVD, and setting all singular values after the $k$th to zero.
Thus the modified $\Sigma$ would be
\begin{equation*}
\Sigma_{\widehat A} = \mbox{diag}(\sigma_1,\sigma_2,\ldots,\sigma_s,0,\ldots,0).
\end{equation*}
Multiplying this matrix with the original $U$ and $V^H$ will give the same $\widehat A$ that was found by computing the truncated SVD directly.
\end{comment}

The \li{scipy.linalg} module has a convenient method to calculate the SVD of a given matrix. We can use
this method to create a lower-rank approximation of a given matrix. Execute the following code.
\begin{lstlisting}
>>> import numpy as np
>>> from scipy.linalg import svd, norm
>>> A = np.array([[1,1,3,4], [5,4,3,7], [9,10,10,12], [13,14,15,16], [17,18,19,20]])
>>> U,s,Vt = svd(A, full_matrices=False)
\end{lstlisting}
In that last line of code, we included the keyword argument \li{full_matrices=False} to calculate the
compact SVD rather than the full SVD. The arrays \li{U} and \li{Vt} correspond to the matrices
$U_1$ and $V_1^H$ discussed earlier in the lab. The array \li{s} simply gives the nonzero singular values
of the matrix \li{A}, and we can find the rank of \li{A} by inspecting the number of entries in \li{s} (in this
example, we have a rank 4 matrix).

Next, we calculate a rank 3 approximation.
We omit the last singular value from the calculation along with the last column of \li{U} and last row of \li{Vt}.

\begin{lstlisting}
>>> S = np.diag(s[:-1])
>>> Ahat = U[:,:-1].dot(S).dot(Vt[:-1,:])
>>> norm(A-Ahat)
\end{lstlisting}
Note that $\widehat A$ is ``close'' to the original matrix $A$, but that its rank is 3 instead of 4. More
precisely, $\widehat A$ is the best rank 3 approximation of $A$ with respect to both the induced 2-norm and
the Frobenius norm.

\begin{problem}
Write a function \li{svd_approx} that takes as input a matrix $A$ and a positive integer $k$ and returns
the best rank $k$ approximation to $A$ with respect to the induced 2-norm.
\label{prob:svd_approx}
\end{problem}

\begin{comment}
The reduced form of the SVD also provides a way to approximate a matrix with another one of lower rank.
This idea is used in many areas of applied mathematics including signal processing, statistics, semantic indexing (search engines), and control theory.
If we are given a matrix $A$ of rank $r$, we can find an approximate matrix $\widehat A$ of rank $s<r$ by taking the SVD of $A$ and setting all of its singular values after $\sigma_s$ to zero, that is,
\begin{equation*}
\Sigma_{s} = \mbox{diag}(\sigma_1,\sigma_2,\ldots,\sigma_s,0,\ldots,0)
\end{equation*}
and then multiplying the matrix back together again.
The more singular values we keep, the closer our approximation is to $A$.
The number of singular values we decide to preserve depends on how close of an approximation we need and what our size requirements are for $U_1$, $\Sigma_{\widehat A}$, and $V_1$.
Try plotting the singular values.
\end{comment}

\subsection*{Application to Imaging}

In Figure \ref{fig:hubble} we present an image and a plot of its singular values.
Matrix rank is on the x-axis and the singular values are on the y-axis.
Note that the SVD orders the singular values from greatest to least.
The more singular values we keep, the closer the approximation but the more data we have to store.
By looking at the graph in Figure \ref{fig:svals_plot} we can have a rough idea of how many singular values we need to preserve to have a good approximation of $A$.
The matrix rank of the image is $670$.
However, we could easily approximate the image using only the first half of the singular values as the plot shows that all the values after are miniscule.

%The greatest singular values contribute most to the image while the smallest singular values hardly contribute anything to the final approximation. %said this earlier

In Figure \ref{fig:rankvalues}, we can see different rank approximations of the image in Figure \ref{fig:hubble}.

\begin{figure}
\centering
\begin{subfigure}[b]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{hubble_red}
\caption{NGC 3603 (Hubble Space Telescope).}
\end{subfigure}
\begin{subfigure}[b]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{hubble_svals}
\caption{Singular values from greatest to smallest.}
\label{fig:svals_plot}
\end{subfigure}
\caption{An image and its singular values.}
\label{fig:hubble}
\end{figure}

\begin{figure}
\centering
\begin{subfigure}[b]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{rank1.jpg}
\caption{Rank 1 approximation}
\end{subfigure}
\begin{subfigure}[b]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{rank14.jpg}
\caption{Rank 14 approximation}
\end{subfigure}

\begin{subfigure}[b]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{rank27.jpg}
\caption{Rank 27 approximation}
\end{subfigure}
\begin{subfigure}[b]{.49\textwidth}
\centering
\includegraphics[width=\textwidth]{rank40.jpg}
\caption{Rank 40 approximation}
\end{subfigure}
\caption{Different rank approximations for SVD based compression.  Notice that higher rank is needed to resolve finer detail.}
\label{fig:rankvalues}
\end{figure}

To read in a image, convert to balck and white, and show it use the code below.
Then use your function from Problem \ref{prob:svd_approx} to calculate an approximation.
%Enter the following into IPython (note that any image you might have will work):
\begin{lstlisting}
>>> import matplotlib.pyplot as plt
>>> X = plt.imread('fingerprint.png')[:,:,0].astype(float)
>>> X.nbytes      #number of bytes needed to store X
>>> plt.imshow(X)
>>> plt.show()
\end{lstlisting}
\begin{comment}
Computing the SVD of your image is simple.
Remember to make the singular values a diagonal matrix before multiplying.
\begin{lstlisting}
>>> U,s,Vt = svd(X, full_matrices=False)
>>> S = sp.diag(s)
\end{lstlisting}
In the next code block, $n$ represents the desired rank of the output.
\begin{lstlisting}
>>> n = 50
>>> u1, s1, vt1 = U[:,0:n], S[0:n,0:n], Vt[0:n,:]
>>> Xhat = u1.dot(s1).dot(vt1)
>>> (u1.nbytes + np.diag(s1).nbytes + vt1.nbytes) - X.nbytes   #should be negative
>>> plt.imshow(Xhat)
>>> plt.show()
\end{lstlisting}
\end{comment}

Recall that the error between the best rank $s$ approximation $\widehat{A_s}$ to $A$ with respect to the induced
2-norm is given by
$$
\|A - \widehat{A_s}\|_2 = \sigma_{s+1},
$$
where $\sigma_{s+1}$ is the $(s+1)$-th singular value of $A$.

\begin{problem}
Sometimes there is not enough available bandwidth to transmit a full resolution photograph.
You aim to reduce the amount of data that needs to be transmitted from a remote location such that loss of image detail is minimal, but the amount of data that needs to be sent has reduced as much as possible.
Write a function \li{lowest_rank_approx} that takes as input a matrix $A$ and a positive number $e$ and returns
the lowest rank approximation of $A$ with error less than $e$ (with respect to the induced 2-norm).
\end{problem}


\newpage
\onecolumn
\section{Proof of Lemma~\ref{lem:robustness}}
\label{sec:proof_robustness}
To prove Lemma~\ref{lem:robustness}, we first prove a more generalized version in Lemma~\ref{lem:robustness_general}. By restricting $f$ to be a one dimensional quadratics function, the generalized curvature $h_t$ itself is the only eigenvalue. We can prove Lemma~\ref{lem:robustness} as a straight-forward corollary. Lemma~\ref{lem:robustness_general} also implies, in the multiple dimensional correspondence of~\eqref{equ:one_dim_22_rec}, the spectral radius $\rho(\mat{A}_t)=\sqrt{\mu}$ if the curvature on all eigenvector directions (eigenvalue) satisfies~\eqref{eqn:robust_region}.

\begin{lemma}
\label{lem:robustness_general}
Let the gradients of a function $f$ be described by
\begin{equation}
	\nabla f(\mat{x}_t) = \mat{H}(\mat{x}_t) (\mat{x}_t - \mat{x}^*),
\end{equation}
with $\mat{H}(\bm{x}_t) \in \mathbb{R}^n \mapsto \mathbb{R}^{n\times n}$.
Then the momentum update can be expressed as a linear operator:
\begin{align}
{\begin{pmatrix}
\mat{y}_{t+1}\\
\mat{y}_t \\
\end{pmatrix}}
=
{\begin{pmatrix}
\mat{I}-\alpha \mat{H}(\mat{x}_t) + \mu \mat{I} & - \mu \mat{I} \\
\mat{I} & \mat{0} \\
\end{pmatrix}}
{\begin{pmatrix}
\mat{y}_t \\
\mat{y}_{t-1} \\
\end{pmatrix}}
=\mat{A}_t
{\begin{pmatrix}
\mat{y}_t \\
\mat{y}_{t-1} \\
\end{pmatrix}},
\end{align}
where $\mat{y}_t\triangleq \mat{x}_t - \mat{x}^*$.
Now, assume that the following condition holds for all eigenvalues $\lambda(\mat{H}(\bm{x}_t))$ of $\mat{H}(\bm{x}_t)$:
\begin{align}
{(1-\sqrt{\mu})^2\over \alpha} &\leq \lambda(\mat{H}(\bm{x}_t)) \leq {(1+\sqrt{\mu})^2\over \alpha}.
\label{equ:control_condition}
\end{align}
then the spectral radius of $\mat{A}_t$ is controlled by momentum with
$	\rho(\mat{A}_t) = \sqrt{\mu}.$

\begin{proof}
Let $\lambda_t$ be an eigenvalue of matrix $\mat{A}_t$, it gives
$\det\left(\mat{A}_t - \lambda_t \mat{I} \right) = 0$.
We define the blocks in $\mat{A}_t$ as $\mat{C} = \mat{I} - \alpha \mat{H}_t + \mu \mat{I} - \lambda_t \mat{I}$, $\mat{D} = -\mu \mat{I}$,
$\mat{E} = \mat{I}$ and $\mat{F} = -\lambda_t \mat{I}$ which gives
\[
\det \left( \mat{A}_t - \lambda_t \mat{I}\right) = \det{\mat{F}} \det{\left(\mat{C} - \mat{D} \mat{F}^{-1}
\mat{E} \right)} = 0
\]
assuming generally $\mat{F}$ is invertible. Note we use $\mat{H}_t\triangleq\mat{H}(\mat{x}_t)$ for simplicity in writing. The equation $\det{\left(\mat{C} - \mat{D} \mat{F}^{-1}
\mat{E} \right)} = 0$ implies that
\begin{equation}
\det \left( \lambda_t^2\mat{I} - \lambda_t \mat{M}_t + \mu \mat{I} \right) = 0
\label{equ:control_condition_2}
\end{equation}
with $\mat{M}_t = \left( \mat{I} - \alpha \mat{H}_t + \mu \mat{I} \right)$. In other words, $\lambda_t$ satisfied that $\lambda_t^2 - \lambda_t \lambda(\mat{M}_t) + \mu = 0$ with $\lambda(\mat{M}_t)$ being one eigenvalue of $\mat{M_t}$. I.e.
\begin{equation}
	\lambda_t = \frac{\lambda(\mat{M}_t) \pm \sqrt{\lambda(\mat{M}_t)^2 - 4\mu}}{2}
\end{equation}

On the other hand,~\eqref{equ:control_condition} guarantees that $(1 - \alpha \lambda(\mat{H}_t) + \mu)^2 \leq 4\mu$. We know both $\mat{H}_t$ and $\mat{I} - \alpha \mat{H}_t + \mu \mat{I}$ are symmetric. Thus for all eigenvalues $\lambda(\mat{M}_t)$ of $\mat{M}_t$, we have $\lambda(\mat{M}_t)^2 = (1 - \alpha \lambda(\mat{H}_t) + \mu)^2 \leq 4\mu$ which guarantees $| \lambda_t | = \sqrt{\mu}$ for all $\lambda_t$. As the spectral radius is equal to the magnitude of the largest eigenvalue of $\mat{A}_t$, we have the spectral radius of $\mat{A}_t$ being $\sqrt{\mu}$.

\end{proof}

\end{lemma}

\section{Proof of Lemma~\ref{lem:main_lemma}}
We first prove Lemma~\ref{lem:bias_rec} and Lemma~\ref{lem:var_rec} as preparation for the proof of Lemma~\ref{lem:main_lemma}. After the proof for one dimensional case, we discuss the trivial generalization to multiple dimensional case.
\begin{lemma}
\label{lem:bias_rec}
	Let the $h$ be the curvature of a one dimensional quadratic function $f$ and $\overline{x}_t = \E x_t$. We assume, without loss of generality, the optimum point of $f$ is $x^{\star}=0$. Then we have the following recurrence
	\begin{equation}
		\begin{pmatrix}
			\overline{x}_{t + 1} \\
			\overline{x}_t
		\end{pmatrix} =
		\begin{pmatrix}
			1-\alpha h + \mu & - \mu\\
			1 & 0 \\
		\end{pmatrix}^{t}
		\begin{pmatrix}
			x_1 \\
			x_0
		\end{pmatrix}
		\label{equ:bias_rec}
	\end{equation}
	\begin{proof}
		From the recurrence of momentum SGD, % in~\eqref{equ:momentum_sgd},
		we have
		\begin{equation*}
			\begin{aligned}
				\E x_{t + 1} = & \E [ x_{t} - \alpha \nabla f_{S_t} (x_t) + \mu (x_t - x_{t - 1} ) ]\\
							= & \E_{x_{t}}	[ x_{t} - \alpha \E_{S_t} \nabla f_{S_t} (x_t) + \mu (x_t - x_{t - 1} ) ] \\
							= & \E_{x_{t}}	[ x_{t} - \alpha h x_t + \mu (x_t - x_{t - 1} ) ] \\
							= & (1 - \alpha h + \mu)\overline{x}_t - \mu\overline{x}_{t - 1}
			\end{aligned}
		\end{equation*}
		By putting the equation in to matrix form,~\eqref{equ:bias_rec} is a straight-forward result from unrolling the recurrence for $t$ times. Note as we set $x_1 = x_0$ with no uncertainty in momentum SGD, we have $[\overline{x}_0, \overline{x}_1] = [x_0, x_1]$.
	\end{proof}
\end{lemma}

\begin{lemma}
\label{lem:var_rec}
	Let $U_t=\E ( x_t - \overline{x}_t ) ^2$ and $V_t= \E (x_t - \overline{x}_t)(x_{t-1} - \overline{x}_{t-1})$ with $\overline{x}_t$ being the expectation of $x_t$. For quadratic function $f(x)$ with curvature $h \in \mathbb{R}$, We have the following recurrence
		\begin{equation}
		\begin{pmatrix}
			U_{t+1} \\
			U_t \\
			V_{t + 1}
		\end{pmatrix} =
		(\mat{I} - \mat{B}^{\top})(\mat{I} - \mat{B})^{-1}
		\begin{pmatrix}
			\alpha^2 C \\
			0 \\
			0
		\end{pmatrix}
	\end{equation}
	where
	\begin{equation}
		\mat{B} =
		\begin{pmatrix}
		(1-\alpha h + \mu)^2 &  \mu^2 & -2\mu(1-\alpha h + \mu)\\
		1 & 0 & 0 \\
		1-\alpha h + \mu & 0 & - \mu
		\end{pmatrix}
	\end{equation}
	and $C = \E ( \nabla f_{S_t}(x_t) - \nabla f(x_t) )^2$ is the variance of gradient on minibatch $S_t$.

	\begin{proof}
		We prove by first deriving the recurrence for $U_t$ and $V_t$ respectively and combining them in to a matrix form. For $U_t$, we have
		\begin{equation}
		\begin{aligned}
			U_{t + 1} = & \E ( x_{t+1} - \overline{x}_{t + 1} )^2\\
			 = & \E ( x_{t} - \alpha \nabla f_{S_t}(x_t) + \mu (x_{t} - x_{t - 1} ) - (1 - \alpha h + \mu) \overline{x}_t + \mu \overline{x}_{t - 1} )^2 \\
			 = & \E ( x_{t} - \alpha \nabla f(x_t) + \mu (x_{t} - x_{t - 1} ) - (1 - \alpha h + \mu) \overline{x}_t + \mu \overline{x}_{t - 1}  + \alpha (\nabla f(x_t) - \nabla f_{S_t}(x_t)) )^2 \\
			 = & \E ( (1 - \alpha h + \mu) (x_t - \overline{x}_t)  - \mu(x_{t - 1} - \overline{x}_{t - 1} ) )^2 + \alpha^2 \E ( \nabla f(x_t) - \nabla f_{S_t}(x_t) )^2 \\
			 = & (1 - \alpha h + \mu)^2 \E ( x_t - \overline{x}_t )^2 -2 \mu (1 - \alpha h + \mu) \E (x_t - \overline{x}_t)(x_{t - 1} - \overline{x}_{t - 1} ) \\
			 & + \mu^2\E ( x_{t-1} - \overline{x}_{t-1} )^2+ \alpha^2 C
		\end{aligned}
		\label{equ:U_term}
		\end{equation}
		where the cross terms cancels due to the fact $\E_{S_t} [\nabla f(x_t) - \nabla f_{S_t}(x_t)]=0$ in the third equality.

		For $V_t$, we can similarly derive
		\begin{equation}
		\begin{aligned}
			V_t = & \E (x_t - \overline{x}_t) (x_{t-1} - \overline{x}_{t-1} ) \\
			= & \E ( (1 - \alpha h + \mu) (x_{t-1} - \overline{x}_{t-1} ) - \mu (x_{t-2} - \overline{x}_{t-2}) + \alpha (\nabla f(x_t) - \nabla f_{S_t}(x_t) ) ) (x_{t-1} - \overline{x}_{t-1} ) \\
			= & (1 - \alpha h + \mu)\E ( x_{t-1} - \overline{x}_{t-1} )^2 - \mu \E (x_{t-1} - \overline{x}_{t-1})(x_{t-2} - \overline{x}_{t-2})
		\end{aligned}
		\label{equ:V_term}
		\end{equation}
		Again, the term involving $\nabla f(x_t) - \nabla f_{S_t}(x_t)$ cancels in the third equality as a results of $\E_{S_t} [\nabla f(x_t) - \nabla f_{S_t}(x_t)]=0$.~\eqref{equ:U_term} and~\eqref{equ:V_term} can be jointly expressed in the following matrix form
		\begin{equation}
		\begin{aligned}
			\begin{pmatrix}
			U_{t+1} \\
			U_t \\
			V_{t + 1}
		\end{pmatrix}= \mat{B}
		\begin{pmatrix}
			U_t \\
			U_{t-1} \\
			V_t
		\end{pmatrix} +
		\begin{pmatrix}
			\alpha^2 C \\
			0 \\
			0
		\end{pmatrix}
		=\sum\limits_{i = 0}^{t-1} \mat{B}^{i} \begin{pmatrix}
			\alpha^2 C \\
			0 \\
			0
		\end{pmatrix} + \mat{B}^t \begin{pmatrix}
			U_1 \\
			U_0 \\
			V_1
		\end{pmatrix}
		= (\mat{I} - \mat{B}^t)(\mat{I} - \mat{B})^{-1}
		\begin{pmatrix}
			\alpha^2 C \\
			0 \\
			0
		\end{pmatrix}.
		\end{aligned}
		\end{equation}
		Note the second term in the second equality is zero because $x_0$ and $x_1$ are deterministic. Thus $U_1\!=\!U_0\!=\!V_1\!=\!0$.
	\end{proof}
\end{lemma}

According to Lemma~\ref{lem:bias_rec} and~\ref{lem:var_rec}, we have $\E ( \overline{x}_t - x^{*} )^2 = (\mat{e}^{\top}_1 \mat{A}^t [x_1, x_0]^{\top})^2$ and $\E ( x_t - \overline{x}_t )^2=\alpha^2 C \mat{e}^{\top}_1 (\mat{I} - \mat{B}^t)(\mat{I} - \mat{B})^{-1}\mat{e}_1$ where $\mat{e}_1 \in \mathbb{R}^n$ has all zero entries but the first dimension. Combining these two terms, we prove Lemma~\ref{lem:main_lemma}. Though the proof here is for one dimensional quadratics, it trivially generalizes to multiple dimensional quadratics. Specifically, we can decompose the quadratics along the eigenvector directions, and then apply Lemma~\ref{lem:main_lemma} to each eigenvector direction using the corresponding curvature $h$ (eigenvalue). By summing quantities in~\eqref{equ:squared_dist_exact} for all eigenvector directions, we can achieve the multiple dimensional correspondence of~\eqref{equ:squared_dist_exact}.

\section{Proof of Lemma~\ref{lem:spectral_var_control}}
Again we first present a proof of a multiple dimensional generalized version of Lemma~\ref{lem:spectral_var_control}. The proof of Lemma~\ref{lem:spectral_var_control} is a one dimensional special case of Lemma~\ref{lem:spectral_var_control_multi}. Lemma~\ref{lem:spectral_var_control_multi} also implies that for multiple dimension quadratics, the corresponding spectral radius $\rho(\mat{B})=\mu$ if ${(1-\sqrt{\mu})^2\over \alpha} \leq h \leq {(1+\sqrt{\mu})^2\over \alpha}$ on all the eigenvector directions with $h$ being the eigenvalue (curvature).
\begin{lemma}
\label{lem:spectral_var_control_multi}
Let $\mat{H}\in\mathbb{R}^{n\times n}$ be a symmetric matrix and $\rho(\mat{B})$ be the spectral radius of matrix
\begin{equation}
	%\rho_V(\alpha, \mu) = \rho \left(
\mat{B} = {\begin{pmatrix}
(\mat{I}-\alpha \mat{H} + \mu \mat{I})^{\top}(\mat{I}-\alpha \mat{H} + \mu \mat{I}) &  \mu^2 \mat{I} & -2\mu(\mat{I}-\alpha \mat{H} + \mu \mat{I})\\
\mat{I} & \mat{0} & \mat{0} \\
\mat{I}-\alpha \mat{H} + \mu \mat{I} & \mat{0} & - \mu \mat{I}
\end{pmatrix}}
	%\right)
\end{equation}
We have $\rho(\mat{B})=\mu$ if all eigenvalues $\lambda(\mat{H})$ of $\mat{H}$ satisfies
\begin{equation}
{(1-\sqrt{\mu})^2\over \alpha} \leq \lambda(\mat{H}) \leq {(1+\sqrt{\mu})^2\over \alpha}.
\label{equ:control_condition_var}
\end{equation}

\begin{proof}
	Let $\lambda$ be an eigenvalue of matrix $\mat{B}$, it gives
$\det\left(\mat{B} - \lambda \mat{I} \right) = 0$ which can be alternatively expressed as
\begin{equation}
\det \left( \mat{B} - \lambda \mat{I}\right) = \det{\mat{F}} \det{\left(\mat{C} - \mat{D} \mat{F}^{-1}
\mat{E} \right)} = 0
\label{equ:control_condition_var_1}
\end{equation}
assuming $\mat{F}$ is invertible, i.e. $\lambda + \mu \neq 0$, where the blocks in $\mat{B}$
\begin{equation*}
		\mat{C} = \left( { \begin{array}{c c}
 			\mat{M}^{\top}\mat{M} - \lambda \mat{I} &  \mu^2 \mat{I} \\
 			\mat{I} & - \lambda \mat{I}
 		\end{array} } \right),
 		\mat{D} = \left( { \begin{array}{c}
 			-2\mu \mat{M} \\
 			\mat{0}
 		\end{array}}\right),
 		\mat{E} = \left( {\begin{array}{c}
 			\mat{M} \\
 			\mat{0}
 		\end{array}} \right)^{\top},
 		\mat{F} = -\mu \mat{I} - \lambda \mat{I}
	\end{equation*}
	with $\mat{M}=\mat{I}-\alpha \mat{H} + \mu \mat{I}$.~\eqref{equ:control_condition_var_1} can be transformed using straight-forward algebra as
	\begin{equation}
		\det \left( \begin{array}{c c}
 			(\lambda - \mu) \mat{M}^{\top}\mat{M} - (\lambda + \mu) \lambda \mat{I} & (\lambda + \mu)\mu^2 \mat{I} \\
 			(\lambda + \mu) \mat{I} & -(\lambda + \mu)\lambda \mat{I}
 		\end{array} \right) = 0
		\label{equ:control_condition_var_2}
	\end{equation}
	Using similar simplification technique as in~\eqref{equ:control_condition_var_1}, we can further simplify into
	\begin{equation}
		(\lambda - \mu)\det \left( (\lambda + \mu)^2 \mat{I} - \lambda \mat{M}^{\top}\mat{M} \right) = 0
	\end{equation}
	if $\lambda \neq \mu$, as $(\lambda + \mu)^2 \mat{I} - \lambda \mat{M}^{\top}\mat{M}$ is diagonalizable, we have $(\lambda + \mu)^2 - \lambda \lambda(\mat{M})^2 = 0$ with $\lambda(\mat{M})$ being an eigenvalue of symmetric $\mat{M}$. The analytic solution to the equation can be explicitly expressed as
	\begin{equation}
		\lambda = \frac{\lambda(\mat{M})^2 - 2\mu \pm \sqrt{(\lambda(\mat{M})^2 - 2\mu)^2 - 4\mu^2}}{2}.
		\label{equ:control_condition_var_3}
	\end{equation}

	When the condition in~\eqref{equ:control_condition_var} holds, we have $\lambda(M)^2=(1 - \alpha \lambda(\mat{H}) + \mu)^2 \leq 4\mu$. One can verify that

	\begin{equation}
		\begin{aligned}
			(\lambda(\mat{M})^2 - 2\mu)^2 - 4\mu^2 & = && (\lambda(\mat{M})^2 - 4\mu)\lambda(\mat{M})^2 \\
			& = &&\left( (1 - \alpha \rho(\mat{H} ) + \mu)^2 - 4\mu\right)\lambda(\mat{M})^2 \\
			& \leq && 0
		\end{aligned}
	\end{equation}
	Thus the roots in~\eqref{equ:control_condition_var_3} are conjugate with $| \lambda | = \mu$. In conclusion, the condition in~\eqref{equ:control_condition_var} can guarantee all the eigenvalues of $\mat{B}$ has magnitude $\mu$. Thus the spectral radius of $\mat{B}$ is controlled by $\mu$.
\end{proof}

\end{lemma}


% Created 2019-10-02 mié 15:23
\documentclass[letterpaper]{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{fixltx2e}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{marvosym}
\usepackage{wasysym}
\usepackage{amssymb}
\usepackage{hyperref}
\tolerance=1000
\usepackage{khpreamble}
\usepackage{khpreamble}
\newcommand*{\shift}{\operatorname{q}}
\author{Kjartan Halvorsen}
\date{\today}
\title{System identification using matlab}
\hypersetup{
  pdfkeywords={},
  pdfsubject={},
  pdfcreator={Emacs 25.3.50.2 (Org mode 8.2.10)}}
\begin{document}

\maketitle

\section{Least-squares parameter estimation}
\label{sec-1}

The model structure used is the so-called ARX (Auto-Regressive with eXogenous input) model:
  \begin{center}
  \begin{tikzpicture}[block/.style={rectangle, draw, minimum height=12mm, minimum width = 17mm}, sumnode/.style={circle, draw, inner sep=2pt}, node distance=30mm]

  \node[coordinate] (input) {};
  \node[block, right of=input] (H) {$\frac{B(\shift)}{A(\shift)}$};
  \node[sumnode, right of=H] (sum) {\small $\Sigma$};
  \node[coordinate, right of=sum] (output) {};
  \node[block, above of=sum, node distance=16mm] (He) {$\frac{\shift^n}{A(\shift)}$};
  \node[coordinate, above of=He, node distance=16mm] (dist) {};

  \draw[->] (input) -- node[above, near start] {$u$} (H);
  \draw[->] (H) -- node[above] {$$} (sum);
  \draw[->] (dist) -- node[left, near start] {$e$} (He);
  \draw[->] (He) -- node[left] {$$} (sum);
  \draw[->] (sum) -- node[above, near end] {$y$} (output);

  \end{tikzpicture}
  \end{center}
where \(u(k)\) is a known input signal, \(y(k)\) is the output signal and \(e(k)\) is a disturbance (an unknown input signal) in the form of a zero-mean white noise sequence.

The model can be written
\[ A(\shift) y(k) = B(\shift)u(k) + \shift^n e(k)\]
\[ (\shift^n + a_1\shift^{n-1} + \cdots + a_n)y(k) = (b_0\shift^{m} + b_1\shift^{m-1} + \cdots + b_m)u(k) + \shift^n e(k)\]
\[ y(k+n) + a_1 y(k+n-1) + \cdots + a_n y(k) = b_0u(k+m) + b_1u(k+m-1) + \cdots + b_m u(k) + e(k+n)\]
\[ y(k+1) + a_1y(k) + \cdots + a_n y(k-n+1) = b_0 u(k+m-n+1) + b_1u(k+m-n) + \cdots + b_m u(k-n+1)) + e(k+1)\]
The one-step-ahead predictor for this model becomes
\begin{align*}
\hat{y}(k+1) &= -a_1 y(k) - a_2 y(k-1) - \cdots - a_n y(k-n+1) \\ &\qquad + b_0 u(k+m-n+1) + b_1 u(k+m-n) +  \cdots + b_m u(k-n+1)\\
             & = \underbrace{\bbm -y(k) & \cdots & -y(k-n+1) & u(k+m-n+1) & \cdots & u(k-n+1)\ebm}_{\varphi\transp(k+1)} \underbrace{\bbm a_1\\\vdots\\a_n\\b_0\\\vdots\\b_m\ebm}_{\theta}\\
 &= \varphi\transp(k+1)\theta.
\end{align*}
Note that the white noise term \(e(k+1)\) by definition cannot be predicted from knowledge of previous values in the sequence (which we don't know) nor from previous output values \(y(t), \; t \le k\) (which could have been used to estimate \(\hat{e}(k)\)). Therefore \(e(k+1)\) is predicted by its mean value which is zero. Note also that if our model with \(\theta = \theta^*\) is perfect (\(\theta^*\) contains the true parameters for the system which generated the data), then the prediction error equals the white noise disturbance: \(\epsilon(k+1) = y(k+1) - \varphi\transp(k+1)\theta^* = e(k+1)\). Therefore, we can check how good a models is by testing how close the prediction errors resembles a white noise sequence.

The system of equations in the unknown system parameters \(\theta\) is
\[ \Phi \theta = y, \]
where
\begin{align*}
\Phi &= \bbm \varphi\transp(n+1)\\\varphi\transp(n+2)\\\vdots\\\varphi\transp(N)\ebm,\\
y &= \bbm y(n+1)\\y(n+2)\\\vdots\\y(N)\ebm.
\end{align*}

The least-squares solution to this system of equations is, by definition, the solution \(\hat{\theta}\) which minimizes the sum of squares of the residuals \(\epsilon = y-\Phi\theta\), i.e. the solution that minimizes the criterion
\[ J(\theta) = \epsilon\transp\epsilon = \sum_i \epsilon_i^2. \]
It is given by
\[ \hat{\theta}_{LS} = \underbrace{(\Phi\transp\Phi)^{-1}\Phi\transp}_{\Phi^+} y, \]
where \(\Phi^+\) is called the \emph{Moore-Penrose invers} of the (typically) non-square, tall matrix \(\Phi\).

\subsection{Instructions}
\label{sec-1-1}
\begin{enumerate}
\item Download the data: \url{http://alfkjartan.github.io/files/sysid_exercise_data.mat} The data consist of 12 vectors \texttt{u1}, \texttt{y1}, \texttt{u1\_val}, \texttt{y1\_val}, \texttt{u2}, \texttt{y2}, \texttt{u2\_val}, \texttt{y2\_val}, \texttt{u3}, \texttt{y3}, \texttt{u3\_val}, \texttt{y3\_val} corresponding to three different LTI systems.
\begin{description}
\item[{\texttt{ui}}] input data for system \(i\)
\item[{\texttt{yi}}] output data for system \(i\)
\item[{\texttt{ui\_val}}] validation input data for system \(i\)
\item[{\texttt{yi\_val}}] validation output data for system \(i\)
\end{description}
The systems are
\begin{enumerate}
\item \( H_1(z) = \frac{b}{z+a} = \frac{0.2}{z -0.8} \)
\item \( H_2(z) = \frac{b_0 z + b_1}{z(z+a_1)} = \frac{0.0625z + 0.0375}{z(z - 0.9)} \)
\item \( H_3(z) = \frac{b}{z^2 + a_1z + a_2} = \frac{0.18}{z^2 - 1.4z + 0.58} \)
\end{enumerate}
\item For each case estimate a model using the input-output data. Formulate the one-step ahead predictor \( \hat{y}(k) = \varphi\transp(k)\theta \) and the system of equations
\( \Phi \theta = y \), and find the least-squares solution to the system. Compare with the true parameters.
\item Validate your model using the validation data. Let's say you have obtained estimates of the numerator \(\hat{B}(z)\) and denominator \(\hat{A}(z)\) of your model. You can then simulate the model using the validation data (a data set which is \textbf{not} used for parameter estimation) like this
\begin{verbatim}
H = tf(B, A, 1); % Create the discrete-time LTI model

% Simulate the model.
ysim = lsim(H, u_val);
% The above is also known as infinite-step ahead prediction
% and works only for stable models. It is often better to
% validate the model using a k-step ahead predictor.

e = y_val - ysim; % The error

% Calculate the root mean square of the error
RMSerror =      %Your code here

% Plot
figure()
subplot(121)
stairs(y_val)
hold on
stairs(ysim)
legend('Validation output', 'Simulated output')
subplot(122)
stairs(e)
title(sprintf('Simulation error. RMS=%f', RMSerror))
\end{verbatim}
\end{enumerate}

\subsection{About the validation simulation}
\label{sec-1-2}
In general it is preferred to compare the validation output to the output from a k-step ahead predictor, where k is chosen to correspond to a typical time interval of the data. For control models where we have used the rule-of-thumb of 4-10 sampling periods per rise time, a choice of k=10 is reasonable. A choice of k=1 (one-step ahead predictor) gives unreliable validation, since the trivial predictor \(\hat{y}(k+1) = y(k)\) can give good predictions if the sampling period is short ("the weather in one minute from now will be equal to the weather now"). Choosing k extremely large corresponds to pure simulation of the model (the predicted output sequence depends only on the input sequence) and will not work for unstable models. Also, for models with integration a small bias in the input sequence will be accumulated and give poor validation result, even for a good model.

The system identification toolbox provides such a function that computes the k-step ahead prediction. See  \url{https://www.mathworks.com/help/ident/ref/predict.html}. An implementation that uses only basic matlab functions can be downloaded here \url{http://alfkjartan.github.io/files/predictlti.m}.
% Emacs 25.3.50.2 (Org mode 8.2.10)
\end{document}


\chapter{Propagation Algorithms}
\label{AppendixAlgos}

\section{Chebychev Propagator}

For the evaluation of the time evolution operator
$f(\pm\hat{A}\,dt) = \ee^{\pm\ii \hat{A}\,dt}$, the Chebychev
propagation~\cite{KosloffJCP88, NdongJCP09}
is implemented by Algorithm~\ref{al:ChebyProp}. The operator $\hat{A}$ may be
a Hamiltonian, in which case the state vectors $\vec{v}$ are Hilbert space
states, or $\hat{A}$ may be a Liouvillian with no dissipators, in which case
$\vec{v}$ is a density matrix.

A re-calculation of the Chebychev coefficients in
line~\ref{Cheby:CallExpChebyCeoffs} is necessary only if the spectral
radius $\Delta$ and minimum eigenvalue $E_{\min}$ have changed, otherwise, the
coefficients from a previous calculation can be re-used. For a more general
function $f(\hat{A})$, instead of \Call{ExpChebyCoeffs}{}, the routine
\Call{ChebyCoeffs}{} defined in Algorithm~\ref{al:ChebyCoeffs} may be used,
together with an appropriate pre-factor in lines~\ref{Cheby:H_norm1}
and~\ref{Cheby:H_norm2}. The coefficients $[a_0 \dots a_n]$ are the same for
both a forward and a backward propagation.
They are real for the exponential function, but may be complex for an arbitrary
$f(\hat{A})$.

\begin{algorithm}
  \caption{{\sc Chebychev-Propagator}
  Evaluate $\vec{w} = f(\pm\hat{A}\,dt) \vec{v}$,
  with $f(\pm\hat{A}\,dt) = \ee^{\pm\ii \hat{A}\,dt}$.
  \label{al:ChebyProp}
  }
  \begin{algorithmic}[1]
    \Statex
    \Require{input vector $\vec{v} \in \Complex^N$;
             operator $\hat{A} \in \Complex^{N \times N}$;
             time step $dt$;}
    \Ensure{Approximation of propagated vector
            $\vec{w}= e^{-i \hat{A}\,dt} \vec{v} \in \Complex^N$}
    \Statex
    \Procedure{Cheby}{$\vec{v}$, $\hat{A}$, $dt$}
     \State $\Delta =$ spectral radius of $\hat{A}$
     \State $E_{\min} =$ minimum eigenvalue of $\hat{A}$
     \State $[a_0 \dots a_n] =$ \Call{ExpChebyCoeffs}{$\Delta$, $E_{\min}$, $dt$}
            \label{Cheby:CallExpChebyCeoffs}
     \State $d = \frac{1}{2} \Delta$;
            $\beta = d + E_{\min}$
     \State $\vec{v}_0 = \vec{v}$
     \State $\vec{w}^{(0)} = a_0 \vec{v}_0$
     \State $\vec{v}_1 = \pm\frac{\ii}{d}\left(
                            \hat{A} \vec{v}_0 - \beta \vec{v}_0
                         \right)$
            \label{Cheby:H_norm1}
     \State $\vec{w}^{(1)} = \vec{w}^{(0)} + a_1 \vec{v}_1$
     \For{$i = 2:n$}
       \State $\vec{v}_i = \pm \frac{2 \ii}{d}\left(
                            \hat{A} \vec{v}_{i-1} - \beta \vec{v}_{i-1}
                            \right) + \vec{v}_{i-2}$
              \label{Cheby:H_norm2}
       \State $\vec{w}^{(i)} = \vec{w}^{(i-1)} + a_i \vec{v}_i$
     \EndFor
     \State \Return $e^{\pm \ii\,\beta\,dt} \vec{w}^{(n)}$
   \EndProcedure
  \end{algorithmic}
\end{algorithm}

\begin{algorithm}
  \caption{{\sc ChebychevCoefficients}
  for $f(\pm \hat{A}\,dt) = e^{\pm i \hat{A}\,dt}$.
  \label{al:ExpChebyCoeffs}
  }
  \begin{algorithmic}[1]
    \Statex
    \Require{spectral radius $\Delta$ of $\hat{A}$;
             minimum eigenvalue $E_{\min}$ of $\hat{A}$;
             time step $dt$}
    \Ensure{Array of Chebychev coefficients $[a_0\dots a_n]$ allowing to
    approximate $f(\hat{A}\,dt)$ to pre-defined precision.}
    \Statex
    \Procedure{ExpChebyCoeffs}{$\Delta$, $E_{\min}$, $dt$}
     \State $\alpha = \frac{1}{2} \Delta \, dt$
     \State $a_0 = J_0(\alpha)$
     \Comment 0'th order Bessel-function of first kind
     \For{$i=1: n_{\max} \approx 4 \lfloor\alpha\rfloor$}
       \State $a_{i} = 2 J_i(\alpha)$
       \Comment $i$'th order Bessel-function of first kind
       \IIf{$\Abs{a_i} < $~limit} exit loop with $n=i$
     \EndFor
     \State \Return $[a_0, \dots a_n]$
   \EndProcedure
  \end{algorithmic}
\end{algorithm}

\clearpage

\section{Newton Propagator with Restarted Arnoldi}

For the evaluation of the time evolution operator
$f(\pm\hat{A}\,dt) = \ee^{\pm\ii \hat{A}\,dt}$, where $\hat{A}$ has complex
eigenvalues, the Newton algorithm using a restarted Arnoldi
scheme~\cite{Tal-EzerSJSC2007} can be used, as discussed
in section~\ref{subsec:newton} of chapter~\ref{chap:numerics}. The
operator $\hat{A}$ is typically a dissipative Liouvillian, but may also be
a Hamiltonian with non-Hermitian terms. The propagation scheme is implemented by
algorithm~\ref{al:RestartedNewton}, using the Arnoldi
algorithm~\ref{al:Arnoldi} as a central component.

The algorithm proceeds in iterations, with each iteration adding $m$ new terms
to the Newton expansion of the propagator. In each iteration, starting from
a vector $\vec{v}_s$, the Arnoldi algorithm is performed to obtain the Arnoldi
vectors spanning the $m$-dimensional Krylov subspace, as well as the Hessenberg
matrix $\hat{H}$, i.e.\ the projection of $\hat{A}$ into the Krylov space. From
the eigenvalues of $\hat{H}$ (the ``Ritz values''),
\index{Ritz values}
new sampling points for the Newton polynomials are chosen, in Leja ordering to
maximize numerical stability, using algorithm~\ref{al:ExtendLeja}. Then, new
Newton coefficients are calculated using
algorithm~\ref{al:ExtendNewtonCoeffs}. Lastly, the coefficients and Leja points
are used to calculate the contribution to the propagation result, and the
starting vector for the next iteration.

\begin{algorithm}
  \caption{{\sc RestartedNewton:}
  Evaluate $\vec{w} = f(\pm\hat{A}\,dt) \vec{v}$,
  with $f(\pm\hat{A}\,dt) = \ee^{\pm \ii \hat{A}\,dt}$.
  \label{al:RestartedNewton}
  }
  \begin{algorithmic}[1]
    \Statex
    \Require{input vector $\vec{v} \in \Complex^N$;
             operator $\hat{A} \in \Complex^{N \times N}$;
             time step $dt$; maximum size $m$ of Hessenberg matrices}
    \Ensure{Approximation of propagated vector
            $\vec{w}= \ee^{\pm \ii \hat{A}\,dt} \vec{v} \in \Complex^N$}
    \Statex
    \Procedure{RestartedNewton}{$\vec{v}$, $\hat{A}$, $dt$, $m$}
    \State $A_0 = \emptyset$; $Z_0 = \emptyset$
    \Comment{$A$, $Z$ use zero-based indexing}
    \State $\vec{w}^{(0)} = \vec{0} \in \Complex^N$
    \State $\vec{v}_0 = \vec{v} \in \Complex^N$
    \State $\beta = \Norm{\vec{v}_0}$
    \State $\vec{v_0} = \vec{v}_0/\beta$
    \State $s = 0$
    \While{not converged}
    \Comment{Iteration $s \rightarrow s+1$}
      \State $U$, $\hat{H}$, $Z$, $m$ = \Call{Arnoldi}{$\hat{A}$, $dt$, $\vec{v}_{s}$, $m$} \label{RN:arnoldi_call}
      \If{$m = 0$, $s = 0$} \label{RN:eigenstate}
        \State \Return $\ee^{\pm \ii \beta H_{1,1}} \vec{v}_s$
      \EndIf
      \State Normalize $Z$ with center $c$ and radius $\rho$
             \label{RN:normalization}
      \State $n_s = \Abs{Z_{s}}$
      \State $Z_{s+1} =$ \Call{ExtendLeja}{$Z_{s}, Z, m$}
             \label{RN:ExtendLejaCall}
      \State $A_{s+1} =$ \Call{ExtendNewtonCoeffs}{$A_{s}, Z_{s+1}, \rho, c$}
             \label{RN:ExtendNewtonCoeffsCall}
      \State $\vec{r}_{0} = \beta \vec{e}_1 \in \Complex^{m+1}$
      \Comment $\vec{e}_1$ is unit vector
      \State $\vec{p}_{0} = a_{n_s} \vec{r}_{0}$
      \For{k=1:m-1}
        \State
          $\vec{r}_{k}
           = (\frac{1}{\rho}\hat{H} - (z_{n_s+k-1}) +\frac{c}{\rho})
           \vec{r}_{k-1}$
        \State $\vec{p}_{k}
                = \vec{p}_{k-1} + a_{n_s+k} \vec{r}_{k}$
      \EndFor \label{RN:end_r_loop}
      \State $w_{+}^{(s)} = \sum_{i=1}^{m} \left[\vec{p}_{m-1}\right]_{i} \, \vec{u}_i$
      \Comment{$\vec{u}_i \in U$}
      \State $\vec{w}^{(s+1)} = \vec{w}^{(s)} + \vec{w}_{+}^{(s)}$
      \State  $\vec{r}_{m}
              =  \left(
                  \frac{1}{\rho}\hat{H}
                  - \left(z_{n_s+m-1} + \frac{c}{\rho}\right) \identity
                \right)\vec{r}_{m-1}$
      \State $\beta = \Norm{\vec{r}_{m}}$
      \State $\vec{r}_{m} = \vec{r}_{m} / \beta$
      \State $\vec{v}_{s+1} = \sum_{i=1}^{m+1}
                              \left[ \vec{r}_m\right]_{i} \, \vec{u}_i$
      \Comment{$\vec{u}_i \in U$}
      \State $s = s + 1$
      \State converged if
             $\frac{\Norm{\vec{w}_{+}^{s}}}{\Norm{\vec{w}^{s}}} < $~limit
    \EndWhile
    \State \Return $\vec{w}^{(s)}$
    \EndProcedure
  \end{algorithmic}
\end{algorithm}

Moreover:
\begin{itemize}[noitemsep]

  \item The check in line~\ref{RN:eigenstate} catches if $\vec{v}_s$ is an
  eigenstate of $\hat{A}$.

  \item For the normalization in line~\ref{RN:normalization}, the
  normalization radius $\rho$ and center $c$  should be calculated in the
  \emph{first} iteration and the re-used in subsequent iterations. For the
  set of Leja points $Z_0 = [z_{0}:z_{m-1}]$, they are calculated as
  \begin{equation}
    c = \frac{1}{m} \sum_{j=0}^{m-1} z_j\,, \qquad
    \rho = \Pi_{j=0}^{m-1} \Abs{c - z_j}^{\frac{1}{m}}\,.
  \end{equation}
  %Alternatively, to ensure that all the Leja points are in the unit circle, use
  %\begin{equation}
  %  \rho = \max_j \Abs{c - z_j}
  %\end{equation}

  \item The call to \Call{ExtendLeja}{} in line~\ref{RN:ExtendLejaCall}
  adds Leja points $z_{n_s}\dots z_{n_s+m-1}$.

  \item The call to \Call{ExtendNewtonCoeffs}{} in
  line~\ref{RN:ExtendNewtonCoeffsCall} adds the coefficients $a_{n_s}\dots
  a_{n_s+m-1}$

  \item The loop ending in line~\ref{RN:end_r_loop} implements the formula
  \begin{gather}
    \vec{r}_{m-1} = \Pi_{j=1}^{k}(\hat{H} - z_{n_s+j-1} \identity)\vec{e}_1\,, \\
    \vec{p}_{m-1} = \sum_{k=0}^{m-1} a_{n_s+k} \, \vec{r}_{k}\,.
  \end{gather}

\end{itemize}

\clearpage

\begin{algorithm}
  \caption{{\sc Arnoldi:}
  Obtain the $m \times m$ Hessenberg matrix for an operator $\hat{A} dt$ by
  projecting it into the Krylov space starting from a vector $\vec{v}$.
  \label{al:Arnoldi}
  }
  \begin{algorithmic}[1]

    \Statex
    \Require{%
    Operator $\hat{A}$,
    Time step $dt$;
    input vector $\vec{v}$;
    maximum order $m_{\max}$}

    \Ensure{%
    Array of $m+1$ (extended) Arnoldi vectors, each of the same dimension as
    $\vec{v}$; Extended $(m+1) \times (m+1)$ Hessenberg matrix $\hat{H}$;
    Accumulated Ritz values $Z$};
    Dimension $m$ of Hessenberg matrix
    \Statex

    \Procedure{Arnoldi}{$\hat{A}$, $dt$, $\vec{v}$, $m_{\max}$}
      \State $\beta = \Norm{\vec{v}}$; $\vec{u}_1 = \vec{v} / \beta$;
              $U = [ \vec{u}_{1} ]$; $Z = \emptyset $; $m = m_{\max}$
      \State $\hat{H}_{1:(m+1),1:(m+1)} = 0$
      \For{$j = 1:m_{\max}$}
        \State $\vec{u}_{j+1} = \hat{A} \vec{u}_j$
        \For{$i = 1:j$}
          \State $H_{i,j} = dt \, \Braket{\vec{u}_i | \vec{u}_{j+1}}$
                 \label{Ar:Hess_element}
          \State $\vec{u}_{j+1} = \vec{u}_{j+1} - \frac{H_{i,j}}{dt} \vec{u}_i$
        \EndFor \label{Ar:EndGramSchmidt}
        \State $Z = Z \cup \text{eigenvalues}(\hat{H}_{1:j,1:j})$
               \label{Ar:eigenvalues}
        \State $h_{next} = \Abs{\vec{u}_{j+1}}$
        \State if $h_{next} \approx 0$: $m = j$, exit loop
        \State $\vec{u}_{j+1} = \vec{u}_{j+1} / h_{next}$
        \State $U = U \cup \vec{u}_{j+1}$
        \State $H_{j+1,j} = h_{next} \, dt$
      \EndFor
      \State \Return $U = [ \vec{u}_{1} : \vec{u}_{m+1}]$,
                     $\hat{H}_{1:(m+1), 1:(m+1)}$, $Z$, $m$
    \EndProcedure
  \end{algorithmic}
\end{algorithm}

Notes:
\begin{itemize}[noitemsep]

  \item At the end of the loop in line~\ref{Ar:EndGramSchmidt}, $\hat{H}$ is
  complete as a $j \times j$ matrix. The loop itself is a Gram-Schmidt
  orthonormalization of the Arnoldi vectors $\vec{u}_j$.

  \item The eigenvalues of the Hessenberg matrix in line~\ref{Ar:eigenvalues}
  can be obtained with the QR method, implemented in LAPACK as {\tt ZHSEQR}.
  %\Comment{sub-matrix $\hat{H}_{1:m,1:m}$ is proper Hessenberg matrix}

\end{itemize}

\begin{algorithm}
  \caption{{\sc ExtendLeja:}
  Choose $m$ new points from a set of Ritz values to extend an existing array of
  Leja points
  \label{al:ExtendLeja}
  }
  \begin{algorithmic}[1]

    \Statex

    \Require{%
    Array $Z_s$ of $n_s$ existing Leja points;
    Array $Z$ of new candidate points (Ritz values);
    Number $m$ of points to pick from $Z$.}

    \Ensure{%
    Array $Z_{s+1}$ of $n_s + m$ Leja-ordered points}
    \Statex

    \Procedure{ExtendLeja}{$Z_s, Z, m$}
      \State $n_0 = 1$; $Z_{s+1} = Z_{s}$
      \If{$Z_{s} = \emptyset$}
        \State $z = \max(\abs(Z))$
        \State $Z_{s+1} = Z_{s_{1}} \cup z$; remove $z$ from $Z$
        \State $n_0 = 2$
      \EndIf
      \For{$n = n_0:m$}
        \State Select $z_i \in Z$ that maximizes
               $\Pi_{z_j \in Z_{s+1}} \Abs{z_i - z_j}$
        \State $Z_{s+1} = Z_{s+1} \cup z_i$; remove $z_i$ from $Z$
      \EndFor
      \State \Return $Z_{s+1}$
    \EndProcedure
  \end{algorithmic}
\end{algorithm}

\begin{algorithm}
  \caption{{\sc ExtendNewtonCoeffs:}
  Choose $m$ new points from a set of Ritz values to extend an existing array of
  Leja points
  \label{al:ExtendNewtonCoeffs}
  }
  \begin{algorithmic}[1]

    \Statex

    \Require{%
    Array $A_{s} = [a_{0} \dots a_{n_s-1} ]$ of $n_s$ Newton coefficients from
    previous iteration;
    Array $Z_{s+1} = [z_{0} \dots z_{n_s-1+m}]$ of Leja points;
    Normalization radius $\rho$; Normalization center $c$
    }

    \Ensure{%
    Array $A_{s+1} = [a_{0} \dots \dots a_{n_s-1+m}]$ of $n_s+m$ Newton
    coefficients
    }
    \Statex
    \Procedure{ExtendNewtonCoeffs}{$A_{s}, Z_{s+1}, \rho, c$}
      \State $A_{s+1} = A_{s}$;
             $n_0 = n_s = \Abs{A_{s}}$,
             $m = \Abs{Z_{s+1}} - n_s$;
      \State Define $f(z) = \ee^{\pm\ii (\rho z + c)}$
      \If{$n_s = 0$}
        \State $a_{0} = f(z_0)$
        \State $A_{s+1} = A_{s+1} \cup a_{0}$
        \State $n_0 = 1$
      \EndIf
      \For{$k = n_0:n_s-1+m$}
        \State $a_k = \frac{f(z_k) - a_0 - \sum_{n=1}^{k-1} a_{n} \Pi_{j=0}^{n-1} (z_k - z_j)}
                           {\Pi_{j=0}^{k-1} (z_k - z_j)}$
        \State $A_{s+1} = A_{s+1} \cup a_k$
      \EndFor
      \State \Return $A_{s+1}$
    \EndProcedure
  \end{algorithmic}
\end{algorithm}


\chapter{Simulating a Call-By-Value $\lambda$-Calculus Machine}
\label{chap:heap}

\newcommand{\subst}[3]{{#1}^{#2}_{#3}}
\newcommand{\Ter}{\MS{Ter}}
\newcommand{\red}{\succ}

In this chapter, we present a large benchmark for our framework.  We implement a multi-tape Turing machine that simulates another abstract machine.
First, we define and motivate the semantics of this machine.  After that, we implement and verify the simulator Turing machine.  We show that the
halting problem of the abstract machine reduces to the halting problem of multi-tape Turing machines.

\section{Heap Machine}
\label{sec:heap}

The abstract machine we present here is a variant of the heap machine in Kunze et~al.~\cite{KunzeEtAl:2018:Formal}.  They formally show in Coq that
their variant of the machine can simulate the language $L$, which implements the call-by-value subset of the $\lambda$-calculus.  Thus, we implement a
simulator that also simulates $L$.  Here we define the language $L$ only briefly.  For more a more thorough treatment of this language, we refer
to~\cite{forster2017weak} and~\cite{KunzeEtAl:2018:Formal}.  The concrete definition of the machine we are considering is due to F.~Kunze.

\subsubsection{Call-By-Value $\lambda$-Calculus}
\label{sec:L}

Terms of this language are De~Bruijn terms, and are inductively defined by:
\begin{align*}
  s,t,u,v &~:~\Ter~::=~ (n:\Nat) ~|~ s~t ~|~ \lambda s
\end{align*}
The language $L$ uses simple substitution:
\begin{align*}
  \subst kku &~:=~u \\
  \subst nku &~:=~n &&\text{if}~n\neq k \\
  \subst{(st)}ku &~:=~(\subst sku)(\subst tku) \\
  \subst{(\lambda s)}ku &~:=~\lambda(\subst s{Sk}u)
\end{align*}
The reduction relation $s \red t$ is inductively defined on terms:
\begin{mathpar}
  \inferrule*{~}{(\lambda s) (\lambda t)\red\subst s0{\lambda t}} \and%
  \inferrule*{s\red s'} {st\red s't} \and%
  \inferrule* {t\red t'} {(\lambda s)t\red(\lambda s)t'}%
\end{mathpar}

\subsubsection{Heap Machine}
\label{sec:heap-def}
\setCoqFilename{ProgrammingTuringMachines.TM.LM.Semantics}%

Instead of substituting expressions, the heap machine works with closures.  A closure is a program together with an environment.  Variable bindings
are implemented as pointers to a linked list of closures, called \textit{heap}.  Programs are lists of commands:
\begin{definition}[Program][Pro]
  \label{def:Com}
  \begin{alignat*}{2}
    n:   &~Var  &~:=~& \Nat \\
    c:   &~Com  &~::=~& \MS{VAR}(n:Var) ~|~ \MS{APP} ~|~ \MS{LAM} ~|~ \MS{RET} \\
    P,Q: &~Pro  &~:=~& \List(Com)
  \end{alignat*}
\end{definition}

Programs essentially are linearised expressions of $L$.  The function $\gamma$ translates terms of $L$ to programs of the heap machine:
\begin{alignat*}{2}
  &\gamma&~n           &:= [\MS{VAR}~n] \\
  &\gamma&~(s~t)       &:= \gamma~s \app \gamma~t \app [\MS{APP}] \\
  &\gamma&~(\lambda s) &:= \MS{LAM} \cons \gamma~s \app [\MS{RET}]
\end{alignat*}

The heap is implemented as a list of heap entries.  A heap entry may be empty or contain a closure and a pointer to the next heap entry.  Pointers are
implemented as list indices on the heap entry list.
\begin{definition}[Closures and heaps][Heap]
  \label{def:Heap}
  ~
  \begin{alignat*}{2}
    a,b: &~Add  &~:=~& \Nat \\
    g:   &~Clos &~:=~& Add \times Pro \\
    e:   &~Entr &~:=~& \Option(Clos \times Add) \\
    H:   &~Heap &~:=~& \List(Entr)
  \end{alignat*}
\end{definition}

States of the abstract machine are triples of two closure lists and a heap:
\[
  (T, V, H) ~:~ \List(Clos) \times \List(Clos) \times Heap.
\]
$T$ is called the control stack.  It contains the closures that the machine has to process.  The second stack $V$ is called the argument stack.  For
example, the $\MS{APP}$ command fetches the functions and arguments from that stack.

We first explain what each rule of the reduction predicate $(T,V,H) \succ (T',V',H')$ does, and then we define it formally.

\paragraph{Application Rule}
When the first closure on the control stack is $(a, \MS{APP} \cons P)$, the machine fetches two closures $g$ and $(b,Q)$ from the argument stack.  The
closure $g$ corresponds to the argument of the application.  The second closure $(b,Q)$ corresponds to the called function, where $b$ is the pointer
to the environment in that the argument is free.  It binds the argument to $g$, by putting a new heap entry $(g,b)$ on the heap.  The heap machine
continues executing $(c,Q)$ and the ``rest closure'' $(a,P)$, where $c$ is the pointer to the new heap entry.

\paragraph{$\lambda$-Rule}
If the head control closure is $(a, \MS{LAM} \cons P)$, the machine splits the linearised program $P$ into the body $Q$ of the $\lambda$-expression
and the rest program $P'$.  Then it pushes the closure of the rest program $(a,P')$ on the control stack, and the ``body closure'' $(a,Q)$ on the
argument stack.  The splitting is realised with the function $\phi : Pro \to \Option(Pro \times Pro)$.  For example, {\small
  \[
    \phi[\MS{VAR}(0); \MS{LAM}; \MS{APP}; \MS{RET}; \MS{RET}; \MS{VAR}(1)] = \Some{([\MS{VAR}(0); \MS{LAM}; \MS{APP}; \MS{RET}], [\MS{VAR}(1)])}.
  \]
}%
The first $\MS{RET}$ matches to the $\MS{LAM}$ in the program above, so it is part of the first half.  $\phi$ is formally defined with a
tail-recursive auxiliary function:
\begin{definition}[$\phi$][jumpTarget]
  \label{def:jumpTarget}
  We define $\phi~P :=~ \phi'~0~\nil~P$ with the auxiliary function $\phi' : \Nat \to Pro \to Pro \to \Option(Pro \times Pro)$ which is defined by
  recursion on $P:Pro$:
  \begin{alignat*}{4}
    &\phi'&~0~   &~Q&~(\MS{RET} \cons P) &&~:=~& \Some{(Q,P')} \\
    &\phi'&~(S~k)&~Q&~(\MS{RET} \cons P) &&~:=~& \phi'~k~(Q \app [\MS{RET}])~P \\
    &\phi'&~k    &~Q&~(\MS{LAM} \cons P) &&~:=~& \phi'~(S~k)~(Q \app [\MS{LAM}])~P \\
    &\phi'&~k    &~Q&~(c        \cons P) &&~:=~& \phi'~k~(Q \app [c])~P \qquad \text{if $c \neq \MS{RET} \neq \MS{LAM}$} \\
    &\phi'&~k    &~Q&~              \nil &&~:=~& \None
  \end{alignat*}
\end{definition}

\paragraph{Variable Rule}
If the first command on the first closure of the control stack is $\MS{VAR}(n)$, the machine looks up the $n$th entry in the environment on the heap
at the address $a$.  Then, it pushes this closure to the argument stack.  The function $lookup : Heap \to Add \to Var \to \Option(Clos)$ starts at the
heap entry at the address $a$, and gets the $n$th entry. We write $H[a,n]$ for $lookup~H~a~n$.  The function is defined by recursion on~$n$:
\begin{definition}[$lookup$][lookup]
  \label{def:lookup}
  $H[a,n]$ is defined by recursion on $n$:
  \[
    H[a,n] :=
    \begin{cases}
      \Some{g} & H[a]=\Some{\Some{(g,b)}} \land n=0 \\
      H[b,n-1] & H[a]=\Some{\Some{(g,b)}} \land n>0 \\
      \None & \text{else}
    \end{cases}
  \]
  where $H[\cdot] : \Option(Entr)$ is the standard list lookup function.
\end{definition}

We formally define the reduction rules.  Note that there is no reduction rule for the command $\MS{RET}$, because the purpose of $\MS{RET}$ is solely
to mark the end of the encoding of the body of a $\lambda$-expression.
\begin{definition}[Semantics of the heap machine][step]
  {\small
    \begin{alignat*}{4}
      & \left((a, (\MS{APP} \cons P)) \cons T, g \cons (b, Q) \cons V, H\right) &~\red~& \left((c, Q) \cons (a,P) \cons_{tr} T, V, H'\right)
      \quad&& \text{if $put~H~\Some{(g,b)} = (c,H')$} \\
      & \left((a, (\MS{LAM} \cons P)) \cons T, V, H\right)                      &~\red~& \left((a, P') \cons_{tr} T, (a,Q) \cons V, H\right)
      \quad&& \text{if $\phi~P = \Some{(Q, P')}$} & \\
      & \left((a, (\MS{VAR}(n) \cons P)) \cons T, V, H\right)                   &~\red~& \left((a, P) \cons_{tr} T, g \cons V, H\right)
      \quad&& \text{if $H[a,n]=\Some{g}$} &
    \end{alignat*}
  }
  with $put~H~c := (\length{H}, H \app [c]) $.\\
  The tail-recursion optimisation $(g \cons_{tr} T) : \List(Clos)$ is defined as follows:
  \begin{alignat*}{3}
    & (a, \nil) &~\cons_{tr}~& T &~:=~& T \\
    & (a, P   ) &~\cons_{tr}~& T &~:=~& (a, P) \cons T \qquad\text{if $P \neq \nil$}
  \end{alignat*}

  We write for $\red^*$ for the reflexive transitive closure of $\red$ (cf.\ Definition~\ref{def:Kleene}) and $\red^k$ for the relational power (cf.\
  Definition~\ref{def:pow}).  Furthermore, we use the following notations:
  \begin{align*}
                         halt(T,V,H) &:= \forall T'~V'~H'.~\lnot \left((T,V,H) \red (T',V',H')\right) \\
    (T,V,H) \terminates^k (T',V',H') &:= (T,V,H) \red^k (T',V',H') \land halt(T',V',H') \\
    (T,V,H) \terminates^* (T',V',H') &:= (T,V,H) \red^* (T',V',H') \land halt(T',V',H')
  \end{align*}
\end{definition}

The tail-recursion optimisation ($\cons_{tr}$) makes sure that no closures with empty programs are pushed on a closure stack.  This optimises space
usage for tail recursive programs.

\begin{lemma}[Basic properties about $\red$]
  \label{lem:heap-red}
  ~
  \begin{enumerate}
    \coqitem[step_iff] \label{lem:step-iff}%
    The relation $\red$ is functional and computable, i.e.\ there is a function
    $step : \List(Clos) \times \List(Clos) \times Heap \to \Option\left(\List(Clos) \times \List(Clos) \times Heap\right)$, such that:
    \[
      step(T,V,H)=\Some{(T',V',H')} ~\iff~ (T,V,H) \red (T',V',H')
    \]
    \coqitem[is_halt_state_correct] \label{lem:halt_state_dec}%
    $halt(T,V,H)$ is decidable, i.e.\ there is a function $isHalt : \List(Clos) \times \List(Clos) \times Heap \to \Bool$, such that:
    \[
      isHalt(T,V,H) = \true ~\iff~ halt(T,V,H)
    \]
  \end{enumerate}
\end{lemma}

\section{Implementation of a Simulator}
\label{sec:heap-implementation}

\setCoqFilename{ProgrammingTuringMachines.TM.LM.Alphabets}%

First, we notice that all types in Definition~\ref{def:Heap}, except commands ($Com$), are encodable on minimal alphabets, according to
Definition~\ref{def:basic-encodings}:
\begin{definition}[Encoding of heaps][]
  \label{def:Heap-encode}
  The type $Com$ is isomorphic to the sum of $Var$ and $ACom := \MS{APP} ~|~ \MS{LAM} ~|~ \MS{RET}$.  $ACom$ is encodable on itself (since it is
  finite), and $Var = \Nat$ is encodable on $\Sigma_{Var} := \Sigma_\Nat$.  Then $Com$ is encodable on $\Sigma_{Com} := \Sigma_{\Nat+ACom}$.  All
  other types are encodable according to Definition~\ref{def:basic-encodings}.
  For completeness, the alphabets are:\\
  $\Sigma_{Pro} := \Sigma_{\List(Com)}$, $\Sigma_{Add} := \Sigma_\Nat$, $\Sigma_{Clos}:=\Sigma_{Add \times Pro}$,
  $\Sigma_{Entr} := \Sigma_{\Option(Clos \times Add)}$, and $\Sigma_{Heap} := \Sigma_{\List(Entr)}$.
\end{definition}

First, we derive constructor and deconstructor machines for commands.  As the type of commands essentially is the sum of two encodable types ($ACom$
and $\Nat$), we can combine constructors and deconstructors from Section~\ref{sec:constructors-deconstructors}.
\setCoqFilename{ProgrammingTuringMachines.TM.LM.CaseCom}%
\begin{definition}[$\MS{CaseCom}$][CaseCom]
  $\MS{CaseCom} : \TM_{\Sigma^+_{Com}}^1(\Option(ACom))$ is defined by
  \begin{alignat*}{2}
    & \MS{CaseCom} &~:=~& \MS{If}~\MS{CaseSum} \\
    &               &    & \MS{Then}~   \Return{\Nop}{\None} \\
    &               &    & \MS{Else}~~~ \MS{Relabel}~(\LiftAlphabet{\MS{CaseFin}}{RetrInl})~(\Some\cdot)
  \end{alignat*}
\end{definition}
Note that $RetrInl : \Sigma_{Var} \hookrightarrow \Sigma_{Com}$.
\begin{lemma}[Correctness of $\MS{CaseCom}$][CaseCom_Sem]
  \label{lem:CaseCom_Sem}
  $\MS{CaseCom} \RealiseIn{11} CaseCom$ with
\begin{lstlisting}[style=semicoqstyle]
$CaseComRel :=$
  $\lambda t~(l, t').~\forall (c:Com).~ t[0] \simeq c \rightarrow \MS{match}~l,c$
      [$\Some{\MS{APP}}, \MS{APP} \Rightarrow isRight~t'[0]$
      |$\Some{\MS{LAM}}, \MS{LAM} \Rightarrow isRight~t'[0]$
      |$\Some{\MS{RET}}, \MS{RET} \Rightarrow isRight~t'[0]$
      |$\None, \MS{VAR}~n \Rightarrow t'[0] \simeq n$
      |_,_=>$\False$
      ].
\end{lstlisting}
\end{lemma}
The $\MS{VAR}$ constructor is defined with $\MS{ConstrInl}$ and the constructor for $ACom$ is defined with $\MS{WriteSymbol}$.

For the functions $\phi$ and $lookup$, we define machines that \textit{compute} these functions in a lesser strict sense than
Definition~\ref{def:Computes2_Rel}.  The Turing machines only save the input-values that a caller machine needs again.  For example, we have a machine
$\MS{Nth'}$ that saves the list, but not the index.  The second change is that for functions with optional outputs, like $nth$, we do not extend the
alphabet with $\Sigma_{\Option(X)}$.  Instead, we encode whether the output value is $\Some{\cdot}$ or $\None$ with the label type $\Bool$.  In case
that the output is $\None$, the machine terminates in $\false$ and the output tape stays right.

After we have defined and verified machines that compute $\phi$ and $lookup$, we define a machine $\MS{Step}$ that simulates a single step of the heap
machine.

\subsection{$\MS{Lookup}$}
\label{sec:Lookup}
\setCoqFilename{ProgrammingTuringMachines.TM.LM.LookupTM}%

\enlargethispage{0.5cm}

\begin{figure}[t]
  \centering
  \small
  \begin{tikzcd}
    & \underset{\text{\footnotesize(1)}}{\Sigma_{Add}} \arrow[r, "RetrFst", hook] & \Sigma_{Clos} \arrow[r, "RetrFst", hook] &%
    \Sigma_{Clos \times Add} \arrow[r, "RetrOpt", hook] &
    \Sigma_{\Option(Clos \times Add)} \arrow[r, "RetrList", hook] & \Sigma_{Heap} \arrow[d, "f_{Heap}", hook] \\
    \underset{\text{\footnotesize(2)}}{\Sigma_{Var}} \arrow[r, "RetrInl", hook] & \Sigma_{Com} \arrow[r, "RetrList", hook] &%
    \Sigma_{Pro} \arrow[u, "RetrSnd", hook] &%
    \underset{\text{\footnotesize(3)}}{\Sigma_{Add}} \arrow[u, "RetrSnd", hook] & \Sigma_{Clos} \arrow[r, "f_{Clos}", hook] & \Sigma
  \end{tikzcd}
  \caption{The retractions of $\MS{Lookup}$.}
  \label{fig:retracts-Lookup}
\end{figure}

The machine $\MS{Lookup} : \TM_{\Sigma^+}^5(\Bool)$ realises the heap $lookup$ function.  It uses the machine $\MS{Nth}' : \TM_{\Sigma^+}^4(\Bool)$
which is like $\MS{Nth}$ from Section~\ref{sec:Nth}, but with the two changes mentioned above.

First we consider the alphabet of $\MS{Lookup}$.  We can encode closures on the heap alphabet $\Sigma_{Heap}$.  However, when $\MS{Step}$ calls
$H[a,n]$, $a$ and $n$ are encoded on an alphabet for the closures on the closure stack.  Thus, we define $\MS{Lookup}$ on an alphabet $\Sigma$, with
retractions $f_{Heap} : \Sigma_{Heap} \hookrightarrow \Sigma$ and $f_{Clos} : \Sigma_{Clos} \hookrightarrow \Sigma$.  The second retraction correspond
to the closure alphabet where $a$ and $n$ are encoded.  The retraction
\[
  \coqlink[retr_clos_lookup_heap]{f_{Clos'}} : \Sigma_{Clos} \hookrightarrow \Sigma := f_{Heap} \circ RetrList \circ RetrOpt \circ RetrFst
\]
is the canonical retraction of closures from the heap.  In Figure~\ref{fig:retracts-Lookup}, the three relevant retractions
$\Sigma_\Nat \hookrightarrow \Sigma$ are visualised:
\begin{enumerate}
\item a heap address of a closure on the stack alphabet:
  \[
    \coqlink[retr_nat_lookup_clos_ad]{f_{add}} := f_{Clos'} \circ RetrFst
  \]
\item a variable in a command of a closure on the stack alphabet:
  \[
    \coqlink[retr_nat_lookup_clos_var]{f_{var}} := f_{Clos'} \circ RetrSnd \circ RetrList \circ RetrInl
  \]
\item a pointer to the next heap entry:
  \[
    \coqlink[retr_nat_lookup_entry]{f_{next}} :=  f_{Heap} \circ RetrList \circ RetrOpt \circ RetrSnd
  \]
\end{enumerate}

Since the function $lookup$ is tail-recursive, we define the step machine with the type $LookupStep : \TM_{\Sigma^+}^5(\Option(\Bool))$.  It first
calls the list lookup machine ($\MS{Nth'}$).  If it failed, $\MS{Lookup}$ immediately terminates in $\false$.  Else, it makes a case-distinction on
the heap entry $e : \Option(Clos \times Add)$.  In case it is $\None$, $\MS{Lookup}$ also terminates in $\false$.  If $e = \Some{e'}$, it destructs
$e' = (g,b)$ and makes a case-distinction over $n$.  If $n=0$, $\MS{LookupStep}$ resets the tapes for $b$ and $n$, translates $g$ from $f_{Clos'}$ to
$f_{Clos}$ and returns $\true$.  Else it translates $b$ from $f_{next}$ to $f_{add}$, moves it to the tape that contained $a$, and repeats the loop.
We visualise the execution in the latter two cases in Tables~\ref{tab:exec-LookupStep-S} and~\ref{tab:exec-LookupStep-0}.

% \enlargethispage{0.5cm}

\begin{table}[H]
  \scriptsize
  \begin{tabular}{l||l|l|l|l|l|l|l|l}
    Input       & $\MS{Nth'}$      & $\MS{CaseOption}$       & $\MS{CasePair}$     & $\MS{CaseNat}$ & $\MS{CopyValue}$  & $\MS{Translate}$ & $\MS{Reset}$ & $\MS{Reset}$ \\ \hline
    $0:H$       & $0:H$            &                       &                      &                 &                   &                  &              &              \\
    $1:a$       & $1:~\dashv$      &                       &                      &                 & $1:b$             & $0:b$            &              &              \\
    $2:n$       &                  &                       &                      & $0:n'$          &                   &                  &              &              \\
    $3:~\dashv$ &                  &                       & $1: g$               &                 &                   &                  &              & $0:~\dashv$  \\
    $4:~\dashv$ & $2:\Some{(g,b)}$ & $0:(g,b)$             & $0:b$                &                 & $0:b$             &                  & $0:~\dashv$  &              \\
  \end{tabular}
  \caption{Execution protocol of $\MS{LookupStep}$ in case $H[a] = \Some{\Some{(g,b)}}$ and $n=S~n'$.  It terminates in the label $\None$.  The
    translation is from $f_{next}$ to $f_{add}$.}
  \label{tab:exec-LookupStep-S}
  \begin{tabular}{l||l|l|l|l|l|l|l}
    Input       & $\MS{Nth'}$      & $\MS{CaseOption}$       & $\MS{CasePair}$     & $\MS{CaseNat}$ & $\MS{Reset}$ & $\MS{Reset}$ & $\MS{Translate}$ \\ \hline
    $0:H$       & $0:H$            &                       &                      &                 &              &              &                  \\
    $1:a$       & $1:~\dashv$      &                       &                      &                 &              &              &                  \\
    $2:n$       &                  &                       &                      & $0:0$           &              & $0:~\dashv$  &                  \\
    $3:~\dashv$ &                  &                       & $1: g$               &                 &              &              & $0:g$            \\
    $4:~\dashv$ & $2:\Some{(g,b)}$ & $0:(g,b)$             & $0:b$                &                 & $0:~\dashv$  &              &                  \\
  \end{tabular}
  \caption{Execution protocol of $\MS{LookupStep}$ in case $H[a] = \Some{\Some{(g,b)}}$ and $n=0$.  It terminates in the label $\Some\true$.  The
    translation is from $f_{Clos'}$ to $f_{Clos}$.}
  \label{tab:exec-LookupStep-0}
\end{table}

\begin{definition}[$\MS{LookupStep}$][Lookup_Step]
  \label{def:Lookup_Step}
  We define the machine $\MS{LookupStep} : \TM_{\Sigma^+}^5(\Option(\Bool))$.
  \begin{align*}
    & \MS{LookupStep} := \\
    & \quad \MS{If}~(\MS{Nth'}~f_{heap}~f_{add}) \\
    & \quad \MS{Then}~\MS{If}~(\LiftBoth{\MS{CaseOption}}{f_{heap} \circ RetrList}{\Vector{4}}) \\
    & \quad \phantom{\MS{Then}}~\MS{Then}~\LiftBoth{\MS{CasePair}}{f_{heap} \circ RetrList \circ RetrOpt}{\Vector{4;3}} \Seq \\
    & \quad \phantom{\MS{Then}~\MS{Then}}~\MS{If}~(\LiftBoth{\MS{CaseNat}}{f_{var}}{\Vector{2}}) \\
    & \quad \phantom{\MS{Then}~\MS{Then}}~\MS{Then}~ \MS{Return}~\bigl(
      \LiftTapes{\MS{CopyValue}}{\Vector{4;1}} \Seq
      \LiftTapes{\MS{Translate}~f_{next}~f_{add}}{\Vector{1}} \Seq \\
    & \quad \phantom{\MS{Then}~\MS{Then}~\MS{Then}~\MS{Return}\bigl(}
      \LiftTapes{\MS{Reset}}{\Vector{4}} \Seq
      \LiftTapes{\MS{Reset}}{\Vector{3}}
      \bigr)~{\None} \\
    & \quad \phantom{\MS{Then}~\MS{Then}}~\MS{Else}~~~ \MS{Return}~\bigl(
      \LiftTapes{\MS{Reset}}{\Vector{4}} \Seq
      \LiftTapes{\MS{Reset}}{\Vector{2}} \Seq \\
    & \quad \phantom{\phantom{\MS{Then}~\MS{Then}}~\MS{Else}~~~\MS{Return}~\bigl(}
      \LiftTapes{\MS{Translate}~f_{Clos'}~f_{Clos}}{\Vector{3}}
      \bigr)~{\Some\true} \\
    & \quad \phantom{\MS{Then}}~\MS{Else}~~~ \Return{\Nop}{\Some\false} \\
    & \quad \MS{Else}~~~ \Return{\Nop}{\Some\false}
  \end{align*}
\end{definition}

The step machine terminates in $\Some\false$ in two cases.  Either the list lookup failed, or it returned the empty heap entry $H[a]=\Some\None$.  In
the successfully termination case (cf.\ Table~\ref{tab:exec-LookupStep-0}), $\MS{LookupStep}$ resets the afterwards unneeded tapes.

Because no initialisation or clean-up is needed before or after the loop, we can define $\MS{Lookup} := \While~\MS{LookupStep}$.  The correctness
relation of $\MS{Lookup}$ says that if it terminated in the label $\true$, we have $H[a,n] = \Some{g}$ for some closure $g$, and tape~$3$ contains $g$
w.r.t.\ the retraction $f_{Clos}$.  Furthermore, the input tape~$0$ still contains $H$ and all other tapes are resetted.  In case $\MS{Lookup}$
terminated in $\false$, the postcondition only commits that $H[a,n]=\None$.  Note that there are two possible reasons for that: Either $H[a]=\None$ or
$H[a]=\Some\None$, but we do not separate these two failure cases.

\enlargethispage{1cm}

\begin{lemma}[Correctness of $\MS{Lookup}$][Lookup_Step_Realise]
  \label{lem:Lookup_Realise}
  $\MS{Lookup} \Realise LookupRel$
  with
  \small
  \begin{alignat*}{1}
    & LookupRel := \\
    &\quad \lambda t~(l, t').~\forall H~a~n.~ t[0] \simeq H \rightarrow t[1] \simeq_{f_{add}} a \rightarrow t[2] \simeq_{f_{var}} n \rightarrow isRight~t[3] \rightarrow isRight~t[4] \rightarrow \\
    &\quad\quad \MS{if}~l~\pthen~\bigl(\exists g.~H[a,n] = \Some{g} \land \\
    &\quad\quad \phantom{\MS{if}~l~\pthen~\bigl(\exists g.~} t'[0] \simeq H \land isRight~t'[1] \land isRight~t'[2] \land t'[3] \simeq_{f_{Clos}} g \land isRight~t[4]\bigr) \\
    &\quad\quad \phantom{\MS{if}~l}~\pelse~H[a,n]=\None.
\end{alignat*}
\end{lemma}

\subsection{$\MS{SplitBody}$}
\label{sec:SplitBody}
\setCoqFilename{ProgrammingTuringMachines.TM.LM.JumpTargetTM}%

The machine $\MS{SplitBody} : \TM_{\Sigma^+_{Pro}}^5(\Bool)$ computes the function $\phi$ (cf.\ Definition~\ref{def:jumpTarget}).  The
implementation of this machine is straight-forward, because $\phi'$ is defined tail-recursively.  The machine
$\MS{SplitBodyLoop} := \While~\MS{SplitBodyStep}$ computes $\phi'$, and $\MS{SplitBody}$ initialises the accumulators before executing the loop.

The step machine $\MS{SplitBodyStep} : \TM_{\Sigma^+_{Pro}}^5(\Option(\Bool))$ first makes a case-distinction over the program on tape $0$.  In the
$\nil$ case, it returns $\Some{\false}$ so that the loop returns $\false$.  In the cons-case, it makes a case-distinction over the command, using
$\MS{CaseCom}$.  In the case the head command is $\MS{RET}$, it does another case-distinction over $k$.  In the case that the head command is
$\MS{VAR}(n)$, the machine applies the $\MS{VAR}$ constructor to $n$ again and appends this command to $Q$.  The case-machines use the parametrised
auxiliary machine $\MS{AppACom}~t$ that appends a finite command (i.e.\ $t$ is either $\MS{APP}$, $\MS{LAM}$, or $\MS{RET}$) to the command list, and
the machine $\MS{AppCom}$ that appends a command on a tape to $Q$.

\begin{definition}[$\MS{SplitBodyStep}$][JumpTarget_Step]
  \label{def:JumpTarget}
  We define the canonical retraction $f_{Var} : Var \hookrightarrow \Sigma_{Pro}$ by $f_{Var} := RetrList \circ RetrInl$.
  %% This is a bad hack, but I think it looks nice...
  %% This hack assumes that $\MS{LAM}$ is the wider than of $\MS{RET}$ and $\MS{APP}$
  \begin{alignat*}{1}
    & \MS{SplitBodyStep} := \\
    & \quad \MS{If}~(\LiftTapes{\MS{CaseList}}{\Vector{0;3}}) \\
    & \quad \MS{Then}~\Switch~(\LiftBoth{\MS{CaseCom}}{RetrList}{\Vector{3}}) \\
    & \quad \phantom{\MS{Then}}~(\lambda (c : \Option(ACom)).~\MS{match}~c\\
    & \quad \phantom{\MS{Then}~(\lambda}~[~ \phalign{\Some{\MS{LAM}}}{\Some{\MS{RET}}} \Rightarrow \MS{If}~(\LiftBoth{\MS{CaseNat}}{f_{Var}}{\Vector{2}}) \\
    & \quad \phantom{\MS{Then}~(\lambda~[~ \Some{\MS{LAM} \Rightarrow}}~\MS{Then}~ \Return{(\LiftTapes{\MS{AppACom~\MS{RET}}}{\Vector{1;4}})}{\None} \\
    & \quad \phantom{\MS{Then}~(\lambda~[~ \Some{\MS{LAM} \Rightarrow}}~\MS{Else}~~~ \Return{(\LiftTapes{\MS{Reset}}{\Vector{2}})}{\Some\true} \\
    & \quad \phantom{\MS{Then}~(\lambda}~|~ \phalign{\Some{\MS{LAM}}}{\Some{\MS{LAM}}} \Rightarrow \Return{(\LiftBoth{\MS{ConstrS}}{f_{Var}}{\Vector{2}} \Seq \MS{AppACom}~\MS{LAM})}{\None} \\
    & \quad \phantom{\MS{Then}~(\lambda}~|~ \phalign{\Some{\MS{LAM}}}{\Some{\MS{APP}}} \Rightarrow \Return{(\MS{AppACom}~\MS{APP})}{\None} \\
    & \quad \phantom{\MS{Then}~(\lambda}~|~ \phalign{\Some{\MS{LAM}}}{\None} \Rightarrow \Return{(\LiftBoth{\MS{ConstrVar}}{f_{Var}}{\Vector{3}} \Seq \LiftTapes{\MS{AppCom}}{1;3;4})}{\None} \\
    & \quad \phantom{\MS{Then}~(\lambda}~]) \\
    & \quad \MS{Else}~~~ \Return{\Nop}{\Some\false}
  \end{alignat*}
\end{definition}

\begin{table}[t]
  % \centering
  \begin{tabular}{l||l|l|l|l}
    Input       & $\MS{CaseList}$ & $\MS{CaseCom}$ & $\MS{CaseNat}$ & $\MS{AppACom}~\MS{RET}$ \\ \hline
    $0:P$       & $0:P'$           &                 &                 &                         \\
    $1:Q$       &                  &                 &                 & $0: Q \app [\MS{RET}]$  \\
    $2:k$       &                  &                 & $0: k'$         &                         \\
    $3:~\dashv$ & $1: \MS{RET}$    & $0:~\dashv$     &                 &                         \\
    $4:~\dashv$ &                  &                 &                 & $1:~\dashv$             \\
  \end{tabular}
  \caption{Execution protocol of $\MS{SplitBodyStep}$ for $P=\MS{RET} \cons P'$ and $k=S~k'$.  The step machine terminates in the label $\None$, thus
    the loop continues.  Note that tape~$4$ is only used as an internal tape for $\MS{AppACom}$.}
  \label{tab:exec-JumpTarget-RET}
% \end{table}
% \begin{table}
%   \centering
  \begin{tabular}{l||l|l|l|l}
    Input       & $\MS{CaseList}$ & $\MS{CaseCom}$ & $\MS{ConstrVar}$ & $\MS{AppCom}$              \\ \hline
    $0:P$       & $0:P'$           &                 &                  &                            \\
    $1:Q$       &                  &                 &                  & $0: Q \app [\MS{VAR}(n)]$  \\
    $2:k$       &                  &                 &                  &                            \\
    $3:~\dashv$ & $1: \MS{VAR}(n)$ & $0: n$          & $0: \MS{VAR}(n)$ & $1:~\dashv$                \\
    $4:~\dashv$ &                  &                 &                  & $2:~\dashv$                \\
  \end{tabular}
  \caption{Execution protocol of $\MS{SplitBodyStep}$ for $P=\MS{VAR}(n) \cons P'$.  The step machine terminates in the label $\None$.}
  \label{tab:exec-JumpTarget-VAR}
\end{table}

Execution protocols for the step machine are visualised in Table~\ref{tab:exec-JumpTarget-RET} and Table~\ref{tab:exec-JumpTarget-VAR}.

Note that the step machine resets the tape for $k$ before it breaks out of the loop and returns $\true$.  Thus, no resetting is needed after the loop.
The loop machine halts in $\true$ if and only if $\phi'~k~Q~P$ is not $\None$.  If it is $\Some{(P',Q')}$, then after the execution the first two
tapes contain $P'$ and $Q'$, and the other tapes are right.  If $\phi'~k~Q~P = \None$, the correctness relation only commits that the machine
terminates in the label $\false$.
\begin{lemma}[Correctness of $\MS{SplitBodyLoop}$][JumpTarget_Loop_Realise]
  $\MS{SplitBodyLoop} \Realise SplitBodyLoopRel$.
  \begin{alignat*}{1}
    & SplitBodyLoopRel := \\
    &\quad \lambda t~(l, t').~\forall P~Q~k.~t[0] \simeq P \rightarrow t[1] \simeq Q \rightarrow t[2] \simeq k \rightarrow isRight~t[3] \rightarrow isRight~t[4] \rightarrow \\
    &\qquad \MS{if}~l~\pthen~\exists P'~Q'.~ \phi'~k~Q~P = \Some{(Q', P')} \land \\
    &\qquad \phantom{\MS{if}~l~\pthen}~t'[0] \simeq P' \land t'[1] \simeq Q' \land \left(\forall(i:\Fin_3).~isRigth~t'[2+i]\right) \\
    &\qquad \phantom{\MS{if}~l}~\pelse~\phi'~k~Q~P=\None
\end{alignat*}
\end{lemma}

Before the loop, the tapes for $Q$ and $k$ are initialised to $\nil$ and $0$:
\begin{definition}[$\MS{SplitBody}$][JumpTarget]
  We define $\MS{SplitBodyLoop} := \While~\MS{SplitBodyStep}$ and
  \[
    \MS{SplitBody} := \LiftTapes{\MS{ConstrNil}}{\Vector{1}} \Seq \LiftBoth{\MS{ConstrO}}{f_{Var}}{\Vector{2}} \Seq \MS{SplitBodyLoop}
  \]
\end{definition}
\begin{lemma}[Correctness of $\MS{SplitBody}$][JumpTarget_Realise]
  $\MS{SplitBody} \Realise SplitLoopRel$ with
  \begin{alignat*}{1}
    & SplitBodyRel := \\
    &\quad \lambda t~(l, t').~\forall P.~t[0] \simeq P \rightarrow isRigth~t[1] \rightarrow (\forall(i:\Fin_3).~isRigth~t[2+i]) \rightarrow \\
    &\qquad \MS{if}~l~\pthen~\exists P'~Q.~ \phi~P = \Some{(Q, P')} \land \\
    &\qquad \phantom{\MS{if}~l~\pthen}~t'[0] \simeq P' \land t'[1] \simeq Q \land \left(\forall(i:\Fin_3).~isRigth~t'[2+i]\right) \\
    &\qquad \phantom{\MS{if}~l}~\pelse~\phi~P=\None.
\end{alignat*}
\end{lemma}

\subsection{$\MS{Step}$}
\label{sec:heap-Step}
\setCoqFilename{ProgrammingTuringMachines.TM.LM.StepTM}%

We define a machine $\MS{Step}_\Sigma^{11} : \TM(\Option(\Unit))$ that simulates steps of the heap machine.  The tapes $0$, $1$, and $2$ contain the
control stack $T$, the argument stack $V$, and the heap $H$, respectively.  If the heap machine does a step $(T,V,H) \red (T',V',H')$, $\MS{Step}$
terminates in $\None$ and the resulting tapes contain $T',V',H'$.  Otherwise, if $(T,V,H)$ is a terminating state, $\MS{Step}$ terminates in the label
$\Some\unit$.  Furthermore, if $\MS{Step}$ terminates in $\None$, there is a successor state $(T',V',H')$.

We implement auxiliary machines for each of the three step rules.  $\MS{Step}$ first destructs $T$.  For example, in the case that
$T=(a, (\MS{APP} \cons P)) \cons T'$, the auxiliary machine for $\MS{APP}$ destructs $V$ further and then calls a machine $\MS{App}$ that realises
list-appending and a machine $\MS{Lenght}$ that computes the length of a list.  After that, it pushes the new closures to $T$ and $V$.  As another
example, if $T=(a, (\MS{VAR}(n) \cons P)) \cons T'$, the auxiliary machine for $\MS{VAR}$ calls $\MS{Lookup}$ and pushes the output closure to the
argument stack and the reset closure to the control stack.  If $\MS{Lookup}$ failed, $\MS{Step}$ immediately terminates in $\Some\unit$, indicating
that the state was a halting state.

The machine $\MS{Step} : \TM_{\Sigma^+}^{11}(\Option(\Unit))$ is defined on an arbitrary finite alphabet $\Sigma^+$ with retractions
$f_{Stack} : \Sigma_{List(Clos)} \hookrightarrow \Sigma$ and $f_{Heap} : \Sigma_{Heap} \hookrightarrow \Sigma$.  We omit the definition of $\MS{Step}$
here.  Although it is quite complex, no new innovations are required, and the designing of the machine was reasonable easy.  The machine first matches
on the control stack.  If it is empty, it immediately terminates in $\Some\unit$.

\begin{lemma}[Correctness of $\MS{Step}$][Step_Realise]
  \label{lem:Step_Realise}
  $\MS{Step} \Realise StepRel$ with
  \begin{alignat*}{1}
    & StepRel := \\
    &\quad \lambda t~(l, t').~\forall T~V~H.~ t[0] \simeq T \rightarrow t[1] \simeq V \rightarrow t[2] \simeq H \rightarrow \left(\forall (i:\Fin_8).~isRight~t[3+i]\right) \rightarrow \\
    &\qquad \MS{if}~l=\None~\pthen~\exists T'~V'~H'.~(T,V,H) \red (T',H',V') ~\land~ \\
    &\qquad \phantom{\MS{if}~l=\None~\pthen}~ t'[0] \simeq T' \land t'[1] \simeq V' \land t'[2] \simeq H' \land \left(\forall (i:\Fin_8).~isRight~t'[3+i]\right) \\
    &\qquad \phantom{\MS{if}~l=\None}~\pelse~ halt(T,V,H) ~\land~ T=\nil \rightarrow \\
    &\qquad \phantom{\MS{if}~l=\None~\pelse}~t'[0] \simeq \nil \land t'[1] \simeq V \land t'[2] \simeq H \land \left(\forall (i:\Fin_8).~isRight~t'[3+i]\right)
  \end{alignat*}
\end{lemma}

For each developed machine, we also have running time proofs, but we omitted the running time lemmas for shortness.  We have a function
\[
  stepSteps : \List(HClos) \to \List(HClos) \to Heap \to \Nat
\]
such that $\MS{Step}$ terminates in $stepSteps~T~V~H$ steps when the first three tapes contain $T$, $V$, $H$, and the internal tapes are right.
\begin{lemma}[Running time of $\MS{Step}$][Step_Terminates]
  \label{lem:Step_Terminates}
  $\MS{Step} \TerminatesIn StepT$ with
  \begin{alignat*}{3}
    & StepT &~:=~& \lambda t~k.~\exists T~V~H.~t[0] \simeq V \land t[1] \simeq T \land t[2] \simeq H \land \left(\forall i:\Fin_8.~isRight~t[3+i]\right) \land \\
    &       &    & \phantom{\lambda t~k.}~stepSteps~T~V~H \leq k
  \end{alignat*}
\end{lemma}

\subsection{$\MS{Loop}$}
\label{sec:heap-Loop}
\setCoqFilename{ProgrammingTuringMachines.TM.LM.HaltingProblem}%

We instantiate $\Sigma := \Sigma_{\List(Clos)} + \Sigma_{Heap}$ and define $\MS{Loop} := \While~\MS{Step}$.  This machine terminates if and only if
the heap state that is encoded on the tapes is a terminating state.  Moreover, if the heap state terminates with an empty control stack
$(\nil, V', H')$, the tapes contain $(\nil, V', H')$ after the execution.

\begin{lemma}[Correctness of $\MS{Loop}$][Loop_Realise]
  \label{lem:Loop_Realise}
  $\MS{Loop} \Realise LoopRel$ with
  \begin{alignat*}{1}
    & LoopRel := \\
    &\quad \lambda t~t'.~\forall T~V~H.~ t[0] \simeq T \rightarrow t[1] \simeq V \rightarrow t[2] \simeq H \rightarrow \left(\forall (i:\Fin_8).~isRight~t[3+i]\right) \rightarrow \\
    &\qquad \exists T'~V'~H'.~ \left(T,V,H\right) \terminates^* \left(T',V',H'\right) \land \\
    &\qquad T' = nil \rightarrow t'[0] \simeq \nil \rightarrow t'[1] \simeq V' \rightarrow t'[2] \simeq H' \rightarrow \left(\forall (i:\Fin_8).~isRight~t'[3+i]\right)
  \end{alignat*}
\end{lemma}

For the running time function $loopSteps$, we need the functions $step$ and $isHalt$ from Lemma~\ref{lem:heap-red}.  The running time-function of
$\MS{Loop}$ is defined per recursion over the number $k'$ of reduction steps of the heap machine.
\begin{lemma}[Running time of $\MS{Loop}$][Loop_Terminates]
  \label{lem:Loop_Terminates}
  $\MS{Loop} \TerminatesIn LoopT$ with
  \begin{alignat*}{2}
    &LoopT &~:=~& \lambda t~k.~\exists T~V~H.~\exists T'~V'~H'~k'.~ \left(T,V,H\right) \terminates^{k'} \left(T',V',H'\right) \\
    &      &    & \phantom{\lambda t~k.}~t[0] \simeq T \land t[1] \simeq V \land t[2] \simeq H \land \left(\forall i:\Fin_8.~isRight~t[3+i]\right) \land \\
    &      &    & \phantom{\lambda t~k.}~loopSteps~T~V~H~k' \leq k
  \end{alignat*}
  with
  \small
\begin{lstlisting}[style=semicoqstyle]
$loopSteps~T~V~H~k := ~$
  match$k$
  [$0$ => $stepSteps~T~V~H$
  |$S~k'$ =>
   match$step(T, V, H)$
   [$\Some{(T',V',H')}$ =>
    if $isHalt(T',V',H')$
    then $1 + stepSteps~T~V~H + stepSteps~T'~V'~H'$
    else $1 + stepSteps~T~V~H + loopSteps~T'~V'~H'~k'$
   |$\None$ => $stepSteps~T~V~H$
   ]
  ].
\end{lstlisting}
\end{lemma}

Lemma~\ref{lem:Loop_Realise} says that if $\MS{Loop}$ with tapes that encode $(T,V,H)$ terminates, then heap machine also terminates.  Dually,
Lemma~\ref{lem:Loop_Terminates} says that if heap machine terminates, so does $\MS{Loop}$ when we encode $(T,V,H)$ on the tapes.

The last step of the reduction of the halting problem of the heap machine to the halting problem of multi-tape Turing machines is to define a function
\[
  initTapes : \List(Clos) \times \List(Clos) \times Heap \to \Tape_\Sigma^{11}
\]
such that the first three tapes of $initTape(T,V,H)$ contain $T$, $V$, or $H$, respectively, and the rest tapes are right.

\begin{definition}[$initTapes$][initTapes]
  \label{def:initTapes}
  ~
  \[
    initTapes(T,V,H) := initValue(T) \cons initValue(V) \cons initValue(H) \cons initRight^8
  \]
  \setCoqFilename{ProgrammingTuringMachines.TM.Code.CodeTM}%
  With $\coqlink[initValue]{initValue}(x) := \MS{midtape}~\nil~\MS{START}~(encode(x) \app [\MS{STOP}])$ and
  $\coqlink[initRight]{initRight} := \MS{midtape}~\nil~\MS{STOP}~\nil$.  \setCoqFilename{ProgrammingTuringMachines.TM.LM.HaltingProblem}%
\end{definition}

\begin{theorem}[Halting problem reduction][HaltingProblem]
  \label{lem:HaltingProblem}
  Let $(T,V,H)$ be a heap state.  Then
  \[
    \left(\exists T'~V'~H'.~(T,V,H) \terminates^* (T',V',H') \right) \iff \left(\exists c~k.~\MS{Loop}(initTapes(T,V,H)) \terminates^k c)\right)
  \]
\end{theorem}

%%% Local Variables:
%%% mode: LaTeX
%%% TeX-master: "thesis"
%%% End:


\section{Closed Jackson Networks}
\label{sec:Closed-Jackson-Networks}

We examine only the Batch Jackson Networks.
See \cite{harchol2013performance} to examine Interactive Jackson Networks.

\begin{definition}[Batch Jackson Network]
	\label{def:Batch-Jackson-Network}
	A Batch Jackson Network is a closed network of queues such that:

	\begin{itemize}
		\item there are $m$ $M/M/1$ queues;
		\item the multiprogramming level is $N$;
		\item the $i$-th server service is Exponentially distributed with rate $\mu_{i}$;
		\item each server may receive only internal arrivals;
		\item when the $i$-th server completes a job, it is routed to the $j$-th server (internal arrival) with probability $P_{i,j}$.
	\end{itemize}
\end{definition}

%\begin{figure}[tp]
%\label{fig:Batch-Jackson-Network}
%	\centering
%	\includegraphics{fig/Batch-Jackson-Network}
%	\caption{A Batch Jackson Network and its corresponding CTMC.}
%\end{figure}

The \textit{response time} is defined as the time from when the job arrives to the system until it leaves it, including possible multiple visitation to the same server.

The \textit{total arrival rate} $\lambda_{i}$ to the $i$-th server is also the total departure rate from the same server.

In such networks it not possible to determine a unique solution for all the $\lambda_{i}$'s, because we face with a $k-1$-order system of equations. So, we have to choose a $\lambda_{i}$ at random. Typically we fix $\lambda_{1}$ at random and then resolve the others \footnote{Since the $\lambda_{i}$'s rely on a made-up value, the $\varrho_{i}$'s do not represent real servers utilizations}.

\begin{theorem}[Batch Jackson Network Product Form]
\label{thm:Batch-Jackson-Network-Product-Form}
	A Closed Jackson Network with $m$ servers has the following product form

	\begin{equation}
	\label{eqn:Batch-Jackson-Network-Product-Form}
	\pi_{n_{1},...,n_{m}} = C \cdot \prod_{i=1}^{m} \varrho_{i}^{n_{i}} (1-\varrho_{i})
	\end{equation}

	where $C$ is the \textit{Normalizing Constant} determined as the solution of

	$
	\sum_{(n_{1},...,n_{m}) \\ \in States} \pi_{n_{1},...,n_{m}} = 1
	$

	\begin{proof}
		The demonstration makes use of the \textit{Mean Value Analysis}.
		For a formal demonstration, see \cite{harchol2013performance}.
	\end{proof}
\end{theorem}

The complexity of determining state probabilities with \Cref{eqn:Batch-Jackson-Network-Product-Form} growths exponentially with $N$ and $m$
\footnote{The total number of states in a Closed Jackson Network is equal to the number of ways of dividing $N$ jobs among $m$ servers, that is
	\begin{equation*}
	\label{eqn:Batch-Jackson-Networks-Number-States}
	Number \! of \! States = \binom{N+m-1}{m-1}
	\end{equation*}
}.

When $N$ and/or $m$ get high, it is better to apply the \textit{Mean Value Analysis}.