\chapter{Probabilistic Programs}\label{chap:preliminaries}

This section formalises necessary ingredients and the~problem statement for probabilistic synthesis.
We will first introduce a~discrete-time Markov chain as the~simplest operation model for probabilistic programs.
Further,~we will describe a~Markov decision process representing a~similar model as a~Markov chain,~but it contains additional non-determinism.
These models play a~crucial role when synthesising probabilistic programs,~as we will see later in Chapter~\ref{chap:synthesis_methods}.
The~following definitions are inspired from the~existing sources,~mainly from~\cite{tacas21,Quatmann2016},~where a~more detailed description can also be found.

\section{Probabilistic Models}

% \MC{Navrhuju napsat nejakou uvodni vetu, ze MC je semantika prob programu a podobne trochu nize ze MDP nam dovoluji uvazovat o mnozine prog programu.}
We denote a~finite set of \textit{parameters} as $V$.
We consider \textit{parameters} over the~domain $\real$ ranged over by $x,y,z$ for the~following definitions.
Let $u: V \rightarrow \real$ denote a~\textit{valuation} for \textit{parameters} $V$.
We denote a~set of \textit{multi-affine multivariate polynomials} as $\mathbb{Q}_V$,~where polynomial $f$ is defined over parameters $V$ and equal to $\sum_{i \leq M}{\prod_{v\in V_i}v \cdot a_i}$,~for appropriate $\mathcal{M} \in \nat$,~$a_i \in \rat$,~and $\forall \, 0 \leq i \leq m. \; V_i \subseteq V$.
We note that this set includes only the~polynomials where the~variables have the~maximal degree equal to one,~i.e.,~$y^3 \notin \mathbb{Q}_V$,~but $y \cdot z \in \mathbb{Q}_V$.
When we apply the~valuation $u$ to polynomial $f \in \mathbb{Q}_V$,~the~resulting instance yields a~real number $f[u] \in \real$,~in which all occurrences of variable $x \in f$ are replaced by $u(x)$.
We consider various types of (parametric) discrete probabilistic models.
We can look at them as the~transition systems with the~given state space where the~transitions are marked with polynomials in $\mathbb{Q}_V$~\cite{Quatmann2016}.

\begin{definition}[pMDP]
A~parametric Markov decision process (pMDP) $\mathcal{M}$ is a tuple $(S, V, s_0, Act, \mathcal{P})$, where $S$ is a~finite set of states,~$V$ is a~finite set of parameters over $\real$,~$s_0 \in S$ is an initial state,~$Act$ is a~non-empty finite set of actions,~and $\mathcal{P}: S \times Act  \times S \nrightarrow \mathbb{Q}_V$ is a~transition function.
\end{definition}
\noindent
A~set $\mathit{Act(s) = \{ a~\in Act \; \lvert \; \exists s' \in S. \mathcal{P}(s, a, s') \neq \; \perp \}}$ for state $s \in S$ represents the~\textit{available} actions.
The~model do not contains deadlock states,~since for each state $s \in S$ holds that the~action set $Act(s)$ is non-empty.
When holds $\lvert \mathit{Act(s)} \rvert = 1$ for each state $s \in S$,~such pMDP straightforward induces an~parametric Markov chain (pMC),~which has enabled only one action at each state.

\begin{figure*}[h!]
\centering
\includegraphics[width=1.0\textwidth]{figures/param_models.pdf}
\caption{The~introduced kinds of parametric probabilistic models \,--\, pMDP and pMC.}%
\label{fig:param_models}%
\end{figure*}

\begin{example}[Parametric models]
We depict (a) pMDP and (b) pMC with parameters $\{x, y\}$ in Figure~\ref{fig:param_models}.
We mark an~initial state $s_0 \in S$  with an~arrow,~and the~target state $s_3 \in S$ with double lines.
We label the~transition from state $s \in S$ to $s' \in S$ by action $\alpha$ and corresponding $\mathcal{P}(s, \alpha, s')$,~if such transition is executable.
\end{example}

\begin{definition}[Distribution]
\cite{tacas21}
    A~\textit{discrete} distribution over a~finite or countably infinite set $X$ is a~function $\mu: X \rightarrow [0,1]$ s.t. $\sum_{x \in X} \mu(x) = \mu(X) = 1$.
    The~set of all distributions on $X$ is denoted $Distr(X)$.
    The~support of a~distribution $\mu$ is $supp(\mu) = \{ x \in X \, \lvert \; \mu(x) > 0 \}$.
    % A~distribution is \textit{Dirac} if $\lvert supp(\mu) \rvert = 1$.
\end{definition}

\begin{example}[Parametric probabilistic models]
    Let $X = \{x_0, x_1, x_0\}$ be a~finite set.
    Let function $\mu: X \rightarrow [0,1]$ defined as $\mu: [x_0 \mapsto \frac{1}{2}, x_1 \mapsto 0, x_2 \mapsto \frac{1}{2}]$ be a~\textit{probability distribution} on $X$, i.e. $\mu \in Distr(X)$.
    The~support of $\mu$ is $supp(\mu) = \{x_0, x_2 \}$, and for simplification,~we writes such distributions as $\mu = \frac{1}{2} : x_0 + \frac{1}{2} : x_2$.
\end{example}

\begin{definition}[MDP]
    A~pMDP $\mathcal{M}$ is Markov decision process (MDP) $\mathcal{M}$ if $\mathcal{P}: S \times Act \times S \rightarrow Distr(S)$,~so for each state $s \in S$ and each its action $\alpha~\in Act(s)$ holds that $\sum_{s' \in S}{\mathcal{P}(s, \alpha, s')} = 1$.
\end{definition}
\noindent
We can define in the~same way a~MCs as the~special instance of pMCs.
We say,~that the~model is \textit{parameter-free} when for each its transition probability holds,~that it contains only constant values and not undefined parameters.
When we apply the~valuation $u$ to the~parametric model $\mathcal{M}$,~then we obtain the~instantiation of $\mathcal{M}$ at $u$ ($\mathcal{M}[u]$),~where each polynomial $f \in \mathcal{M}$ is replaced by $f[u]$.
We use the~valuation $u$ typically to substitute the~transition function $f$ by the~concrete probability $f[u]$.
When the~instantiated model $\mathcal{M}[u]$ yields an~MDP or MC,~we denote the~valuation $u$ as \textit{well-defined} since it returns the~probability distributions.

\begin{figure*}[h!]
\centering
\includegraphics[width=1.0\textwidth]{figures/models.pdf}
\caption{The~introduced kinds of probabilistic models \,--\, MDP and MC.}%
\label{fig:models}%
\end{figure*}

\begin{example}[Probabilistic models]
Figure~\ref{fig:param_models} depicts an pMDP and an pMC.
Let us consider the~following valuation $u: \{ x \mapsto 0.5, y \mapsto 0.75 \}$.
By applying this valuation to an~pMDP results in parameter-free MDP depicted in Figure~\ref{fig:models},~and the~same in the~case of MC instantiated from relevant pMC.
\end{example}

Instinctively,~we can imagine an~$MC$ as a~state-transition system with the~following semantics.
A~probability distribution for each state $\forall s \in S: \mathcal{P}(S)$ represents a~stochastic choice of firing the~transition from such state $s \in S$ to one of its successors states $s' \in supp(\mathcal{P(S)})$.
Consequently,~an~$MC$ defined with such semantics has a~\textit{Markov property} (memorylessness),~which is essential when modelling systems and for efficient analysis.
This property declares that the~probability of the~transition from state $s \in S$ to state $s' \in supp(\mathcal{P(S)})$ depends only on the~current state,~and it is independent of the~taken path of chain to state $s$.
We can see that each state of an $MC$ disposes of a~unique probability distribution over its possible successor states.
MDPs define an~extension of $MCs$ introducing a~non-deterministic choice between several probability distributions over each state.

A~(in)finite sequence $\pi = s_0 \overset{a_0}{\rightarrow} s_1 \overset{a_1}{\rightarrow} \dots$,~where $s_i \in S$,~$a_i \in Act(s_i)$,~and $\mathbb{P}(s_i, a_i)(s_{i+1}) \neq 0$ for all $i \in \mathbb{N}$ represents the~\textit{path} of an~MDP M.
For finite $\pi$, $last(\pi)$ denotes the~last state of $\pi$,~and the~set of (in)finite paths of M we denotes as $Paths_{fin}^{M}\,(Paths^M)$.
When an~$MDP$ is currently in the~state $s \in S$,~it has a~non-deterministic choice of an action $a \in Act(s)$ leading to the~one possible probability distribution $\mathcal{P}(s)(a)$ over its successors' states.
These actions cause the~non-deterministic behaviour of an~$MDP$.
Still,~this property can be suppressed by applying a~\textit{scheduler},~which selects one specific action in each state,~i.e.,~transforms an~$MDP$ to an~$MC$.

% \begin{definition}[MDP]
% \cite{roman-DP}
%     A Markov decision process (MDP) $M$ is a~quadruple $\mdp$ ~where $S$ is a~finite set of states,~$s_0 \in S$ is an~initial state,~$Act$ is a~finite set of actions,~and $\mathcal{P}: S \times Act  \nrightarrow Distr(S)$ represents a~(partial) transition probability function.
% \end{definition}

% A~set $\mathit{Act(s) = \{ a \in Act \; \lvert \; \mathcal{P}(s, a) \neq \; \perp \}}$ for state $s \in S$ represents the~\textit{available} actions.
% When holds $\lvert \mathit{Act(s)} \rvert = 1$ for each state $s \in S$,~such MDP straightforward induces an~MC.

\begin{definition}[Scheduler]
A~scheduler for an pMDP $\mathcal{M} = (S, V, s_0, Act, \mathcal{P})$ is a~function $\sigma: Paths_{fin}^{M} \rightarrow Act$ such that $\sigma(\pi) \in Act(last(\pi))$ for all $\pi \in Paths_{fin}^{M}$.
Scheduler $\sigma$ is memory-less if $last(\pi) = last(\pi') \Longrightarrow \sigma(\pi) = \sigma(\pi')$ for all $\pi, \pi' \in Paths_{fin}^{M}$.
The~set of all schedulers of M is $\Sigma^M$.
\end{definition}
\noindent
When we apply a~scheduler to an~pMDP,~then we obtain an~induced parametric Markov chain,~which does not contain any non-determinism.
In other words,~the~transition probabilities are obtained concerning the~choice of actions.

\begin{definition}[Induced pMC] \label{def:incuded_mc}
\cite{cegar}
An~pMC induced by pMDP $\mathcal{M} = (S, V, s_0, Act, \mathcal{P})$ and $\sigma \in \Sigma^M$ is defined by $\mathcal{M}_{\sigma} = ( Paths_{fin}^{M}, s_0, \mathbf{P}^{\sigma})$ where:
\begin{align*}
    \mathbf{P^{\sigma}}(\pi, \pi') =
    \begin{cases}
        \mathcal{P}(last(\pi), \sigma(\pi))(s') \quad & if \; \pi' = \pi \overset{\sigma(\pi)}{\rightarrow} s' \\
         0 \quad & otherwise.
    \end{cases}
\end{align*}
\end{definition}

% \section{Probabilistic Synthesis}
% \MC{Potrebujeme tu definici sketch -- navrhuji to vynechat a definici ruznych syn problemu udelat az v 2.4. nad family of MC -- potom v te sekci pridat veticku, ze modeluje pomoci sketching language, ktery je popsan v 5.2}
% We use a~\textit{sketch}~\cite{sketching1,sygus},~which represents a~incomplete high-level description of a~probabilistic system,~to describe the~family of MCs.
% It usually describes the~modelled system's fixed behaviour,~but it also contains the~undefined one represented by \textit{holes}.
% They represent the~incomplete parts of a~program that must be instantiated so that the~final description satisfies a~given specification.
% We note that the~holes correspond to parameters,~and their instantiation yields a~concrete Markov chain.
% At follow,~we formalise the~sketch defining the~set of designs:

% \begin{definition}[Sketch]
% % \todo{Add parameters}
% Let $\sketch$ be a~sketch containing holes from the~set $\mathcal{H} = \left\{ H_k \right \}_k$ with $R_k$ being the~set of options available for hole $H_k$.
% Let $\rlzf = \prod_k R_k$ denote the~set of all hole assignments (realizations),~$\sketch[r]$ denote the~program induced by a~substitution $r \in \rlzf$ and $\mathcal{D}_r$ denote the~underlying MC.
% Note that the~set $\rlzf$ is exponential in $\lvert \mathcal{H} \rvert$.
% \end{definition}
% \noindent

\section{Region Model Checking} \label{sec:regional_model}
% \MC{Pridat nejakou motivacni vetu, proc je to dulezite v kontextu syntezy -- dovoluje nam to efektivne analyzovat mnoziny MC}
When considering the~model checking of pMCs at parameter synthesis problem,~we have to analyse the~whole set of MCs representing the~individual parameters substitution.
For this purpose,~we introduce the~region model checking,~a~technique to practical analyses such sets of MCs.
We consider valuation sets that mapping parameter to the~value within a~specified interval.
We introduce a~technique for approximate checking pMC instances concerning the~valuation in such a~set.
Firstly,~we introduce definitions for \textit{regions} and the~considered task of model checking.
Subsequently,~we define the~construction of \textit{over-approximation},~which will be subsequently \textit{reduce} to an~MDP problem.
The~definitions in this section are inspired mainly by~\cite{Quatmann2016}.

\subsection{Regions}
\begin{definition}[Region]
Let $V = \{ v_1, v_2, \dots, v_n \}$ be a~set of parameters and $\forall \, v_i \in V. \; \mathcal{B}(v_i) = \{b_1, b_2 \}$ rational bounds for each parameter $v_i$.
These parameter bounds induce for parameter $v_i$ its interval $I(v_i) = [b1, b2]$ where holds $b1 \leq b2$.
Let $U = \{ u \; \lvert \; \forall \, v_i \in V. \; u(v_i) \in I(v_i) \}$ be a~set of valuations,~which we call a~region for parameters set $V$.
\end{definition}
\noindent
Note that regions represent instances $\mathcal{M}[u]$ derived from the~parametric model $\mathcal{M}$.
As we presented above,~these model instances are \textit{well-defined} when they hold some assumptions,~so we shift them also to regions.

\begin{definition}[Well-defined region]
Let $\mathcal{M}$ be a~parametric probabilistic model.
Let $r$ be a~region for set of parameters $V$,~we call the~region $v$ \textit{well-defined} for $\mathcal{M}$,~when holds for each \textit{polynomial} $f \in \mathcal{M}$ either $f[u] > 0$ or  $f$ can be reduced to 0 $f = 0$,~and for each \textit{valuation} $u \in r$ holds that $u$ is \textit{well-defined} for model $\mathcal{M}$.
\end{definition}
\noindent
The~first condition guarantees that both models $\mathcal{M}$ and $\mathcal{M}[u]$ have the~same topology,~and the~second one says that $\mathcal{M}[u]$ represents a~probabilistic model (MDP or MC).

\begin{figure*}[h!]
\centering
\includegraphics[width=1.0\textwidth]{figures/param_instance.pdf}
\caption{A~\textit{parametric} MC $\mathcal{D}$,~and instantiated MC $\mathcal{D}[u]$ from it.}%
\label{fig:param_instance}%
\end{figure*}

\begin{example}[Region]
Let us consider the~pMC $\mathcal{D}$ depicted in Figure~\ref{fig:param_instance}(a),~the~region $r$ defined as $r = [0.1, 0.5] \times [0.5, 0.6]$ and valuation derived from it $u = (0.1, 0.55)$.
Applying a~valuation $u$ to parametric model $\mathcal{D}$,~yields a~parameter-free MC $\mathcal{D}[u]$ depicted in Figure~\ref{fig:param_instance}(b).
Note,~that this instantiated model $\mathcal{D}[u]$ has the~same topology as the~original pMC.
We can say that the~considered region $r$ is \textit{well-defined},~since this holds for each possible valuation $u' \in R$,~respectively for an~instance $\mathcal{D}[u']$ derived from it.
At the~contrary,~the~region $r' = [0.0, 0.5] \times [0.75, 1.0]$ is not \textit{well-defined},~because for example the~valuation $v' = (0.0, 1.0)$ yields an~MC which has no transition from $s_0$ to $s_1$,~or from $s_1$ to $s_3$,~so it has different topology as original pMC $\mathcal{D}$.
\end{example}

We will introduce a~\textit{satisfaction relation} for regions,~which defines whether the~given region $r$ and parametric model $\mathcal{M}$ satisfies the~specification $\varphi$.
We denote $\mathcal{M}, r \models \varphi$ when each instance derived from the~parametric model $\mathcal{M}$ described with a~region $r$ satisfies the~analysed specification $\varphi$.

\begin{definition}[Region satisfaction relation]
The~relation $\models$ for a~well-defined region $r$,~a~parametric model $\mathcal{M}$,~and a~given specification $\varphi$ is defined as follows:
\begin{align*}
    \mathcal{M}, r \models \varphi \Longleftrightarrow \forall u \in r. \; \mathcal{M}[u] \models \varphi
\end{align*}
\end{definition}
\noindent
When holds $\mathcal{M}, r \not\models \varphi$,~then this statement implies $\exists u \in r. \; \mathcal{M}[u] \not\models \varphi$.
On the~contrary,~when holds $\mathcal{M}, r \models \neg\varphi$ which implies $\forall u \in r. \; \mathcal{M}[u] \not\models \varphi$,~this is different from the~first case.
Let $\mathcal{D} = (S, V, s_0, \mathcal{P})$ be an pMC,~$r$ a~well-defined region for this pMC $\mathcal{D}$,~and $\varphi = \reachability{\geq}{\lambda}{T}$ a~given reachability property.
When we want to conclude whether the~region $r$ is \textit{live} or not,~we need to compute for each valuation $u \in r$ the~corresponding (\textit{minimal} or \textit{maximal}) reachability probability.
We define the~following equivalences to detect whether given \textit{region} $r$ is \textit{live} or not:
\begin{align*}
    \mathcal{D},~r \models \varphi \Longleftrightarrow (\argmin_{u \in r} \prob[\mathcal{D}[u] \models F \; T]) \geq \lambda \\
    \mathcal{D},~r \models \neg\varphi \Longleftrightarrow (\argmax_{u \in r} \prob[\mathcal{D}[u] \models F \; T]) < \lambda \\
\end{align*}
Since the~corresponding probability $\prob[\mathcal{D}[u], r \models F \; T]$ can be represented as a~rational function $f$,~and the~region $r$ is well-defined,~there exists the~valuation inducing the~minimal or maximal reachability probability on considered region $r$.

\begin{example}[Region model checking]
Let $\mathcal{D}$ be an~MC depicted in Figure~\ref{fig:param_instance}(a),~and we  consider a~region $r = [0.1, 0.5] \times [0.5, 0.6]$.
Let us suppose,~that we want to obtain a~valuation $u \in r$ maximising $\prob[\mathcal{D}[u], r \models F \; \{s_3 \}]$,~so the~probability that from an~initial state $s_0$ will be reached the~state $s_3$.
Let us observe,~that the~single state from which the~state $s_3$ is not reachable is the~state $s_4$,~which is reachable only from state $s_2$.
Therefore,~to achieve the~highest probability will be the~best strategy minimising the~probability to reach state $s_2$.
We can conclude,~that the~parameter $x$ should has its value $u(x)$ as high as possible,~i.e.,~$u(x)= 0.5$.
Now,~we consider the~state $s_1$,~when we want to reach state $s_3$ from it,~then the~parameter $y$ should has its value $u(x)$ as low as possible.
However,~at the~same time we would need the~highest possible value for the~parameter $y$ to reach state $s_1$ from state $s_2$.
An~exhaustive analysis is required to find an~optimal value for parameter $y$,~due to this trade-off.
\end{example}

\subsection{Substitution of pMCs}
We will introduce a~technique based on the~dropping of parameters dependencies in the~different states.
Thanks to the~removal of these dependencies,~we can then effectively compute an~\textit{optimal} solution.
In order to compute minimal (or maximal) probability we have to make a~discrete choice over valuations $\mathit{v}: V \rightarrow \real$ with $\mathit{v}(v_i) \in \mathcal{B}(v_i)$.
The~key idea consists of doing such a~choice locally at each state,~which allow us to construct the~\textit{parameter-free MDP} out of the~pMC $\mathcal{D}$ and the~region $r$.
The~non-deterministic choices in this constructed MDP represent each valuation which is required to consider.

\begin{definition}[Substitution-pMC] \label{def:sub_mc}
Let $\mathcal{D}_{sub}^{r} = (S, s_0, Act_{sub}, \mathcal{P}_{sub})$ be an MDP representing the~parameter-substitution of a~pMC $\mathcal{D} = (S, V, s_0), \mathcal{P})$ and a~region $r$,~where holds $Act_{sub} = \biguplus_{s \in S}\{ \mathit{v}: V_s \rightarrow \real \; \lvert \; \mathit{v}(v_i) \in \mathcal{B}(v_i) \}$ and
\begin{align*}
    \mathcal{P}_{sub}(s, \mathit{v}, s') =
    \begin{cases}
        \mathcal{P}(s, s')[\mathit{v}] \quad & if \; v \in Act_s \\
        0 \quad & otherwise.
    \end{cases}
\end{align*}
\end{definition}
\noindent
Intuitively,~the~parameter-substitution $\mathcal{D}_{sub}^r$ of a~pMC $\mathcal{D}$ and a~region $r$ chooses an~action $\mathit{v}$ in state $s$ which corresponds to the~assigning of extremal values $\mathcal{B}(v_i)$ to the~parameter $v_i$.
Note that the~non-deterministic choices introduced by this substitution depend only on the~values $\mathcal{B}(v_i)$ of the~parameter $v_i$ in the~region $r$.

\begin{example}[Substitution-pMC]
Let us consider pMC $\mathcal{D}$ depicted in Figure~\ref{fig:param_instance}(a) and also the~introduced region $r = [0.1, 0.5] \times [0.5, 0.6]$ as above.
Figure~\ref{fig:sub_mc}(a) depicts the~\textit{substitution} of MC $\mathcal{D}$,~which includes non-deterministic choices instead of outgoing transition from states $s_0, s_1, s_2$ in the~original pMC $\mathcal{D}$.
In more precise,~we choose the~\textit{upper} or \textit{lower} bound for the~corresponding interval of the~parameter in each state.
The~green lines depict the~transitions that belong to the~action for the~lower bound,~and red lines for the~upper bound.
We note,~that states $s_3$ and $s_4$ contains only constant outgoing transition and therefore they have not any choices.
Figure~\ref{fig:sub_mc}(b) depicts an~MC $(\mathcal{D}_{sub_r})^\sigma$ which is induced by the~scheduler $\sigma$ on MDP $\mathcal{D}_{sub_r}$.
\end{example}

\begin{figure*}[h!]
\centering
\includegraphics[width=1.0\textwidth]{figures/sub_mc.pdf}
\caption{A~\textit{parameter-substitution} MDP $\mathcal{D}_{sub_r}$,~and instantiated MC $(\mathcal{D}_{sub_r})^\sigma$.}%
\label{fig:sub_mc}%
\end{figure*}
\noindent
According to Theorem 1 which was defined and proved in~\cite{Quatmann2016},~we list the~following:

\begin{theorem}[]
Let $\mathcal{D} = (S, V, s_0, \mathcal{P})$ be an~pMC,~$r$ a~region defined over this pMC $\mathcal{D}$,~and $T \subseteq S$ a~set of target states of $\mathcal{D}$,~then holds:
\begin{align*}
    \argmin_{u \in r} \prob[\mathcal{D}[u] \models F \; T] \geq \argmin_{\sigma \in \Sigma^{\mathcal{D}_{sub}^{r}}} \prob[(\mathcal{D}_{sub}^{r})^\sigma \models F \; T] \\
    \argmax_{u \in r} \prob[\mathcal{D}[u] \models F \; T] \leq \argmax_{\sigma \in \Sigma^{\mathcal{D}_{sub}^{r}}} \prob[(\mathcal{D}_{sub}^{r})^\sigma \models F \; T]
\end{align*}
\end{theorem}
\noindent
We define the~following consequence straightforward from this statement:
\begin{theorem} \label{def:sub_mc_check}
Let $\mathcal{D} = (S, V, s_0, \mathcal{P})$ be an pMC,~$r$ a~well-defined region defined over this pMC $\mathcal{D}$,~then holds:
\begin{align*}
    \mathcal{D}_{sub}^{r} \models \reachability{\leq}{\lambda}{T} \Longrightarrow \mathcal{D}, r \models & \; \reachability{\leq}{\lambda}{T} \\
    \mathcal{D}_{sub}^{r} \models \reachability{>}{\lambda}{T} \Longrightarrow \mathcal{D}, r \models & \; \neg\reachability{\leq}{\lambda}{T}
\end{align*}
\end{theorem}
\noindent
Therefore,~we can use the~standard approaches for MDP model checking to check whether $\mathcal{D}, r \models \varphi$,~by applying these techniques to model $\mathcal{D}_{sub}^r$.
When the~model checking of this considered over-approximation yield an~inconclusive results,~we refine the~regions to sub-regions and repeat the~control.

\subsection{Substitution of pMDPs}
In the~previous section,~we introduced a~technique to apply the~standard MDP model checking to the~parametric MCs.
This technique substitutes outgoing transitions with non-deterministic choices representing the~relevant bounds of the~parameter related to this transition.
Now,~we generalise this technique to parametric MDPs,~or in general,~systems that include non-determinism.
In addition to the~choices over actions that the~MDP implicitly contains,~this technique also adds options over valuations concerning the~considered parameters.
This added level of non-determinism leads to a~game with two players.
The~first player represents the~original MDP non-determinism,~and the~second player the~abstracted parameters,~which results in a~stochastic game.

\begin{example}[MDP Substitution]
Let $\mathcal{M} = (S, V, s_0, Act, \mathcal{P})$ be an~pMDP depicted in Figure~\ref{fig:sub_mdp}(a).
Note that an~initial state $s_0$ can choice from two actions $\alpha$ and $\beta$.
When we consider the~scheduler $\sigma = \{ s \mapsto \beta \}$,~then by applying it to pMDP $\mathcal{M}$ we obtain pMC,~which we can subsequently transform to parameter-substitution MC,~as depicted in Figure~\ref{fig:sub_mdp}(b).
The~parameter substitution of a~MDP $\mathcal{M}$ returns an~SG $\mathcal{G}$ which is depicted in Figure~\ref{fig:sub_mdp}(d).
\end{example}

\begin{figure*}[h!]
\centering
\includegraphics[width=1.0\textwidth]{figures/mdp_sub.pdf}
\caption{Illustration of the~substitution of a~pMDP.}%
\label{fig:sub_mdp}%
\end{figure*}
\noindent
When constructing the~\textit{parameter substitution} of a~pMDP,~we first need to construct its substitution,~which separates probabilistic choices from non-deterministic actions.
In the~first step,~we split each state $s \in S$ into set $\{s \} \; \uplus \; \{ (s, \alpha) \; \lvert \; \alpha \in Act(s) \}$,~and then we add a~transition with the~probability of $1$ from state $s$ to state $(s, \alpha)$ and move the~probabilistic choice concerning action $\alpha$ from $s$ to $(s, \alpha)$.
After applying of these operations,~we obtain a~pMDP $\mathcal{M}'$,~depicted in Figure~\ref{fig:sub_mdp}(c).
We can see that this MDP has clear non-deterministic choices leading to the~newly added states of the~form $(s, \alpha)$,~which on the~contrary,~have clear probabilistic choices.
As we said above,~by introducing parameter substitution on the~probabilistic states,~we obtain the~stochastic game $\mathcal{G}$,~depicted in Figure~\ref{fig:sub_mdp}(d).
The~first player represents the~original pMDP non-determinism,~while the~second one decides whether the~lower or upper bounds of the~parameters will be set.

\begin{definition}[Substitution-pMDP] \label{def:sub_mdp}
Let $\mathcal{M} = (S, V, s_0, Act, \mathcal{P})$ be an~pMDP,~and $r$ well-defined region.
A~\textit{stochastic game} $\mathcal{M}_{sub}^r = (S_{\circ} \uplus S_{\square}, s_0, Act_{sub}, \mathcal{P}_{sub})$ is the~parameter-substitution of $\mathcal{M}$ over $r$ if $S_{\circ} = S$, $S_{\square} = \{ (s, \alpha) \; \lvert \; \alpha \in Act(s) \}$, $Act_{sub} = Act \uplus (\biguplus_{(s, \alpha) \in S_{\square}}Act_s^\alpha)$ where $Act_s^\alpha = \{\mathit{v}: V_s^\alpha \rightarrow \real \; \lvert \; \mathit{v}(v_i) \in \mathcal{B}(v_i) \}$,~and
\begin{align*}
    \mathcal{P}_{sub}(t, \beta, t') =
    \begin{cases}
        1 \quad & if \; t \in S_\circ \; and \; t' = (t, \beta) \in S_\square, \\
        \mathcal{P}(s, \alpha, t')[\beta] \quad & if \; t = (s, \alpha) \in s_\square, \beta \in Act_s^\alpha, and \; t' \in S_\circ, \\
        0 \quad & otherwise.
    \end{cases}
\end{align*}
\end{definition}
\noindent
The~constructed stochastic game $\mathcal{G} = \mathcal{M}_{sub}^r$ induced by various schedulers for the~first player $\circ$ relates with the~substitution in the~pMCs induced by schedulers from $\mathcal{M}$.
In more precise,~the~schedulers $\sigma \in \Sigma^M_\circ$ for the~first player $\circ$ coincide with the~schedulers in $\mathcal{M}$.
Therefore,~the~models $\mathcal{G}^\sigma$ and ($\mathcal{M}^\sigma)_{sub}^r$ induced by the~scheduler $\sigma$ induce the~same reachability probabilities.
Thus,~we can substitute $\mathcal{M}$ directly and preserve the~reachability probability,~instead of performing the~substitution on the~pMC induced by $\mathcal{M}$ and $\sigma$.

\begin{theorem} \label{the:mdp_stochastic}
Let $\mathcal{M} = (S, V, s_0, Act, \mathcal{P})$ be an pMDP,~$r$ a~well-defined region defined over this pMC $\mathcal{D}$,~then holds:
\begin{align*}
    \mathcal{M}_{sub}^{r} \models_{\emptyset} \reachability{\leq}{\lambda}{T} \Longrightarrow \mathcal{M}, r \models & \; \reachability{\leq}{\lambda}{T} \\
    \mathcal{M}_{sub}^{r} \models_{\circ} \reachability{>}{\lambda}{T} \Longrightarrow \mathcal{M}, r \models & \; \neg\reachability{\leq}{\lambda}{T}.
\end{align*}
\end{theorem}
\noindent
Therefore,~similar as in the~introduced pMC case,~we can determine whether $\mathcal{M}_{sub}^r \models \varphi$ by analysing a~stochastic game.
For more details about proofs of listed definitions and theorems please refer to~\cite{Quatmann2016}.

\section{Families of Markov Chains}
This thesis considers a~parametric transition probability function as an~explicit representation of an~MCs family.
Such explicit representation relieves the~presentation and provides to describe exciting and practical problems for probabilistic synthesis.
On the~other hand,~arbitrary probabilistic programs permit the~modelling of more complex and independent parameter structures~\cite{cegar}.
In this thesis and our implementation,~we consider a~more flexible high-level modelling language,~see Section~\ref{sec:pris_language}.

We want to focus on families of (parametric) Markov chains having different topologies of the~state space and having undefined transition probabilities.
Such families have different sets of reachable states and also different probabilities of reaching individual states.
More specifically,~we consider the~family,~which is parametrised by a~finite set of discrete parameters $K$ and a~finite set of parameters $V$ over the~domain $\real$.

\begin{definition}[Family of pMCs]
\cite{cegar}
    A~\emph{family of pMCs} $\fml$ is a~quintuple $\family$  where $S$ is a~finite set of \emph{states},~$\sinit \in S$ is an~\emph{initial state},~$K$ is a~finite set of parameters with domains $T_k \subseteq S$ for each $k \in K$,~$V$ is a~finite set of parameters over the~domain $\real$,~and $\fpm : S \times K \nrightarrow \mathbb{Q}_V$ is a~family of parametric transition probability functions.
\end{definition}

The~transition probability function $\fpm$ of pMCs family $\fml$ maps each state $s \in S$ to the~probability distribution over parameters from $K$,~where the~transitions are labelled with polynomials in $\mathbb{Q}_V$.
As we mentioned above,~these parameters from both sets $K$ and $V$ represent undefined system specifications when the~probabilistic synthesis is considered.
This function $\fpm$ yields a~specific pMC when we instantiate each parameter $k \in K$ with the~specific value from its domain $T_k \subseteq S$.
We call such instantiated pMC as \textit{realisation},~and the~following definition describes it.

\begin{definition}[Realisation]
\cite{cegar}
A~\emph{realization} of a~family $\fml = \family$ of pMCs is a~function $r: K \rightarrow S$ s.t.~$\forall k \in K :  r(k) \in T_k$.
A~realization~$r$ yields a~pMC $\mathcal{D}_r = (S, V, \sinit,\fpm(r))$,~where $\fpm(r): S \times K \nrightarrow \mathbb{Q}_V$ represents the~parametric transition probability matrix where each parameter $k \in K$ is substituted with $r(k)$.
Let $\rlzf$ denote the~set of all (parametric) realisations for $\fml$.
\end{definition}

A~set $\prod_{k \in K} T_k$ representing all possible parameter combinations values has the~same semantics as a~set $\rlzf$ representing all family realisations with different topology of the~state space.
In other words,~we can define the~\textit{size of the~family} of pMCs in the~following way: $\lvert \fml \rvert := \lvert \prod_{k \in K} T_k \rvert = \lvert \rlzf \rvert = \prod_{k \in K} \lvert T_k \rvert$.
The~family size $\lvert \fml \rvert$ is finite because of the~finiteness of each parameter domain $T_k \subseteq S$,~but it is exponential in the~number of parameters $\lvert K \rvert$.
The~individual pMCs within the~family share the~same state space $S$,~but their set of reachable states can vary.
We emphasise that,~from the~point of view of program synthesis,~such defined size of the~family does not indicate the~maximum number of realisations that must be examined.
This size indicates the~number of parametric realisations with different topologies,~which,~however,~may still be inconclusive due to continuous parametric regions.
We note $\rlz$ as the~sub-families induced from the~whole family $\rlzf$.

\begin{example}[Family of MCs]\label{exam:mcfamily}
Let $\fml = \family$ be a~family of pMCs,~where $S = \{s_0, s_1, s_2, s_3\}$,~$K = \{ k_0, k_1, k_2, k_3 \}$ with domains $T_{k_0} = T_{k_1} = \{s_1, s_2\}$,~$T_{k_2} = \{ s_0 \}$,~and $T_{k_3} = \{ s_3 \}$,~and $V = \{ x, y \}$.
The~parametric transition function $\fpm$ is defined as follows:
\begin{align*}
    \fpm(s_0) &= x : k_0 + (1-x) : k_1  \\
    \fpm(s_1)  &= y : k_2  + (1-y) : k_3 \\
    \fpm(s_2) &= 1.0 : k_0  \\
    \fpm(s_3)  &= 1.0 : k_3
\end{align*}
Figure~\ref{fig:mcfamily} draws the~pMCs family $\fml$ that correspond to all possible realisations: $\lvert T_{k_0} \rvert \cdot \lvert T_{k_1} \rvert = 2 \cdot 2 = 4$.
We note that these pMCs have a~different topology of the~underlying state space,~resulting in different sets of reachable states.
\end{example}

\begin{figure}[ht!]
\centering
\includegraphics[width=0.9\textwidth]{figures/MCFamily.pdf}
\caption{A~family $\fml$ of four various realisations. Unreachable states are greyed out.}%
\label{fig:mcfamily}%
\end{figure}

\begin{definition}[Specification]
In our work,~we focus on the~conjunctions of specifications with \textit{reachability} and \textit{expected rewards}.
Let $T$ be a~set of target states,~then the~reachability property $\varphi \equiv \reachability{\bowtie}{\lambda}{T}$ where $\bowtie \in \{<, \leq, >, \geq\}$ and $0 \leq \lambda \leq 1$ expresses that the~probability of reaching $T$ refers to $\lambda \in [0,1]$ in agreement with $\bowtie$.
Expected reward property $\phi \equiv \reward {\bowtie}{\lambda}{T}$ expresses that the~expected reward accumulated before $T$ is reached refers to $\lambda \in \mathbb{R}^+$ in accordance with $\bowtie \, \in \{<, \leq \}$.
Let $\mathcal{P}[r]$ be a~program induced by the~realisation $r$,~we denote $\mathcal{P}[r] \models \varphi$ when this program satisfies the~specification $\varphi$.
Let $\varPhi = \{ \varphi_i \}_{i \in I}$ be a~finite set of specifications,~when $\forall i \in I: \mathcal{P}[r] \models \varphi_i$ then we write $\mathcal{P}[r] \models \varPhi$.
\end{definition}

We aim to two kinds of synthesis tasks for a~given probabilistic program described with a~realisation set $\rlzf$,~and a~specification set $\varPhi$.
The~first task tries to identify just one realisation $r \in \rlzf$ that satisfy the~given specification set $\varPhi$.
This task represents a~special instance of the~threshold synthesis,~which tries to divide the~realisation set $\rlzf$ into two subsets based on their satisfiability.
We do not address this synthesis task in our work.
The~second task on which we focus our attention is finding a~realisation that minimises or maximises a~given objective.

\begin{definition}[Feasibility]
Find a~realisation $r \in \rlzf$ such that $\mathcal{P}[r] \models \varPhi$.
\end{definition}

\begin{definition}[Minimality] \label{def:minimality}
For property $\phi_{\min}$,~find a~realisation $r^* \in \rlzf$ such that:
$$r^* \in \argmin_{r \in \rlzf} \left \{ \prob[\sketch[r] \models \phi_{\max}] \mid \sketch[r] \models \varPhi \right \}.$$.
\end{definition}

We defined only minimal synthesis task for probability property,~but its variants for maximisation and expected rewards may be defined analogously.
Moreover,~we focus also to a~relaxed variant of minimal synthesis,~the~so-called \textit{$\varepsilon$-minimal synthesis},~defined as follows: $\mathcal{P}[r^*] \models \varPhi \; and \;
\mathbb{P}[\mathcal{P}[r^*] \models \varphi_{min}] \leq (1 - \varepsilon) \cdot \min_{r \in \mathcal{\overline{R}}} \{ \mathbb{P}[\mathcal{P}[r] \models \varphi_{min}] \; \lvert \; \mathcal{P}[r] \models \varPhi \}.$

\begin{example} (Synthesis problem)
Assume an~MCs family $\fml$ from Example~\ref{exam:mcfamily} and the~specification $\phi = \reachability{\geq}{0.1}{\{s_1\}}$.
The~solution to the~feasibility synthesis problem is,~for example,~the~realisation $r_0$,~since $\mathcal{D}_{r_0}$ has a~probability of $\frac{2}{3}$ to reach state $s_1$.
For $\varPhi = [F \; \{s1\}]$,~the~solution to the~maximal synthesis problem on MCs family $\fml$ is the~realisation $r_2$,~as MC $\mathcal{D}_{r_2}$ has a~probability equal to one to reach state $s_1$.
\end{example}

\textit{Quotient} pMDP $\mathcal{M}^\fml$ simulates the~behaviours of each member of the~family $\fml$ and can even pass between them during the~execution.
In more precise,~when the~path $s_0s_1s_2 \dots$ is executable in some family member $\mathcal{D}_r$,~then it is also executable in $\mathcal{M}^\fml$ as $s_0 \overset{r}{\rightarrow} s_1 \overset{r}{\rightarrow} \dots$.
However,~there may exist a~path $\pi$ that is executable in $\mathcal{M}^\fml$,~but it is not realisable in neither of the~family members.
We can conclude that \textit{quotient} MDP over-approximates the~behaviours of the~members of family $\fml$.

\begin{definition}[Quotient pMDP] \label{def:quotient_mdp}
\cite{roman-DP}
Let $\fml = \family$ be an~pMCs family.
A~\textit{quotient} pMDP $\mathcal{M}^\fml$ of the~family $\fml$ is a~tuple $(S, V, s_{init}, \mathcal{R}^\fml, \mathcal{P})$,~where $\mathcal{P}(\cdot)(r) \equiv \mathcal{B}_r$.
\end{definition}

A~restriction of a~\textit{quotient} pMDP concerning $\rlz \subseteq \rlz^\fml$ induces an~pMDP which takes into account only transitions associated with $\rlz$.
We define the~usage of \textit{consistent} schedulers ensuring that execution of a~\textit{quotient} pMDP always selects the~concrete realisation.
We note that a~\textit{consistent} scheduler yields a~specific family member for a~\textit{quotient} pMDP,~which directly follows from Definition~\ref{def:quotient_mdp}.

\begin{definition}[Consistent scheduler]
\cite{roman-DP}
Let $\fml = \family$ be a~pMCs family and let $\mathcal{M}^\fml = (S, V, s_{init}, \mathcal{R}^\fml, \mathcal{P})$ be a~quotient pMDP of the~family $\fml$.
For $r \in \rlz^\fml$,~a~(memory-less) scheduler $\sigma_r \in \sigma^{M^\fml}$ is called \textit{r-consistent} iff $\forall s \in S: \, \sigma(s) = r$.
A~scheduler is called \textit{consistent} iff it is consistent for some $r \in \rlz^\fml$.
\end{definition}


\documentclass{article}

\usepackage[stdmargin, noindent]{../../rajeev}
\usepackage{mathtools}
\usepackage{dirtytalk}

\pagestyle{fancy}
\rhead{\today}
\lhead{Math 291H HW \#3}

\begin{document}

\begin{center}
    \Large \textbf{Math 291H Homework \#3}
  \end{center}
\begin{center}
    \Large Rajeev Atla
  \end{center}

Honors Pledge Statement: \say{The writeup of this submission is my own work alone.}

\problem{2.1}

\subsection*{a}

\begin{align*}
  \bm{x}' \pars{t} &= \pars{x' \pars{t}, y' \pars{t}} \\
                   &= \boxed{\pars{1, 2t}} \\
  \bm{x}'' \pars{t} &= \boxed{\pars{0, 2}} \\
\end{align*}

\subsection*{b}

\begin{align*}
  v \pars{t} &= \norm{\bm{v} \pars{t}} \\
             &= \boxed{\sqrt{1 + 4t^2}} \\
  \bm{T} \pars{t} &= \frac{\bm{v} \pars{t}}{v \pars{t}} \\
  &= \boxed{\frac{1}{\sqrt{1 + 4t^2}} \pars{1, 2t} } \\
\end{align*}

\subsection*{c}
At $t=1$, $\bm{x} = \pars{2, 1}$.

\begin{align*}
  \bm{y} \pars{t} & \approx \pars{2, 1} + \pars{t-1} \pars{1, 2} \\
  &= \boxed{\pars{1, -1} + t \pars{1, 2}} \\
\end{align*}

\problem{2.3}
We need to prove two preliminary results in order to solve this problem: both the product and sum of two continuous functions are continuous.

\subsection*{Sums}

If $\epsilon > 0$, then we must necessarily have $\frac{\epsilon}{2} > 0$.
If $\bm{x} \pars{t}$ and $\bm{y} \pars{t}$ are both continuous over $\RR$,

\begin{align*}
  \forall \epsilon > 0, \exists \delta_{\bm{x}} > 0 &: \abs{t - t_0} < \delta_{\bm{x}} \implies \norm{ \bm{x} \pars{t} - \bm{x} \pars{t_0}} < \frac{\epsilon}{2} \\
  \forall \epsilon > 0, \exists \delta_{\bm{y}} > 0 &: \abs{t - t_0} < \delta_{\bm{y}} \implies \norm{\bm{y} \pars{t} - \bm{y} \pars{t_0}} < \frac{\epsilon}{2} \\
\end{align*}

Define $\bm{z} \pars{t} := \bm{x} \pars{t} + \bm{y} \pars{t}$.
Using the triangle inequality,

\begin{align*}
  \norm{\bm{z} \pars{t} - \bm{z} \pars{t_0}} & = \norm{\bm{x} \pars{t} + \bm{y} \pars{t} - \bm{x} \pars{t_0} - \bm{y} \pars{t_0}} \\
                                             & = \norm{\bm{x} \pars{t} - \bm{x} \pars{t_0} + \bm{y} \pars{t} - \bm{y} \pars{t_0}} \\
                                             & \leq \norm{\bm{x} \pars{t} - \bm{x} \pars{t_0}} + \norm{\bm{y} \pars{t} - \bm{y} \pars{t_0}} \\
                                            & < 2 \pars{\frac{\epsilon}{2}} \\
                                            & = \epsilon \\
\end{align*}

Define $\delta_{\bm{z}} := \min \set{\delta_{\bm{x}}, \delta_{\bm{y}}}$.
We have

$$
\forall \epsilon > 0, \exists \delta_{\bm{z}} > 0 : \abs{t - t_0} < \delta_{\bm{z}} \implies \norm{ \bm{z} \pars{t} - \bm{z} \pars{t_0}} < \epsilon
$$
This shows that $\bm{z} \pars{t}$ is continuous.

\subsection*{Products}

Let $x \pars{t}, y \pars{t}$ be real-valued continuous functions over $\RR$.
Then,

\begin{align*}
    \forall \epsilon_{x} > 0, \exists \delta_{x} > 0 &: \abs{t - t_0} < \delta_{x} \implies \abs{ x \pars{t} - x \pars{t_0}} < \epsilon_x \\
  \forall \epsilon_{y} > 0, \exists \delta_{y} > 0 &: \abs{t - t_0} < \delta_{y} \implies \abs{y \pars{t} - y \pars{t_0}} < \epsilon_y \\
\end{align*}

Define $w \pars{t} := x \pars{t} y \pars{t}$.
We wish to show that

$$
\forall \epsilon_{w} > 0, \exists \delta_{w} > 0 : \abs{t - t_0} < \delta_{w} \implies \abs{ w \pars{t} - w \pars{t_0}} < \epsilon_w
$$

We have

\begin{align*}
  \abs{w \pars{t} - w \pars{t_0}} & = \abs{x \pars{t} y \pars{t} - x \pars{t_0} y \pars{t_0}} \\
                                  & = \abs{x \pars{t} y \pars{t} - x \pars{t} y \pars{t_0} + x \pars{t} y \pars{t_0} - x \pars{t_0} y \pars{t_0}} \\
                                  & \leq \abs{x \pars{t} y \pars{t} - x \pars{t} y \pars{t_0}} + \abs{x \pars{t} y \pars{t_0} - x \pars{t_0} y \pars{t_0}} \\
                                  & = \abs{x \pars{t}} \abs{y \pars{t} - y \pars{t_0}} + \abs{y \pars{t_0}} \abs{x \pars{t} - x \pars{t_0}} \\
                                  & < \epsilon_y \abs{x \pars{t}} + \epsilon_x \abs{y \pars{t_0}} \\
\end{align*}

Here, we have used the triangle inequality and the above definition of the continuity of $x$ and $y$.
Using the continuity of $x \pars{t}$, we can pick $\delta_x$, using $k \in \RR$ to avoid having a denominator of 0.

\begin{align*}
  \epsilon_x &= \frac{\epsilon_w}{2 \abs{y \pars{t_0}} + k},\ k > 0 \\
  \epsilon_x \abs{y \pars{t_0}} &= \frac{\epsilon_w \abs{y \pars{t_0}}}{2 \abs{y \pars{t_0}} + k} \\
  & < \frac{\epsilon_w}{2} \\
\end{align*}

We then note by the reverse triangle inequality,

$$
  \abs{x \pars{t} - x \pars{t_0}} \geq \abs{x \pars{t}} - \abs{x \pars{t_0}} \\
$$

In addition, using continuity, we can pick $\delta_x$ such that $\abs{x \pars{t} - x \pars{t_0}} < \epsilon_w$.
Combining with the above inequality, we see that

\begin{align*}
  \abs{x \pars{t}} - \abs{x \pars{t_0}} &< \epsilon_w \\
  \abs{x \pars{t}} &< \abs{x \pars{t_0}} + \epsilon_w \\
\end{align*}

We then pick $\delta_y$ such that

\begin{align*}
  \epsilon_y &= \frac{\epsilon_w}{2 \pars{\epsilon_w + \abs{x \pars{t_0}}}} \\
  \epsilon_y \abs{x \pars{t}} &= \frac{\epsilon_w \abs{x \pars{t}}}{2 \pars{\epsilon_w + \abs{x \pars{t_0}}}} \\
             & < \frac{\epsilon_w}{2} \pars{\frac{\epsilon_w + \abs{x \pars{t_0}}}{\epsilon_w + \abs{x \pars{t_0}}}} \\
             &= \frac{\epsilon_w}{2} \\
\end{align*}

We then have

\begin{align*}
  \epsilon_y \abs{x \pars{t}} + \epsilon_x \abs{y \pars{t_0}} & < 2 \pars{\frac{\epsilon_w}{2}} \\
  &= \epsilon_w
\end{align*}

Putting together all the inequalities, we have

$$
\abs{w \pars{t} - w \pars{t_0}} < \epsilon_w
$$

This establishes that $w \pars{t}$ is a continuous function in $\RR$.

\subsection*{Dot Product}
We show that the dot product is a combination of additive and multiplicative operations on the entry functions of $\bm{x} \pars{t}$ and $\bm{y} \pars{t}$.
Recall that in order for $\bm{x} \pars{t}$, each of $\set{x_i \pars{t}}$ must also be a continuous function.
The same goes for $\bm{y} \pars{t}$.

Define $v \pars{t} := \bm{x} \pars{t} \cdot \bm{y} \pars{t}$.
By definition

\begin{align*}
  v \pars{t} &= \bm{x} \pars{t} \cdot \bm{y} \pars{t} \\
             &= \sum \limits_{i=1}^{i=n} x_i \pars{t} y_i \pars{t} \\
\end{align*}

Further, define

$$
v_{k} \pars{t} := \sum \limits_{i=1}^{k \leq n} x_i \pars{t} y_i \pars{t}
$$

We see that $v_1 \pars{t} = x_1 \pars{t} y_1 \pars{t}$, the product of two continuous functions, making it a continuous function.
Similarly,
\begin{align*}
  v_2 \pars{t} &= x_1 \pars{t} y_1 \pars{t} + x_2 \pars{t} y_2 \pars{t} \\
               &= v_1 \pars{t} + x_2 \pars{t} y_2 \pars{t} \\
\end{align*}

Since both $x_2 \pars{t}$ and $y_2 \pars{t}$ are both continuous functions, their product is also a continuous function.
In addition, we have already shown that $v_1 \pars{t}$ is a continuous function.
Therefore, $v_2 \pars{t}$ is a continuos function.
In general for $k \leq n$,

$$
v_k \pars{t} = v_{k-1} \pars{t} + x_k \pars{t} y_k \pars{t}
$$

Each term is a continouus function, so $v_k \pars{t}$ is also a continuous function.
In the $k=n$ case, this becomes the dot product, completing the proof.

\subsection*{Cross Product}
In $\RR^3$, the cross product is

\begin{align*}
  \bm{x} \pars{t} \times \bm{y} \pars{t} &=
                                           \begin{vmatrix}
                                             \bm{e}_1 & \bm{e}_2 & \bm{e}_3 \\
                                             x_1 \pars{t} & x_2 \pars{t} & x_3 \pars{t} \\
                                             y_1 \pars{t} & y_2 \pars{t} & y_3 \pars{t} \\
                                           \end{vmatrix} \\
                                         &= \pars{x_2 \pars{t} y_3 \pars{t} - x_3 \pars{t} y_2 \pars{t}} \bm{e}_1 + \pars{x_3 \pars{t} y_1 \pars{t} - x_1 \pars{t} y_3 \pars{t}} \bm{e}_2 + \pars{x_1 \pars{t} y_2 \pars{t} - x_2 \pars{t} y_1 \pars{t}} \bm{e}_3 \\
\end{align*}

A vector function is continuous if and only if each of its entry functions is also continuous.
Therefore, it suffices to show that each entry function is continuous.
Each entry function is a combination of products of continuous functions and differences between these products.
Therefore, the cross product is continuous.

\problem{2.4}

\subsection*{a}

\begin{align*}
  \bm{v} \pars{t} &= \boxed{\pars{- \sin \pars{t}, \cos \pars{t}, \frac{1}{r}}} \\
  \bm{a} \pars{t} &= \boxed{\pars{- \cos \pars{t}, - \sin \pars{t}, 0}} \\
\end{align*}

\subsection*{b}

\begin{align*}
  v \pars{t} &= \sqrt{\sin^2 t + \cos^2 t + \frac{1}{r^2}} \\
             &= \boxed{\sqrt{1 + \frac{1}{r^2}}} \\
  \bm{T} \pars{t} &= \frac{\bm{v} \pars{t}}{v \pars{t}} \\
             &= \boxed{ \frac{r}{\sqrt{1+r^2}} \pars{- \sin \pars{t}, \cos \pars{t}, \frac{1}{r}}} \\
\end{align*}

\subsection*{c}

\begin{align*}
  \bm{a}_{\parallel} \pars{t} &= v' \pars{t} \bm{T} \pars{t} \\
  &= 0 \\
  \bm{a}_{\perp} \pars{t} &= v \pars{t} \bm{T}' \pars{t} \\
                              &= \pars{\sqrt{1 + \frac{1}{r^2}}} \pars{\frac{r}{\sqrt{1+r^2}} \pars{- \cos \pars{t}, - \sin \pars{t}, 0}} \\
                              &= \pars{- \cos \pars{t}, - \sin \pars{t}, 0} \\
  \bm{N} \pars{t} &= \frac{\bm{a}_{\perp} \pars{t}}{\norm{\bm{a}_{\perp} \pars{t}}} \\
                              &= \boxed{\pars{- \cos \pars{t}, - \sin \pars{t}, 0}} \\
  \kappa \pars{t} &= \frac{ \norm{\bm{a}_{\perp} \pars{t}} }{v^2 \pars{t}} \\
                              &= \frac{1}{1 + \frac{1}{r^2}} \\
                              &= \boxed{\frac{r^2}{1+r^2}} \\
  \bm{B} \pars{t} &= \bm{T} \pars{t} \times \bm{N} \pars{t} \\
                              &= \boxed{\frac{1}{\sqrt{1+r^2}} \pars{\sin \pars{t}, - \cos \pars{t}, r}} \\
  \tau \pars{t} &= \frac{1}{v^6 \pars{t} \kappa^2 \pars{t}} \pars{\bm{x}' \pars{t} \times \bm{x}'' \pars{t}} \cdot \bm{x}''' \pars{t} \\
                              &= \frac{1}{\pars{1 + \frac{1}{r^2}}^3} \pars{1 + \frac{1}{r^2}}^2 \pars{\pars{- \sin \pars{t}, \cos \pars{t}, \frac{1}{r}} \times \pars{- \cos \pars{t}, - \sin \pars{t}, 0}} \cdot \pars{\sin \pars{t}, - \cos \pars{t}, 0} \\
                              &= \frac{1}{1 + \frac{1}{r^2}} \pars{\frac{1}{r} \sin \pars{t}, - \frac{1}{r} \cos \pars{t}, 1} \cdot \pars{\sin \pars{t}, - \cos \pars{t}, 0} \\
                              &= \frac{r^2}{1+r^2} \pars{\frac{1}{r} \sin^2 \pars{t} + \frac{1}{r} \cos^2 \pars{t}} \\
                              &= \boxed{\frac{r}{1+r^2}} \\
\end{align*}

\subsection*{d}

\begin{align*}
  \boldsymbol{\omega} \pars{t} &= \tau \pars{t} \bm{T} \pars{t} + \kappa \pars{t} \bm{B} \pars{t} \\
                               &= \pars{\frac{r}{1+r^2}} \pars{\frac{r}{\sqrt{1 + r^2}} \pars{- \sin \pars{t}, \cos \pars{t}, \frac{1}{r}}} + \pars{\frac{r^2}{1+r^2}} \pars{\frac{1}{\sqrt{1+r^2}} \pars{\sin \pars{t}, - \cos \pars{t}, r}} \\
                               &= \frac{r^2}{\pars{1+r^2}^{\frac{3}{2}}} \pars{0, 0, r + \frac{1}{r}} \\
  &= \boxed{\frac{r}{\sqrt{1+r^2}} \pars{0, 0, 1}} \\
\end{align*}

\subsection*{e}

\subsubsection*{Tangent Line}

\begin{align*}
  \bm{x} \pars{t} & \approx \bm{x} \pars{\frac{\pi}{4}} + \pars{t - \frac{\pi}{4}} \bm{v} \pars{\frac{\pi}{4}}\\
                  &= \pars{\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}, \frac{\pi}{4r}} + \pars{t - \frac{\pi}{4}} \pars{- \frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}, \frac{1}{r}} \\
                  &= \boxed{\pars{\frac{4+\pi}{4 \sqrt{2}}, \frac{4 - \pi}{4 \sqrt{2}} , 0} + t \pars{- \frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}, \frac{1}{r}}} \\
\end{align*}

\subsubsection*{Osculating Plane}
\begin{align*}
  \bm{B} \pars{\frac{\pi}{4}} \cdot \pars{\bm{x} \pars{t} - \bm{x} \pars{\frac{\pi}{4}}} &= 0 \\
  \bm{B} \pars{\frac{\pi}{4}} \cdot \bm{x} \pars{t} &= \bm{B} \pars{\frac{\pi}{4}} \cdot \bm{x} \pars{\frac{\pi}{4}} \\
  \frac{1}{\sqrt{1+r^2}} \pars{\frac{\sqrt{2}}{2}, - \frac{\sqrt{2}}{2}, r} \cdot \bm{x} \pars{t} &= \frac{1}{\sqrt{1+r^2}} \pars{\frac{\sqrt{2}}{2}, - \frac{\sqrt{2}}{2}, r} \cdot \pars{\frac{\sqrt{2}}{2}, \frac{\sqrt{2}}{2}, \frac{\pi}{4r}} \\
  \pars{2\sqrt{2}, - 2\sqrt{2}, 4r} \cdot \bm{x} \pars{t} &= \pi \\
  \boxed{2 x\sqrt{2} - 2y\sqrt{2} + 4rz = \pi} \\
\end{align*}

\subsubsection*{Intersection}

\begin{align*}
  x \pars{t} &= \frac{4+\pi}{4 \sqrt{2}} - \frac{t}{\sqrt{2}} \\
  y \pars{t} &= \frac{4-\pi}{4 \sqrt{2}} + \frac{t}{\sqrt{2}} \\
  z \pars{t} &= \frac{t}{r} \\
  2 x\sqrt{2} - 2y\sqrt{2} + 4rz &= \pi \\
\end{align*}

We see that upon substitution, we are left with a contradiction, meaning that the line and plane never intersect.

\problem{2.5}

\subsubsection*{a}

\begin{align*}
  \bm{v} \pars{t} &= \pars{e^t \cos t - e^t \sin t, e^t \sin t + e^t \cos t, e^t} \\
  v \pars{t} &= e^t \sqrt{\pars{\cos t - \sin t}^2 + \pars{\cos t + \sin t}^2 + 1} \\
  &= e^t \sqrt{3} \\
  s \pars{t} &= \int \limits_{t=0}^{t} e^t \sqrt{3}\ dt \\
  &= \pars{e^t -1 } \sqrt{3} \\
\end{align*}

In order to write an arc-length parametrization, we invert $s \pars{t}$ to get $t \pars{s}$.

\begin{align*}
  s &= \pars{e^t - 1} \sqrt{3} \\
  \frac{s}{\sqrt{3}} &= e^t - 1 \\
  t \pars{s} &= \ln \pars{\frac{s}{\sqrt{3}} + 1} \\
\end{align*}

Finally, we find $\bm{x} \pars{t \pars{s}}$.

\begin{align*}
  \bm{x} \pars{t \pars{s}} &= \pars{\pars{\frac{s}{\sqrt{3}} + 1} \cos \pars{\pars{\frac{s}{\sqrt{3}} + 1}}, \pars{\frac{s}{\sqrt{3}} + 1} \sin \pars{\pars{\frac{s}{\sqrt{3}} + 1}}, \frac{s}{\sqrt{3}} + 1} \\
\end{align*}

\subsection*{b}
\begin{align*}
  \bm{T} \pars{t} &= \frac{\bm{v} \pars{t}}{v \pars{t}} \\
                  &= \frac{1}{\sqrt{3}} \pars{\cos t - \sin t, \cos t + \sin t, 1} \\
  \bm{T}' \pars{t} &= \frac{1}{\sqrt{3}} \pars{- \sin t - \cos t, - \sin t + \cos t, 0} \\
  \bm{a}_{\perp} \pars{t} &= v \pars{t} \bm{T}' \pars{t} \\
                  &= e^t \pars{- \sin t - \cos t, - \sin t + \cos t, 0} \\
  \norm{\bm{a}_{\perp} \pars{t}} &= e^t \sqrt{2} \\
  \kappa \pars{t} &= \frac{\norm{\bm{a}_{\perp} \pars{t}}}{v^2 \pars{t}} \\
                  &= \frac{e^t \sqrt{2}}{3 e^{2t}} \\
                  &= \boxed{e^{-t} \frac{\sqrt{2}}{3}} \\
\end{align*}

\begin{align*}
  \bm{N} \pars{t} &= \frac{\bm{a}_{\perp}}{\norm{\bm{a}_{\perp}}} \\
                  &= \frac{1}{\sqrt{2}} \pars{- \sin t - \cos t, - \sin t + \cos t, 0} \\
  \bm{B} \pars{t} &= \bm{T} \pars{t} \times \bm{N} \pars{t} \\
                  &= \frac{1}{\sqrt{6}} \pars{\sin t - \cos t, - \sin t - \cos t, 2} \\
  \bm{B}' \pars{t} &= \frac{1}{\sqrt{6}} \pars{\cos t + \sin t, - \cos t + \sin t, 0} \\
  \bm{B}' \pars{t} &= - v \pars{t} \tau \pars{t} \bm{N} \pars{t} \\
  \frac{1}{\sqrt{6}} \pars{\cos t + \sin t, - \cos t + \sin t, 0} &= - \tau \pars{t} \pars{e^t \sqrt{3}} \pars{\frac{1}{\sqrt{2}} \pars{- \sin t - \cos t, - \sin t + \cos t, 0}} \\
  \tau \pars{t} &= \boxed{\frac{e^{-t}}{3 \sqrt{2}}} \\
\end{align*}

\subsection*{c}

\begin{align*}
  \bm{x} \pars{0} &= \pars{1, 0, 1} \\
  \bm{B} \pars{0} &= \frac{1}{\sqrt{6}} \pars{-1, -1, 2} \\
  \bm{B} \cdot \bm{x} &= \bm{B} \cdot \bm{x} \pars{0} \\
  \pars{-1, -1, 2} \cdot \pars{x, y, z} &= \pars{-1, -1, 2} \cdot \pars{1, 0, 1} \\
  -x - y + 2z &= 1 \\
\end{align*}

\problem{2.7}

\subsection*{a}

\begin{align*}
  \bm{x} \pars{t} &= \pars{2t, t^2, \frac{t^3}{3}} \\
  \bm{v} \pars{t} &= \pars{2, 2t, t^2} \\
  v \pars{t} &= \sqrt{4 + 4t^2 + t^4} \\
                  &= t^2 + 2 \\
  v' \pars{t} &= 2t \\
  \bm{T} \pars{t} &= \pars{\frac{2}{t^2+2}, \frac{2t}{t^2+2}, \frac{t^2}{t^2+2}} \\
  \bm{T}' \pars{t} &= \pars{- \frac{4t}{\pars{t^2+2}^2}, \frac{2 \pars{2- t^2}}{\pars{t^2+2}^2}, \frac{4}{\pars{t^2+2}^2}} \\
  \bm{a}_{\parallel} \pars{t} &= \pars{\frac{4t}{t^2+2}, \frac{4t^2}{t^2+2}, \frac{2t^3}{t^2+2}} \\
  \bm{a}_{\perp} \pars{t} &= \pars{- \frac{4t}{t^2 + 2}, \frac{2 \pars{2-t^2}}{t^2 + 2}, \frac{4}{t^2 + 2}} \\
  \norm{\bm{a}_{\perp} \pars{t}} &= \frac{2 \sqrt{t^4 + 8}}{t^2 + 2} \\
  \bm{N} \pars{t} &= \pars{-\frac{2t}{\sqrt{t^4 + 8}}, \frac{2-t^2}{\sqrt{t^4 + 8}}, \frac{2}{\sqrt{t^4 + 8}}} \\
  \bm{B} \pars{t} &= \pars{\frac{t \pars{t^3 - 2t + 2}}{\pars{t^2 + 2} \sqrt{t^4 + 8}}, - \frac{2 \pars{t^3 + 2}}{\pars{t^2 + 2} \sqrt{t^4 + 8}}, \frac{2}{\sqrt{t^4 + 8}} } \\
  \bm{B} \pars{1} &= \pars{\frac{1}{9}, - \frac{2}{3}, \frac{2}{3}} \\
  \bm{x} \pars{1} &= \pars{1, \frac{1}{2}, \frac{1}{3}} \\
  \bm{B} \pars{1} \cdot \bm{x} \pars{t} &= \bm{B} \pars{1} \cdot \bm{x} \pars{1} \\
  \pars{\frac{1}{9}, - \frac{2}{3}, \frac{2}{3}} \cdot \pars{x, y, z} &= \frac{1}{6} \\
  \boxed{\frac{1}{9} x - \frac{2}{3} y + \frac{2}{3} z = \frac{1}{6}} \\
\end{align*}

\subsection*{b}
The minimal distance is

\begin{align*}
  \abs{\pars{\bm{x} \pars{1} - \pars{0, 0, 0}} \cdot \bm{B} \pars{1}} &= \boxed{\frac{1}{6}} \\
\end{align*}

\problem{2.9}

\subsection*{a}

We substitute the parametrized equation into the Cartesian equation to verify tht the parametrization is valid.

\begin{align*}
  \frac{x^2}{a^2} + \frac{y^2}{b^2} &= 1 \\
  \frac{a^2 \cos^2 t}{a^2} + \frac{b^2 \sin^2 t}{b^2} &= 1\\
  \cos^2 t + \sin^2 t &= 1 \\
\end{align*}

This trigonometric identity is obviously true regardless of the value of $t$, so the parametrization works.

\subsection*{b}

\begin{align*}
  \bm{x} \pars{t} &= \pars{a \cos t, b \sin t} \\
  \bm{v} \pars{t} &= \pars{- a \sin t, b \cos t} \\
  v \pars{t} &= \sqrt{a^2 \sin^2 t + b^2 \cos^2 t} \\
  \bm{T} \pars{t} &= \pars{- \frac{a \sin t}{\sqrt{a^2 \sin^2 t + b^2 \cos^2 t}}, \frac{b \cos t}{\sqrt{a^2 \sin^2 t + b^2 \cos^2 t}}} \\
  \bm{T}' \pars{t} &= \pars{-\frac{ab^2 \cos t}{\pars{a^2 \sin^2 t + b^2 \cos^2 t}^{\frac{3}{2}}}, -\frac{a^2 b \sin t}{\pars{a^2 \sin^2 t + b^2 \cos^2 t}^{\frac{3}{2}}}} \\
  \bm{a}_{\perp} \pars{t} &= \pars{-\frac{ab^2 \cos t}{a^2 \sin^2 t + b^2 \cos^2 t}, -\frac{a^2 b \sin t}{a^2 \sin^2 t + b^2 \cos^2 t}} \\
  \norm{\bm{a}_{\perp} \pars{t}} &= \frac{ab}{\sqrt{a^2 \sin^2 t + b^2 \cos^2 t}} \\
  \kappa \pars{t} &= \boxed{\frac{ab}{\pars{a^2 \sin^2 t + b^2 \cos^2 t}^{\frac{3}{2}}}} \\
  \kappa ' \pars{t} &= \frac{3ab \pars{b^2 - a^2} \sin t \cos t}{\pars{a^2 \sin^2 t + b^2 \cos^2 t}^{\frac{5}{2}}} \\
\end{align*}

Let $n \in \ZZ$.
We see that the critical points are
$$
\kappa \pars{\pi n} = \frac{a}{b^2}
$$
and

$$
\kappa \pars{\frac{\pi}{2} \pars{2n+1}} = \frac{b}{a^2}
$$

There are 3 main cases:
\begin{itemize}
\item When $a=b$, the ellipse degenerates into a circle and the curvature is constant throughout.
\item When $a>b$, the ellipse is oriented along the $x$-axis.
  Here, $\frac{a}{b^2} > \frac{b}{a^2}$.
  This means that $t=\pi n$ are the maxima and $t = \frac{\pi}{2} \pars{2n+1}$ are the minima.
  These $t$-coordinates correspond to the points $\pars{\pm a, 0}$ and $\pars{0, \pm b}$.
\item When $a < b$, the ellipse is oriented along the $y$-axis.
  Here, $\frac{a}{b^2} < \frac{b}{a^2}$.
  This means that $t=\pi n$ are the minima and $t = \frac{\pi}{2} \pars{2n+1}$ are the maxima.
  These $t$-coordinates correspond to the points $\pars{\pm a, 0}$ and $\pars{0, \pm b}$.
\end{itemize}

\problem{2.11}

\begin{align*}
  \bm{x} \pars{t} &= \pars{t+1, t^2} \\
  \bm{v} \pars{t} &= \pars{1, 2t} \\
  v \pars{t} &= \sqrt{1+4t^2} \\
  s &= \int \limits_{-1}^{0} \sqrt{1+4t^2}\ dt \\
  & \boxed{\approx 1.479} \\
\end{align*}

\end{document}


\vfill \eject
\par
\section{Serial Solution of $A X = Y$ using an $LU$ factorization}
\label{section:LU-serial}
\par
The user has some representation of the data which represents the
linear system, $AX = Y$.  The user wants the solution $X$.
The {\bf SPOOLES} library will use $A$ and $Y$
and provide $X$ back to the user.
\par
The {\bf SPOOLES} library is based on an object
oriented design philosophy.  The first object that the user
must interact with is {\tt InpMtx}\footnote{
{\tt InpMtx} stands for {\tt Inp}ut {\tt M}a{\tt t}ri{\tt x},
for it is the object into which the user inputs the matrix
entries.}.
The {\tt InpMtx} object is where
the {\bf SPOOLES} representation of $A$ is assembled.
The user can input the representation of $A$
into the {\tt InpMtx} object with methods for
single matrix entry (consisting of the row index, the column index,
and the value), for an array of entries, for a set of
entries in a specified row or column, and for a dense sub-matrices
(useful
for finite element applications).  All of these methods can be used
interchangeably with each other.
\par
A complete listing of a sample program
is found in Section~\ref{section:LU-serial-driver}.
We will now begin to
work our way through the program to illustrate the use of {\bf
SPOOLES}
to solve a system of linear equations.
\par
\subsection{Reading the input parameters}
\label{subsection:serial:input-data}
\par
The program starts by declaring a variety of variables
and pointers for the program.  It then reads
the following parameters from standard input.
\begin{itemize}
\item
The variable {\tt msglvl} controls the
level of output generated by the program and by {\bf SPOOLES}.
\item
The printed output is sent to {\tt messageFile}.
\item
Whether the matrix is real or complex is controled by {\tt type}
(1 for real, 2 for complex).
\item
Similarly, {\tt symmetryflag} controls whether the matrix is
symmetric (0), Hermitian (1), or nonsymmetric (2).
\item
The matrix data will be read from the file {\tt matrixFileName}.
The matrix data has a simple format with the first line
containing the number of rows ({\tt nrow}), the number of
columns ({\tt ncol}), and the number of entries ({\tt nent}).
The remaining {\tt nent} lines on the file contain the row
number, the column number, and value for each nonzero in the sparse
matrix.
In our sample case, the matrix is symmetric so only the entries
in the upper triangle are given on the file.
If the matrix is complex, there would be 2 values,
one for the real part and one for the imaginary part.
{\bf SPOOLES} follows the C language convention for indexing
all arrays starting with 0.
So the row and column labels for a matrix of order {\tt neqns} range
from 0 to {\tt neqns-1}.
\item
The right hand side matrix $Y$
will be read from the file {\tt rhsFileName}.
The first line of this file has two numbers:
{\tt nrow}, the number of rows of $Y$
that are present in the file,
followed by {\tt nrhs}, the number of columns of $Y$.
(The number of rows of $Y$ in the file may be different from the
number of rows in $Y$, since often right hand side matrices are
sparse.
This allows us the option of only reading in nonzero rows of $Y$.)
The remaining lines of the file have the following format:
the row id, followed by either {\tt nrhs} floating point numbers if
the system is real, or {\tt 2*nrhs} numbers if the system is complex.
\item
The {\tt seed} parameter is a random number seed used in the
ordering process.
\end{itemize}
\par
\subsection{Communicating the data for the problem}
\label{subsection:serial:communicating-data}
\par
The following code segment from the full sample program opens the
file {\tt matrixFileName}, reads the first line of the file,
and then initializes the {\tt InpMtx} object.
The program continues by reading each line of
the input matrix data and uses either the method
{\tt InpMtx\_inputRealEntry()} or {\tt InpMtx\_inputComplexEntry()}
to place that entry into the {\tt InpMtx} object.
Finally this code segment closes the file.
finalizes the input to {\tt InpMtx} by converting the internal
storage of the matrix entries to a vector form.
(This is necessary for later steps.)
\begin{verbatim}
inputFile = fopen(matrixFileName, "r") ;
fscanf(inputFile, "%d %d %d", &nrow, &ncol, &nent) ;
neqns = nrow ;
mtxA = InpMtx_new() ;
InpMtx_init(mtxA, INPMTX_BY_ROWS, type, nent, neqns) ;
if ( type == SPOOLES_REAL ) {
   double   value ;
   for ( ient = 0 ; ient < nent ; ient++ ) {
      fscanf(inputFile, "%d %d %le", &irow, &jcol, &value) ;
      InpMtx_inputRealEntry(mtxA, irow, jcol, value) ;
   }
} else {
   double   imag, real ;
   for ( ient = 0 ; ient < nent ; ient++ ) {
      fscanf(inputFile, "%d %d %le %le", &irow, &jcol, &real, &imag) ;
      InpMtx_inputComplexEntry(mtxA, irow, jcol, real, imag) ;
   }
}
fclose(inputFile) ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n input matrix") ;
   InpMtx_writeForHumanEye(mtxA, msgFile) ;
   fflush(msgFile) ;
}
\end{verbatim}
\par
The {\tt InpMtx} object is created via a call to {\tt InpMtx\_new()}
and initialized via a call to {\tt InpMtx\_init()}.
The arguments to {\tt InpMtx\_init()}
are the pointer to the {\tt InpMtx}
object created by {\tt InpMtx\_new()} followed by four integers,
{\tt coordType},
{\tt inputMode}, {\tt maxnent}, and {\tt maxnvector}.
\begin{itemize}
\item
The second argument {\tt coordType} = {\tt INPMTX\_BY\_ROWS}
represent a general purpose mode that is well suited
for most users.\footnote{Note that {\bf SPOOLES} has some
pre-defined parameters such as {\tt INPMTX\_BY\_ROWS} for some objects.
These parameters are always uppercase and either begin with the name of
the object which they apply to, or the library name, e.g.,
{\tt SPOOLES\_REAL}.
They are described in the reference manual in the section for the
particular object.}
Some users may want to use other settings for {\tt coordType}
whose complete descriptions are found in the reference
manual.
\item
The third argument {\tt inputMode} controls whether
the matrix is real or complex.
One use of {\bf SPOOLES} not illustrated here is that
the {\tt InpMtx} object can have no values.
This allows {\bf SPOOLES}
to be used to generate an ordering for use by another package.
\item
The fourth argument {\tt maxnent} is an estimate of the number of
nonzero entries in the matrix.
\item
The fifth argument {\tt maxnvector} is an estimate of the number of
number of vectors that will be used, e.g., number of rows or
numbers of columns.
\end{itemize}
The {\tt maxnent} and {\tt maxnvector} arguments only
have to be estimates as they are used in the initial sizing of the
object.
Either can be 0.
The {\tt InpMtx} object resizes itself
as required to handle the linear system.
\par
Every object in {\bf SPOOLES} has print methods to output the
contents of that object.  This is illustrated in this code segment
by printing the input matrix as contained in the {\tt InpMtx} object,
{\tt mtxA}.
To shorten this chapter we will from now on omit the part of
the code that prints debug output to {\tt msgFile} for the various
code segments.
The complete sample program in Section~\ref{section:LU-serial-driver}
contains all of the debug print statements.
\par
After the matrix $A$ has been read in from the file and placed in an
{\tt InpMtx} object, the right hand matrix $Y$ is read in from a
file and placed in a {\tt DenseMtx} object.
The following code fragment does this operation.
\begin{verbatim}
inputFile = fopen(rhsFileName, "r") ;
fscanf(inputFile, "%d %d", &nrow, &nrhs) ;
mtxB = DenseMtx_new() ;
DenseMtx_init(mtxB, type, 0, 0, neqns, nrhs, 1, neqns) ;
DenseMtx_zero(mtxB) ;
if ( type == SPOOLES_REAL ) {
   double   value ;
   for ( irow = 0 ; irow < nrow ; irow++ ) {
      fscanf(inputFile, "%d", &jrow) ;
      for ( jrhs = 0 ; jrhs < nrhs ; jrhs++ ) {
         fscanf(inputFile, "%le", &value) ;
         DenseMtx_setRealEntry(mtxB, jrow, jrhs, value) ;
      }
   }
} else {
   double   imag, real ;
   for ( irow = 0 ; irow < nrow ; irow++ ) {
      fscanf(inputFile, "%d", &jrow) ;
      for ( jrhs = 0 ; jrhs < nrhs ; jrhs++ ) {
         fscanf(inputFile, "%le %le", &real, &imag) ;
         DenseMtx_setComplexEntry(mtxB, jrow, jrhs, real, imag) ;
      }
   }
}
fclose(inputFile) ;
\end{verbatim}
The dense matrix object is created by a call to {\tt DenseMtx\_new()}
and initialized via a call to {\tt DenseMtx\_init()}.
There are seven arguments to {\tt DenseMtx\_init()}, not counting
the initial pointer argument.
\begin{itemize}
\item
The second argument specifies the type of the matrix, real or
complex.
\item
The third and fourth arguments specify row and column ids of the
matrix.
This is useful when the dense matrix is a submatrix of a larger
block matrix, but this feature is not used in the present context.
\item
The fifth and sixth arguments are the number of rows and columns in
the matrix, here equal to {\tt neqns} and {\tt nrhs}.
\item
The seventh and eighth arguments are the row stride and column
stride for the matrix entries.
For our application we require a column major matrix, and so
the row stride is {\tt 1} and the column stride is the number of
rows, or {\tt neqns}.
\end{itemize}
The initialization step allocates storage for the matrix entries,
but it does not fill them with any values.
This is done explicitly via the {\tt DenseMtx\_zero()} method,
which places zeroes in all the entries.
This is necessary since the right hand side matrix $Y$ may be
sparse, and so the number of rows in the file may not equal the
number of equations.
\par
The right hand side entries are then in, row by row, and placed
into their locations via one of the two ``set entries'' methods.
Note, the nonzero rows can be read from the file in any order.
\par
\subsection{Reordering the linear system}
\label{subsection:serial:reordering}
\par
The first step is to find the permutation matrix $P$, and then
permute $AX = Y$ into $(PAP^T)(PX) = PY$.
The result of the {\bf SPOOLES} ordering step is not just $P$
or its permutation vector, it is a {\it front tree} that defines
not just the permutation, but the blocking of the factor matrices,
which in turn specifies the data structures and the computations
that are performed during the factor and solves.
To determine this {\tt ETree} {\it front tree} object takes three
step, as seen in the code fragment below.
\begin{verbatim}
adjIVL = InpMtx_fullAdjacency(mtxA) ;
nedges = IVL_tsize(adjIVL) ;
graph = Graph_new() ;
Graph_init2(graph, 0, neqns, 0, nedges, neqns, nedges, adjIVL,
            NULL, NULL) ;
frontETree = orderViaMMD(graph, seed, msglvl, msgFile) ;
\end{verbatim}
The ordering modules requires a graph of $A + A^T$.
(The {\bf SPOOLES} $LU$ factorization works with matrices of
symmetric structure.)
The {\tt Graph} object represents the graph of the matrix.
Its internal representation uses adjacency lists, one for each
vertex, which in turn are stored in an {\tt IVL} object.
The {\tt Graph} and {\tt InpMtx} objects are at a high level
in the object hierarchy.
To promote independence of the objects, the two do not know about
each other, so we cannot create one from the other.
Instead, the {\tt InpMtx} object creates the lower level {\tt IVL}
object\footnote{{\tt IVL} stands for {\tt I}nteger {\tt V}ector
{\tt L}ist, i.e., a list of integer vectors.}, which is then used
in the initialization step for the {\tt Graph} object.
The {\tt Graph} object is quite general, and can be used to
describe a graph with unit or non-unit vertices and edges.
We refrain from describing all the input parameters to initialize
the {\tt Graph} object and instead refer the reader to the
reference manual.
\par
Once a {\tt Graph} object has been created, it is ordered via the
multiple minimum degree method, whose return value is a front tree
object.
The minimum degree method is the simplest of the ordering methods
provided in the {\bf SPOOLES} library.
For more information on ordering, please see the user document
{\it ``Ordering Sparse Matrices and Transforming Front Trees''}.
\par
\subsection{Non-numeric work}
\label{subsection:serial:non-numeric}
\par
The next phase is to obtain the permutation matrix $P$, (stored
implicitly in a permutation vector), and apply it to the front
tree, the matrix $A$ and the right hand side $Y$.
This is done by the following code fragment.
\begin{verbatim}
oldToNewIV = ETree_oldToNewVtxPerm(frontETree) ;
oldToNew = IV_entries(oldToNewIV) ;
newToOldIV = ETree_newToOldVtxPerm(frontETree) ;
newToOld   = IV_entries(newToOldIV) ;
ETree_permuteVertices(frontETree, oldToNewIV) ;
InpMtx_permute(mtxA, oldToNew, oldToNew) ;
if (  symmetryflag == SPOOLES_SYMMETRIC
   || symmetryflag == SPOOLES_HERMITIAN ) {
   InpMtx_mapToUpperTriangle(mtxA) ;
}
InpMtx_changeCoordType(mtxA, INPMTX_BY_CHEVRONS) ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
DenseMtx_permuteRows(mtxB, oldToNewIV) ;
\end{verbatim}
The {\tt oldToNewIV} and {\tt newToOldIV} variables are {\tt IV}
objects that represent an integer vector.
The {\tt oldToNew} and {\tt newToOld} variables are pointers to
{\tt int}, which point to the base address of the {\tt int} vector
in an {\tt IV} object.
\par
Once we have the permutation vector, we apply it to the front tree,
by the {\tt ETree\_permuteVertices()} method, and then to the
matrix with the {\tt InpMtx\_permute()} method.
If the matrix $A$ is symmetric or Hermitian, we expect all nonzero
entries to be in the upper triangle.
Permuting the matrix yields $PAP^T$, which may not have all of its
entries in the upper triangle.
If $A$ is symmetric or Hermitian, the call to
{\tt InpMtx\_mapToUpperTriangle()} ensures that all entries of
$PAP^T$ are in its upper triangle.
Permuting the matrix destroys the internal vector structure, which
has to be restored.
But first we need to change the coordinate type of the {\tt InpMtx}
object, from rows into {\it chevrons}.\footnote{The $i$-th
chevron of $A$ consists of the diagonal entry $A_{i,i}$,
the $i$-th row of the upper triangle of $A$,
and the $i$-th column of the lower triangle of $A$.}
This is necessary in order to assemble entries of $PAP^T$ during
the numerical factorization.
At this point the {\tt InpMtx} object holds $PAP^T$ in the form
required by the factorization.
What remains is to transform $Y$ into $PY$, which is done via a
call to {\tt DenseMtx\_permuteRows()}.
\par
The final step is to compute the symbolic factorization,
which is stored in an {\tt IVL} object.
\begin{verbatim}
symbfacIVL = SymbFac_initFromInpMtx(frontETree, mtxA) ;
\end{verbatim}
\par
\subsection{The Matrix Factorization}
\label{subsection:serial:factor}
\par
The numeric factorization step begins by initializing the {\tt
FrontMtx} object with the {\tt frontETree} and {\tt symbacIVL}
objects created in early steps.
The {\tt FrontMtx} object holds the actual factorization.
The code segment for the initialization is found below.
\begin{verbatim}
frontmtx = FrontMtx_new() ;
mtxmanager = SubMtxManager_new() ;
SubMtxManager_init(mtxmanager, NO_LOCK, 0) ;
FrontMtx_init(frontmtx, frontETree, symbfacIVL, type, symmetryflag,
              FRONTMTX_DENSE_FRONTS, pivotingflag, NO_LOCK, 0, NULL,
              mtxmanager, msglvl, msgFile) ;
\end{verbatim}
Here is a brief description of the initialization method and its
input parameters.
\begin{itemize}
\item
The fourth parameter is the matrix type, real or complex.
\item
The fifth parameter specifies whether the matrix is symmetric,
Hermitian or nonsymmetric.
\item
The sixth parameter defines whether the fronts in the factor matrix
are stored as dense or sparse matrices.
The latter is necessary for an approximate factorization.
\item
The seventh parameter says whether pivoting is enabled for
numerical stability.
\item
The eighth, nine and ten parameters are used during a multithreaded
or MPI factorization.
Their present values are for a serial factorization.
\item
The eleventh parameter, {\tt mtxmanager} is a {\tt SubMtxManager}
object, an object used to manage instances of submatrices that form
the factor matrices.
The {\tt FrontMtx} object does not concern itself with finding
storage for the factor matrices, instead it asks the {\tt
SubMtxManager} object for submatrices.
While this seems awkward, it allows the {\tt FrontMtx} to operate
in serial, multithreaded and MPI environments with little internal
code differences, and it is the hook we have left in the library to
extend its capabilities to out-of-core factors and solves.
\item
The twelveth and thirteenth parameters define the message level and
message file for the factorization.
\end{itemize}
\par
The numeric factorization is performed by the
{\tt FrontMtx\_factorInpMtx()} method.
The code segment from the sample program for the numerical
factorization step is found below.
\begin{verbatim}
chvmanager = ChvManager_new() ;
ChvManager_init(chvmanager, NO_LOCK, 1) ;
DVfill(10, cpus, 0.0) ;
IVfill(20, stats, 0) ;
rootchv = FrontMtx_factorInpMtx(frontmtx, mtxA, tau, droptol,
             chvmanager, &error, cpus, stats, msglvl, msgFile) ;
ChvManager_free(chvmanager) ;
\end{verbatim}
Working storage used during the factorization is found in the form
of block {\it chevrons}, in a {\tt Chv} object, which hold the partial
frontal matrix for a front.
Much as with the {\tt SubMtx} object, the {\tt FrontMtx} object does
not concern itself with managing working storage, instead it relies
on a {\tt ChvManager} object to manage the {\tt Chv} objects.
We now discuss the arguments to the factor method.
\begin{itemize}
\item
The third argument is used when pivoting for numerical stability is
enabled.
Each entry in $L$ and $U$ is bounded above in magnitude by {\tt tau}.
We recommend a value of 100 for this parameter.
\item
The fourth argument is a drop tolerance
that is not relevant for this case.
When used with approximate factorizations, this argument is a lower
bound on the magnitude of the entries that are stored in the front
matrices.
\item
The sixth argument is an error flag.
After a successful factorization, {\tt error < 0} implies that the
factorization finished.
If {\tt error > 0}, then the factorization failed at front {\tt error}.
\item
The seventh and eighth arguments are vectors to be filled with
statistics and breakdown of cpu times.
\end{itemize}
The return value of the factorization is a {\tt Chv} object, which
will be {\tt NULL} if the factorization succeeded.
We have left this as a hook for future extensions where only
portions of the factor matrices are created.
\par
The factorization is performed using a one-dimensional
decomposition of the factor matrices.
Keeping the factor matrices in this form severely limits the amount
of parallelism for the forward and backsolves.
We perform a post-processing step to convert the one-dimensional
data structures to submatrices of a two-dimensional block
decomposition of the factor matrices.
The following code fragment performs this operation.
\begin{verbatim}
FrontMtx_postProcess(frontmtx, msglvl, msgFile) ;
\end{verbatim}
\par
\subsection{The Forward and Backsolves}
\label{subsection:serial:solve}
\par
The following code fragment solves the linear system
$(PAP^T) (PX) = PY$,
and permutes the solution $PX$ back into the original ordering,
yielding $X$.
\begin{verbatim}
mtxX = DenseMtx_new() ;
DenseMtx_init(mtxX, type, 0, 0, neqns, nrhs, 1, neqns) ;
DenseMtx_zero(mtxX) ;
FrontMtx_solve(frontmtx, mtxX, mtxB, mtxmanager,
               cpus, msglvl, msgFile) ;
DenseMtx_permuteRows(mtxX, newToOldIV) ;
\end{verbatim}
First we initialize a new {\tt DenseMtx} object to hold $X$
(and also $PX$).
(Note, in all cases {\it but} a nonsymmetric matrix with pivoting
enabled in an MPI environment, $X$ may overwrite $Y$, and so we
can use the same {\tt DenseMtx} object for $X$ and $Y$.)
We then solve the linear system with a call to
{\tt FrontMtx\_solve()}.
Note that one of the arguments is the {\tt mtxmanager} object,
first created for the numerical factorization.
The solve requires working submatrices, and so we continue the
convention of having the {\tt FrontMtx} ask the manager object for
working storage.
The last step is to permute the rows of the {\tt DenseMtx}
from the new ordering into the old ordering.
\par
\subsection{Sample Matrix and Right Hand Side Files}
\label{subsection:serial:input-files}
\par
Immediately below are two sample files:
{\tt matrix.input} holds the matrix input
and {\tt rhs.input} holds the right hand side.
This example is for a symmetric Laplacian operator
on a $3 \times 3$ grid.
Only entries in the upper triangle are stored.
The right hand side is the $9 \times 9$ identity matrix.
Note how the indices are zero-based as for C, instead of one-based
as for Fortran.
\begin{center}
\begin{tabular}{|l|}
\multicolumn{1}{c}{\tt matrix.input} \\ \hline
\begin{minipage}[t]{0.75 in}
\begin{verbatim}
9 9 21
0 0  4.0
1 1  4.0
2 2  4.0
3 3  4.0
4 4  4.0
5 5  4.0
6 6  4.0
7 7  4.0
8 8  4.0
0 1 -1.0
1 2 -1.0
3 4 -1.0
4 5 -1.0
6 7 -1.0
7 8 -1.0
0 3 -1.0
1 4 -1.0
2 5 -1.0
3 6 -1.0
4 7 -1.0
5 8 -1.0
\end{verbatim}
\end{minipage}
\\ \hline
\end{tabular}
\qquad
\begin{tabular}{|l|}
\multicolumn{1}{c}{\tt rhs.input} \\ \hline
\begin{minipage}[t]{2.75 in}
\begin{verbatim}
9 9
0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0
3 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0
4 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0
6 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
\end{verbatim}
\end{minipage}
\\ \hline
\end{tabular}
\end{center}
\par


\title{Classes of Inference}

{{navbar}}

\subsubsection{Classes of Inference}

Inference is broadly classified under three classes: variational
inference, Monte Carlo, and exact inference.
We highlight how to use inference algorithms from each class.

As an example, we assume a mixture model with latent mixture
assignments \texttt{z}, latent cluster means \texttt{beta}, and
observations \texttt{x}:
\begin{equation*}
p(\mathbf{x}, \mathbf{z}, \beta)
=
\text{Normal}(\mathbf{x} \mid \beta_{\mathbf{z}}, \mathbf{I})
~
\text{Categorical}(\mathbf{z}\mid \pi)
~
\text{Normal}(\beta\mid \mathbf{0}, \mathbf{I}).
\end{equation*}

\subsubsection{Variational Inference}

In variational inference, the idea is to posit a family of approximating
distributions and to find the closest member in the family to the
posterior \citep{jordan1999introduction}.
We write an approximating family,
\begin{align*}
q(\beta;\mu,\sigma) &= \text{Normal}(\beta; \mu,\sigma), \\[1.5ex]
q(\mathbf{z};\pi) &= \text{Categorical}(\mathbf{z};\pi),
\end{align*}
using TensorFlow variables to represent its parameters
$\lambda=\{\pi,\mu,\sigma\}$.
\begin{lstlisting}[language=Python]
from edward.models import Categorical, Normal

qbeta = Normal(loc=tf.Variable(tf.zeros([K, D])),
               scale=tf.exp(tf.Variable(tf.zeros[K, D])))
qz = Categorical(logits=tf.Variable(tf.zeros[N, K]))

inference = ed.VariationalInference({beta: qbeta, z: qz}, data={x: x_train})
\end{lstlisting}
Given an objective function, variational inference optimizes the
family with respect to \texttt{tf.Variable}s.

Specific variational inference algorithms inherit from
the \texttt{VariationalInference} class to define their own methods, such as a
loss function and gradient.
For example, we represent
MAP
estimation with an approximating family (\texttt{qbeta} and
\texttt{qz}) of \texttt{PointMass} random variables, i.e., with all
probability mass concentrated at a point.
\begin{lstlisting}[language=Python]
from edward.models import PointMass

qbeta = PointMass(params=tf.Variable(tf.zeros([K, D])))
qz = PointMass(params=tf.Variable(tf.zeros(N)))

inference = ed.MAP({beta: qbeta, z: qz}, data={x: x_train})
\end{lstlisting}
\texttt{MAP} inherits from \texttt{VariationalInference} and defines a
loss function and update rules; it uses existing optimizers inside
TensorFlow.

\subsubsection{Monte Carlo}

Monte Carlo approximates the posterior using samples
\citep{robert1999monte}. Monte Carlo is an inference where the
approximating family is an empirical distribution,
\begin{align*}
q(\beta; \{\beta^{(t)}\})
&= \frac{1}{T}\sum_{t=1}^T \delta(\beta, \beta^{(t)}), \\[1.5ex]
q(\mathbf{z}; \{\mathbf{z}^{(t)}\})
&= \frac{1}{T}\sum_{t=1}^T \delta(\mathbf{z}, \mathbf{z}^{(t)}).
\end{align*}
The parameters are $\lambda=\{\beta^{(t)},\mathbf{z}^{(t)}\}$.
\begin{lstlisting}[language=Python]
from edward.models import Empirical

T = 10000  # number of samples
qbeta = Empirical(params=tf.Variable(tf.zeros([T, K, D]))
qz = Empirical(params=tf.Variable(tf.zeros([T, N]))

inference = ed.MonteCarlo({beta: qbeta, z: qz}, data={x: x_train})
\end{lstlisting}
Monte Carlo algorithms proceed by updating one sample
$\beta^{(t)},\mathbf{(z)}^{(t)}$ at a time in the empirical approximation.
%
Markov chain Monte Carlo does this sequentially to update
the current sample (index $t$ of \texttt{tf.Variable}s) conditional on
the last sample (index $t-1$ of \texttt{tf.Variable}s).
%
Specific Monte Carlo samplers determine the update rules;
they can use gradients such as in Hamiltonian Monte Carlo
\citep{neal2011mcmc} and graph
structure such as in sequential Monte Carlo \citep{doucet2001introduction}.

\subsubsection{Non-Bayesian Methods}

As a library for probabilistic modeling (not necessarily Bayesian
modeling), Edward is agnostic to the paradigm for inference.  This
means Edward can use frequentist (population-based) inferences,
strictly point estimation, and alternative foundations for parameter
uncertainty.

For example, Edward supports non-Bayesian methods such as generative
adversarial networks (GANs)
\citep{goodfellow2014generative}.
For more details, see the \href{/tutorials/gan}{GAN tutorial}.

In general, we think opening the door to non-Bayesian approaches is a
crucial feature for probabilistic programming. This enables advances
in other fields such as deep learning to be complementary: all is in
service for probabilistic models and thus it makes sense to combine
our efforts.

\subsubsection{Exact Inference}

In order to uncover conjugacy relationships between random variables
(if they exist), we use symbolic algebra on nodes in the computational
graph.  Users can then integrate out variables to automatically derive
classical Gibbs \citep{gelfand1990sampling},
mean-field updates \citep{bishop2006pattern}, and exact inference.

For example,  can calculate a conjugate posterior analytically by
using the \texttt{ed.complete_conditional} function:

\begin{lstlisting}[language=Python]
from edward.models import Bernoulli, Beta

# Beta-Bernoulli model
pi = Beta(1.0, 1.0)
x = Bernoulli(probs=pi, sample_shape=10)

# Beta posterior; it conditions on the sample tensor associated to x
pi_cond = ed.complete_conditional(pi)

# Generate samples from p(pi | x = NumPy array)
sess.run(pi_cond, {x: np.array([0, 1, 0, 0, 0, 0, 0, 0, 0, 1])})
\end{lstlisting}

\subsubsection{References}\label{references}


\documentclass[11pt]{article}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{bm}
\usepackage{fancyvrb}
\usepackage{float}
\usepackage{graphicx}
\begin{document}

\title{APPM 5510 HW 6}
\author{Zane Jakobs}
\date{}
\maketitle

\subsection*{1(a)} Since the samples are i.i.d. and have finite variance, the sample mean follows a Student t-distribution with expected value $\mu$, and the expected value of a sample from the posterior is $\mu$, so the expected error is zero (i.e. the estimator is unbiased).
\subsection*{1(b)} As $N\to\infty$, the variance of the sample mean, $\frac{\sigma^2}{N}$, goes to zero, but, independently of the draws used to form the estimate, the true value has a variance of $\sigma^2$, so since the error is the difference between the sample mean and the true value, which are independent, its variance is the sum of the variances of the sample mean and true value, $\sigma^2 (1 + \frac{1}{N})$. Thus, the variance does not go to zero as $N\to\infty$.

\subsection*{2(a)}
NOTE: all code is attached to the end of this assignment. We first get this plot:\\
\begin{figure}[H]
\includegraphics[scale=0.5]{/Users/zanejakobs/Desktop/p2a.png}
\end{figure}
and we have non-Gaussianity on 12\% of timesteps, and error greater than observation error on 14\%.
\subsection*{2(b)}
Running with the observation error variance set to 0.01, we get this plot:\\
\begin{figure}[H]
\includegraphics[scale=0.5]{/Users/zanejakobs/Desktop/p2b.png}
\end{figure}
with non-Gaussianity 60\% of the time, and error greater than observation error 8.5\% of the time.
\subsection*{3(a)} Letting $\bm{1}$ be a vector of ones and $\bm{\varepsilon}$ be the vector of perturbations, condition (i) implies that $\bm{1}^T\bm{\varepsilon} = 0.$ Now, given condition (i), the expected values of the perturbation and state ensembles are both $\mu\bm{1}$, so the cross-covariance matrix is
\[
\begin{aligned}
\mathbb{E}[(\bm{y} - \mu\bm{1})(\bm{y} + \bm{\varepsilon} - \mu\bm{1})^T] &= \mathbb{E}[\bm{y}\bm{y}^T + \bm{y}\bm{\varepsilon}^T - \mu\bm{y}\bm{1}^T - \mu\bm{1}\bm{y}^T - \mu\bm{1}\bm{\varepsilon}^T + \mu^2 \bm{1}\bm{1}^T]\\
&= \bm{y}\bm{y}^T- \mu\bm{y}\bm{1}^T - \mu\bm{1}\bm{y}^T + \mu^2 \bm{1}\bm{1}^T + \mathbb{E}[\bm{y}\bm{\varepsilon}^T - \mu\bm{1}\bm{\varepsilon}^T]\\
&= \bm{y}\bm{y}^T- \mu\bm{y}\bm{1}^T - \mu\bm{1}\bm{y}^T + \mu^2 \bm{1}\bm{1}^T + (\bm{y} - \mu\bm{1})\bm{\varepsilon}^T. \end{aligned}
\]
In order for the cross-covariance to be zero, we need this matrix to equal the identity matrix $\bm{I}$--that is, we need
\[
(\bm{y} - \mu\bm{1})\bm{\varepsilon}^T = \bm{I} - \bm{y}\bm{y}^T+ \mu\bm{y}\bm{1}^T + \mu\bm{1}\bm{y}^T - \mu^2 \bm{1}\bm{1}^T.
\]
To simultaneously satisfy conditions (i) and (ii), we need for the following two equations to hold:
\[
\begin{aligned}
\bm{1}^T\bm{\varepsilon} &= 0\\
(\bm{y} - \mu\bm{1})\bm{\varepsilon}^T &= \bm{I} - \bm{y}\bm{y}^T+ \mu\bm{y}\bm{1}^T + \mu\bm{1}\bm{y}^T - \mu^2 \bm{1}\bm{1}^T
\end{aligned}
\]
\subsection*{3(b)} Solutions to this system are not unique--that is, there are multiple combinations of actual values of perturbations that each independently satisfy the constraints.

\subsection*{4} All of the eigenvalues fit on one histogram, shown here:\\
\begin{figure}[H]
\includegraphics[scale=0.6]{/Users/zanejakobs/Desktop/p4hist.png}
\end{figure}
The largest eigenvalue is well-approximated, but the smaller ones are not quite as well-approximated (they skew upwards).

\begin{Verbatim}[xleftmargin=-2cm]
import numpy as np
from scipy.stats import anderson
import matplotlib.pyplot as plt

def advance_and_observe(forecast, x0, obs_var):
    obs_error = np.random.normal(0.0, np.sqrt(obs_var))
    return forecast(x0) + obs_error

def forward_model(step_model, x0, n):
    obs = [x0]
    for i in range(n):
        obs.append(step_model(obs[-1]))

    return np.array(obs)

def draw_ensemble(mean, var, n):
    return np.random.normal(mean, np.sqrt(var), n)

def create_ensemble(obs, ensemble_mean, ensemble_var, n):
    return obs + draw_ensemble(ensemble_mean, ensemble_var, n)

def EnKF_Step(forecast, ensemble, obs, obs_var, n, inflation=1.0, H=None, adthreshold=0.05):

    for i in range(n):
        #forecast ensemble
        ensemble[i] = forecast(ensemble[i])
    #check if prior is Gaussian with Anderson-Darling
    stat, crit_vals, sig_lvls = anderson(ensemble, 'norm')
    #5% is the third critical value
    if stat >= crit_vals[2]:
        print("Ensemble fails Anderson-Darling test (is rejected as Normal) with test statistic %f and critical value %f" % (stat, crit_vals[2]))
        failed=True
    else:
        failed=False
    if not inflation == 1.0:
        ensemble *= np.sqrt(inflation)
    #covariance
    amean = np.mean(ensemble) #analysis mean

    C = np.sum((ensemble - amean) ** 2) / (n-1)

    #update ensemble
    if H is None:

        K = C / (C + obs_var)

        ensemble += K * (obs + draw_ensemble(0.0, obs_var, n) - ensemble)

        C -= K * C

        amean = np.mean(ensemble) #analysis mean
        aspread = np.sqrt(C)

    return amean, aspread, ensemble, failed

def EnKF_Run(forecast, init_ensemble, obs, obs_var, timesteps, n,inflation=1.0, H=None, adthreshold=0.05):
    ameans = []
    apreads = []
    nfailed = 0
    for t in range(timesteps):
        am, asp, init_ensemble, failed = EnKF_Step(forecast, init_ensemble, obs[t], obs_var, n, inflation=1.0, H=None, adthreshold=0.05)
        if failed:
            nfailed += 1
        ameans.append(am)
        apreads.append(asp)

    return ameans, apreads, nfailed

if __name__ == '__main__':

    def advance(x):
        xp = 3.95 * x * (1-x)
        if xp < 0.0:
            xp = 0.0
        elif xp > 1.0:
            xp = 1.0

        return xp

    #generate obs
    n = 200
    x0 = 0.25
    x = forward_model(advance, x0, n-1)

    obs_var = 0.01
    ensemble_size = 50
    xerr = draw_ensemble(0.0, obs_var, n)

    obs = x + xerr

    iensemble = draw_ensemble(1.0/3.0, 0.01, ensemble_size)

    emeans, evars, nfailed = EnKF_Run(advance, iensemble, obs, obs_var, n, ensemble_size)

    print("non-gaussian on fraction of timesteps" , (float(nfailed)/200.0), '\n')

    err = np.abs(obs - emeans)

    bigerr=0

    for i in range(n):
        if err[i] > np.sqrt(obs_var):
            bigerr += 1

    print("Error greater than observation error on fraction of timesteps", float(bigerr)/200.0, '\n')

    plt.plot(np.array(list(range(n))), err)
    plt.title('EnKF Error vs Timestep')

    #random matrix
    A = np.random.rand(100,100)

    Q, R = np.linalg.qr(A)

    D = np.identity(100)
    D[0,0] = np.sqrt(100)

    Broot = Q @ D @ Q.T

    x = np.empty((100,110))

    for i in range(110):
        x[:,i] = Broot @ np.random.standard_normal(100)

    C = np.cov(x)

    lbda, v = np.linalg.eig(C)

    lbda = lbda[np.argsort(lbda)[-100:]] #10 largest

    plt.figure()
    plt.hist(lbda, bins=50)

    plt.show()

\end{Verbatim}

\end{document}


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{March 10, 2014}
\maketitle
\section*{homework}
\#30:
$\{12,13,14,23,24,34\}$ is only antichain. Not $1234$ or $\emptyset$ since they are comparable to everything.

\section*{our counting box from way back}
\begin{tabular}{|r|c|c|c|}
  \hline
  &none&unlimited&restricted supply\\
  \hline
  ordered&$\frac{n!}{(n-r)!}$&$n^r$&$\frac{n!}{(n-r)!x_1!x_2!\dots x_n!}$\\
  \hline
  unordered&$\frac{n!}{(n-r)!r!}=\binom{n}{r}$&$\binom{r+n-1}{r}$&*use inclusion-exclusion\\
  &$k_1=k_2=\dots=k_n=1$&$k_1=k_2=\dots=k_n=r$ (or $\infty$)&$1\le k_i$\\
  \hline
\end{tabular}

unify bottom row into the folowing problem:

What is the number of r-combinations taken from the multiset $\{k_1\cdot a_1,k_2\cdot a_2,\dots,k_n\cdot a_n\}$?

$=\{\underbrace{a_1,a_1,\dots,a_1}_{k_1\text{ copies}},\underbrace{a_2,\dots,a_2}_{k_2},\dots,\underbrace{a_n,\dots,a_n}_{k_n}\}$

pick r elts from above
\subsection*{example}
in how many ways could you choose 12 pieces of candy if there are at the store: 13 butterscotch, 4 root beer barrels, 8 lemon heads, 5 cinnamon.

our multiset is $\{13\cdot b,4\cdot r, 8\cdot l, 5\cdot c\}$. butterscotch$>12=\infty$ so $\{\infty\cdot b,4\cdot r, 8\cdot l, 5\cdot c\}$ lest $S=\{12-\text{combinations of }\{\infty\cdot b,\infty\cdot r,\infty\cdot l,\infty\cdot c\}\}$
\begin{align*}
  \left\lvert S\right\rvert&=\binom{12+4-1}{12}\\
  &=\binom{15}{12}\\
\text{Let }A_r&=\{12-\text{combinations in $S$ with }\ge5\; r\text{'s}\}\\
\text{Let }A_l&=\{12-\text{combinations in $S$ with }\ge9\; l\text{'s}\}\\
\text{Let }A_c&=\{12-\text{combinations in $S$ with }\ge6\; c\text{'s}\}\\
\end{align*}
$\left\lvert S\right\rvert-\left\lvert A_r\cup A_l\cup A_c\right\rvert$ do this with inclusion exclusion

$A_r=\{rrrrr\text{ and 7 others}\}=$7-combinations from 4 types (infinite supply). $\left\lvert A_r\right\rvert=\binom{7+4-1}{7}$. $\left\lvert A_l\right\rvert=\binom{3+4-1}{3}$, $\left\lvert A_c\right\rvert=\binom{6+4-1}{6}$. similarly with intersections.

Rewrite example as follows: what is the \# of nonnegative integral solutions to
\begin{align*}
  x_1+x_2+x_3+x_4=12
\end{align*}
where $0\le x_2\le4,x_3\le8,x_4\le5,x_1\le\infty$

exam question?
equivalently to $y_1+y_2+y_3+y_4=$ where $\le y_1\le\quad \le y_2\le\quad\le y_3\le$

equivalently to $y_1+y_2+y_3+y_4=$ where $4\le y_1\le\infty\quad 1\le y_2\le5\quad-1\le y_3\le7\quad 2\le y_4\le7$

so we've done $y_1=x_1+4,y_2=x_2+1,y_3=x_3-1,y_4=x_4+2$
\end{document}


\chapter{Introduction}

    \section*{Landscape and Motivation}
        This paper investigates a natural proof search introduced by \citet{petri-nets}.
        Given a formula composed of conjunctions and disjunctions of variables, it falls into one of three categories: provably true, satisfiably true/false, provably false.
        Satisfiably true or false terms are equivalent to those studied in the boolean satisfiability problem (SAT).
        Since the set of provably false formulae forms the complement to the set of satisfiably true formulae, with provably true similarly forming the complement to satisfiably false, the question of proof search and its complexity ties directly into problems of P versus NP\@.
        The motivating question was `What is the complexity of the chosen proof search'.

    \section*{Research and Results}
        The paper begins with a run-through of classical logic and sequent proofs before examining the proof search algorithm in question.
        The research entails studying the properties of this algorithm and implementation details that affect computational complexity, in particular a natural optimisation that gives great performance benefits for certain classes of formulae.
        Finally, the essence of the problem is dissected --- how can the complexity of proof search be bounded by the structure of the formula?

    \section*{Implementation and Validation}
        A C implementation of the described algorithm complements the research and may be found at \url{https://gitlab.com/adamlassiter/petri-nets}.
        This is broken down into:
        \begin{itemize}
            \item An additive linear logic (ALL) formula parser with some additional inferences for some operators ($(a \implies b) \equiv (\neg a \vee b)$ etc.)
            \item An implementation of the coalescence algorithm using a derivative of petri nets, including a described optimisation on the complexity bound for certain formulae
            \item A system to backtrack over petri nets fired through coalescence to their equivalent sequent proofs --- a bug remains present in this code where proofs are occasionally backtracked incorrectly
        \end{itemize}
        While this remains somewhat of a work in progress, the code as is provides consistent results and moderately good performance --- when benchmarked against Naoyuki Tamura's Prolog sequent prover\footnote{\url{http://bach.istc.kobe-u.ac.jp/seqprover/}} saw a decrease in runtime of up to 25x for some formulae.