% !TEX root = ../master/master.tex

%% Dexter Barrows, 2016
%% dbarrows.github.io

\section{S-maps}

    A family of forecasting methods that shy away from the mechanistic model-based approaches outlined in the previous sections have been developed by George Sugihara and collaborators \cite{Sugihara1990}\cite{Sugihara1994}\cite{Hsieh2008}\cite{Glaser2014} over the last several decades. As these methods do not include a mechanistic model in their forecasting process, they also do not attempt to perform parameter estimation or inference. Instead they attempt to reconstruct the underlying dynamical process as a weighted linear model from a time series.

    One such method, the sequential locally weighted global linear maps (S-map), builds a global linear map model and uses it to produce forecasts directly. Despite relying on a linear mapping, the S-map does not assume the time series on which it is operating is the product of linear system dynamics, and in fact was developed to accommodate non-linear dynamics. The linear component of the method only comes into play when combining forecast components together to produce a single estimate

    The S-map works by first constructing a time series embedding of length $E$, known as the library and denoted $\{\mathbf{x_i}\}$. Consider a time series of length $T$ denoted $x_1, x_2,..., x_T$. Each element in the time series with indices in the range $E,E+1,...,T$ will have a corresponding entry in the library such that a given element $x_t$ will correspond to a library vector of the form $\mathbf{x_i} = (x_t, x_{t-1},...,x_{t-E+1})$. Next, given a forecast length $L$ (representing $L$ time steps into the future), each library vector $\mathbf{x_i}$ is assigned a prediction from the time series $y_i = x_{t+L}$, where $x_t$ is the first entry in $\mathbf{x_i}$. Finally, a forecast ${\hat{y_t}}$ for specified predictor vector $\mathbf{x_t}$ (usually from the library itself), is generated using an exponentially weighted function of the library $\{\mathbf{x_i}\}$, predictions $\{y_i\}$, and predictor vector $\mathbf{x_t}$.

    This function is defined as follows:

    First construct a matrix $A$ and vector $b$ defined as

    \begin{equation}\label{AB}
    	\begin{array}{rl}
        \displaystyle
            A(i, j) & = w (||\mathbf{x_i}-\mathbf{x_t}||) \mathbf{x_i}(j) \\
            b(i) 	& = w (||\mathbf{x_i}-\mathbf{x_t}||) y_i
        \end{array}
    \end{equation}

    where $||\cdot||$ is the Euclidean norm, $i$ ranges over 1 to the length of the library, and $j$ ranges over $[0,E]$. In the above equations and the ones that follow, we set $x_t(0) \equiv 1$ to account for the linear term in the map.

	The weighting function $w$ is defined as

	\begin{equation}
		w(d) = \exp \left( \frac{-\theta d}{\bar{d}} \right) ,
	\end{equation}

	where $d$ is the euclidean distance between the predictor vector and library vectors in Equation [\ref{AB}] and $\bar{d}$ is the average of these distances. We can then see that $\theta$ serves as a way to specify the appropriate level of penalization applied to poorly-matching library vectors -- if $\theta$ is 0 all weights are the same (no penalization), and increasing $\theta$ increases the level of penalization.

	Now we solve the system $Ac = b$ to obtain the linear weightings used to generate the forecast according to

	\begin{equation}
		\hat{y_t} = \sum_{j = 0}^{E} c_t(j) \mathbf{x_t}(j) .
	\end{equation}

	In this way we have produced a forecast value for a single time. This process can be repeated for a sequence of times $T + 1, T + 2, ...$ to project a time series into the future.

	In essence what we are doing is generating a series of forecasts from every vector in the library, weighting those forecasts based on how similar the corresponding library vector is to our predictor vector, obtaining a solution to the system that maps components of a predictor vector to its library vector's forecasted point (the mapping), then applying that mapping to our predictor variable to obtain a forecast.

\section{S-map Algorithm}

    The above description can be summarized in Algorithm [\ref{smap}].

    \begin{algorithm}

        \BlankLine

        \SetKwInOut{Input}{Input}
        \SetKwInOut{Output}{Output}
        \DontPrintSemicolon

        \tcc{Select a starting point}
        \Input{Time series $x_1, x_2, ..., x_T$, embedding dimension $E$, distance penalization $\theta$, forecast length $L$, predictor vector $\mathbf{x_t}$}

        \BlankLine

        \tcc{Construct library $\{{\mathbf{x_i}}\}$}
        \For{$i = E:T$}{
        	$\mathbf{x_i} = (x_i, x_{i-1}, ...,x_{i-E-1})$
        }

        \BlankLine

        \tcc{Construct mapping from library vectors to predictions}
        \For{$i = 1:(T_E+1)$}{
        	\For{$j = 1:E$}{
	        	$A(i,j) = w(||\mathbf{x_i} - \mathbf{x_t}||) \mathbf{x_i}(j)$
	        }
        }
        \For{$i = 1:(T_E+1)$}{
	        $b(i) = w(||\mathbf{x_i} - \mathbf{x_t}||) y_i$
        }

        \BlankLine

        \tcc{Use SVD to solve the mapping system, Ac = b}
        $SVD(Ac = b)$

        \BlankLine

        \tcc{Compute forecast}
        $\hat{y_t} = \sum_{j = 0}^{E} c_t (j) \mathbf{x_t} (j)$

        \BlankLine

        \tcc{Forecasted value in time series}
        \Output{Forecast $\hat{y_t}$}

        \BlankLine

        \caption{S-map \label{smap}}

    \end{algorithm}

\section{SIRS Model}

	In an epidemic or infectious disease context, the S-map algorithm will only really work on time series that appear cyclic. While there is nothing mechanically that prevents it from operating on a time series that do not appear cyclic, S-mapping requires a long time series in order to build a quality library. Without one the forecasting process would produce unreliable data.

	Given, the S-map's data requrements, we need to specify a modified version of the SIR model. As IF2 and HMC in principle should be able operate on any reasonably well-specified model, the easiest way to compare the efficacy of S-mapping to IF2 or HMC is to generate data from a SIRS model with a seasonal component, and have all methods operate on the resulting time series.

	The basic skeleton of the SIRS model is similar to the stochastic SIR model described previously, with one small addition. The deterministic ODE component of the model is as follows.

	\begin{equation}
		\begin{aligned}
			\frac{dS}{dt} & = - \Gamma(t) \beta S I + \alpha R \\
			\frac{dI}{dt} & = \Gamma(t) \beta S I - \gamma I \\
			\frac{dR}{dt} & = \gamma I - \alpha R,
		\end{aligned}
	\end{equation}

	There are two new features here. We have a rate or waning immunity $\alpha$ through which people become able to be reinfected, and a seasonality factor function $\Gamma (t)$ defined as

	\begin{equation}
		\Gamma(t) = \exp \left[ 2 \left( \cos \left(  \frac{2 \pi}{1 \, \text{year}} t \right) - 1 \right) \right],
	\end{equation}

	where $t$ is in days. This function oscillates between 1 and $e^{-4}$ (close to 0) and is meant to represent transmission damping during the off-season, for example summer for influenza. Further, it displays flatter troughs and sharper peaks to exaggerate its effect in peak season.

	As before, $\beta$ is allowed to walk restricted by a geometric mean, described by

	\begin{equation}
		\beta_{t+1} = \exp \left( \log(\beta_{t}) + \eta (\log(\bar{\beta}) - \log(\beta_{t})) + \epsilon_{t} \right).
	\end{equation}

	Figure [\ref{sirsdataplot}] shows the SIRS model simulated for the equivalent of 5 years (260 weeks) and adding noise drawn from $\mathcal{N}(0,\sigma)$.

	\begin{figure}
        \centering
        \captionsetup{width=.8\linewidth}
        \includegraphics[width=0.8\textwidth]{./images/dataplot.pdf}
        \caption{Five cycles generated by the SIRS function. The solid line the the true number of cases, dots show case counts with added observation noise. The parameter values were $\mathcal{R}_0 = 3.0$, $\gamma = 0.1$, $\eta = 1$, $\sigma = 5$, and 10 initial cases. \label{sirsdataplot}}
    \end{figure}

    Figure [\ref{smap_project}] shows how the S-map can reconstruct the next cycle in the time series.

    \begin{figure}[H]
        \centering
        \captionsetup{width=.8\linewidth}
        \includegraphics[width=0.8\textwidth]{./images/smap-project.pdf}
        \caption{S-map applied to the data from the previous figure. The solid line shows the infection counts with observation noise from the previous plot, and the dotted line is the S-map forecast. Parameters chosen were $E = 14$ and $\theta = 3$. \label{smap_project}}
    \end{figure}

    The parameters used in the S-map algorithm to obtain the forecast used in Figure [\ref{smap_project}] were obtained using a grid search of potential parameters outlined in \cite{Glaser2014}. The script to perform this optimisation procedure is included in the appendices.

\section{SIRS Model Forecasting}

	Naturally we wish to compare the efficacy of this comparatively simple technique against the more complex and more computationally taxing frameworks we have established to perform forecasting using IF2 and HMC.

	To do this we generated a series of artificial time series of length $260$ meant to represent 5 years of weekly incidence counts and used each method to forecast up to 2 years into the future. Our goal here was to determine how forecast error changed with forecast length.

	Figure [\ref{sirssseplot}] shows the results of the simulation.

	\begin{figure}
        \centering
        \captionsetup{width=.8\linewidth}
        \includegraphics[width=0.8\textwidth]{./images/sseplot.pdf}
        \caption{Error as a function of forecast length. \label{sirssseplot}}
    \end{figure}

    Interestingly, all methods produce roughly the same result, which is to say the spikes in each outbreak cycle are difficult to accurately predict. IF2 produces better results than either HMC and the S-map for the majority of forecast lengths, with the S-map producing the poorest results with the exception of the second rise in infection rates where it outperforms the other methods.

    While the S-map may not provide the same fidelity or forecast as IF2 or HMC, it shines when it comes to running time. Figure [\ref{sirstimeplot}] shows the running times over 20 simulations.

    \begin{figure}
        \centering
        \captionsetup{width=.8\linewidth}
        \includegraphics[width=0.8\textwidth]{./images/timeplot.pdf}
        \caption{Runtimes for producing SIRS forecasts. The box shows the middle 50th percent, the bold line is the median, and the dots are outliers. Note that these are not ``true'' outliers, simply ones outside a threshold based on the interquartile range. \label{sirstimeplot}}
    \end{figure}

    It is clear from Figure [\ref{sirstimeplot}] that the S-map running times are minute compared to the other methods, but to emphasize the degree: The average running time for the S-map is about $0.15$ seconds, for IF2 it is about $47,000$, and for HMC it is about $9,200$. This is a speed-up of over 316,000x compared to IF2 and over 61,800x compared to HMC.

    Additionally, we are interested in coverage. While a full coverage analysis would require roughly a 100-fold increase in computational complexity, we can use the trajectories generated by IF2 and HMC to display forecast coverage across data sets, given particular weeks in the forecast.

    Figure [\ref{sirscoverage}] shows such plots for forecasts 10 and 45 weeks ahead. Week 10 represents the approximate first epidemic ``spike'' we are attempting to predict, a typically difficult task, and week 45 represents the first ``trough'', which should be easier to predict. We can see that the error bars are much wider when attempting to predict the spike, and much smaller for the trough. Also the trough coverage appears slightly better than the spike coverage.

    \begin{figure}
        \centering
        \captionsetup{width=.8\linewidth}
        \includegraphics[width=0.8\textwidth]{./images/coverage.pdf}
        \caption{Coverage plots for forecast weeks 10 (top) and 45 (bottom). Black bars are from IF2 forecast trajectories, and grey bars are from HMC trajectories. \label{sirscoverage}}
    \end{figure}


\onecolumn
\section{Implementation details}
\label{sec:implementation}

\section{Theoretical proofs for the sampling procedures}
\label{sec:sampling}

Let's refresh notation from \refsec{method}.

Let $\sX$ be a universe of possible outputs (e.g.\ relation instances),
  $\sY \subseteq \sX$ be an unknown subset of this universe corresponding to the correct elements in $\sX$ and
  $X_1, \ldots X_m \subseteq \sX$ be known subsets that correspond to the predicted output from $m$ systems,
  and $Y_1, \ldots, Y_m$ be the intersection of $X_1, \ldots, X_m$ with $\sY$.
Furthermore,
  let $\Xh_i$ be a mulit-set of $n_i$ independent samples drawn from $X_i$ with the distribution $p_i$,
  $\Yh_i$ be the intersection of these sets with $\sY$, and
  $\Yh_0$ be a sample drawn from $\sY$ according to an unknown distribution $p'(x)$.

We would like to evaluate precision, $\pi_i$, and recall, $\recall_i$:
\begin{align*}
  \pi_i  &\eqdef \E_{x \sim X_i}[f(x)] &
  \recall_i &\eqdef \E_{x \sim \sY}[g_i(x)],
\end{align*}

In this section, we'll provide proofs that show that the joint estimators proposed in \refsec{method} are indeed unbiased, and we will characterize their variance.

\subsection{Estimating precision}

In \refsec{method}, we proposed the following estimator for $\pi_i$:
\begin{align*}
  \pih_i &\eqdef \sum_{j=1}^m \frac{w_{ij}}{n_j} \sum_{x \in \Xh_j} \frac{p_i(x) f(x)}{q_i(x)},
\end{align*}
where $q_i(x) = \sum_{j=1}^m w_{ij} p_j(x)$ and $w_{ij} \ge 0$ are mixture parameters such that $\sum_{j=1}^m w_{ij} = 1$ and $q_i(x) > 0$ wherever $p_i(x) > 0$.

\begin{theorem}[Statistical properties of $\pih_i$]
\label{thm:pih}
  $\pih_i$ is an unbiased estimator of $\pi_i$ and has a variance of:
  \begin{align*}
    \Var{\pih_i} &= \sum_{j=1}^m \frac{w_j^2}{n_j} \E_{p_j}\left[\frac{{p_i(x)}^2{f(x)}^2 - \pi_{ij}p_i(x)f(x)q_i(x)}{{q_i(x)}^2} \right],
  \end{align*}
  where $\pi_{ij} \eqdef \E_{p_j}\left[\frac{p_i(x)f(x)}{q_i(x)} \right]$.
\end{theorem}
\begin{proof}
  Let $\Xh = (\Xh_1, \ldots, \Xh_m)$ which is drawn from the product distribution of $p_1 \times p_m$.
  By independence and the linearity of expectation,
  \begin{align*}
    \E_{\Xh}\left[\sum_{j=1}^m f(\Xh_j) \right] &= \sum_{j=1}^m \E_{\Xh_j}[f(\Xh_j)].
  \end{align*}

  First, let's show that $\pih_i$ is unbiased:
  \begin{align*}
    \E_{\Xh}[\pih_i]
    &= \E_{\Xh}\left[\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Xh_j} \frac{p_i(x) f(x)}{q_i(x)} \right] \\
    &= \sum_{j=1}^m \frac{w_j}{n_j} \E_{\Xh_j}\left[\sum_{x \in \Xh_j} \frac{p_i(x) f(x)}{q_i(x)} \right] \\
    &= \sum_{j=1}^m \frac{w_j}{n_j} n_j \E_{p_j}\left[\frac{p_i(x) f(x)}{q_i(x)}\right] \\
    &= \sum_{j=1}^m w_j \sum_{x \in \sX} p_j(x) \frac{p_i(x) f(x) }{q_i(x)}\\
    &= \sum_{x \in \sX} \sum_{j=1}^m w_j p_j(x) \frac{p_i(x) f(x) }{q_i(x)}\\
    &= \sum_{x \in \sX} q_i(x) \frac{p_i(x) f(x)}{q_i(x)} \\
    &= \sum_{x \in \sX} p_i(x) f(x) \\
    &= \pi_i.
  \end{align*}

  Now let's compute the variance.
  \begin{align*}
    \Var{\pih_i}
    &= \sum_{j=1}^m \frac{w_j^2}{n_j} \E_{p_j}\left[\frac{{p_i(x)}^2{f(x)}^2}{{q_i(x)}^2} \right]
    - \sum_{j=1}^m \frac{w_j^2}{n_j} {\E_{p_j}\left[\frac{p_i(x)f(x)}{q_i(x)} \right]}^2 \\
    &= \sum_{j=1}^m \frac{w_j^2}{n_j} \E_{p_j}\left[\frac{{p_i(x)}^2{f(x)}^2}{{q_i(x)}^2} - \frac{\pi_{ij} p_i(x) f(x)}{q_i(x)} \right] \\
    &= \sum_{j=1}^m \frac{w_j^2}{n_j} \E_{p_j}\left[\frac{{p_i(x)}^2{f(x)}^2 - \pi_{ij}p_i(x)f(x)q_i(x)}{{q_i(x)}^2} \right],
  \end{align*}
  where $\pi_{ij} \eqdef \E_{p_j}\left[\frac{p_i(x)f(x)}{q_i(x)} \right]$.
\end{proof}

\subsection{Estimating recall}

In \refsec{method}, we used the fact that the recall of system $i$, $\recall_i$, can be expressed as the recall of $i$ within the pool, $\nu_i$ and the recall of the pool itself $\theta$: $\recall_i = \theta \nu_i$:
\begin{align*}
  \nu_i &= \E_{x \sim \sY \given Y}[g_i(x)] &
  \theta &= \E_{x \sim \sY}[g(x)],
\end{align*}
where $x$ is sampled under the distribution $p'(x \given x \in Y)$ and $p'(x)$ respectively and
  $g(x) \eqdef \I[x \in \Union_{i=1}^m X_i] = \max_{j \in [1,m]} g_j(x)$ is the indicator function for $x$ belonging to the pool.

Ideally, to estimate the pooled recall, $\nu_i$, we need to take expectations with respect to $x \sim Y$.
However, we only have samples drawn from individual $X_i$.
To correct for this bias, we'll use a self-normalizing estimator for $\nu_i$:
\begin{align*}
  \nuh_i &\eqdef \frac{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p_0(x) g_i(x)}{q(x)}}{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p_0(x)}{q(x)}},
\end{align*}
where $p'(x) \propto p_0(x)$, $q(x) = \sum_{j=1}^m w_{j} p_j(x)$ and $w_{j} \ge 0$ are mixture parameters such that $\sum_{j=1}^m w_{j} = 1$.

The pool recall $\theta$ can be estimated as follows:
\begin{align*}
  \thetah &\eqdef \sum_{x \in \Yh_0} g(x),
\end{align*}
where $g(x) \eqdef \I\left[ x \in \Union_{i=1}^m X_i \right] = \max_{j \in [1,m]} g_j(x)$.

Finally, we proposed the following estimator for recall $\recall_i$:
\begin{align*}
  \recallh_i &\eqdef \thetah\nuh_i.
\end{align*}

Let's start by showing that $\nu_i$ is unbiased.
\begin{theorem}[Statistical properties of $\nuh_i$]
\label{thm:nuh}
  $\nuh_i$ is a \textit{consistent} estimator of $\nu_i$.
\end{theorem}
\begin{proof}
  We have that $p'_Y(x) = \frac{w(x)}{Z_Y}$.
  While we do not know the value of $Z_Y$, we can divide both the numerator and denominator of $\nuh_i$ by this quantity:
  \begin{align*}
    \nuh_i
    &= \frac{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p_0(x) g_i(x)}{Z_Y q(x)}}{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p_0(x)}{Z_Y q(x)}} \\
    &= \frac{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p'_Y(x) g_i(x)}{q(x)}}{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p'_Y(x)}{q(x)}}.
  \end{align*}

  As the number of samples $n_i \to \infty$,
  \begin{align*}
    \E_{X}[\nuh_i]
    &= \E_{X}\left[ \frac{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p'_Y(x) g_i(x)}{q(x)}}{\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p'_Y(x)}{q(x)}} \right] \\
    &= \frac{\E_{X}\left[ \sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p'_Y(x) g_i(x)}{q(x)} \right]}
    {\E_{X}\left[\sum_{j=1}^m \frac{w_j}{n_j} \sum_{x \in \Yh_j} \frac{p'_Y(x)}{q(x)}\right]}.
  \end{align*}

  Following similar arguments as in the proof of \refthm{pih}, the numerator and denominator are unbiased estimators of $\E_{x \sim \sY \given Y}[g_i(x)]$ and $\E_{x \sim \sY \given Y}[1] = 1$ respectively.
  Thus,
  \begin{align*}
    \E_{X}[\nuh_i]
    &= \E_{x \sim \sY \given Y}[g_i(x)] \\
    &= \nu_i.
  \end{align*}
  $\nuh_i$ is an unbiased estimator of $\nu_i$.

  % TODO: derive variance which will look ugly as hell.
\end{proof}

Finally, we turn to studying $\recallh$:
\begin{theorem}[Statistical properties of $\recallh_i$]
\label{thm:rhoh}
  $\recallh_i$ is an unbiased estimator of $\recall_i$ with variance
  \begin{align*}
    \Var \recallh_i &= \theta \Var \nuh_i + \nu_i \Var\thetah + \Var\thetah \Var\nuh_i.
  \end{align*}
\end{theorem}
\begin{proof}
  First, let's show that $\recall_i = \theta \nu_i$:
  \begin{align*}
    \recall_i
    &\eqdef= \E_{x \sim \sY}[g_i(x)] \\
      &= p'(Y_i) \\
      &= p'(Y \band Y_i) \\
      &= p'(Y) p'(Y_i \given Y) \\
      &= \E_{x\sim \sY}[g(x)] \E_{x \sim \sY\given Y}[g_i(x)] \\
      &= \theta \nu_i.
  \end{align*}

  From \refthm{nuh}, we have that $\nuh_i$ is an unbiased estimator of $\nu_i$.
  It is evident that $\thetah$ is an unbiased estimator of $\theta$.
  $\nuh_i$ and $\thetah$ are estimated using independent samples ($\Yh$ and $\Yh_0$ respectively), and hence
  \begin{align*}
    \E_{Y_0, Y}[\recallh]
      &= \E_{Y_0, Y}[\thetah \nuh_i] \\
      &= \E_{Y_0}[\thetah] \E_Y[\nuh_i] \\
      &= \theta \nu_i \\
      &= \recallh.
  \end{align*}

  By \reflem{variance-product},
  \begin{align*}
    \Var \recallh_i &= \theta \Var\nuh_i + \nu_i \Var\thetah + \Var\thetah \Var\nuh_i.
  \end{align*}
\end{proof}

\subsection{Picking heuristic $w_{ij}$.}
\label{sec:heuristic-wij}

\subsection{Picking optimal number of samples for a new system}
\label{sec:optimal-samples}

In \refsec{joint}, we outlined a method to pick the optimal number of samples to draw and evaluate for a new system:
  we pick the minimum number of samples $n_m$ required to evaluate system $m$ within a target variance using a conservative estimate of the variance of $\pih_m^{\text{(joint)}}$.
  In particular, we use the following estimate for variance using the result from \refthm{pih}:
\begin{align*}
  \widehat{\Var}{\pih_m} &= \sum_{j=1}^{m-1} \frac{w_j^2}{n_j} \sum_{x \in \Xh_j} \frac{1}{n_j} \left[\frac{{p_i(x)}^2{f(x)}^2 - \pi_{ij}p_i(x)f(x)q_i(x)}{{q_i(x)}^2} \right] + \frac{w_m^2}{n_m} \sum_{x \in X_m} p_m(x) {\left[ \frac{p_m(x)}{q(x)} \right]}^2,
\end{align*}
where the first $m-1$ terms are an empirical estimate of variance and the last term is an upper bound on the variance.
We note that the actual output of each system, $X_j$, and the samples drawn from previous systems, $\Xh_{j}$, is known.
Thus, the only variable in computing $\widehat{\Var}{\pih_m}$ is $n_m$.
Furthermore, $\widehat{\Var}{\pih_m}$ is a monotonically decreasing in $n_m$, so we can easily solve for the minimum number of samples required to estimate $\pih_m^{\text{(joint)}}$ within a confidence interval $\epsilon$ by using the bisection method \citep{burden1985bisection}.


\chapter{AES in Cryptol}\label{app:aes}

In this appendix we present the Cryptol code for the AES\indAES in its
entirety for reference purposes.
\hyperref[chapter:aes]{Chapter~\ref*{chapter:aes}} has a detailed
discussion on how AES works, and the construction of the Cryptol model
below.

In the below code, simply set {\tt Nk} to be 4 for AES128, 6 for
AES192, and 8 for AES256 on line 19. No other modifications are
required for obtaining these AES variants. Note that we have
rearranged the code given in \autoref{chapter:aes} below for ease
of reading.
%%  and eliminated the theorem declarations for simplicity.

\fvset{fontsize=\footnotesize}
\VerbatimInput[numbers=left]{aes/AES.cry}
\commentout{
\begin{code}
AESEncapsulated = {
include "AES.cry";
};
\end{code}
}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../main/Cryptol"
%%% End:


% !TEX root = template.tex

% some macros
\newcommand*{\x}{\boldsymbol{x}}
\newcommand*{\y}{\boldsymbol{y}}
\newcommand*{\z}{\boldsymbol{z}}
\newcommand*{\xm}{\bar{\x}}

\section{Processing Pipeline}
\label{sec:processing-pipeline}

\begin{figure*}[h]
	\centering
	\includegraphics[width=1\textwidth]{images/processing_pipeline.jpg}
	\caption{Our proposed signal processing pipeline \& learning strategy}
	\label{fig:processing-pipeline}
\end{figure*}

As reported in \cite{stisen2015smart}, three major types of heterogeneities yield impairment in HAR:
\begin{itemize}
	\item \textbf{Sensor Biases (SB)}: To keep the overall cost of a smartphone low, cheap accelerometer and gyroscope sensors are used, yielding a poor-calibrated, inaccurate and range/granularity limited acquired signals. In these situation we could observe low precision, resolution, range and also biases.
	\item \textbf{Sampling Rate Heterogeneity (SRH)}: Often popular smartphones vary in terms of default and supported sampling frequencies for accelerometer and gyroscope sensors. In the dataset proposed in \cite{stisen2015smart} for example, the sampling frequency varies from 50Hz to 200Hz.
	\item \textbf{Sampling Rate Instability (SRI)}: This phenomenon is specific to a single device and regards the regularity of time span between successive measurements. Different factors could accentuate this problem, including heavy multitasking or high I/O load in the mobile device. In particular, multitasking operations extremely affects the sensor sampling rate because smartphone usually prioritizes among various running tasks. For example, in our collected dataset with a 100 Hz sampling rate (which means a time-span between consecutive samples of $10ms$), we observe a time-span ranging mainly between $3ms$ and $15ms$ with an average of $7ms$, even if the smartphone was in \textit{airplane mode} to reduce at minimum this effect. The Fig. \ref{time-span} shows the amount of different time-span between consecutive measurements contained in our dataset.
\end{itemize}

\begin{figure}[h]
	\centering
	\includegraphics[width=0.4\textwidth]{images/interval_samples.png}
	\caption{Histogram of different time span between consecutive samples included in our collected dataset}
	\label{time-span}
\end{figure}

Furthermore, if we are in a real use case scenario, we must also consider that smartphones can be positioned and oriented in different ways in human body. For example a smartphone could lay in trouser pockets (back and front) or maybe inside a pouch or a bag with different orientations. These different problem settings have huge effects in prediction accuracy of an HAR predictor, especially if the model has been trained on a dataset consisting of activity measurement coming from one fixed position and orientation, as is usually the case in the publicity available datasets.

To tackle all these problems we decided to adopt in our pipeline three main preprocessing blocks, as shown in Fig. \ref{fig:processing-pipeline}.

The first block called Linear Interpolator is in charge of mitigate the problems regarding the SRH and SRI heterogeneity discussed previously. It's main purpose is to down-sample the input data to a fixed sampling rate, in our case 50 samples/second equally spaced in time.

The second block called Orientation Independent Transformation (OIT) is used to represent data coming from different smartphone orientations in a new space whose orientation is independent of that of the smartphone, and aligned with gravity and the direction of motion. In this way an user can place his smartphone in whatever position he or she wants, mitigating the problem of different position and rotations of smartphone, as we will see in Sec \ref{sec:results}.

Our last block consists of a data centering operation which center the signal among y-axis that are used as input only for the Convolutional Layer. The reason of this choice would be clear in the Sec. \ref{sec:results}. As reported in \cite{ignatov2018real}, time series centering standardize the input data, making the task for the CNN easier. Data normalization instead must be avoided because it does not help in this situation since it significantly distorts time series shape, removing magnitude information which is critical for activities differentiation.

After the preprocessing pipeline we adopt a novel learning framework. It is composed of CNN augmented with features coming from the code of an autoencoder. As discussed in \cite{ignatov2018real} CNNs learns filters that are applied to small sub-regions of the data, and therefore they are able to capture local data pattern and their variations. Additionally, due to a small number of connections and high parallelism the amount of computations and running time of CNNs is significantly lower compared to other deep learning algorithm. This means that such a model is perfect for real-time HAR applications even on constrained environment like smartphones. The only drawback of CCNs is that they fall behind in capturing global properties of the signal. In \cite{ignatov2018real}, authors resolve this problem by augmenting CNNs with some basic statistical features. But as opposed to what was done in this latter work where they used few hand-made features, we decided to opt for automatically extracted features. These are created by an autoencoder which should provide more robust representation of information. As implication, we can train the autoencoder separately and then use the trained model in combination with CNN.

\section{Signals and Features}
\label{sec:signals-and-features}

\subsection{Dataset \& Measurement Setup}
\label{subsec:dataset-measurement-setup}

Many datasets are available for this type of applications in
literature: \cite{HETEROGENEITY, UCI, WISDM, EXTRASENSORY}.  However,
there are non-trivial problems in real-word applications: one above
all, the environment in not controlled. So we should be capable to
deal with technical details such as smartphone orientation or sensor
accuracy as reported in Sec. \ref{sec:processing-pipeline}.

For our application we decided to use the \textit{Heterogeneity
  Dataset (HD)} \cite{HETEROGENEITY} described in
\cite{stisen2015smart}, where data are collected directly from
smartphone sensors. The dataset is composed by nine users (from 25 to
30 years). All users followed a scripted set of activities in
\textit{sit}, \textit{stand}, \textit{walk}, \textit{bike},
\textit{stairs up}, \textit{stairs down} while carrying eight
smartphones and four smart watches. All eight smartphones were kept in
a tight pouch and carried by the users around their waist, whereas two
smart watches were worn on each arm. Each participant conducted five
minutes of each activity for each device. In our case we were
interested only in smartphones acquired signals, leaving out from our
experiments the smartwatch evaluations.

We also collected a novel dataset called \textit{Oriented Dataset
  (OD)}. The goal of this new dataset is to see whether the model can
generalize well on new unseen users and also to test how OIT
preprocessing could increase the performance in this new context. This
dataset is collected with a single smartphone, at a sampling rate of
100 Hz in seven different positions: top, bottom, left, right are
collected in the same way as in work \cite{stisen2015smart}, but
instead of a fixed orientation we rotated the smartphone inside the
pouch to each possible orientation. Then we also perform the same
activities with the smartphone hold in hand and in trousers pocket, to
see to what extent the model is capable of generalize activity
recognition in new unseen positions. In this case we condensed
\textit{sit} and \textit{stand} activities into \textit{no activity}
for two main reasons: first, we are not interested in distinguishing
between the two, and second, they are made indistinguishable by OIT.

\subsection{Signal preprocessing}

\textbf{Notation}: With $\boldsymbol{x} \in {\rm I\!R}^{n}$ we define a column vector as $\boldsymbol{x}=(x_{1}, x_{2}, ..., x_{n})^{T}$ where with superscript $^T$ we mean the transpose operator. With $\boldsymbol{||x||}$ we mean the L2-norm operator $||\x|| = (\sum_{i=1}^{n} x_{i}^{2})^{1/2}$ and with $\bar{x} = (\sum_{i=1}^{n} x_{i}) / n$ the mean of a vector. With $\boldsymbol{x} \cdot \boldsymbol{y} =\boldsymbol{x}^{T}\boldsymbol{y} $ we mean the inner product of two vectors. With $\boldsymbol{\vec{x}}$ we mean a 3D vector $\boldsymbol{\vec{x}} = (x_{1}, x_{2}, x_{3})^{T}$ and with $\boldsymbol{\hat{x}}$ the corresponding 3D versor $\boldsymbol{\hat{x}}=\boldsymbol{\vec{x}}/ ||\boldsymbol{\vec{x}}||$. For any two 3D vector $\boldsymbol{\vec{x}}$ and $\boldsymbol{\vec{y}}$ we indicate their cross-product as $\boldsymbol{\vec{x}} \times \boldsymbol{\vec{y}}$. We represent with $\boldsymbol{a_{x}}, \boldsymbol{a_{y}}, \boldsymbol{a_{z}}$ the x, y and z components of the accelerometer signal, as well as the gyroscope signal is represented with $\boldsymbol{g_{x}}, \boldsymbol{g_{y}}, \boldsymbol{g_{z}}$. We define also matrices with uppercase and bold letters. For example to define a $3 \times n$ matrix composed of 3 vectors \mbox{$\boldsymbol{x}, \boldsymbol{y}, \boldsymbol{z} \in {\rm I\!R}^{n}$} we use the notation \mbox{$\boldsymbol{M} = [\boldsymbol{x}, \boldsymbol{y}, \boldsymbol{z}]^{T}$}\\

\textbf{Linear Interpolation \& Down-sampling.} As described in
Sec. \ref{subsec:dataset-measurement-setup}, our dataset is composed
by signals collected with different sampling rates from many mobile
phones and they are usually out of sync due to the SRI. For this
reason we apply to all signals a simple preprocessing pipeline block
composed by two steps: linear interpolation and down-sampling. With
linear interpolation we project signals with different sampling
frequencies, i.e 50, 100, 200 Hz, to a 50 Hz signal which mean that
we are also applying at the same time down-sampling. Linear/nearest
interpolations are usually preferred w.r.t. cubic spline or smooth
cubic spline as investigated in \cite{stisen2015smart}, because they
usually mitigate the introduction of noise or artifacts. In linear
interpolation the value at time $t$ is the piecewise-linear
interpolation, i.e. the linear interpolation between the input samples
adjacent to $t$ in the sequence of input sample timestamps. Let $(x_0,
y_0)$ and $(x_1, y_1)$ be two known points, the linear interpolant is
the straight line between this points. For a value $x$ in the interval
$(x_0, x_1)$ the value $y$ along the straight line is given from the
equation of slopes
\begin{equation}
  \label{eq:linear-interpolation}
  \frac{y - y_0}{x - x_0} = \frac{y_1 - y_0}{x_1 - x_0}.
\end{equation}
The linear interpolation on a set of data points $(x_0, y_0), (x_1, y_1), \ldots, (x_n, y_n)$ is defined as the concatenation of linear interpolants between each pair of data points.

% In our case the downsampling is applied in combination with linear interpolation, but it is not always the case (see Sec. EXPERIMENTS). It allows to standardize data at fixed sampling frequency in order to extract time windows and feature vectors for learning models and, above all, it reduces noise and artifacts introduced by upsampling. Downsampling simply disregards points by resampling the signal at a certain frequency (50 Hz in our case). Please note that the downsample frequency should be a sub-multiple of linear interpolation frequency.

\vspace{1em}
\textbf{Orientation Independent Transformation.}
To project the signals acquired during an activity in a new space independent of the rotation of the smartphone, three orthogonal versors need to be found. We decided to adopt the technique proposed in \cite{gadaleta2018idnet}, although many other works proposed a similar solution as in \cite{kunze2009way, henpraserttae2011accurate}. In summary we have to find these 3 orthogonal versor namely vertical versor $\boldsymbol{\hat{v}}$, horizontal versor $\boldsymbol{\hat{h}}$ and lateral versor $\boldsymbol{\hat{l}}$. As the names suggest the vertical versor is aligned with user torso, pointing up, the horizontal versor is aligned with the direction of motion, pointing forward, and the last versor tracks lateral movements and it is orthogonal to the other two.

Starting with the vertical versor we need to find where the gravity vector $\boldsymbol{\vec{p}}$ lie in the original space. Although the gravity vector is a constant vector in stationary conditions, during a user activity it continuously changes in the original coordinate system of the smartphone, therefore we are only able to consider the mean direction of the gravity within the current user activity. So, to estimate it we must considered only the accelerometer signal: \mbox{$ \boldsymbol{\vec{p}} = (\bar{a_{x}}, \bar{a_{y}}, \bar{a_{z}})^{T}$}  and we now could find the vertical versor with \mbox{$ \boldsymbol{\hat{v}} = \boldsymbol{\vec{p}} / ||\boldsymbol{\vec{p}}|| $}.

This is the first axis of our new space, and we can project datas onto this new versor to obtain our first component axis. To do that we define the acceleration matrix \mbox{$\boldsymbol{A} = [ \boldsymbol{a_{x}}, \boldsymbol{a_{y}}, \boldsymbol{a_{z}} ]^{T}$} and the gyroscope matrix \mbox{$\boldsymbol{G} = [ \boldsymbol{g_{x}}, \boldsymbol{g_{y}}, \boldsymbol{g_{z}} ]^{T} $}. Now we could project the data onto $\boldsymbol{\hat{v}}$ by:

\begin{equation}
	\label{v-axis eq}
	 \boldsymbol{a_{v}} = \boldsymbol{A} \cdot \boldsymbol{\hat{v}} ,\;\; \boldsymbol{g_{v}} = \boldsymbol{G} \cdot \boldsymbol{\hat{v}} \,.
\end{equation}

Now we have to find an horizontal plane, parallel to the floor, where the activity motion mostly occurs. To do so we have to remove the  $\boldsymbol{a_{v}}$ component from the original data. We represent the accelerometer data that lie on this new plane as $\boldsymbol{M}$ representing the so called motion plane. To find $\boldsymbol{M}$ we have to: \mbox{$\boldsymbol{M} = \boldsymbol{A} - \boldsymbol{\hat{v}} \boldsymbol{a_{v}}^{T} $}. In this new plane, we could see that the direction with the largest variance of projected data, represents the main direction of motion, i.e in which direction the user is currently performing the activity w.r.t the current smartphone orientation. By applying PCA \cite{rao1964use}, we are able to find the direction along which the variance of measurements is maximized. This is the horizontal vector $\boldsymbol{\vec{h}}$. We now could compute the horizontal versor \mbox{$ \boldsymbol{\hat{h}} = \boldsymbol{\vec{h}} / ||\boldsymbol{\vec{h}}|| $} and we are now able to project our data onto this second new axis:

\begin{equation}
	\label{h-axis eq}
	\boldsymbol{a_{h}} = \boldsymbol{A} \cdot \boldsymbol{\hat{h}} , \;\; \boldsymbol{g_{h}} = \boldsymbol{G} \cdot \boldsymbol{\hat{h}} \,.
\end{equation}

To find the last axis its sufficient to apply a cross product between the two last obtained versors, so $ \boldsymbol{\hat{l}} = \boldsymbol{\hat{v}} \times \boldsymbol{\hat{h}} $. We are now been able to project the data among this new axis:

\begin{equation}
	\label{l-axis eq}
	\boldsymbol{a_{l}} = \boldsymbol{A} \cdot \boldsymbol{\hat{l}} , \;\; \boldsymbol{g_{l}} = \boldsymbol{G} \cdot \boldsymbol{\hat{l}} \,.
\end{equation}

Combining Eq. (\ref{v-axis eq}), Eq. (\ref{h-axis eq}) and Eq. (\ref{l-axis eq}) lead us to the final accelerometer and gyroscope transformed signals represented in this new orientation independent space, that are $[\boldsymbol{a_{v}}, \boldsymbol{a_{h}}, \boldsymbol{a_{l}}]^T $ and $ [\boldsymbol{g_{v}}, \boldsymbol{g_{h}}, \boldsymbol{g_{l}}]^T $

\vspace{1em}
\textbf{Centering.}
Our last signal preprocessing block involves data centering. As proved in \cite{ignatov2018real} this could slightly
improve performance within a CNN based learning model because
centering the time series make the task easier for the CNN. We denote with $\x^{c}$ the centered vector of $\x$, i.e
\begin{equation}
  \label{eq:centering-func}
  \x^{c} = \x - \bar{x} = (x_1-\bar{x}, x_2-\bar{x},..., x_n-\bar{x})^T.
\end{equation}
Centering is applied for each time-window only on accelerometer data, thus the new centered acceleration matrix is:
\begin{equation}
  \label{eq:centering-accelerometer-data}
  \boldsymbol{A}^{c} = [\boldsymbol{a_{x}}^{c}, \boldsymbol{a_{y}}^{c}, \boldsymbol{a_{z}}^{c}]^T \,.
\end{equation}

\subsection{Feature vector}
\label{subsec:feature-vector}

\textbf{Time windows.} When dealing with time-based signals (or
time-series in general) we must keep in mind that correlation between
samples occurs and we have to handle this useful information. However,
in the domain of HAR, even if samples can be correlated in time, the
correlation do not persist over a long period. In literature many time
window intervals were experimented \cite{ignatov2018real} and they
discovered that the best time window interval is from $1$s to
$2.5$s. For this reason we adopt a sliding window approach with fixed
window length of $2.5$s with $50$\% overlap between two successive
windows.

\textbf{Features}. Features are the basic elements that every machine
learning algorithm need in order to learn something. Everything fed
into a machine learning algorithm can be considered a feature, but
here we use three different types of feature targeting different types
of information.

\begin{itemize}
\item \textit{Raw features}. These enables the model to learn directly
  from data and hopefully from it's shape in order to generalize and
  classify activities. Raw features are represented by a $6 \text{x}
  125$ matrix where rows are accelerometer and gyroscope $x$, $y$ and
  $z$ dimensions while columns are samples over time.
\item \textit{Manual features}. Manual features take into account the
  statistical mode of the signal and are used in
  \cite{ignatov2018real} and \cite{anguita2013public} with great
  results. We exploit manual features like mean, standard deviation,
  sum of the absolute values and the histogram of each input data
  channel computed on local time-windows applied only for
  accelerometer signal $\boldsymbol{a}$. As defined in
  \cite{ignatov2018real}, we produce a basic feature vector
  $\boldsymbol{f_{b}} = (\bar{a_x}, \bar{a_y}, \bar{a_z},
  \sigma_{\boldsymbol{a_{x}}}, \sigma_{\boldsymbol{a_{y}}},
  \sigma_{\boldsymbol{a_{z}}}, \tilde{a_x}, \tilde{a_y}, \tilde{a_z},
  \psi_{\boldsymbol{a_{x}}\boldsymbol{a_{y}}\boldsymbol{a_{z}}}, $ $
  \boldsymbol{h}_{\boldsymbol{a_{x}}}^T,
  \boldsymbol{h}_{\boldsymbol{a_{y}}}^T,
  \boldsymbol{h}_{\boldsymbol{a_{z}}}^T)^T$ where for a generic column
  vector $\x = (x_1, x_2,..., x_n)^T$ with \mbox{$\x^c = (x_1^c,
    x_2^c,..., x_n^c)^T$:}
  \begin{align}
    \sigma_{\x} &= \sqrt{ \frac{\sum_{i=1}^{n}(|x^c_i|)^2}{n} }\,,
    \\ \tilde{x} &= \frac{\sum_{i=1}^{n}(|x^c_i|)}{n}\,,
    \\ \psi_{\x\y\z} &= \frac{\sum_{i=1}^{n} \sqrt{x_i^2 + y_i^2 +
        z_i^2}} {n}\,,
  \end{align}
  and $\boldsymbol{h}_\x$ is column vector that sort the values of
  $\x$ into $10$ equally spaced bins along the x-axis between the
  minimum and maximum values of $\x$.\footnote{For reference, see
    \texttt{np.histogram} documentation.}  This lead to a manual
  features vector composed of $40$ manual extracted features.
\item \textit{Autoencoder features}. Autoencoder features are
  automatically extracted from the signal. The autoencoder, as
  described in Sec. \ref{subsec:autoencoder}, is capable to compress
  data and learn a good representation of features keeping only
  interesting data. The code size we use is made up of 36 features.
\end{itemize}

\section{Learning Framework}
\label{sec:learning-framework}

\begin{figure*}[h]
	\centering
	\includegraphics[width=1\textwidth]{images/full_architecture.jpg}
	\caption{Our proposed learning framework, CL1 is a convolutional-1D layer, ML1 is a max-pooling layer, FL1 are the fully connected layer which represents previous extracted features that are concatenated with the autoencoder features, FL2 and FL3 are fully connected layers}
	\label{fig:proposed-architecture}
\end{figure*}

\subsection{Autoencoder}
\label{subsec:autoencoder}

In this section we describe the autoencoder model based on
\cite{vincent2010stacked}, \cite{gu2018locomotion} and
\cite{gao2019human}. An autoencoder is a type of artificial neural
network used to learn efficient data codings in an unsupervised
manner. The aim of an autoencoder is to learn a representation
(encoding) for a set of data by training the network to ignore signal
noise. Given $\boldsymbol{X} = [ \boldsymbol{a}_v, \boldsymbol{a}_h,
  \boldsymbol{a}_l, \boldsymbol{g}_v, \boldsymbol{g}_h,
  \boldsymbol{g}_l ]^T$ as a $6\text{x}125$ matrix and $\x$ as the
column vector $750\text{x}1$ obtained by flattening $\boldsymbol{X}$ we
can define the autoencoder in two blocks. The first block is the
encoder which is a function of the input:
\begin{align}
  f_{\theta}(\x) &= \sigma(\boldsymbol{W_{2}} f'_\theta(\x) + \boldsymbol{b_{2}}) \\
  f'_{\theta}(\x) &= \textit{ReLU}(\boldsymbol{W_{1}}\x + \boldsymbol{b_{1}})
\end{align}
where $\theta_{i}$ is a vector of $\boldsymbol{W_{i}}$ and $\boldsymbol{b_{i}}$
which are, respectively, the weight matrix and the bias
vector for layer $i$. We
can derive $\y = f_\theta(\x)$ as the output of this
block. The second block is the decoder where the input is
reconstructed back starting from the code obtained by the encoder
\begin{align}
  g_{\theta}(\y) &= \boldsymbol{W_{4}} g'_\theta(\y) + \boldsymbol{b_{4}} \\
  g'_{\theta}(\y) &= \textit{ReLU}(\boldsymbol{W_{3}}\y + \boldsymbol{b_{3}})
\end{align}
Thus, the reconstructed input
is $\z = g_{\theta}(\y)$. What we want to minimize is the distance
between $\x$ and $\z$ w.r.t. a distance measure with a loss function
like MSE.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.48\textwidth]{images/encoder.png}
	\caption{(Auto)-encoder structure}
	\label{fig:encoder-structure}
\end{figure}

The main goal of the autoencoder for this application is to extract
useful representations of input signal in few features and this is
done by looking at the autoencoder's code, i.e., the output of the
encoder block:
\begin{equation}
  \boldsymbol{\Phi_{\text{e}}} = \y.
\end{equation}

The autoencoder follows the structure reported in
Fig. \ref{fig:encoder-structure} and is based on a Neural Network
architecture. The encoder is built with an input layer of
$6\text{x}125$ neurons flattened into a vector $750\text{x}1$, one hidden
layer with 150 neurons and \textit{ReLU} activation function, and an output
layer, i.e, the code, of $36$ neurons with sigmoid activation
function. The decoder replicates the same structure of the encoder but
reversed, where the activation function on the output layers is the
linear activation, i.e., the identity.

\subsection{Convolutional Neural Network}
\label{subsec:cnn}
The final CNN architecture proposed in this work is shown in Fig. \ref{fig:proposed-architecture}. We decided to start with the architecture proposed in work \cite{ignatov2018real} and we tried to improve it, by augmenting the CNN with the encoder extracted features. CNN architecture are very powerful when dealing with images, but they were proven to be a good features extractors also for motion data. A CNN is composed of essentially two parts: the first-one is in charge of extract features performing a dimensionality reduction over the input data, though a series of convolution and max pooling layers, and the second part of the CNN is responsible to give the final classification with usually a single fully-connected layer. Original data, collected from smartphone sensors, are pre-processed accordingly to what previously described in Sec. \ref{sec:processing-pipeline}. Our input matrix presented to our CNN is the matrix \mbox{$ \boldsymbol{X} = [ \boldsymbol{a_{v}}^{c}, \boldsymbol{a_{h}}^{c}, \boldsymbol{a_{l}}^{c}, \boldsymbol{g_{v}}^{c}, \boldsymbol{g_{h}}^{c}, \boldsymbol{g_{l}}^{c}]$} which is formed by accelerometer and gyroscopes signals, linearly interpolated at 50 samples/second per channel, with a fixed time-window of $2.5s$, with OIT and data centering. Note that this matrix has a dimension $n \times 6$, due to the fact that TensorFlow Convolutional 1D Layers work with input dimension of $(batch\_size, samples, channel)$.

In detail we have the following stacked layers (CL=Convolutional Layer, ML=Max-pooling Layer, FL= Fully-connected Layer):
\begin{itemize}
	\item \textbf{CL1}: The first convolutional layer is performing a Convolution 1D over the input signal. It is composed of 196 filters of size ($1\times16$). Since, a convolution is defined over all the input channels, in this layer we are also capturing relation among accelerometer and gyroscope signals. The stride is set to 1, and the padding is configured as 'same' to have the same output dimension as the original one. After the convolution operation, we apply a $ReLU(x)=\max(0,x)$ activation function to learn non-linear correlation between signals and extract richer features. Furthermore, even if we are dealing with a small architecture, $ReLU$ activation function could prevent vanishing gradient problems, is less prone to overfitting since it induces the sparsity in the hidden units and it is extremely fast to compute, making it perfect when dealing with low computational resources \cite{ignatov2018real}.
	\item \textbf{ML1}: After the convolutional layer, usually it is apply a max-pooling or average-pooling layer to reduce and summarize the obtained representation. We decided to use max-pooling layer of size ($1\times4$), reducing by 4 times the original input shape. We call this new features representation with $ \boldsymbol{\Phi_{\text{c}} }$
	\item \textbf{FL1}: After the convolutional layer and the max-pooling layer, the output of these layer is flattened into 6272 neurons. At this stage we decided to concatenate the encoder extracted features $ \boldsymbol{\Phi_{\text{e}}} $ with the Convolutional ones named $\boldsymbol{\Phi_{\text{c}}}$, creating our complete features vector $\boldsymbol{\Phi} = (\boldsymbol{\Phi_{\text{c}}}, \boldsymbol{\Phi_{\text{e}}})^T$ that is presented to the final fully-connected layers to perform the classification.
	\item \textbf{FL2-3}: These two final layers are composed of 64 dense neuron and 6 dense neurons, respectively. These two last layers are used to perform the classification of the activity. For the FC2 we use the $ReLU$ activation function and for the final classification we apply the \textit{soft-max} activation function, which computes probability distribution over the predicted classes.
\end{itemize}

As for optimization techniques we decided to use dropout and $l_2$-regularization. These latter techniques are commonly used in these types of architectures, yielding better performances in the test set and commonly in generalization capabilities, preventing overfitting. We apply a dropout rate of 0.05 and an $l_2$-regularization of $5e^{-5}$ to the FL2 layer. We have try with different hyper-parameters values for these two techniques, without any substantial changes in classification performance on the test set. Finally the parameters of the network are optimized using Adam: a modification of stochastic gradient descent that also incorporates Momentum. Finally, the network is trained to minimize the Categorical cross-entropy loss function.


\section{$\cHH^\ast$ as a cohomology theory}
Office hours: today, Hood in 4-390 from 1:30 to 3:30 and Miller in 4-478 from 1-3 on Tuesday. Note that pset 6 is due Wednesday. Also, Wednesday we'll have a lightning review of $\pi_1$ and covering spaces.

We're coming to the end of the course, and there are going to be oral exams. I have some questions that I'd like to ask you. They won't be super advanced, detailed questions -- they'll be basic things. I'll post a list of examples of questions. I won't select questions from that list, that's cruel and isn't the point. The oral will be 40 minutes. It'll be fun -- better than a written exam. It's much better than grading a written exam!

PLEASE DRAW A PICTURE WHEN READING THIS IF YOU DIDN'T COME TO CLASS!
\subsection{$\cHH^\ast$ versus $H^\ast$}
Let's come up with an example that distinguishes $\cHH^\ast$ and $H^\ast$. This is a famous example -- the topologist's sine curve. The topologist's sine curve is defined as follows. Consider the graph of $\sin(\pi/x)$ where $0<x\leq 1$, where we draw a continuous curve from $(0,-1)$ to $(1,0)$. This is a counterexample for a lot of things, you've probably seen it in 18.901.

What is $H_\ast$ of the topologist sine curve? Use Mayer-Vietoris! I can choose the bottom half to be some connected portion of the continuous curve from $(0,-1)$ to $(1,0)$, and the top half to be the rest of the space that intersects the bottom half (in two spots). The top is the union of two path components, each contractible.

To see this, suppose I have $\sigma:I\to\mathrm{top}$ such that $\sigma(0)=(0,b)$ for some $-3/2 < b\leq 1$ (the $-3/2$ is arbitrary, just choose something $\leq 0$). If $b<1$, pick $\epsilon>0$ such that if we write $\sigma=(\sigma_1,\sigma_2)$, then $\sigma_2(t)<1$ for all $t\in[0,\epsilon)$, where we use continuity. Then $\sigma|_{[0,\epsilon)}$ can't be on the sine curve. If $b=1$, pick $\epsilon>0$ such that $\sigma_2(t)>-1$ for all $t\in[0,\epsilon)$. Similarly, it can't be on the sine curve.

Thus, $H_1$ is simple: it fits into $0\to H_1(X)\to H_0(\mathrm{top}\cap\mathrm{bottom})\to H_0(\mathrm{top})\oplus H_0(\mathrm{top})\to H_0(X)\to 0$. This is basically $0\to H_1(X)\xrightarrow{\partial}\to\Z\oplus\Z\hookrightarrow\Z\oplus\Z\oplus\Z\to\Z\to 0$, so $\partial=0$. This means that $H_\ast(X)\cong H_\ast(\ast)$ and this implies that $H^\ast(X)\cong H^\ast(\ast)$.

How about $\cHH^\ast$? Let $X\subset U$ be an open neighborhood. The interval is contained in some $\epsilon$-neighborhood that's contained in $U$. This implies that there exists a neighborhood $X\subseteq V\subseteq U$ such that $V\sim S^1$. Therefore, $\varinjlim_{U\in\mathcal{U}_X}H^\ast(U)\cong H^\ast(S^1)$ by ``cofinality''. So $\cHH^\ast$ and $H^\ast$ differ.
\subsection{Cofinality}
Let $\cI$ be a directed set. Let $A:\cI\to \mathbf{Ab}$ be a functor. If I have a functor $f:\cK\to\cI$, then I get $Af:\cK\to\mathbf{Ab}$, i.e., $(Af)_j=A_{f(j)}$.

I can form $\varinjlim_{\cK}Af$ and $\varinjlim_{I}A$. I claim you have a map $\varinjlim_{\cK}Af\to\varinjlim_{\cI}A$. All I have to do is the following:
\begin{equation*}
\xymatrix{
	\varinjlim_{J}Af\ar[r] & \varinjlim_{I}A\\
	A_{f(j)}\ar[u]^{\mathrm{in}_j}
}
\end{equation*}
So I have to give you maps $A_{f(j)}\to\varinjlim_{I}A$ for various $j$. I know what to do, because I have $\mathrm{in}_{f(j)}:A_{f(j)}\to\varinjlim_{I}A$. Are they compatible when I change $j$? Suppose I have $j^\prime\leq j$. Then I get a map $f(j^\prime)\to f(j)$, so I have a map $A_{f(j^\prime)}\to A_{f(j)}$, and thus the maps are compatible. Hence I get:
\begin{equation*}
\xymatrix{
	\varinjlim_{J}Af\ar[r] & \varinjlim_{I}A\\
	(Af)_j=A_{f(j)}\ar@{-->}[ur]^{\mathrm{in}_{f(j)}}\ar[u]^{\mathrm{in}_j}
}
\end{equation*}
\begin{example}
Suppose $K\supseteq L$ be closed, then I get a map $\cHH^\ast(K)\to\cHH^\ast(L)$. Is this a homomorphism? Well, $\cHH^\ast(K)=\varinjlim_{U\in\mathcal{U}_K}H^\ast(U)$ and $\cHH^\ast(L)=\varinjlim_{V\in\mathcal{U}_L}H^\ast(V)$. This is an example of a $\cI$ and $\cK$ that I care about. Well, $\mathcal{U}_K\subseteq\mathcal{U}_L$, and thus I get a map $\cHH^\ast(K)\to\cHH^\ast(L)$, which is what I wanted.

I can do something for relative cohomology. Suppose:
\begin{equation*}
\xymatrix{K\ar@{^(->}[d] & L\ar@{^(->}[d]\ar@{_(->}[l] \\ K^\prime & L^\prime\ar@{_(->}[l]}
\end{equation*}
I get a homomorphism $\cHH^\ast(K,L)\to \cHH^\ast(K^\prime,L^\prime)$ because I have $\mathcal{U}_{K,L}\to\mathcal{U}_{K^\prime,L^\prime}$.
\end{example}
This isn't exactly what we need:
\begin{question}
When does $f:\cK\to\cI$ induce an isomorphism $\varinjlim_{J}Af\to\varinjlim_{I}A$?
\end{question}
This is a lot like taking a sequence and a subsequence and asking when they have the same limit. There's a cofinality condition in analysis, that has a similar expression here.
\begin{definition}
$f:\cK\to\cI$ is cofinal if for all $i\in\cI$, there exists $j\in\cK$ such that $i\leq f(j)$.
\end{definition}
\begin{example}
If $f$ is surjective.
\end{example}
\begin{lemma}
If $f$ is cofinal, then $\varinjlim_{J}Af\to\varinjlim_{I}A$ is an isomorphism.
\end{lemma}
\begin{proof}
Check that $\{A_{f(j)}\to\varinjlim_{I}A\}$ satisfies the necessary and sufficient conditions:
\begin{enumerate}
\item For all $a\in\varinjlim_{I}A$, there exists $j$ and $a_j\in A_{f(j)}$ such that $a_j\mapsto a$. We know that there exists some $i$ and $a_i\in A$ such that $a_i\mapsto a$. Pick $j$ such that $f(j)\geq i$, so we get a map $a_i\to a_{f(j)}$, and by compatibility, we get $a_{f(j)}\mapsto a$.
\item The other condition is also just as easy.
\end{enumerate}
\end{proof}
This is a very convenient condition.
\begin{example}
I had a perverse way of constructing $\QQ$ by using the divisibility directed system. A much simpler (linear!) directed system is $\Z\xrightarrow{2}\Z\xrightarrow{3}\Z\xrightarrow{4}\Z\to\cdots$. This has the same colimit as the divisibility directed system because $n|n!$, so we have a cofinal map between directed systems.
\end{example}
How about the direct limits in the \v{C}ech cohomology case?
\begin{example}
Do I have a map $\cHH^\ast(K,L)\to\cHH^\ast(K)$? Suppose:
\begin{equation*}
\xymatrix{K\ar@{^(->}[d] & L\ar@{^(->}[d]\ar@{_(->}[l] \\ U & V\ar@{_(->}[l]}
\end{equation*}
Then $\cHH^p(K,L)=\varinjlim_{(U,V)\in\mathcal{U}_{K,L}}H^p(U,V)$ and $\cHH^p(K)=\varinjlim_{U\in\mathcal{U}_K}H^p(U)$. I have a map of directed sets $\mathcal{U}_{K,L}\to\mathcal{U}_K$ by sending $(U,V)\mapsto U$. I didn't have to use cofinality. I want a long exact sequence, though, and I'm going to do this by saying that it's a directed limit of a long exact sequence. I'm going to have to have all of these various \v{C}ech cohomologies as being the directed limit over the \emph{same} indexing set.

I'd really like to say that $\cHH^p(K)=\varinjlim_{U\in\mathcal{U}_K}H^p(U)\cong \varinjlim_{(U,V)\in\mathcal{U}_{K,L}}H^p(U)$. Thus I need to show that $\mathcal{U}_{K,L}\to\mathcal{U}_K$ where $(U,V)\mapsto U$ is cofinal. This is easy, because if $U\in\mathcal{U}_K$, just pick $(U,U)$, i.e., $\mathcal{U}_{K,L}\to\mathcal{U}_K$ is cofinal. How about $\mathcal{U}_{K,L}\to\mathcal{U}_L$ by $(U,V)\mapsto V$; is it cofinal? Yes! For $V\in\mathcal{U}_L$, pick $(X,V)$! This means that $\cdots\cHH^{p-1}(L)\to\cHH^p(K,L)\to\cHH^p(K)\to\cHH^p(L)\to\cHH^{p+1}(K,L)$ is $\varinjlim_{\mathcal{U}_{K,L}}\left(\cdots\to H^{p}(U,V)\to\cdots\right)$, and hence exact.
\end{example}
How about excision? I need this to get to Mayer-Vietoris!
\begin{lemma}
Assume $X$ is normal and $A,B$ are closed subsets. Then $\cHH^p(A\cup B,B)\to\cHH^p(A,A\cap B)$ is an isomorphism.
\end{lemma}
\begin{proof}
Well, $\cHH^p(A\cup B,B)$ is $\varinjlim$ over $\mathcal{U}_{A\cup B,B}$ and $\cHH^p(A,A\cap B)$ is $\varinjlim$ over $\mathcal{U}_{A,A\cap B}$. Let $W\supseteq A$ and $Y\supseteq B$ are neighborhoods. I claim that $\mathcal{U}_A\times\mathcal{U}_B\to\mathcal{U}_{A\cup B,B}$ sending $(W,Y)\mapsto (W\cup Y,Y)$ and $\mathcal{U}_A\times\mathcal{U}_B\to\mathcal{U}_{A,A\cap B}$ sending $(W,Y)\mapsto (W,W\cap Y)$ are cofinal.

If I give you $(U,V)\in \mathcal{U}_{A\cup B,B}$, define $(W,V)\in\mathcal{U}_A\times\mathcal{U}_B$ where $W=U$ and $Y=V$, so $\mathcal{U}_A\times\mathcal{U}_B\to\mathcal{U}_{A\cup B,B}$ is surjective, hence cofinal. The latter is trickier. Let $U\supseteq A$ and $V\supseteq A\cap B$. Here's where normality comes into play. Separate $B-V$ from $A$. Let $T\supseteq B-V$. Shit. \emph{Shit!}

Maybe I'll leave this to you. I'll put this on the board on Wednesday. Anyway, I'll use normality to show that $\mathcal{U}_A\times\mathcal{U}_B\to\mathcal{U}_{A,A\cap B}$ is cofinal, and thus this verifies excision -- so you actually have excision.
\end{proof}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Copyright (c) 2003-2018 by The University of Queensland
% http://www.uq.edu.au
%
% Primary Business: Queensland, Australia
% Licensed under the Apache License, version 2.0
% http://www.apache.org/licenses/LICENSE-2.0
%
% Development until 2012 by Earth Systems Science Computational Center (ESSCC)
% Development 2012-2013 by School of Earth Sciences
% Development from 2014 by Centre for Geoscience Computing (GeoComp)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Stokes Flow}
\label{STOKES FLOW CHAP}
In this section we look at Computational Fluid Dynamics (CFD) to simulate
the flow of fluid under the influence of gravity.
The \class{StokesProblemCartesian} class will be used to calculate the velocity
and pressure of the fluid.
The fluid dynamics is governed by the Stokes equation. In geophysical problems
the velocity of fluids is low; that is, the inertial forces are small compared
with the viscous forces, therefore the inertial terms in the Navier-Stokes
equations can be ignored.
For a body force $f$, the governing equations are given by:
%
\begin{equation}
\nabla \cdot (\eta(\nabla \vec{v} + \nabla^{T} \vec{v})) - \nabla p = -f,
\label{GENERAL NAVIER STOKES}
\end{equation}
%
with the incompressibility condition
%
\begin{equation}
\nabla \cdot \vec{v} = 0.
\label{INCOMPRESSIBILITY}
\end{equation}
%
where $p$, $\eta$ and $f$ are the pressure, viscosity and body forces, respectively.
Alternatively, the Stokes equations can be represented in Einstein summation
tensor notation (compact notation):
%
\begin{equation}
-(\eta(v_{i,j} + v_{j,i})),_{j} + p,_{i} = f_{i},
\label{GENERAL NAVIER STOKES COM}
\end{equation}
%
with the incompressibility condition
%
\begin{equation}
-v_{i,i} = 0.
\label{INCOMPRESSIBILITY COM}
\end{equation}
%
The subscript comma $i$ denotes the derivative of the function with respect to $x_{i}$.
%A linear relationship between the deviatoric stress $\sigma^{'}_{ij}$ and the stretching $D_{ij} = \frac{1}{2}(v_{i,j} + v_{j,i})$ is defined as \cite{GROSS2006}:
%
%\begin{equation}
%\sigma^{'}_{ij} = 2\eta D^{'}_{ij},
%\label{STRESS}
%\end{equation}
%
%where the deviatoric stretching $D^{'}_{ij}$ is defined as
%
%\begin{equation}
%D^{'}_{ij} = D^{'}_{ij} - \frac{1}{3}D_{kk}\delta_{ij}.
%\label{DEVIATORIC STRETCHING}
%\end{equation}
%
%where $\delta_{ij}$ is the Kronecker $\delta$-symbol, which is a matrix wi-th ones for its diagonal entries ($i = j$) and zeros for the remaining entries ($i \neq j$).
The body force $f$ in \eqn{GENERAL NAVIER STOKES COM} is the gravity acting in
the $x_{3}$ direction and is given as $f=-g\rho\delta_{i3}$.
The Stokes equation is a saddle point problem, and can be solved using a Uzawa scheme.
A class called \class{StokesProblemCartesian} in \escript can be used to solve
for velocity and pressure. A more detailed discussion of the class can be
found in Chapter \ref{MODELS CHAPTER}.
In order to keep numerical stability and satisfy the Courant-Friedrichs-Lewy condition (CFL condition)\index{Courant number}\index{CFL condition}, the
time-step size needs to be kept below a certain value.
The Courant number \index{Courant number} is defined as:
%
\begin{equation}
C = \frac{v \delta t}{h}
\label{COURANT}
\end{equation}
%
where $\delta t$, $v$, and $h$ are the time-step, velocity, and the width of
an element in the mesh, respectively. The velocity $v$ may be chosen as the
maximum velocity in the domain. In this problem the time-step size was
calculated for a Courant number of $0.4$.

The following \PYTHON script is the setup for the Stokes flow simulation, and
is available in the \ExampleDirectory as \file{fluid.py}.
It starts off by importing the classes, such as the \class{StokesProblemCartesian}
class, for solving the Stokes equation and the incompressibility condition for
velocity and pressure.
Physical constants are defined for the viscosity and density of the fluid,
along with the acceleration due to gravity.
Solver settings are set for the maximum iterations and tolerance; the default
solver used is PCG (Preconditioned Conjugate Gradients).
The mesh is defined as a rectangle to represent the body of fluid.
We are using $20 \times 20$ elements with piecewise linear elements for the
pressure and for velocity but the elements are subdivided for the velocity.
This approach is called \textit{macro elements}\index{macro elements} and
needs to be applied to make sure that the discretized problem has a unique
solution, see~\cite{LBB} for details\footnote{Alternatively, one can use
second order elements for the velocity and first order elements for pressure
on the same element. You can set \code{order=2} in \class{esys.finley.Rectangle}.}.
The fact that pressure and velocity are represented in different ways is
expressed by
\begin{python}
  velocity=Vector(0., Solution(mesh))
  pressure=Scalar(0., ReducedSolution(mesh))
\end{python}
The gravitational force is calculated based on the fluid density and the
acceleration due to gravity.
The boundary conditions are set for a slip condition at the base and the left
face of the domain. At the base fluid movement in the $x_{0}$-direction
is free, but fixed in the $x_{1}$-direction, and similarly at the left
face fluid movement in the $x_{1}$-direction is free  but fixed in
the $x_{0}$-direction.
An instance of the \class{StokesProblemCartesian} class is defined for the
given computational mesh, and the solver tolerance set.
Inside the \code{while} loop, the boundary conditions, viscosity and body
force are initialized.
The Stokes equation is then solved for velocity and pressure.
The time-step size is calculated based on the Courant-Friedrichs-Lewy condition
(CFL condition)\index{Courant number}\index{CFL condition}, to ensure stable solutions.
The nodes in the mesh are then displaced based on the current velocity and
time-step size, to move the body of fluid.
The output for the simulation of velocity and pressure is then saved to a file
for visualization.
%
\begin{python}
  from esys.escript import *
  import esys.finley
  from esys.escript.linearPDEs import LinearPDE
  from esys.escript.models import StokesProblemCartesian
  from esys.weipa import saveVTK

  # physical constants
  eta=1.
  rho=100.
  g=10.

  # solver settings
  tolerance=1.0e-4
  max_iter=200
  t_end=50
  t=0.0
  time=0
  verbose=True

  # define mesh
  H=2.
  L=1.
  W=1.
  mesh = esys.finley.Rectangle(l0=L, l1=H, order=-1, n0=20, n1=20)
  coordinates = mesh.getX()

  # gravitational force
  Y=Vector(0., Function(mesh))
  Y[1] = -rho*g

  # element spacing
  h = Lsup(mesh.getSize())

  # boundary conditions for slip at base
  boundary_cond=whereZero(coordinates[1])*[0.0,1.0]
      +whereZero(coordinates[0])*[1.0,0.0]

  # velocity and pressure vectors
  velocity=Vector(0., Solution(mesh))
  pressure=Scalar(0., ReducedSolution(mesh))

  # Stokes Cartesian
  solution=StokesProblemCartesian(mesh)
  solution.setTolerance(tolerance)

  while t <= t_end:
    print(" ----- Time step = %s -----"%t)
    print("Time = %s seconds"%time)

    solution.initialize(fixed_u_mask=boundary_cond, eta=eta, f=Y)
    velocity,pressure=solution.solve(velocity,pressure,max_iter=max_iter, \
                                     verbose=verbose)

    print("Max velocity =", Lsup(velocity), "m/s")

    # CFL condition
    dt=0.4*h/(Lsup(velocity))
    print("dt =", dt)

    # displace the mesh
    displacement = velocity * dt
    coordinates = mesh.getX()
    newx=interpolate(coordinates + displacement, ContinuousFunction(mesh))
    mesh.setX(newx)

    time += dt

    vel_mag = length(velocity)

    #save velocity and pressure output
    saveVTK("vel.%2.2i.vtu"%t, vel=vel_mag, vec=velocity, pressure=pressure)
    t = t+1.
\end{python}
%
The results from the simulation can be viewed with \mayavi, by executing the
following command:
\begin{verbatim}
mayavi2 -d vel.00.vtu -m Surface
\end{verbatim}
%
Colour-coded scalar maps and velocity flow fields can be viewed by selecting
them in the menu.
The time-steps can be swept through to view a movie of the simulation.
\fig{FLUID OUTPUT} shows the simulation output.
Velocity vectors and a colour map for pressure are shown.
As the time progresses the body of fluid falls under the influence of gravity.
%
\begin{figure}[ht]
\center
\subfigure[t=1]{\label{FLOW OUTPUT 01}\includegraphics[height=5cm]{stokes-fluid-t01}}
\hspace{1.6cm}
\subfigure[t=20]{\label{FLOW OUTPUT 10}\includegraphics[height=5cm]{stokes-fluid-t10}}
\hspace{1.6cm}
\subfigure[t=30]{\label{FLOW OUTPUT 20}\includegraphics[height=5cm]{stokes-fluid-t20}}\\
\subfigure[t=40]{\label{FLOW OUTPUT 30}\includegraphics[height=5cm]{stokes-fluid-t30}}
\hspace{1cm}
\subfigure[t=50]{\label{FLOW OUTPUT 40}\includegraphics[height=5cm]{stokes-fluid-t40}}
\hspace{1cm}
\subfigure[t=60]{\label{FLOW OUTPUT 50}\includegraphics[height=5cm]{stokes-fluid-t50}}
%\includegraphics[scale=0.25]{stokes-fluid-colorbar}
\caption{Simulation output for Stokes flow. Fluid body starts off as a
rectangular shape, then progresses downwards under the influence of gravity.
Colour coded distribution represents the scalar values for pressure.
Velocity vectors are displayed at each node in the mesh to show the flow field.
Computational mesh used was 20$\times$20 elements.}
\label{FLUID OUTPUT}
\end{figure}
%
The view used here to track the fluid is the Lagrangian view, since the mesh
moves with the fluid. One of the disadvantages of using the Lagrangian view is
that the elements in the mesh become severely distorted after a period of time
and introduce solver errors. To get around this limitation the Level Set
Method can be used, with the Eulerian point of view for a fixed mesh.
%The Level Set Method is discussed in Section \ref{LEVELSET CHAP}.


\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[affil-it]{authblk}
\usepackage{amsmath}
\usepackage{fullpage}
\usepackage{hyperref}
\usepackage{tikz}
\usepackage{calc}
\usepackage{pgfplots}
\usepackage{float}
\usepackage{caption}
\usetikzlibrary{automata}
\title{\textbf{Hidden Markov Model Cheatsheet}}
\author{Kanru Hua%
  \thanks{Email address: \texttt{khua5@illinois.edu}}}
\affil{University of Illinois at Urbana-Champaign}
\date{April 2016}
\begin{document}

\maketitle

\section{Preliminaries}

\begin{align*}
\intertext{\textbf{Conditional (Posterior) Probability}}
P(A|B) &= \frac{P(A, B)}{P(B)}
\intertext{where $P(A, B)$ is called \textbf{joint probability} and $P(B)$ is called \textbf{prior probability}.}
\intertext{\textbf{Statistical Independence}}
\intertext{If A and B are independent events,}
P(A|B) = P(A) &\textrm{\ and\ } P(B|A) = P(B) \\
\textrm{or\ } P(A, B) &= P(A) P(B)
\intertext{\textbf{Marginal Probability}}
P(A) &= \sum_b P(A, B = b)
\intertext{\textbf{Expectation}}
E[X] &= \sum_j P(X = x_j) x_j
\intertext{\textbf{Jensen's Inequality}}
\intertext{(You are \textbf{not} required to know this beforehand)}
E[f(X)] &\leq f(E[X]) \textrm{\ for any concave function $f$} \\
E[f(X)] &= f(E[X]) \textrm{\ if X is constant}
\end{align*}

\section{Markov Chain}

\subsection{Example}

\newcommand{\placeholder}[1]{\phantom{aaa}\makebox[0pt]{#1}\phantom{aaa}}
\begin{center}
\begin{tikzpicture}[->,auto,node distance=4.5cm]
\tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=1]
\node[state]    (A)                     {\placeholder{sunny}};
\node[state]    (B)[below left of=A]    {\placeholder{cloudy}};
\node[state]    (C)[below right of=A]   {\placeholder{rainy}};
\path (A) edge[loop above]     node{0.8}         (A);
\path (A) edge[bend right]     node{0.1}         (B);
\path (A) edge[bend left]      node{0.1}         (C);
\path (B) edge[loop left]      node{0.6}         (B);
\path (B) edge[bend right]     node{0.2}         (A);
\path (B) edge[bend right]     node{0.2}         (C);
\path (C) edge[loop right]     node{0.7}         (C);
\path (C) edge[bend left]      node{0.2}         (A);
\path (C) edge     			   node{0.1}         (B);
\end{tikzpicture}
\end{center}

\subsection{Definition}

\begin{align*}
\textrm{parameter set:\quad} &\lambda = \{\pi, \mathbf{A}\} \\
\textrm{initial probability:\quad}&\pi_j = P(s_1 = j | \lambda) \\
\textrm{transition probability:\quad}&a_{ij} = P(s_t = j | s_{t-1} = i, \lambda)
\intertext{\textbf{Markov Property}}
P(s_t = j | s_1, s_2, ..., s_{t-2}, s_{t-1} = i, \lambda) &= P(s_t = j | s_{t-1} = i, \lambda) = a_{ij}
\end{align*}

A lateral view where each edge has a transition probability, or an initial probability for $t = 1$:
\begin{center}
\begin{tikzpicture}[->,auto,node distance=2.5cm]
\tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=1]
\tikzstyle{ends}=[fill=white,draw=white,thick,text=black,scale=1]
\node[ends]     (0)                {begin};
\node[state]    (A)[right of=0]    {\placeholder{$s_1=1$}};
\node[state]    (B)[right of=A]    {\placeholder{$s_2=1$}};
\node[state]    (C)[right of=B]    {\placeholder{$s_3=1$}};
\node[state]    (D)[right of=C]    {\placeholder{$s_4=1$}};
\node[state]    (A1)[below of=A]   {\placeholder{$s_1=2$}};
\node[state]    (B1)[right of=A1]  {\placeholder{$s_2=2$}};
\node[state]    (C1)[right of=B1]  {\placeholder{$s_3=2$}};
\node[state]    (D1)[right of=C1]  {\placeholder{$s_4=2$}};
\node[state]    (A2)[below of=A1]  {\placeholder{$s_1=3$}};
\node[state]    (B2)[right of=A2]  {\placeholder{$s_2=3$}};
\node[state]    (C2)[right of=B2]  {\placeholder{$s_3=3$}};
\node[state]    (D2)[right of=C2]  {\placeholder{$s_4=3$}};
\node[ends]     (1)[right of=D]    {end};
\path (0) edge (A);\path (0) edge (A1);\path (0) edge (A2);
\path (A) edge (B);\path (A) edge (B1);\path (A) edge (B2);
\path (B) edge (C);\path (B) edge (C1);\path (B) edge (C2);
\path (C) edge (D);\path (C) edge (D1);\path (C) edge (D2);
\path (D) edge (1);\path (D1) edge (1);\path (D2) edge (1);
\path (A1) edge (B1);\path (A1) edge (B);\path (A1) edge (B2);
\path (B1) edge (C1);\path (B1) edge (C);\path (B1) edge (C2);
\path (C1) edge (D1);\path (C1) edge (D);\path (C1) edge (D2);
\path (A2) edge (B2);\path (A2) edge (B);\path (A2) edge (B1);
\path (B2) edge (C2);\path (B2) edge (C);\path (B2) edge (C1);
\path (C2) edge (D2);\path (C2) edge (D);\path (C2) edge (D1);
\end{tikzpicture}
\end{center}

\section{Hidden Markov Model}

\subsection{Definition}

\begin{align*}
\textrm{parameter set:\quad} &\lambda = \{\pi, \mathbf{A}, \mathbf{B}\} \\
\textrm{initial probability:\quad}&\pi_j = P(s_1 = j | \lambda) \\
\textrm{transition probability:\quad}&a_{ij} = P(s_t = j | s_{t-1} = i, \lambda) \\\\
\textrm{output probability (discrete case):\quad}&b_{jk} = P(o_t = k | s_t = j, \lambda) \\
\textrm{output probability density (continuous case):\quad}&b_j(o_t) = p(o_t | s_t = j, \lambda)
\end{align*}

\subsection{Graph}
\begin{center}
\begin{tikzpicture}[->,auto,node distance=2.5cm]
\tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=1]
\tikzstyle{output}=[fill=white,draw=black,thick,text=black,scale=1,minimum height=1cm]
\tikzstyle{ends}=[fill=white,draw=white,thick,text=black,scale=1]
\node[ends]     (0)                {begin};
\node[state]    (A)[right of=0]    {\placeholder{$s_1=?$}};
\node[state]    (B)[right of=A]    {\placeholder{$s_2=?$}};
\node[state]    (C)[right of=B]    {\placeholder{$s_3=?$}};
\node[state]    (D)[right of=C]    {\placeholder{$s_4=?$}};
\node[output]   (Ao)[above of=A]   {\placeholder{$o_1$}};
\node[output]   (Bo)[above of=B]   {\placeholder{$o_2$}};
\node[output]   (Co)[above of=C]   {\placeholder{$o_3$}};
\node[output]   (Do)[above of=D]   {\placeholder{$o_4$}};
\node[ends]     (1)[right of=D]    {end};
\path (0) edge node{$\pi_{s_1}$}   (A);
\path (A) edge node{$a_{s_1,s_2}$} (B);
\path (B) edge node{$a_{s_2,s_3}$} (C);
\path (C) edge node{$a_{s_3,s_4}$} (D);
\path (D) edge node{$1$}           (1);
\path (A) edge node{$b_{s_1}(o_1)$}(Ao);
\path (B) edge node{$b_{s_2}(o_2)$}(Bo);
\path (C) edge node{$b_{s_3}(o_3)$}(Co);
\path (D) edge node{$b_{s_4}(o_4)$}(Do);
\end{tikzpicture}
\end{center}

\subsection{List of Operations \& Algorithms}

What can we do with HMM?

\begin{itemize}
\item Generation
  \begin{itemize}
    \item Random walk
  \end{itemize}
\item Inference
  \begin{itemize}
    \item \textbf{Forward/Backward algorithm}
    \item \textbf{Viterbi algorithm}
    \item Total probability: $P(O | \lambda)$
    \item State occupancy probability: $P(s_t = j | O, \lambda)$
    \item Optimal probability \& state sequence: $\underset{s}{\arg\max\\ } P(O, s | \lambda)$
  \end{itemize}
\item Parameter estimation (training)
  \begin{itemize}
    \item Initialization
    \item Viterbi training
    \item \textbf{Baum-Welch algorithm} (expectation-maximization)
  \end{itemize}
\end{itemize}

\newpage
\section{HMM Inference}

\subsection{Building Blocks}

\begin{align*}
\intertext{What we already know:}
P(s_1 = i | \lambda) &= \pi_i \\
P(s_t = j | s_{t - 1} = i, \lambda) &= a_{ij} \\
P(o_t | s_t = j, \lambda) &= b_j(o_t)
\intertext{Trivial extensions using Markov property:}
P(s_1, s_2, ..., s_t | \lambda) &= \pi_{s_1} \prod_{\tau=2}^t a_{\tau-1,\tau} \\
P(o_1, o_2, ..., o_t | s_1, s_2, ..., s_t, \lambda) &= \prod_{\tau=1}^t b_{s_\tau}(o_\tau)
\intertext{By combining above equations, we get}
P(o_1, o_2, ..., o_t, s_1, s_2, ..., s_t | \lambda) &= \pi_{s_1} b_{s_1}(o_1) \prod_{\tau=2}^t a_{\tau-1,\tau} b_{s_\tau}(o_\tau)
\end{align*}

\subsection{Inference Algorithms}

\begin{align*}
\intertext{\textbf{Forward Algorithm}}
\alpha_t(i) &= P(o_1, o_2, ..., o_t, s_t=i | \lambda) \\
            &= P(o_t | s_t=i, \lambda) \sum_j P(o_1, ..., o_{t-1}, s_{t-1}=j | \lambda) P(s_t=i | s_{t-1}=j, \lambda) \\
            &= b_i(o_t) \sum_j a_{ji} \alpha_{t-1}(j) \\
\alpha_1(i) &= P(o_1 | s_1 = i, \lambda) P(s_1=i | \lambda) = b_i(o_1) \pi_i
\intertext{\textbf{Backward Algorithm}}
\beta_t(i)  &= P(o_{t+1}, o_{t+2}, ..., o_t | s_t=i, \lambda) \\
            &= \sum_j P(o_{t+1} | s_{t+1}=j, \lambda) P(o_{t+2}, ..., o_T | s_{t+1}=j, \lambda) P(s_{t+1}=j | s_t=i, \lambda) \\
            &= \sum_j b_j(o_{t+1}) \beta_{t+1}(j) a_{ij} \\
\beta_T(i)  &= 1
\intertext{\textbf{Viterbi Algorithm}}
\alpha^*_t(i)
            &= \max_{s_1, ..., s_{t-1}} P(o_1, ..., o_t, s_1, ..., s_{t-1}, s_t = i | \lambda) \\
            &= P(o_t | s_t=i, \lambda) \max_j \left( P(s_t=i | s_{t-1}=j, \lambda) \max_{s_1, ..., s_{t-2}} P(o_1, ..., o_{t-1}, s_1, ..., s_{t-2}, s_{t-1}=i | \lambda) \right) \\
            &= b_i(o_t) \max_j a_{ji} \alpha^*_{t-1}(j) \\
p^*_t(i)    &= \underset{j}{\arg\max\ } a_{ji} \alpha^*_{t-1}(j) \\
\alpha^*_1(i)
            &= b_i(o_1) \pi_i \\
p^*_1(i)    &= 0
\intertext{\textbf{Total Probability}}
P(O | \lambda) &= \sum_j P(o_1, ..., o_T, s_t=j | \lambda) = \sum_j \alpha_T(j) \textrm{\quad (from forward probability)} \\
P(O | \lambda) &= \sum_j P(o_1 | s_1=j, \lambda) P(o_2, ..., o_T | s_1=j, \lambda) P(s_1=j | \lambda) \\
               &= \sum_j b_j(o_1) \beta_1(j) \pi_j \textrm{\quad (from backward probability)} \\
P(O | \lambda) &= \sum_j P(o_1, ..., o_t, o_{t+1}, ..., o_T, s_t=j | \lambda) \\
               &= \sum_j P(o_1, ..., o_t, s_t=j | \lambda) P(o_{t+1}, ..., o_T | s_t=j, \lambda) \\
               &= \sum_j \alpha_t(j) \beta_t(j) \textrm{\quad (from both forward and backward probability, for arbitrary $t$)}
\intertext{\textbf{State Occupancy Probability}}
\gamma_t(j) &= P(s_t=j | O, \lambda) \\
            &= \frac{P(o_1, ..., o_t, s_t=j | \lambda) P(o_{t+1}, ..., o_T | s_t=j, \lambda)}{P(O | \lambda)} \\
            &= \frac{\alpha_t(j) \beta_t(j)}{P(O | \lambda)}
\intertext{note that $\sum_j \gamma_t(j) = 1$}
\intertext{\textbf{State Transition Probability} (not to be confused with $a_{ij}$)}
\gamma_t(i, j) &= P(s_{t-1}=i, s_t=j | O, \lambda) \\
               &= \frac{\alpha_{t-1}(i) \beta_t(j) b_j(o_t) a_{ij}}{P(O | \lambda)} \quad\textrm{whose derivation is similar to $\gamma_t(j)$}
\end{align*}

\newpage
\section{HMM Parameter Estimation}

\subsection{Expectation-Maximization Algorithm}

\begin{align*}
\intertext{Goal: obtain maximum likelihood estimation of $\lambda$:}
\lambda^* &= \underset{\lambda}{\arg\max\ } l(\lambda) = \underset{\lambda}{\arg\max\ } \log P(O | \lambda) \\
l(\lambda) &= \log P(O | \lambda) = \log \sum_\mathbf{s} P(O, \mathbf{s} | \lambda)
\intertext{Motivation: find an alternative likelihood function whose derivatives are easier to calculate.}
\intertext{Assume we have a function $Q(\mathbf{s})$ such that $\sum_\mathbf{s} Q(\mathbf{s}) = 1$ and $Q(\mathbf{s}) > 0 \ \forall \mathbf{s}$,}
l(\lambda) &= \log \sum_\mathbf{s} Q(\mathbf{s}) \frac{P(O, \mathbf{s} | \lambda)}{Q(\mathbf{s})} \\
  &= \log E_{\mathbf{s}} [\frac{P(O, \mathbf{s} | \lambda)}{Q(\mathbf{s})}] \\
  &\geq E_{\mathbf{s}} [\log \frac{P(O, \mathbf{s} | \lambda)}{Q(\mathbf{s})}] \quad\textrm{(whose partial derivatives have closed form)}
\intertext{To make the lower bound more ``effective", i.e., we want $\log E_s [...] = E_s [\log (...)]$,}
&\begin{cases}
  \frac{P(O, \mathbf{s} | \lambda}{Q(\mathbf{s})} &= c \\
  \sum_\mathbf{s} Q(\mathbf{s}) &= 1
\end{cases} \ \Rightarrow Q(\mathbf{s}) = P(\mathbf{s} | O, \lambda)
\end{align*}

\subsubsection*{EM Algorithm}

\noindent
Repeat until convergence \{

Expectation: $Q(\mathbf{s}) = P(\mathbf{s} | O, \lambda)$

Maximization: $\lambda^* = \underset{\lambda}{\arg\max\ } \sum_\mathbf{s} Q(\mathbf{s}) \log \frac{P(O, \mathbf{s} | \lambda)}{Q(\mathbf{s})}$

\noindent
\}

\subsection{Baum-Welch Algorithm}

\begin{align*}
\intertext{Error function (in maximization step):}
J(\lambda) &= \sum_\mathbf{s} Q(\mathbf{s}) \log \frac{P(O, \mathbf{s} | \lambda)}{Q(\mathbf{s})} \\
  &= \sum_\mathbf{s} Q(\mathbf{s}) \left( \log \pi_{s_1} + \sum_{t=2}^T \log a_{s_{t-1},s_t} + \sum_{t=1}^T \log b_{s_t}(o_t) - \log(Q(\mathbf{s})) \right)
\intertext{Take partial derivative with respect to, for example, $a_{ij}$,}
\frac{\partial J}{\partial a_{ij}}
  &= \frac{\partial}{\partial a_{ij}} \sum_\mathbf{s} Q(\mathbf{s}) \sum_{t=2}^T \log a_{s_{t-1},s_t} \\
  &= \frac{\partial}{\partial a_{ij}} \sum_{t=2}^T \sum_m \sum_n \log a_{mn} \sum_\mathbf{\substack{s_1,..., s_{t-2} \\ s_{t+1}, ..., s_T}} Q(\mathbf{s}) \\
  &= \frac{\partial}{\partial a_{ij}} \sum_{t=2}^T \sum_m \sum_n \log a_{mn} \underbrace{P(s_{t-1}=m, s_t=n | O, \lambda')}_{\gamma_t(m, n)} \\
  &= \frac{1}{a_{ij}} \sum_{t=2}^T \gamma_t(i, j)
\intertext{To make sure $\sum_j a_{ij} = 1 \ \forall i$, introduce Lagrange multiplier $l$,}
&\begin{cases}
  \sum_{t=2}^T \gamma_t(i, j) &= l a_{ij} \\
  \sum_j a_{ij} &= 1
\end{cases}
\intertext{Solve the equations,}
l &= \sum_{t=2}^T \sum_n \gamma_t(i, n), \quad a_{ij} = \frac{\sum_{t=2}^T \gamma_t(i, j)}{\sum_{t=2}^T \sum_n \gamma_t(i, n)} = \frac{\sum_{t=2}^T \gamma_t(i, j)}{\sum_{t=2}^T \gamma_{t-1}(i)}
\intertext{Similarly for $\pi_i$ and $b_{ik}$ (discrete case) we get,}
\pi_i &= \gamma_1(i) \\
b_{ik} &= \frac{\sum_{t: o_t = k} \gamma_t(i)}{\sum_{t=1}^T \gamma_t(i)}
\intertext{\textbf{Multiple Observation Sequences}}
\pi_i &= \frac{1}{L} \sum_{l=0}^L \gamma_1^l(i) \\
a_{ij} &= \frac{\sum_{l=0}^L \sum_{t=2}^T \gamma_t^l(i, j)}{\sum_{l=0}^L \sum_{t=2}^T \gamma_{t-1}^l(i)}
\quad b_{ik} = \frac{\sum_{l=0}^L \sum_{t: o_t = k} \gamma_t^l(i)}{\sum_{l=0}^L \sum_{t=1}^T \gamma_t^l(i)}
\end{align*}

\subsection{Baum-Welch Algorithm for Continuous Output Distributions}

\begin{alignat*}{3}
\intertext{\textbf{Multivariate Normal Distribution}}
\mu_j &= \frac{\sum_{t=1}^T \gamma_t(j) o_t}{\sum_{t=1}^T \gamma_t(j)} \quad &\Sigma_j &= \frac{\sum_{t=1}^T \gamma_t(j) o_t o_t^T}{\sum_{t=1}^T \gamma_t(j)} - \mu_j \mu_j^T
\end{alignat*}
\begin{align*}
\intertext{\textbf{Multivariate Gaussian Mixture Model}}
\intertext{Gaussian mixture model:}
p(x \in \mathbf{R}^k | \boldsymbol c, \boldsymbol\mu, \boldsymbol\Sigma) &= (2\pi)^{-\frac{k}{2}} \sum_j c_j |\Sigma_j|^{-\frac{k}{2}} e^{-\frac{k}{2} (x - \mu_j)^T \Sigma_j^{-1} (x - \mu_j)}
\intertext{HMM-GMM:}
g_{jk}(o_t) &= p(o_t | s_t=j, m_t = k, \lambda) = p(o_t | \boldsymbol \mu_{jk}, \boldsymbol \Sigma_{jk}) \\
b_j(o_t) &= p(o_t | s_t=j, \lambda) = \sum_k c_{jk} g_{jk}(o_t)
\intertext{HMM-GMM inference:}
\xi_t(j, k) &= p(s_t=j, m_t=k | O, \lambda) = \frac{\sum_i \alpha_{t-1}(i) \beta_t(j) a_{ij}c_{jk}g_{jk}(o_t)}{P(O | \lambda)}
\end{align*}
\begin{alignat*}{3}
\intertext{HMM-GMM parameter estimation:}
c_{jk} &= \frac{\sum_{t=1}^T \xi_t(j, k)}{\sum_{t=1}^T \gamma_t(j)} \\
\mu_{jk} &= \frac{\sum_{t=1}^T \xi_t(j, k) o_t}{\sum_{t=1}^T \xi_t(j, k)} \quad &\Sigma_{jk} &= \frac{\sum_{t=1}^T \xi_t(j, k) o_t o_t^T}{\sum_{t=1}^T \xi_t(j, k)} - \mu_{jk} \mu_{jk}^T
\end{alignat*}

\begin{thebibliography}{99}

\bibitem{mmprta}{Fink, Gernot A. ``Markov models for pattern recognition: from theory to applications". Springer Science \& Business Media, 2014.}

\bibitem{cs229}{Ng, Andrew. ``Mixtures of Gaussians and the EM algorithm." Stanford University. CS229 Lecture Notes (2014).}

\bibitem{bilmes}{Bilmes, Jeff A. ``A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models." International Computer Science Institute 4.510 (1998): 126.}

\end{thebibliography}

\end{document}


\subsection{Numerical Grid}\label{num.sec.grid}
The continuous equations descrining ice physics have to be discretised in order to be solved by a computer (which is inherently finite). This section describes the finite--difference grids employed by the model.
\subsubsection{Horizontal Grid}
The modelled region ($x\in[0,L_x]$, $y\in[0,L_y]$) is discretised using a regular grid so that $x_i=(i-1)\Delta x$ for $i\in[1,N]$ (and similarly for $y_j$). The model uses two staggered horizontal grids in order to improve stability. Both grids use the same grid spacing, $\Delta x$ and $\Delta y$, but are off-set by half a grid (see Fig. \ref{kin.fig.grid}).
\begin{figure}[htbp]
  \begin{center}
    \includegraphics{\dir/figs/grid.eps}
    \caption{Horizontal Grid.}
    \label{kin.fig.grid}
  \end{center}
\end{figure}
Quantities calculated on the $(r,s)$--grid are denoted with a tilde, i.e. $\tilde{F}$. Quantities are transformed between grids by averaging over the surrounding nodes, i.e. a quantity in the $(i,j)$--grid becomes in the $(r,s)$ grid:
\begin{subequations}
  \begin{align}
    \tilde{F}_{r,s}&=\tilde{F}_{i+\frac12,j+\frac12}=\frac14(F_{i,j}+F_{i+1,j}+F_{i+1,j+1}+F_{i,j+1})\\
    \intertext{and similarly for the reverse transformation:}
    F_{i,j}&=F_{r-\frac12,s-\frac12}=\frac14(\tilde{F}_{r-1,s-1}+\tilde{F}_{r,s-1}+\tilde{F}_{r,s}+\tilde{F}_{r-1,s})
  \end{align}
\end{subequations}

In general, horizontal velocities and associated quantities like the diffusivity are calculated on the $(r,s)$ grid, ice thickness, temperatures and vertical velocities are calculated on the $(i,j)$--grid.

Horizontal gradients are calculated on the $(r,s)$--grid, i.e. surface gradients are:
\begin{subequations}
\begin{align}
  \left(\frac{\pd s}{\pd x}\right)_{r,s}=\tilde{s}^x_{r,s}&=\frac{s_{i+1,j}-s_{i,j}+s_{i+1,j+1}-s_{i,j+1}}{2\Delta x}\\
  \left(\frac{\pd s}{\pd y}\right)_{r,s}=\tilde{s}^y_{r,s}&=\frac{s_{i,j+1}-s_{i,j}+s_{i+1,j+1}-s_{i+1,j}}{2\Delta y}
\end{align}
\end{subequations}
Ice thickness gradients, $\tilde{H}^x_{r,s}$ and $\tilde{H}^y_{r,s}$, are formed similarly. Gradients in the $(r,s)$--grid are formed in a similar way, e.g.
\begin{equation}
  \left(\frac{\pd u}{\pd x}\right)_{i,j}=u^x_{i,j}=\frac{\tilde{u}_{r,s-1}-\tilde{u}_{r-1,s-1}+\tilde{u}_{r,s}-\tilde{u}_{r-1,s}}{2\Delta x}
\end{equation}

\subsubsection{Periodic Boundary Conditions}
The model can be run with horizontal periodic boundary conditions, i.e. the western edge of the modelled region is joined with the eastern edge. Figure \ref{num.fig.grid_ew} illustrates the numeric grid when the model is run in torus mode.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.9\textwidth]{\dir/figs/grid_ew.eps}
  \caption{A row of the numeric grid when the model is used in torus mode. Circles indicate points in $(i,j)$--grid and squares indicate points in the $(r,s)$--grid. Points with the same colour are logically the same.}
  \label{num.fig.grid_ew}
\end{figure}

These boundary conditions are enforced by exchanging points for the temperature and vertical velocity calculations. The ice thicknesses are calculated explicitly at the ghostpoints.

\subsubsection{$\sigma$--Coordinate System}\label{num.sec.sigma}
The vertical coordinate, $z$, is scaled by the ice thickness analogous to the $s$--coordinate in numerical weather simulations \citep[e.g.][]{Holton1992}. A new vertical coordinate, $\sigma$, is introduced so that the ice surface is at $\sigma=0$ and the ice base at $\sigma=1$ (see Fig. \ref{kin.fig.scale}), i.e.
\begin{equation}
  \label{kin.eq.vertical_scale}
  \sigma=\frac{s-z}{H}.
\end{equation}

\begin{figure}[htbp]
  \begin{center}
    \includegraphics{\dir/figs/scale.eps}
    \caption[Vertical scaling of the ice sheet model.]{Vertical scaling of the ice sheet model. The vertical axis is scaled to unity. The horizontal coordinates are not changed.}
    \label{kin.fig.scale}
  \end{center}
\end{figure}

The derivatives of a function $f$ in $(x,y,z,t)$ become in the new $(\tilde{x},\tilde{y},\sigma,\tilde{t})$ system:
\begin{subequations}
  \begin{align}
    \frac{\pd f}{\pd x} &= \frac{\pd f}{\pd\tilde{x}}+\frac1H\Delta_{\tilde{x}}\frac{\pd f}{\pd \sigma},\\
    \frac{\pd f}{\pd y} &= \frac{\pd f}{\pd\tilde{y}}+\frac1H\Delta_{\tilde{y}}\frac{\pd f}{\pd \sigma},\\
    \frac{\pd f}{\pd t} &= \frac{\pd f}{\pd\tilde{t}}+\frac1H\Delta_{\tilde{t}}\frac{\pd f}{\pd \sigma},\\
    \frac{\pd f}{\pd z} &= -\frac1H\frac{\pd f}{\pd\sigma},
  \end{align}
\end{subequations}
where  the geometric factors, $\Delta_{\tilde{x}}$, $\Delta_{\tilde{y}}$ and $\Delta_{\tilde{t}}$, are defined by
\begin{subequations}
  \begin{align}
  \Delta_{\tilde{x}}&=\left(\frac{\pd s}{\pd\tilde{x}}-\sigma\frac{\pd H}{\pd\tilde{x}}\right),\\
  \Delta_{\tilde{y}}&=\left(\frac{\pd s}{\pd\tilde{y}}-\sigma\frac{\pd H}{\pd\tilde{y}}\right),\\
  \Delta_{\tilde{t}}&=\left(\frac{\pd s}{\pd\tilde{t}}-\sigma\frac{\pd H}{\pd\tilde{t}}\right).
  \end{align}
\end{subequations}
The integral of $z$ becomes in the $\sigma$--coordinate system:
\begin{equation}
  \int_h^zfdz=-H\int_1^\sigma fd\sigma
\end{equation}

The vertical coordinate is discretised using an irregular grid spacing to reflect the fact that ice flow is more variable at the bottom of the ice column. In the vertical the index $k$ is used.


\documentclass[10pt]{article}

\usepackage{enumerate}
\usepackage{amsmath,amsthm,amssymb,mathtools}
\usepackage{graphicx}
\usepackage{mycommands}
\usepackage{hyperref}

\newcommand{\eV}{\mtx{eV}}
\newcommand{\nm}{\mtx{nm}}
\newcommand{\varep}{\varepsilon}

\setlength{\oddsidemargin}{-0.25in}
\setlength{\evensidemargin}{-0.25in}
\setlength{\topmargin}{-0.25in}
\setlength{\headheight}{-0.25in}
\setlength{\headsep}{0in}
\setlength{\textwidth}{7.0in}
\setlength{\textheight}{9.5in}
\setlength{\parindent}{.25in}
\setlength{\parskip}{0in}

\begin{document}

\title{Computing spin correlations with 2-body density matrix\\ and other misc notes}
\maketitle

Operators are written like variables for simplicity. What is or isn't an operator should be clear from context.

\section{Computing spin correlations with 2-body density matrix}

The prerequisites are that
\begin{align}
  &
  S_\pm = S_x \pm i S_y,
  &&
  S_z = \frac 12 (c_\up^\dag c_\up - c_\dn^\dag c_\dn)
  &
\end{align}
which can be found by Wikipedia, looking for ``spin operator".
Using this,
\begin{align}
  &
  S_x = \frac 12 (S_+ + S_-)
  &&
  S_y = \frac 1{2i} (S_+ - S_-)
  &
\end{align}
You can also derive this by taking $S_\alpha = c^\dag_\sigma \tau_{\alpha,\sigma\sigma'} c_{\sigma'}$, where $\tau_{\alpha,\sigma\sigma'}$ are the Pauli matrices.

The rest is careful algebra after plugging these in.
\begin{align}
  S_i \cdot S_j
  &=
  S_{ix} S_{jx}
  +
  S_{iy} S_{jy}
  +
  S_{iz} S_{jz}
  \\
  &=
  \frac 12 (
    S_{i+} S_{j-} + S_{i-} S_{j+}
  )
  +
  S_{iz} S_{jz}
  \\
  &=
  \frac 12 \lt(
    c_{i\up}^\dag
    c_{i\dn}
    c_{j\dn}^\dag
    c_{j\up}
    +
    c_{i\dn}^\dag
    c_{i\up}
    c_{j\up}^\dag
    c_{j\dn}
  \rt)
  +
  \frac 14 \lt(
    c_{i\up}^\dag
    c_{i\up}
    c_{j\up}^\dag
    c_{j\up}
    +
    c_{i\dn}^\dag
    c_{i\dn}
    c_{j\dn}^\dag
    c_{j\dn}
    -
    c_{i\dn}^\dag
    c_{i\dn}
    c_{j\up}^\dag
    c_{j\up}
    -
    c_{i\up}^\dag
    c_{i\up}
    c_{j\dn}^\dag
    c_{j\dn}
  \rt)
  \\
  &=
  -\frac 12 \lt(
    c_{i\up}^\dag
    c_{j\dn}^\dag
    c_{i\dn}
    c_{j\up}
    +
    c_{j\up}^\dag
    c_{i\dn}^\dag
    c_{j\dn}
    c_{i\up}
  \rt)
  -
  \frac 14 \lt(
    c_{i\up}^\dag
    c_{j\up}^\dag
    c_{i\up}
    c_{j\up}
    +
    c_{i\dn}^\dag
    c_{j\dn}^\dag
    c_{i\dn}
    c_{j\dn}
    +
    c_{j\up}^\dag
    c_{i\dn}^\dag
    c_{i\dn}
    c_{j\up}
    +
    c_{i\up}^\dag
    c_{j\dn}^\dag
    c_{j\dn}
    c_{i\up}
  \rt)
\end{align}
The last step is rearranging the operators according to the chemistry notation~(\ref{eq:chem}).
Using that notation,
\begin{align}
  S_i \cdot S_j
  &=
  -\frac 12 \lt(
    \rho_{ijji}^{\up\dn}
    +
    \rho_{jiij}^{\up\dn}
  \rt)
  -
  \frac 14 \lt(
    \rho_{ijji}^{\up\up}
    +
    \rho_{ijji}^{\dn\dn}
    +
    \rho_{jjii}^{\up\dn}
    +
    \rho_{iijj}^{\up\dn}
  \rt)
\end{align}

\section{Misc notes when working on this}

\verb|PySCF| uses the ``chemistry notation'' when storing the 2-body density matrix. In particular,
\begin{align}
  \label{eq:chem}
  \rho^{\up \dn}_{ijkl}
  =
  \abr{c_{i\up}^\dag c_{k\dn}^\dag c_{l\dn} c_{j\up}}
\end{align}
This is documented in, for example, the FCI \verb|make_rdm12s| docstring.
This notation has the advantage that when $k = j$ the center becomes a density operator.

Using $\{c_i^\dag,c_j\} = 0$, it differs by a sign and \verb|swapaxes| from the physics notation (TODO verify this...it's been a while and this is from memory):
\begin{align}
  \rho^{\up \dn}_{ijkl}
  =
  \abr{c_{i\up}^\dag c_{j\dn}^\dag c_{k\up} c_{l\dn}}
\end{align}

\end{document}