%!TEX root = ../thesis.tex

\chapter{The title of chapter one}

\section{Quick examples}

Example of citation~\cite{dummy-1} and~\cite{dummy-2}.

Write \(\N,\Z,\Q,\R,\C\).

\(\norm{a}, \abs{b}, \ket{c}, \bra{d}, \braket{e}{f}, \inpro{g}{h}, \nicefrac{3}{\pi}\)

\begin{theorem}
  \label{thm:dummy-thm}
  A
\end{theorem}

\begin{theorem*}[Theorem of second dummy]
  A, follows from Theorem~\ref{thm:dummy-thm}
\end{theorem*}

\begin{lemma}
  \label{lem:dummy}
  Statement
\end{lemma}

\begin{example}
  \label{exa:haha}
  Null example.
\end{example}

\begin{equation}
  \label{eq:nontrivial}
  x^2=x^2.
\end{equation}
Note that Equation~\eqref{eq:nontrivial} is not an obvious fact. You need to scratch your head for it.

\begin{definition}
  \label{def:empty}
  This defintion is not empty.
\end{definition}
\begin{remark}
  \label{rem:regarding-def}
  The reader may find Definition~\ref{def:empty} nonempty, however, you can prove that the definition \emph{is} empty.
\end{remark}

List of environments can be found in the .cls file.

Commutative diagramme using tikz-cd.

\[
  \begin{tikzcd}
  T
\arrow[drr, bend left, "x"]
\arrow[ddr, bend right, "y"]
\arrow[dr, dotted, "{(x,y)}" description] & & \\
& X \times_Z Y \arrow[r, "p"] \arrow[d, "q"] & X \arrow[d, "f"] \\
& Y \arrow[r, "g"] &Z
\end{tikzcd}
\]

Example of commutative diagramme using TikZ.
    \begin{center}
      \begin{tikzpicture}[scale=1]
        \draw[->] (1.7,0)--(5.2,0); \draw[->] (0,1.7)--(0,0.2);
        \draw[->](2.5,2)--(4.4,2); \draw[->] (5.8, 1.7)--(5.8,0.24);
        \draw[->, double] (1.8, 0.6)--(4.2,1.4); \node at (6, 0)[
        scale=1]{$\mathfrak{S}(A,D)$}; \node at (6.2, 2)[
        scale=1]{$\mathfrak{S}(A,B)\times \mathfrak{S}(B,D)$}; \node
        at (-0.2, 2)[
        scale=1]{$\mathfrak{S}(A,B)\times \mathfrak{S}(B,C)\times
          \mathfrak{S}(C,D)$}; \node at (2.7,1.1)[scale=0.8]{$\sim$};
        \node at (3.8,0.8)[scale=0.8]{$a(A,B,C,D)$}; \node at (0, 0)[
        scale=1]{$\mathfrak{S}(A,C)\times \mathfrak{S}(C,D)$}; \node
        at (3.45, 2.25)[scale=0.8]{$\textup{Id}\times c(B,C,D)$};
        \node at (-1.1,
        1.0)[scale=0.8]{$ c(A,B,C)\times \textup{Id}$}; \node at (3.1,
        -0.2)[scale=0.8]{$c(A,C,D)$}; \node at (6.6,
        1.0)[scale=0.8]{$c(A,B,D)$};
      \end{tikzpicture}
    \end{center}

    Commutative diagramme using package XY.

    \begin{figure}[htb]
      \[
        \xymatrix@1@C=3em{ (G,\alpha)\quad
      \ar@/^1pc/[r]^{(X,\lambda)}_{}="0"
      \ar@/_1pc/[r]_{(Y,\kappa)}^{}="1" \ar@{=>}"0";"1"^{\phi} &
      \quad(H,\beta)\quad \ar@/^1pc/[r]^{(X',\lambda')}_{}="0"
      \ar@/_1pc/[r]_{(Y',\kappa')}^{}="1" \ar@{=>}"0";"1"^{\phi'}&
      \quad(K,\mu) }
  \]
    \caption{}
    \label{fig:vert-comp}
  \end{figure}

  Refer the packages for details about commutative diagrammes (type
  \verb+ texdoc tikz-cd+ or
  \verb+ texdoc xy+ and enter, or ask the internet).

 Glossaries ---regular glossaries, or glossaries appearing as list of
 symbols, etc--- are supported with package \emph{glossaries}. It can
 be used in the standard way. The package does not have \emph{xindy} option.

 Package \emph{xcolor} is loaded with \emph{x11names} option.

 Avoid working out~\ref{rem:more-details} and~\ref{rem:renaming-env}
 unless you know what you are doing.
 \begin{remark}
   \label{rem:more-details}
   Please have a look at \emph{commands.tex} and \emph{symbols.tex},
   in commands folder, for various shortforms and newcommands. You may
   modify and add new newcommands here. The packages are listed in
   IISERB.cls. You may add or modify the packages as per your need.
 \end{remark}

  \begin{remark}
    \label{rem:renaming-env}
    If you want, you may modify the math-theorem commands to use
    \verb+ \begin{thm}+ instead of
    \verb+ \begin{theorem}+ , and similar
    for other environments like proposition, examples, remarks. Check
    out the \emph{command.tex} in the commands folder.
  \end{remark}

\section{Dummy text}

In mathematics, a group is a set equipped with a binary operation that combines any two elements to form a third element in such a way that four conditions called group axioms are satisfied, namely closure, associativity, identity and invertibility. One of the most familiar examples of a group is the set of integers together with the addition operation, but groups are encountered in numerous areas within and outside mathematics, and help focusing on essential structural aspects, by detaching them from the concrete nature of the subject of the study.

\section{Philosophical discussion about groups}
\label{sec:grp-philo}

Groups share a fundamental kinship with the notion of symmetry. For example, a symmetry group encodes symmetry features of a geometrical object: the group consists of the set of transformations that leave the object unchanged and the operation of combining two such transformations by performing one after the other. Lie groups are the symmetry groups used in the Standard Model of particle physics; Poincaré groups, which are also Lie groups, can express the physical symmetry underlying special relativity; and point groups are used to help understand symmetry phenomena in molecular chemistry.

\subsection{More philosophy!}
\label{subsec:grp-philo}

The concept of a group arose from the study of polynomial equations, starting with Évariste Galois in the 1830s, who introduced the term of group (groupe, in French) for the symmetry group of the roots of an equation, now called a Galois group. After contributions from other fields such as number theory and geometry, the group notion was generalized and firmly established around 1870. Modern group theory—an active mathematical discipline—studies groups in their own right. To explore groups, mathematicians have devised various notions to break groups into smaller, better-understandable pieces, such as subgroups, quotient groups and simple groups. In addition to their abstract properties, group theorists also study the different ways in which a group can be expressed concretely, both from a point of view of representation theory (that is, through the representations of the group) and of computational group theory. A theory has been developed for finite groups, which culminated with the classification of finite simple groups, completed in 2004.Since the mid-1980s, geometric group theory, which studies finitely generated groups as geometric objects, has become an active area in group theory.

\section{A random jump to Hilbert spaces}

\textbf{This jump is because we wish to show you the header of a non-chapter right page.}

The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions. A Hilbert space is an abstract vector space possessing the structure of an inner product that allows length and angle to be measured. Furthermore, Hilbert spaces are complete: there are enough limits in the space to allow the techniques of calculus to be used.

Hilbert spaces arise naturally and frequently in mathematics and physics, typically as infinite-dimensional function spaces. The earliest Hilbert spaces were studied from this point of view in the first decade of the 20th century by David Hilbert, Erhard Schmidt, and Frigyes Riesz. They are indispensable tools in the theories of partial differential equations, quantum mechanics, Fourier analysis (which includes applications to signal processing and heat transfer), and ergodic theory (which forms the mathematical underpinning of thermodynamics). John von Neumann coined the term Hilbert space for the abstract concept that underlies many of these diverse applications. The success of Hilbert space methods ushered in a very fruitful era for functional analysis. Apart from the classical Euclidean spaces, examples of Hilbert spaces include spaces of square-integrable functions, spaces of sequences, Sobolev spaces consisting of generalized functions, and Hardy spaces of holomorphic functions.

Geometric intuition plays an important role in many aspects of Hilbert space theory. Exact analogs of the Pythagorean theorem and parallelogram law hold in a Hilbert space. At a deeper level, perpendicular projection onto a subspace (the analog of "dropping the altitude" of a triangle) plays a significant role in optimization problems and other aspects of the theory. An element of a Hilbert space can be uniquely specified by its coordinates with respect to a set of coordinate axes (an orthonormal basis), in analogy with Cartesian coordinates in the plane. When that set of axes is countably infinite, the Hilbert space can also be usefully thought of in terms of the space of infinite sequences that are square-summable. The latter space is often in the older literature referred to as the Hilbert space. Linear operators on a Hilbert space are likewise fairly concrete objects: in good cases, they are simply transformations that stretch the space by different factors in mutually perpendicular directions in a sense that is made precise by the study of their spectrum.

% \begin{theorem}
%   This is theorem.
% \end{theorem}


\section*{Binary boolean operators}

\subsection*{\href{https://sicp.comp.nus.edu.sg/chapters/8}{Conjunction}}

\[
\textit{expression}_1 \ \textbf{\texttt{\&\&}} \ \textit{expression}_2
\]
stands for
\[
\textit{expression}_1 \ \textbf{\texttt{?}} \ \textit{expression}_2 \ \textbf{\texttt{:}}\ \textbf{\texttt{false}}
\]

\subsection*{\href{https://sicp.comp.nus.edu.sg/chapters/8}{Disjunction}}

\[
\textit{expression}_1 \ \textbf{\texttt{||}} \ \textit{expression}_2
\]
stands for
\[
\textit{expression}_1 \ \textbf{\texttt{?}}\ \textbf{\texttt{true}}\  \textbf{\texttt{:}}\ \textit{expression}_2
\]


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{geometry}
 \geometry{
 a4paper,
 total={170mm,257mm},
 left=20mm,
 top=20mm,
 }

\usepackage{
    algorithm, amssymb, booktabs, graphicx, mathtools,
    multirow, newtxtext, newtxmath, fancyvrb
}

\DeclarePairedDelimiter{\ceil}{\lceil}{\rceil}
\DeclarePairedDelimiter{\floor}{\lfloor}{\rfloor}

\title{ElGamal Experiments Report}
\author{Lucas Perin}
\date{May 2020}

\begin{document}

\maketitle

\section{Introduction}

Brett's questions:

% \begin{enumerate}
%     \item for a fixed $t$ and t-tuple $z$, are there arbitrarily large values of $p$ for which the LB from Theorem 8 is tight?
%     \item for a fixed $t$ and t-tuple $z$, are there arbitrarily large values of $p$ for which the UB from Theorem 8 is tight?
%     \item for a fixed $t$ and t-tuple $z$, are there arbitrarily large values $p$ for which $q$ is not even and the LB from Theorem 8 is tight?
%     \item for a fixed $t$ and t-tuple $z$, are there arbitrarily large values $p$ for which $q$ is not even and the UB from Theorem 8 is tight?
%     \item for a fixed $t$ and t-tuple $z$, are there arbitrarily large values $p$ for which $g$ is not even and the LB from Theorem 8 is tight?
%     \item for a fixed $t$ and t-tuple $z$, are there arbitrarily large values $p$ for which $g$ is not even and the UB from Theorem 8 is tight?
% \end{enumerate}

For a collection of primes $p$ (in millions), define a handful of primitive elements $g$ (preferably first five smallest and  10 additional spanning over the entire range). For each $g$ and with $2 \leq z \leq 15$, plot $\Delta_l = \lambda(z) - \operatorname{lower-bound}$ and $\Delta_u \operatorname{upper-bound} - \lambda(z)$.Repeat experiment with the following restrictions:

\begin{enumerate}
    \item plot only when $q$ is even;
    \item plot only when $q$ is odd;
    \item plot only when $g$ is even;
    \item plot only when $g$ is odd.
\end{enumerate}

In Theorem 8, we have that the occurrence of some arbitrary $t$-tuple in the ElGamal sequence is bounded as
$$ \floor*{\frac{g}{v}}^{t-1} \floor*{\frac{q}{v}} \leq \lambda(z) \leq \ceil*{\frac{g}{v}}^{t-1} \left(\floor*{\frac{q}{v}}+1\right).$$
We plot these bounds with the following code:
\begin{Verbatim}[frame=single]
def tuple_bound(p, v, g, t):
    """
    Bounds defined in Theorem 8 for t-tuples
    """
    q = floor(p/pow(g,t-1))
    lower = pow(floor(g/v),t-1)*floor(q/v)
    upper = pow(ceil(g/v),t-1)*(floor(q/v)+1)
    return lower, upper
\end{Verbatim}

\subsection{Expected bound explosion!}

We could probably stop plotting when (or let it happen a bit):

For the upper bound:

$$ g > \sqrt[t-1]{p}$$

$$ t > \log_g p  $$

for the lower bound:

$$ v > q $$

$$ v > g $$

\section{First Evaluating $t$-tuple bounds for $v = 2$}

In this section, we first investigated real counts and expected count with the bounds, by drawing a line over the plot. The line is truncated when the upper bound exceeds $1.3$ times the largest actual count of the entire plot. This is clear in the figures when $t$ grows. The experiments that follow do not use $\Delta_l$ or $\Delta_u$.

\subsection{Plotting all for each prime individually}

\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2/test_p1020137_v2_t2.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2/test_p1020137_v2_t4.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2/test_p1020137_v2_t6.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2/test_p1020137_v2_t3.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2/test_p1020137_v2_t5.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2/test_p1020137_v2_t7.png}
    \end{minipage}
\end{figure}

\subsection{Plotting when $q$ is odd for each prime individually}
\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qodd/test_p1020137_v2_t2_q_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qodd/test_p1020137_v2_t4_q_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qodd/test_p1020137_v2_t6_q_odd.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qodd/test_p1020137_v2_t3_q_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qodd/test_p1020137_v2_t5_q_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qodd/test_p1020137_v2_t6_q_odd.png}
    \end{minipage}
\end{figure}

\subsection{Plotting when $q$ is even for each prime individually}
\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qeven/test_p1020137_v2_t2_q_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qeven/test_p1020137_v2_t4_q_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qeven/test_p1020137_v2_t6_q_even.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qeven/test_p1020137_v2_t3_q_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qeven/test_p1020137_v2_t5_q_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2qeven/test_p1020137_v2_t7_q_even.png}
    \end{minipage}
\end{figure}

\subsection{Plotting when $g$ is odd for each prime individually}
\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2godd/test_p1020137_v2_t2_g_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2godd/test_p1020137_v2_t4_g_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2godd/test_p1020137_v2_t6_g_odd.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2godd/test_p1020137_v2_t3_g_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2godd/test_p1020137_v2_t5_g_odd.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2godd/test_p1020137_v2_t7_g_odd.png}
    \end{minipage}
\end{figure}

\subsection{Plotting when $g$ is even for each prime individually}
\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2geven/test_p1020137_v2_t2_g_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2geven/test_p1020137_v2_t4_g_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2geven/test_p1020137_v2_t6_g_even.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2geven/test_p1020137_v2_t3_g_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2geven/test_p1020137_v2_t5_g_even.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/1020137v2geven/test_p1020137_v2_t7_g_even.png}
    \end{minipage}
\end{figure}

\subsection{Box and 3d plots}
We may try to compose a set of experiments for the same prime all in one plot as shown below, using 3d plots. The tricky part is that the plotted data will have to be the delta mentioned in the first page --- namely, the distance of the bounds to original count. The main issue with this is that for each pair $(x,y) = (t, \lambda)$ we will have to plot two dots, one for the lower bound and one for the upper bound. Essentially, one dot may end up on top of each other or too far away, making it hard to read. I will investigate further.

\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/other/boxplot_p1020137_v2_t4.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/other/test_p1020137_v2.png}
    \end{minipage}
\end{figure}

\section{Delta plots}

In the following, we show how the bounds have a tendency to ``explode'' for some specific set of parameters.

\begin{figure}[H]\label{fig:delta_explode}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/delta_1020137v2_large/Bound delta p=1020137 v=2 and t=2.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/delta_1020137v2_large/Bound delta p=1020137 v=2 and t=4.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/delta_1020137v2_large/Bound delta p=1020137 v=2 and t=6.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/delta_1020137v2_large/Bound delta p=1020137 v=2 and t=3.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/delta_1020137v2_large/Bound delta p=1020137 v=2 and t=5.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/delta_1020137v2_large/Bound delta p=1020137 v=2 and t=7.png}
    \end{minipage}
\end{figure}

\subsection{Tight delta plots}

Now, we present plots for parameter sets where the bounds are tight, that is, close to the actual count. We do this by considering only parameters such that, for small $v$, $g < \lfloor \sqrt[t-1]{p} \rfloor$ and $t < \lfloor log_g p \rfloor $. For example, let $(t, p, v) = (7, 1020137, 2)$, then we are we have that the bound explodes when $g \geq 10$. The same happens if we set $t = 5$ for $g >= 31$.

\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_2.png}
    \end{minipage}
\end{figure}

As it shows in the examples, these parameter constraints do not correlate to the explosion of the lower bound necessarily. Clearly, since the lower bound cannot be less than zero, eventually it will be stable. However, we do expect it \emph{NOT} to be tight when $v > q$ or $v > g$. While the first requires $v$ to be large when $p$ grows, the second is may occur eventually (even as $p$ grows) for small value $v$. Observe the following display of this behaviour.
\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_3.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_4.png}
    \end{minipage}
\end{figure}

We also remark that the numbering on the $y$ axis makes it difficult to actually understand how tight the bound is. One would always consider the $\Delta$ close to zero to be tight, and zero to be exact. But when $\lambda(z)$ varies in range of hundreds of thousands, looking at $\Delta$ in range of thousands is not so bad. Let us look at the same examples but using a ratio value.

\begin{figure}[H]
    \begin{minipage}{.49\textwidth}
            \centering
             \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_1.2.png}
             \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_3.2.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_2.2.png}
            \includegraphics[width=\textwidth]{../plots/tuples_figures/explosion_example_4.2.png}
    \end{minipage}
\end{figure}

Brett and Daniel. These last two plots are weird. What happens here is that the lower bound is zero but the actual count is not. Let $\Delta_l$ denote the difference of actual count and the lower bound, namely $\min(\lambda(z)) - lb$, then I compute the ratio as follows:
$$ \Delta_l^r = \frac{\min(\lambda(z)) - lb/}{\min(\lambda(z))} = 1 - \frac{lb}{\min(\lambda(z)}$$
and for the upper bound $\Delta_u$
$$ \Delta_u^r = \frac{ub - \max(\lambda(z))}{\max(\lambda(z))} = \frac{ub}{\max(\lambda(z))} - 1 $$

This means that this ratio is bad when it comes to show the lower bound. Any tips?
\end{document}


\subsection{Overview}

Once we have produced this synthetic travel demand data, we model the regional circulation network for traffic assignment. Our integrated pipeline models networks as mathematical graphs consisting of a set of nodes \textit{N} connected to one another by a set of edges \textit{E} \citep{newman_networks:_2010,gastner_spatial_2006}. Specifically, the road network is modeled as a nonplanar directed multigraph with possible self-loops. The data come from OpenStreetMap.

This section describes how we acquire these data, construct a graph model of the network, process its topology, and calculate and impute relevant variables. Next it describes the process of calculating BPR coefficients and the assumptions baked into this calculation. Finally, it explains the process of converting the zone-based travel demand output from ActivitySim to a network node-based demand data set.

\subsection{Network creation, construction, and processing}

The network data in this modeling pipeline come from OpenStreetMap. OpenStreetMap is a public worldwide collaborative mapping project and web platform with over two million users. Anyone may edit or access OpenStreetMap data, but community oversight and standards exist to prevent significant vandalism or inaccurate edits \citep{jokar_arsanjani_openstreetmap_2015}. In general, OpenStreetMap data are of high accuracy and quality, particularly in the United States and Western Europe \citep{corcoran_analysing_2013,over_generating_2010,haklay_how_2010,maron_how_2015}. OpenStreetMap imported the 2005 TIGER/Line roads data set as a foundation, and numerous corrections and additions to these data have been made since \citep{willis_openstreetmap_2008}.

The network graph is constructed using OSMnx, an open-source Python package for working with OpenStreetMap data \citep{boeing_osmnx:_2017}. OSMnx is built on top of NetworkX, a Python package for network analysis developed by researchers at Los Alamos National Laboratory. OSMnx extends NetworkX's network analysis capabilities by working explicitly with spatial infrastructure networks and interfacing with OpenStreetMap's various APIs \citep{boeing_methods_2017}. It can automatically construct topologically-corrected nonplanar directed multigraphs constrained to any polygonal boundaries for anywhere in the world from OpenStreetMap data. OSMnx uses a multistep algorithm to simplify the topology of the graph so that it retains nodes only at intersections and dead-ends, as well as the full geometry of the simplified edges, as shown in Figure \ref{fig:simplification_before_after}.

\begin{figure}[htbp]
    \center
    \includegraphics[width=\textwidth]
    {graphics/simplification_before_after.png}
    \caption{A suburban street network from OpenStreetMap data before OSMnx topology simplification (left) and after (right), with nodes in black and edges in gray. Note that the full spatial geometry is retained as edge metadata even though the edge itself is compressed to a single pair of origin and destination nodes.}
    \label{fig:simplification_before_after}
\end{figure}

We use OSMnx to download the road network for the nine-county San Francisco Bay Area. We use the 2016 US Census Bureau TIGER/Line shapefile of United States counties to define the spatial extents of these nine counties: Alameda, Contra Costa, Marin, Napa, San Francisco, San Mateo, Santa Clara, Solano, and Sonoma. We calculate a convex hull around these geometries to obtain a single polygon for the spatial query---this prevents the query from discarding any network elements that fall within our study area but outside of a county's borders (for example, bridges over the San Francisco Bay).

Next, we use OSMnx to download the drivable road network within this convex hull. OSMnx processes the detailed metadata tags to identify which paths are drivable. It then constructs them into a graph. This initial graph contains 1.2 million nodes and 2.3 million edges.

Next we filter the road network to retain only tertiary roads and larger. In OpenStreetMap terminology, the road types we retain comprise: motorway, motorway\textunderscore link, trunk, trunk\textunderscore link, primary, primary\textunderscore link, secondary, secondary\textunderscore link, tertiary, tertiary\textunderscore link, unclassified, and road. The \enquote{link} types are necessary to retain the connectors (such as on-ramps and off-ramps) between certain roads. The \enquote{road} type is standardly used in the OpenStreetMap community as a null value. The \enquote{unclassified} type technically refers to the British-style roads hierarchy, in which \enquote{unclassified} is the level below \enquote{tertiary} – that is, quaternary. However, it sometimes is used inconsistently in the United States, so we retain it for completeness.

\begin{figure}[htbp]
    \center
    \includegraphics[width=\textwidth]
    {graphics/bay_area_network.png}
    \caption{The nine-county San Francisco Bay Area road network used in our modeling pipeline, created by OSMnx from OpenStreetMap data.}
    \label{fig:bay_area_road_network}
\end{figure}

After filtering out all edges not of these major types, we remove all isolated nodes then keep only the largest weakly-connected component of the graph. Finally, we simplify the graph using OSMnx then save it to disk as a GraphML file. GraphML is a standard, XML-based file format for storing and exchanging complete graph structure data. Our final graph contains 31,000 nodes and 66,000 directed edges representing 20,000 kilometers of roads across these 9 Bay Area counties, as seen in Figure \ref{fig:bay_area_road_network}.

\subsection{BPR coefficients calculation and assumptions}
To define the relationship between edge travel time and edge congestion we use the Bureau of Public Roads (BPR) congestion function. This BPR function has long been used by transportation engineers to model the increased time needed to traverse an edge when congestion is high. Once we have the final processed graph of the Bay Area road network, we calculate the BPR  curves' coefficients $a_0$, $a_1$, $a_2$, $a_3$, and $a_4$ for it. The BPR function is defined in Equation \ref{eq:bpr_function}.

\begin{equation}
    t_i = t^0_i (1 + \alpha (\frac{v_i}{c_i}) ^ \beta)
    \label{eq:bpr_function}
\end{equation}

where:

\begin{itemize}
    \item $t_i$ is the congested flow travel time on edge $i$
    \item $t^0_i$ is the free-flow travel time on edge $i$
    \item $v_i$ is the number of vehicles on edge $i$ per unit of time
    \item $c_i$ is the capacity (i.e., maximum number of vehicles) of edge $i$ per unit of time
    \item $\alpha$ linearly increases the congested travel time with regards to the volume:capacity ratio
    \item $\beta$ exponentially increases the congested travel time with regards to the volume:capacity ratio
\end{itemize}

The $\alpha$ parameter was assigned a value of 0.15 in the original BPR curve, and the $\beta$ parameter was assigned a value of 4 in the original BPR curve. We adopt these default $\alpha$ and $\beta$ parameter values in this integrated modeling pipeline, but note that they are not the only way to parameterize this function. For instance, the TRB's NHCRP Report 716: Travel Demand Forecasting Parameters and Techniques provides various coefficients estimated using the 1985 Highway Capacity Manual \citep[p.~75]{transportation_research_board_highway_1985,transportation_research_board_travel_2012}.

Given this function, we further define the coefficients $a_0$, $a_1$, $a_2$, and $a_3$ as:

\begin{itemize}
    \item $a_0 = t0_i$, that is, the free-flow travel time
    \item $a_1$, $a_2$, and $a_3$ are always zero
\end{itemize}

Then we calculate the coefficient $a_4$ as defined by Equation \ref{eq:bpr_a4_coefficient}:

\begin{equation}
    a_4 = t^0_i \frac{\alpha}{c_i ^ {\beta}}
    \label{eq:bpr_a4_coefficient}
\end{equation}

To calculate these coefficients, we require per-edge data about capacity and free-flow travel time. To assemble these data, we require information about edge lengths, free-flow speed, and number of lanes. We use OSMnx to calculate these edge lengths. OpenStreetMap contains sparse data per-edge on maximum permitted speed and number of lanes. When these data are missing, we infer or impute them as per the defaults\footnote{The authors wish to thank Madeleine Sheehan and Alexander Skabardonis for providing some of these values.} in Tables \ref{table:free_flow_speed_defaults} and \ref{table:capacity_defaults}.

First, for each edge that is missing number of lanes data, we impute the value based on its edge type (which OpenStreetMap always provides as metadata). We calculate this imputed value by taking the median value of all edges of this type. Second, for each edge that is missing maximum permitted speed data, we infer the free-flow speed from a look-up table via edge type and number of lanes (see Table \ref{table:free_flow_speed_defaults}). We can now calculate free-flow travel time per edge as a function of free-flow speed and edge length, as defined in Equation \ref{eq:free_flow_travel_time}:

\begin{equation}
    t^0_i = \frac{d_i}{s_i}
    \label{eq:free_flow_travel_time}
\end{equation}

where:

\begin{itemize}
    \item $t^0_i$ represents free-flow travel time on edge $i$, in units of seconds
    \item $d_i$ represents the length of edge $i$, in units of meters
    \item $s_i$ represents free-flow speed (i.e. the maximum permitted speed of travel) on edge $i$, in units of meters per second
\end{itemize}

Next, we infer each edge's vehicle capacity (per lane per hour) from a look-up table via edge type and number of lanes (see Table \ref{table:capacity_defaults}). We then convert this capacity per lane per hour value to units of capacity per edge per second. Finally, we use these data to calculate the values of the $a0$ and $a4$ coefficients for the BPR curve. These calculations demand a caveat: due to missing data on OpenStreetMap, we are forced to infer or impute variables on many edges. Our assumptions on parameter values and capacity and speed limit defaults propagate through to our BPR coefficients.

\begin{table}[htbp]
\centering
\caption{Free-flow speed defaults by edge type and number of lanes, in units of miles per hour.}
\label{table:free_flow_speed_defaults}
\begin{tabular}{ l r r r r }
    \hline
    Edge type & lanes=1 & lanes=2 & lanes=3 & lanes=4+ \\
    \hline
    motorway & 50 & 50 & 65 & 65 \\
    motorway\textunderscore link & 50 & 50 & 65 & 65 \\
    trunk & 45 & 45 & 45 & 45 \\
    trunk\textunderscore link & 45 & 45 & 45 & 45 \\
    primary & 30 & 30 & 30 & 30 \\
    primary\textunderscore link & 30 & 30 & 30 & 30 \\
    secondary & 25 & 25 & 25 & 25 \\
    secondary\textunderscore link & 25 & 25 & 25 & 25 \\
    tertiary & 20 & 20 & 20 & 20 \\
    tertiary\textunderscore link & 20 & 20 & 20 & 20 \\
    unclassified & 20 & 20 & 20 & 20 \\
    road & 30 & 30 & 30 & 30 \\
    \hline
\end{tabular}
\end{table}

\begin{table}[htbp]
\centering
\caption{Capacity defaults by edge type and number of lanes, in units of vehicles per lane per hour.}
\label{table:capacity_defaults}
\begin{tabular}{ l r r r r }
    \hline
    Edge type & lanes=1 & lanes=2 & lanes=3 & lanes=4+ \\
    \hline
    motorway & 1900 & 2000 & 2000 & 2200 \\
    motorway\textunderscore link & 1900 & 2000 & 2000 & 2200 \\
    trunk & 1900 & 2000 & 2000 & 2000 \\
    trunk\textunderscore link & 1900 & 2000 & 2000 & 2000 \\
    primary & 1000 & 1000 & 1000 & 1000 \\
    primary\textunderscore link & 1000 & 1000 & 1000 & 1000 \\
    secondary & 900 & 900 & 900 & 900 \\
    secondary\textunderscore link & 900 & 900 & 900 & 900 \\
    tertiary & 900 & 900 & 900 & 900 \\
    tertiary\textunderscore link & 900 & 900 & 900 & 900 \\
    unclassified & 800 & 800 & 800 & 800 \\
    road & 900 & 900 & 900 & 900 \\
    \hline
\end{tabular}
\end{table}

\subsection{Linking zone-based travel demand to the network}

The ActivitySim component of our integrated modeling pipeline produces an output dataset of zone-to-zone travel demand data. To model this travel demand (and in turn traffic assignment) on our road network, we must convert the trip origins and destinations from zones to network nodes.

We use the shapefile of Traffic Analysis Zones (TAZs) from the Bay Area Metropolitan Transportation Commission (MTC) to acquire zone spatial extents. Then we calculate the centroid of each zone polygon. Finally, we identify the network node nearest to each zone's centroid by taking the minimum of a vectorized calculation of great-circle distances from the centroid to every node in the network, using the haversine formula defined in Equation \ref{eq:haversine_formula}:

\begin{equation}
    \delta_{gc} = r\cdot \arccos{(\sin{\Phi_1}\cdot\sin{\Phi_2} + \cos{\Phi_1}}\cdot\cos{\Phi_2}\cdot\cos{|\lambda_1 - \lambda_2|})
    \label{eq:haversine_formula}
\end{equation}

where:

\begin{itemize}
    \item $\delta_{gc}$ represents the great-circle distance between the two points, in meters
    \item $r$ represents the radius of the Earth, in meters
    \item $\Phi_1$ and $\Phi_2$ represent the geographical latitudes of the two points, in radians
    \item $\lambda_1$ and $\lambda_2$ represent the geographical longitudes of the two points, in radians
\end{itemize}

Now that we have the road network, its per-edge BPR coefficients, and node-based travel demand data, we are ready to model traffic assignment.


\iffalse
It's unfair to say that mathematicians aren't real doctors, we perform surgeries all the time. In this class we'll introduce the notion of a topological manifold via simplicial (delta) complexes. Spend a day or two doing examples and go over several notions like orientation, cobordism and of course surgery.

Keywords: simplicial complex, manifold, orientation, cobordism, surgery

Type: Lecture
Homework: Recommended
Prereqs: None
\fi

\input{../preamble}
\rhead{\scshape Mathcamp 2017 : All things Manifoldy}
\begin{document}
\title{Heegard splittings of 3 manifolds}
\author{Apurva Nakade}
\thispagestyle{fancy}
\maketitle

It is difficult to imagine 3 dimensional manifolds as they require at least 4 dimensions to be embedded in. Instead we describe 3 dimensional manifolds using ``gluing diagrams'' which are obtained by gluing sides of solid polytopes in 3 dimensions.

\section{The three dimensional sphere $S^3$}
The 3 dimensional sphere is defined as $$S^3 = \{ (x,y,z,t) : x^2 + y^2 + z ^{2} + t ^{2} = 1  \}$$ If we consider the standard sphere $S^2$ of radius 1 and intersect it by a vertical plane $z = c$ we get an $S^1$ if $-1 < c < 1 $, a point if $c = \pm 1$ and an empty set otherwise. Similarly if we intersect $S^3$ with $t=c$ we get a sphere $S^2$ if $ -1 < c < 1$, a point if $c=1$ and an empty set otherwise.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.90\linewidth]{../images/SphereTime}
	\caption{$S^3$ as a sequence of spheres}
\end{figure}

Another way to visualize $S^3$ is by noticing that if we remove one point from $S^3$ we get the space $\R^3$. Conversely this implies that $S^3$ is the one point compactification of $\R^3$ i.e. $S^3 = \R^3 \cup \{ *\}$.

\subsection{Hemispheres and solid tori}
Consider the two hemispheres
\begin{align*}
	S^3_+ & = \{ (x,y,z,t) : x^2 + y^2 + z ^{2} + t ^{2} = 1 , t \ge 0 \} \\
	S^3_- & = \{ (x,y,z,t) : x^2 + y^2 + z ^{2} + t ^{2} = 1 , t \le 0 \}
\end{align*}
Both $S^3_+$ and $S^3_-$ are homeomorphic to the three disc
\begin{align*}
	D^3 & = \{ (x,y,z) : x^2 + y^2 + z ^{2} \le 1
\end{align*}
via the projection maps $S^3_+ \rightarrow D^3$ and $S^3_- \rightarrow D^3$. Hence we can think of $S^3$ as two 3 dimensional discs being glued together along the boundary.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.5\linewidth]{../images/SphereDiscs}
	\caption{$S^3$ as two 3 discs $S^3_+$, $S^3_-$ glued together}
\end{figure}

We can perform surgery on $S^3_+$ and $S^3_-$ by removing and adding a solid cylinder respectively.
\begin{figure}[H]
	\centering
	\includegraphics[width=0.25\linewidth]{../images/HeegardSphere}
	\caption{$S^3$ as two solid tori glued together}
\end{figure}

We can also remove and add multiple solid cylinders to get
\begin{figure}[H]
	\centering
	\includegraphics[width=0.25\linewidth]{../images/HeegardSphere2}
	\caption{$S^3$ as two solid genus $2$ surfaces glued together}
\end{figure}

These are called Heegard splittings of $S^3$.

\subsection{Heegard splittings of 3 manifolds}
Let $M$ be a 3 manifold and suppose we have a ``gluing diagram'' of $M$ consisting of multiple polytopes. However unlike for 2 manifolds, we do not allow different sides of the same polytope to be glued to each other.

For example, $S^3$ has a gluing diagram consisting of two solid cubes with corresponding sides glued.
\begin{figure}[H]
	\centering
	\includegraphics[width=0.25\linewidth]{../images/SphereGluingDiagram}
	\caption{$S^3$ as a sum of two solid cubes}
\end{figure}

Let $X$ be the set of edges of gluing diagram which have been thickened. And let $Y$ be the complement of $X$ in $S^3$, $Y = S^3 \setminus X$.
\begin{figure}[H]
	\centering
	\includegraphics[width=0.15\linewidth]{../images/Heegard1}
	\includegraphics[width=0.15\linewidth]{../images/Heegard2}
	\caption{Heegard splitting of $S^3$, both $X$ and $Y$ are solid genus 5 surfaces.}
\end{figure}

It is not hard to see that both $X$ and $Y$ are solid surfaces of the same genus. This shows that every 3 dimensional manifold $M$ which has a ``gluing diagram'' can be obtained by gluing two solid surfaces of the same genus. This is called the \textbf{Heegard splitting} of the manifold $M$.

\section{Exercises}

\begin{exercise}
	Construct ``gluing diagrams'' for $T \times S^1$ and $S^2 \times S^1	$ using the gluing diagram of $T$ and $S^2$ and thinking of $S^1$ as two endpoints of $[0,1]$ glued together.
\end{exercise}

\begin{exercise}
	Construct Heegard splittings for $T \times S^1$ and $S^2 \times S^1$.
\end{exercise}

\begin{exercise}
	Why does this procedure not work for two dimensional manifolds?
\end{exercise}

\end{document}

01 All things manifoldy/05 - Heegard splittings of 3 manifolds.tex


\section{Real analysis}\label{sec:real_analysis}

Real analysis is concerned with \hyperref[def:function]{functions} with values in \hyperref[def:euclidean_space]{Euclidean spaces}. These include the real line \( \BbbR \), plane \( \BbbR^2 \) or space \( \BbbR^3 \), which are the classic spaces from \fullref{subsec:analytic_geometry_in_the_plane}.

Important aspects of real analysis include:
\begin{itemize}
  \item \hyperref[def:local_continuity]{Continuity}, which we discuss extensively in \fullref{sec:general_topology} and \fullref{sec:metric_spaces}, and aggregate in \fullref{subsec:topology_of_euclidean_spaces} and \fullref{subsec:real_convergence}.

  \item \hyperref[def:differentiability]{Differentiability}, which we discuss in \fullref{subsec:differentiability} from \fullref{sec:functional_analysis} and discuss briefly in \fullref{subsec:differentiability}, and, in a generalized form, in \fullref{subsec:nonsmooth_derivatives}.

  \item Integration, which we discuss in \fullref{subsec:riemann_integration} and \fullref{subsec:line_integrals}.

  \item Special functions, which we delegate to \fullref{sec:complex_analysis} via \fullref{subsec:special_functions}.

  \item Convex functions, which we discuss in \fullref{subsec:convex_functions}.
\end{itemize}


\section{Union-Find Partitioned-Growth decoder}\label{sec:ufbb}
We describe the \emph{Union-Find Partitioned-Growth} decoder in this section. We first introduce the concept of the potential matching weight in \Cref{sec:matchingweight}. The Union-Find Partitioned-Growth decoder uses the potential matching weight as a heuristic to prioritize partitioned growth of clusters. We introduce the node-tree data structure required for this decoder in \Cref{sec:nodeset}, and describe how to approximate the potential matching weight leveraging the node-tree in \Cref{sec:paritydelay}. In \Cref{sec:grownodetrees,sec:nodejoin,sec:inversion}, we show how to approximate efficiently the potential matching weight. The algorithm is described in \Cref{sec:pseudocode}.

\input{sec_algo_a_pwm.tex}
\input{sec_algo_b_nodetree.tex}
\input{sec_algo_c_dfs.tex}
\input{sec_algo_e_pseudocode.tex}
\input{sec_algo_d_grow.tex}
\input{sec_complexity}
\input{sec_algo_f_inversion.tex}


\documentclass{discussion}

\usepackage{framed}
\usepackage[position=b]{subcaption}
\DeclareMathOperator{\Parents}{Parents}
\DeclareMathOperator{\NonDesc}{NonDesc}
\newcommand{\G}{\mathcal{G}}
\newcommand{\I}{\mathcal{I}}

\begin{document}

% Lecture Info
\lecture{7}{Probabilistic Graphical Models}{Benjamin R. Bray}

%\begin{exercise}
%\begin{example}

%%%% INTRODUCTION %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

This week in lecture, we introduced \textbf{Probabilistic Graphical Models}, which are a very elegant and powerful way to model the \textit{independence structure} of complex data.  So far, we have covered only simple probabilistic models like linear regression, logistic regression, and Naive Bayes.   Graphical models provide us with a language for expressing more complex probabilistic models.  Here are a few advantages of using graphical models:

\begin{itemize}
\item \textbf{Transparency.}  All modeling assumptions are explicitly encoded in the graph structure of a probabilistic model and the accompanying distributions.
\item \textbf{Interpretability.} compared to i.e. neural networks or random forest models, whose learned parameters are hard to interpret.  In addition, because graphical models are \textit{probabilistic}, we can easily determine confidence intervals for our estimates, which matters when machine learning algorithms are used to make important real-world decisions (diagnosis, trading, self-driving cars, etc.).
\item \textbf{Modularity.}  Graphical models can be easily combined and modified, and it is easy to adapt old models to new applications.  Sometimes it's even useful to "nest" one graphical model inside another!
\end{itemize}

\begin{framed}
\textbf{Fair Warning:} We'll be using probability extensively in this section of the course, and it's important that you have a good grasp of the basics!  Conditional independence and Bayes' rule will be especially important.  Check Piazza for a list of concepts we expect you to know.
\end{framed}

%%%% CONDITIONAL INDEPENDENCE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Bayesian networks, which we define below, can be interpreted as a compact way of encoding conditional independence assumptions.  First, recall the concept of independence:

\begin{definition}
We say events or variables $X$ and $Y$ are \textbf{independent}, written $(X \perp Y)$, if
    \begin{equation*}
    P(X \mid Y) = P(X) \quad \text{or} \quad P(Y) = 0
    \end{equation*}
\end{definition}

While independence is a useful property, it is not often that we encounter two independent events in practice.  More commonly, two events are independent only if we have information about a third event.

\begin{definition}
We say events or variables $X$ and $Y$ are \textbf{conditionally independent} given $Z$, which we write as $(X \perp Y \mid Z)$, if
    \begin{equation*}
    P(X|Y,Z) = P(X|Z)
    \end{equation*}
\end{definition}

% Example:  Stanford/MIT
\begin{example} (Koller, \textit{Probabilistic Graphical Models})
For example, suppose we want to reason about the chance that a student is accepted to graduate studies at Stanford or MIT.  Denote by \textit{Stanford} the event "admitted to Stanford" and by \textit{MIT} the event "admitted to MIT."  In most reasonable distributions, these two events are not independent.  If we learn that a student was admitted to Stanford, then our estimate of her probability of bveing accepted at MIT is now higher, since it is a sign that she is a promising student.

Now, suppose both universities base their decisions only on the student's grade point average, and we know that our student has a GPA of A.  In this case, we might argue that learning that the student was admitted to Stanford should not change the probability that she will be admitted to MIT:  her GPA already encodes all relevant information.  Formally, we have
    \begin{equation*}
    P(MIT \mid Stanford, GradeA) = P(MIT \mid GradeA)
    \end{equation*}
In this case, we say \textit{MIT} is conditionally independent of \textit{Stanford} given \textit{GradeA}.
\end{example}

\begin{exercise}
Let $X,Y,Z,W$ be events or random variables.  Show that conditional independence satisfies the following properties:
    \begin{itemize}
    \item \textbf{Symmetry:}  $(X \perp Y \mid Z) \implies (Y \perp X \mid Z)$
    \item \textbf{Decomposition:} $(X \perp Y,W \mid Z) \implies (X \perp Y \mid Z)$
    \item \textbf{Weak Union:} $(X \perp Y,W \mid Z) \implies (X \perp Y \mid Z,W)$
    \item \textbf{Contraction:} $(X \perp W \mid Z,Y) \wedge (X \perp Y \mid Z) \implies (X \perp Y,W \mid Z)$
    \end{itemize}
\end{exercise}

%%%% BAYESIAN NETWORKS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Bayesian Networks}

A Bayesian network is one type of graphical model that encodes a set of independence assumptions among a set of random variables with a graph.  At a high level, the structure of a Bayesian network tells us which relationships between variables we believe to be most important for solving the problem at hand.  Here is the formal definition of a Bayesian network from lecture:

\begin{definition}
A \textbf{Bayesian network} $\G$ is a directed acyclic graph whose nodes represent random variables $X_1, \dots, X_n$.  Let $\Parents(X_k)_\G$ denote the parents of $X_k$ in $\G$ and $\NonDesc(X_k)_\G$ denote the variables in the graph that are not descendants of $X_k$.  Then $\G$ encodes the following set of conditional independence assumptions, called the \textbf{local independencies}, and denoted by $\I_{\ell}(\G)$.
    \begin{equation*}
    \text{For each variable $X_k$}:  (X_k \perp \NonDesc_\G(X_k) \mid \Parents_\G(X_k))
    \end{equation*}
\end{definition}%

\noindent This definition looks scary and technical, but over the next few weeks we will work with many examples and working with these graphs will become second-nature.  For now, whenever you get confused, remember this one important fact:

\begin{framed} \centering
Every variable $X_k \in \G$ is conditionally independent of its nondescendants, given its parents.
\end{framed}

% Exercise:  Cycles
\begin{exercise}
Why must we require that the graphs be acyclic?  What happens if there are cycles?
\end{exercise}

% Exercise:  Parents & Nondescendants
\begin{exercise}
Identify the parents and nondescendants for each node in every graph in Figure \ref{fig:pgms}.
\end{exercise}

%% Naive Bayes --------------------------------------------------------------------------------
\subsection{Example: Naive Bayes}

Naive Bayes is the simplest nontrivial example of a graphical model.  The corresponding Bayesian network, shown in Figure \ref{fig:naive-bayes}, was briefly discussed in lecture, but let's go through it a little more carefully.

% Exercise:  Naive Bayes Local Independencies
\begin{exercise}
List all local independencies $\I_\G$ encoded by the graph in Figure \ref{fig:naive-bayes}.
\end{exercise}

\begin{example}
Show directly that the independence assumptions made by Bayesian network in Figure \ref{fig:naive-bayes} correspond exactly the Naive Bayes assumption.
\end{example}

\begin{figure*}[h!tb]
    \centering
    \begin{subfigure}[t]{0.33\textwidth}
        \centering
        \includegraphics[height=1.2in]{images/naive-bayes}
        \caption{Naive Bayes \label{fig:naive-bayes}}
    \end{subfigure}%
    ~
    \begin{subfigure}[t]{0.33\textwidth}
        \centering
        \includegraphics[height=1.2in]{images/topological}
        \caption{Topological Sort}
    \end{subfigure}%
    ~
    \begin{subfigure}[t]{0.33\textwidth}
        \centering
        \includegraphics[height=0.3in]{images/markov-chain}
        \caption{Markov Chain \label{fig:markov-chain}}
    \end{subfigure}%
    \caption{\label{fig:pgms} Three simple Bayesian networks.}
\end{figure*}

%% Factorization Theorem ----------------------------------------------------------------------
\pagebreak
\section{From Networks to Factorizations}

\begin{theorem}[Factorization]
Suppose the network $\G$ is an I-map for $P$, that is, the distribution $P$ satisfies the independence assumptions contained in $\G$.  $X_1, \dots, X_n$ is any \textbf{topological sorting} of the nodes of $\G$, then $P$ \textbf{factorizes} according to $\G$ in the following way:
    \begin{equation*}
    P(X_1,\dots,X_n)
    = \prod_{k=1}^n P(X_k \mid \Parents_\G(X_k))
    \end{equation*}
\end{theorem}

\begin{proof}
Once we have topologically sorted nodes $X_1, \dots, X_n$, the chain rule for probability yields
    \begin{equation*}
    P(X_1,\dots,X_n)
    = P(X_1) P(X_2 \mid X_1) \cdots P(X_n | X_1, \dots, X_{n-1})
    = \prod_{k=1}^n P(X_k \mid X_1, \dots, X_{k-1})
    \end{equation*}
The nodes are topologically sorted, so all parents of $X_k$ are in the range $X_1, \dots, X_{k-1}$, and the rest of these nodes must be nondescendants.  Because $X_k$ is conditionally independent of its nondescendants given its parents, we can delete the nondescendants frmo this list when conditioning.  Therefore, $P(X_k \mid X_1, \dots, X_{k-1}) = P(X_k \mid \Parents_\G(X_k))$.  For a more rigorous proof, see the lecture notes.
\end{proof}

% Example:  Naive Bayes
\begin{example}
After establishing the factorization theorem, it is clear that Figure \ref{fig:naive-bayes} is indeed Naive Bayes.
\end{example}

The factorization theorem is incredibly useful for building \textbf{generative models}, which describe our assumptions about how the observed data is generated.  Using the factorization theorem, we can interpret any Bayesian network as a generative model.

%% Example:  Markov Chains -------------------------------------------------------------------
\subsection{Example:  Markov Chains}

A \textbf{Markov chain} is a simple probabilistic model used for describing sequences, usually discretized in either time or space.  The corresponding graphical model is shown in Figure \ref{fig:markov-chain}.  Markov chains are often called \textit{local} or \textit{memoryless} models because if the value at position $k$ in the chain is known, then past values do not influence our predictions of future values.  While not entirely true in general, this assumption can simplify inference in complex settings like weather forecasing, finance, and natural language processing.  Usually, we assume all variables in a Markov chain have the same type.

\begin{exercise}
List all local independence assumptions made by the Markov Chain model in Figure \ref{fig:markov-chain}.
\end{exercise}

\begin{exercise}
By the factorization theorem, the joint distribution for a Markov chain factorizes as
    \begin{equation*}
    P(X_1,\dots,X_n)
    = \prod_{k=1}^n P(X_k \mid \Parents_\G(X_k))
    = \prod_{k=1}^n P(X_k \mid X_{k-1})
    \end{equation*}
Using this factorization, interpret Markov Chains as a generative model.  It may help to think of the random variables $X_1, \dots, X_n$ as words in a sentence, a robot's position at time $t$, or the weather forecast $\{ Rainy, Sunny \}$ on day $d$.  Draw a transition diagram for the model.
\end{exercise}

%% From Factorizations to Networks -----------------------------------------------------------
\section{From Factorizations to Networks}

If we know the independence structure of a joint distrubution over some variables, we can write down a corresponding Bayesian network that encodes these independencies.  Remember that there are multiple valid answers!  The complete graph on $n$ nodes is a valid Bayesian network for any variables $X_1, \dots, X_n$.

\begin{exercise}
Draw a Bayesian network for each factored joint distribution below.
    \begin{itemize}
    \item $P(A,B,C,D)$
    \item $P(A,B,C) = P(A)P(B|A)P(C|B)$
    \item $P(A,B,C,D,E) = P(A)P(B)P(C|A,B)P(D)P(E|C,D)$
    \end{itemize}
\end{exercise}

We can also be more concrete!  Suppose we are given a generative model, complete with distributions that tell us how to generate data.  We can use this information to come up with a Bayesian network that fits the independence assumptions of the generative model.

\begin{example}
TODO
\end{example}

\end{document}


\section{Types}

\subsection{Use of Generics}
\begin{lstlisting}[language=Scala]
//+A means, Queue[A] is covariant. If A is a subtype of B, then Queue[A] is also a subtype of Q[B]
//Define a constructor in this class which declares a field of type List[A] that can be used to set the element list. Lists of type List are immutable
// private (elements: List[A]) is the private primar constructor.
class Queue[+A] private (elements: List[A]) {
  def dequeue(): (A, Queue[A]) = {
    return (elements.head, new Queue(elements.tail));
  }

  //Lower Bound: T should be a subtype of A, or A itself
  def enqueue[T >: A](x: T) : Queue[T] = {
    //You can use method :+ to append an element to a list.
    return new Queue(elements :+ x);
  }
}

object Queue{
  //A parameter of type A* stands for an arbitrary number of arguments
  def apply[A](elements: A*) : Queue[A] ={
    new Queue(elements.toList);
  }
}
\end{lstlisting}

\clearpage
\subsection{Pair}
\begin{lstlisting}
// immutable:
class Pair[+T,+S](val first: T, val second: S) {
  def replaceFirst[U >: T](newFst: U): Pair[U,S] = new Pair(newFst, second)
  def replaceSecond[U >: S](newSnd: U): Pair[T,U] = new Pair(first, newSnd)
}

// mutable:
//private[this] var f means that the field f is only accessible from methods of the "this" object only, not from other objects of class PairM.
class PairM[T,S](private[this] var f: T, private[this] var s: S) {
  def first = f
  def second = s
  def replaceFirst (newFst: T) { f = newFst }
  def replaceSecond (newSnd: S) { s = newSnd }
}

//Define a generic method swap that takes an immutable instance of class Pair and that returns a new instance of class Pair with exchanged parts.
object Pair {
  def swap[F, S](p: Pair[F,S]): Pair[S,F] = new Pair(p.second, p.first)
  // result type: Pair[S,T]
}

//Define an analog generic method for mutable instances of class PairM that executes the swap in-place on the argument, i.e. which does not create a new object.
object PairM {
  def swap[S](p: PairM[S, S]) : Unit = {
    val first = p.first
    val second = p.second
    p.replaceFirst(second)
    p.replaceSecond(first)
  }
// This doesn't return anything. Instead, you are able to call this method only if your PairM consists of two elements with the same type. It will swap the elements of the object.
}
\end{lstlisting}

\textbf{The type parameters of the class PairM cannot be defined covariant. Give an example that shows that it would not be type-safe to declare the class PairM covariant.} \\

The mutable pair allows to change the values on the existing object itself. If the parameter of the class would be covariant, one would be allowed to also store values of subtypes of the existing pair and change the type of the existing object. This would not be type safe anymore.

\textbf{Show analogously with a counterexample that the type parameters cannot be declared contravariant either.} \\

Like in the answer of the covariant question: One would be able to store supertypes of the existing object's parameters and this would change the type of the object itself.

\clearpage


\subsection{Doobit™}
\label{sec:doobit}

Working on the Arduino with signals very quickly turned into a constant battle
of resolution and storage, limited by its relatively tiny memory. After msot
optimizations we went through, managing memory manually very closely and
ensuring sequentialized execution contexts, we were teribly bottlenecked by the
storage. To work around this physical limit, we manage each byte of the stored
signal manually and instead of one, store two signal data points in every byte.
This reduces our data resolution by way of being limited to 4 bits, but grants
us unmatched robustness to noise by comparison, by doubling possible data
processing capabilities. The system has been affectionately named Doobit.

\begin{lstlisting}[language=C++]
// bit mask storage
struct doobit{
    uint_fast8_t data;

    doobit(int8_t x = 0, int8_t y = 0){ // handles all our casts too
        this->storelow(x);
        this->storehigh(y);
    }

    void storelow(int8_t);
    void storehigh(int8_t);

    int8_t getlow();
    int8_t gethigh();

    int16_t operator*(doobit& b);
};
\end{lstlisting}

A \texttt{struct} provides us fast access with very little memory and
performance overhead, something that only becomes more and more negligible as we
increase our (now doubled!) data numbers.

Unpacking the code block, \texttt{data} is the actual storage, an unsigned 8-bit
type, chosen this way to avoid accidental signed interpretation and any unwanted
processing by the compiler. Reliance on unsigneds in a case such as this is
common even within the compiler itself, where it would convert signeds to
unsigneds before evalutation to avoid ambiguity. In particular, a two's
complement could scramble the data beyond recognition quite quickly.

The \texttt{fast} part of \texttt{uint\_fast8\_t} indicates to the compiler that
we are looking for a type that is atleast 8 bits, but is the fastest among
those. On an Arduino Uno, of course, this is just the 8 bit unsigned integer,
but on more exotic embedded systems this could end up being a 9 or 10 bit type,
if not more. This notation allows for some compatibility between systems, though
as is with embedded systems, one would try to create more efficient structures
that exploit the architecture of those systems.

The storage of the signal is handled by the \texttt{storelow()} and
\texttt{storehigh()} functions, which store data into the 4 least significant
and 4 most significantbits of the storage \texttt{data} respectively. We look at
one of the functions:

\begin{lstlisting}[language=C++]
void doobit::storelow(int8_t x){
	this->data &= 0b11110000;  // clear for storage

	x += 7; // remove signed component
	assert((!(x & 0b11110000)) && "doobit range violation");

	this->data |= x;
}
\end{lstlisting}

This function takes in a value \texttt{x}, to be stored in the lower 4 bits of
\texttt{data}, and preparing for it, clears the lower 4 bits via an \texttt{AND}
operation with a bitmask \texttt{11110000}. Following this, a manual conversion
is made to ensure \texttt{x} is unsigned. The operation moves \texttt{x}'s
previous range, -7 to +7, to now 0 to 14, with 15 remaining generally unused,
rushing to somehow occupy this position leads to little benefit and after much
trial to squeeze extra storage out of the system, was abandoned.

The \texttt{assert} exists merely for testing purposes to ensure our data can
indeed fit in 4 bits. Finally, having ensured \texttt{x} has only its lower bits
populated, an \texttt{OR} operation with the storage inserts it in.

The \texttt{storehigh()} function works in a similar manner, albeit with a bit
shift and an inverted bit mask to work on the 4 higher bits.

Now remains the issue of retrieving data from this storage, and is done as very
much the inverse of how it is stored:

\begin{lstlisting}[language=C++]
int8_t doobit::getlow(){
    int8_t x = (data & 0b00001111); // bitmask
    return (x-7); // reinsert sign
}
\end{lstlisting}

The \emph{unrequired} data is removed via a bit mask, and would be moved
rightwards via a bit shift in the case of \texttt{gethigh()}, and its signed
nature is restored by shifting it back to the original range.

The conversion to unsigned is quite important to have to not manage the carry
bits arising from a two's complement operation. The number -7 in C++ could
ambiguously be coming from an \texttt{int} (internally \texttt{int32\_t}) in
which case the signed bit is the 32nd, while it could also be coming from an 8
or 16 bit type, making the signed bit unclear and its extraction slow and
painful. Asking for a change of range was the fastest of the operations we
tested, included several bitwise only operations.

The storage could be optimized for several arithmetic operations, but for our
case in particular, for the correelation setups, we need only multiplication. As
of now, this is done simply by retrieving the numbers individually and
multiplying them pairwise. This is done as opposed to attempting bitwise
operations as (1), multiplication operations are quite optimized on a circuit
level in modern processors anyway, and more importantly (2), 4 bit being such a
limited storage type, would cause an overflow for most possible multiplication
operands.

\begin{lstlisting}[language=C++]
int16_t operator*(doobit& b){
    auto highprod = this->gethigh() * b.gethigh();
    auto lowprod = this->getlow() * b.getlow();
    return (highprod + lowprod);
}
\end{lstlisting}

This definition is obviously not compatible with all arithmetic operations, but
quite specific for our limited operations, kept this way to prevent
overcomplication of the rather heavily utilized routine.

The full code for all the functions may as always be found
\hyperref[sec:arduinocode]{in the Appendix}.


\documentclass{stdlocal}
\begin{document}
\section{Introduction} % (fold)
\label{sec:introduction}

For various mathematical and physical problems, there exists no feasible, deterministic algorithm to solve them.
Especially, the simulations of physical systems with many coupled degrees of freedom, such as fluids and global illumination, seem to be difficult to compute due to their high dimensionality.
Instead, a class of randomized algorithms, called Monte Carlo methods, are used to approximate the actual outcome.
Monte Carlo methods rely on repeated random sampling to obtain a numerical result.
Hence, they are not bound to the \enquote{curse of dimensionality} and are able to quickly evaluate complex equations.
\autocite{pharr2016,bauke2007,mueller2012,landau2014}\footnote{In this thesis, citations concerning a whole paragraph will be given after the last sentence.}

To obtain precise answers with a small relative error, Monte Carlo algorithms have to use a tremendous amount of random numbers.
But the usage of truly random numbers generated by physical processes consists at least of two drawbacks.
First, the output of the algorithm will be non-deterministic and, as a result, untestable.
Second, the generation of truly random numbers is typically based on a slow process and consequently reduces the performance of the entire program.
For that reason, Monte Carlo algorithms usually use so-called pseudorandom number generators (PRNGs).
PRNGs generate a sequence of numbers based on a deterministic procedure and a truly random initial value as seed.
The sequence of numbers is not truly random but fulfills several properties of truly random sequences.
\autocite{bauke2007,intel-drng,lecuyer1994,lecuyer2015,lecuyer2017,volchan2002}

The structure of Monte Carlo methods causes a program to spend most of its time with the construction of random numbers.
Even the application of PRNGs does not change that.
Today's computer processors provide functionality for the parallel execution of code in different ways, mainly single instruction multiple data (SIMD) and multiple instruction multiple data (MIMD).
Hence, to efficiently use the computing power of a processor for Monte Carlo algorithms not bound by memory, PRNGs have to be vectorized and parallelized to exploit such features.
Whereas parallelization takes place at a high level, vectorization has to be done by the compiler or manually by the programmer at a much lower level.
The implementation of PRNGs constrains automatic vectorization due to internal flow and data dependencies.
To lift this restriction, a manual vectorization concerning data dependence and latencies appears to be the right way.
Often, there are already manually vectorized algorithms that need to use random numbers, such as in \textcite{blacher2018}.
As a consequence, vectorizing PRNGs is even essential to provide the utilities for these implementations.
\autocite{lecuyer2017,bauke2007,hennessy2019,patterson2014,barash2017,dolbeau2016,intel-optimization-reference,fog2019a,fog2019b,fog2019c,fog2019d,fog2019e}

The C++ programming language is an adequate candidate for the development of vectorized PRNGs.
It is one of the most used programming languages in the world and can be applied to small research projects as well as large enterprise programs.
C++ allows for the high-level abstraction of algorithms and structures.
On the other hand, it is capable of accessing low-level routines to exploit special hardware features, like Intel's Streaming SIMD Extensions (SSE), Intel's Advanced Vector Extensions (AVX), and threads.
A typical C++ compiler is able to optimize the code with respect to such features automatically.
But we as programmers are not bound to this and can manually optimize the code further.
Every three years, a new standard is published, such as the latest language specification called C++17.
The language is evolving by its communities improvements and therefore it keeps to be modern.
On top of this, other languages, such as Python, usually provide an interface to communicate with the C programming language.
Through the design of an efficient implementation in C++, we can easily add support for other languages as well by providing a standard C interface.
\autocite{stroustrup2014,meyers2014,vandevoorde2018,intel-intrinsics-guide,reddy2011,cppreference,isocpp}

Lots of PRNGs have been implemented by different libraries with different Application Programming Interfaces (APIs), such as the STL of C++, \citetitle{boost}, \citetitle{intel-mkl}, and \citetitle{rngavxlib}.
For example, the STL and \citetitle{boost} provide a large set of robust PRNGs which are not vectorized but well documented.
Their API makes them likely to be used but shows some flaws.
It does not allow to explicitly use the vectorization capabilities of a PRNG and gives you a bad default seeding.
The use of standard distributions is difficult and not adjustable.
\citetitle{rngavxlib} provides highly optimized, open-source, vectorized implementations with bad documentation and difficult-to-use code.
The \citetitle{intel-mkl}, too, provides vectorized PRNGs, but is not available as open-source and uses difficult-to-use interfaces.
Besides complete libraries, there are also standalone implementations of vectorized PRNGs in \textcite{lemire-pcg} and \textcite{lemire-xorshift}.
So far, there has been no easily-accessible, portable, and open-source software library written in C++ which gives a coherent, easy-to-use and consistent interface for vectorized PRNGs.
\autocite{intel-mkl,boost,guskova2016,oneill-blog-api,gcc-libstdcpp}

In this thesis, precisely in sections \ref{sec:design_of_the_api} and \ref{sec:implementation}, we develop a new library, called \citetitle{pxart}, in the C++ programming language \footfullcite{pxart}.
\citetitle{pxart} vectorizes a few already known PRNGs which partly do not exist as vectorized versions and provides a new API for their usage to accommodate the disadvantages of the random facilities of the STL.
The library itself is header-only, open-source, and can be found on GitHub.
It is easily installable on every operating system.
% Additionally, we compare the performance of our vectorized PRNGs to other already accessible implementations in Boost, Intel MKL, Lemire, RNGAVXLIB and others.
% The performance is measured by speed, code size, memory size, complexity, and random properties.
The necessary theoretical background to understand the design- and the implementation-specific aspects is given in the sections \ref{sub:preliminaries}, \ref{sub:simulation_in_physics_and_mathematics}, and \ref{sec:pseudorandom_number_generators}.
Here, we will give a brief introduction to computer architecture, Monte Carlo methods and random sequences.
Especially, section \ref{sec:pseudorandom_number_generators} will give a mathematically rigorous introduction to PRNGs to make the reader familiar with the topic.
Section \ref{sec:previous_work} refers to the previous work concerning the vectorization of PRNGs.
% Meanwhile, we apply the implementations to an example Monte Carlo simulation.
% For this, a small test framework is implemented which allows us to easily test and evaluate PRNGs with respect to stated measures.
To show that our vectorized implementations result in a performance improvement without reducing the statistical quality of generated random numbers, we have used standard test suites, namely \citetitle{dieharder} and \citetitle{testu01-lib}, and created benchmarks to measure the actual speed-up in section \ref{sec:testing_framework}.
At the end, we have designed an application based on Monte Carlo algorithms to simulate essential parts of photon propagation and global illumination to show the usage of our PRNGs in the context of physics.
In the sections \ref{sec:evaluation} and \ref{sec:conclusions}, the evaluation is shown followed by a discussion dealing with further improvements.
\autocite{dieharder,testu01-lib,testu01}

% section introduction (end)
\end{document}


    \filetitle{transpose}{Transpose each page of matrix with names rows and columns}{namedmat/transpose}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
X = transpose(X)
X = X.'
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{X} {[} namedmat {]} - Input matrix or array with named rows
  and columns.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{X} {[} namedmat {]} - Transpose of the input matrix; if it is
  more than 2-dimensional, each page of the matrix is transposed.
\end{itemize}

\paragraph{Description}\label{description}

\paragraph{Example}\label{example}


\documentclass[11pt]{article}
\usepackage{amsmath,amssymb,graphicx}
\usepackage{../setspace}
\addtolength{\textwidth}{1.5in}
\addtolength{\hoffset}{-1in}
\addtolength{\textheight}{1.5in}
\addtolength{\voffset}{-1in}

\title{STAT3401: Lab exercises concerning Hotelling's T$^{2}$ test}
\author{Paul Hewson}
\date{16th November 2007}
\usepackage{/usr/share/R/share/texmf/Sweave}
\begin{document}
\setlength{\parindent}{0pt}
\setlength{\parskip}{12pt}
\sffamily
\maketitle

\section{Two sampling Hotelling's T$^{2}$ test}

We are going to consider an example using data from Flea Beetles reported by Lubischew (1962) and used in Flury (1997).  We are going to use \textbf{R} as a sophisticated calculator and work through a lot of the calculations long hand first.

\begin{Schunk}
\begin{Sinput}
> library(Flury)
> ?flea.beetles
> data(flea.beetles)
\end{Sinput}
\end{Schunk}

It can be seen that there is a factor ``Species'' denoting whether the beetles are from 'oleracea' or 'carduorum'.   There are four numeric variables as follows: 'TG'; Distange of the Transverse Groove to the posterior border of
          the prothorax (microns), 'Elytra'; Length of the Elytra (in units of 0.01mm), 'Second.Antenna'; Length of the second antennal joint (microns) and 'Third.Antenna'; Length of the third antennal joint (microns).   We need to estimate the mean for each sample, and calculate the difference between the two vectors:

\begin{Schunk}
\begin{Sinput}
> mu <- by(flea.beetles[,-1], flea.beetles$Species, colMeans)
> mudiff <- mu[[1]] - mu[[2]]
> p <- dim(flea.beetles)[2] - 1 ## how many variables are we using
\end{Sinput}
\end{Schunk}

The next step is to extract the two covariance matrices:

\begin{Schunk}
\begin{Sinput}
> covmats <- by(flea.beetles[,-1], flea.beetles$Species, cov)
> covmats
\end{Sinput}
\begin{Soutput}
flea.beetles$Species: oleracea
                      TG    Elytra Second.Antenna Third.Antenna
TG             187.59649 176.86257       48.37135     113.58187
Elytra         176.86257 345.38596       75.97953     118.78070
Second.Antenna  48.37135  75.97953       66.35673      16.24269
Third.Antenna  113.58187 118.78070       16.24269     239.94152
------------------------------------------------------------
flea.beetles$Species: carduorum
                      TG    Elytra Second.Antenna Third.Antenna
TG             101.83947 128.06316       36.98947      32.59211
Elytra         128.06316 389.01053      165.35789      94.36842
Second.Antenna  36.98947 165.35789      167.53684      66.52632
Third.Antenna   32.59211  94.36842       66.52632     177.88158
\end{Soutput}
\end{Schunk}

and then to estimate the pooled covariance matrix $\boldsymbol{S}$ for the flea beetle data (where N[1] gives $n_{1}$,  N[2] gives $n_{2}$), can be calculated as:

\begin{Schunk}
\begin{Sinput}
> N <- xtabs(~flea.beetles[,1])
> pooledS <- ((N[1]-1) * covmats[[1]] + (N[2]-1) * covmats[[2]]) / (N[1] + N[2] -2)
> pooledS
\end{Sinput}
\begin{Soutput}
                      TG   Elytra Second.Antenna Third.Antenna
TG             143.55910 151.8034       42.52660      71.99253
Elytra         151.80341 367.7878      121.87653     106.24467
Second.Antenna  42.52660 121.8765      118.31408      42.06401
Third.Antenna   71.99253 106.2447       42.06401     208.07290
\end{Soutput}
\begin{Sinput}
> Sinv <- solve(pooledS)
> Sinv
\end{Sinput}
\begin{Soutput}
                         TG        Elytra Second.Antenna Third.Antenna
TG              0.013257964 -0.0053492256   0.0015134494 -0.0021617878
Elytra         -0.005349226  0.0066679441  -0.0047337699 -0.0005969439
Second.Antenna  0.001513449 -0.0047337699   0.0130490933 -0.0007445297
Third.Antenna  -0.002161788 -0.0005969439  -0.0007445297  0.0060093005
\end{Soutput}
\end{Schunk}

Having calculated the inverse of the pooled correlation matrix we also need the scaling factor $\frac{n_{1} n_{2}}{n_{1} + n_{2}}$.   Hotellings T$^{2}$ is then quite straightforward to calculate:

\begin{Schunk}
\begin{Sinput}
> scaleFact <- (N[1]*N[2]) / (N[1]+N[2])
> Hotellings <-  t(mudiff) %*% Sinv %*% mudiff * scaleFact
> Hotellings
\end{Sinput}
\begin{Soutput}
         [,1]
[1,] 133.4873
\end{Soutput}
\end{Schunk}

which is the value of the T$^{2}$ statistic.   We could work with this value directly, but it is more convenient to transform it into something we can compare with the $F$ distribution.

\begin{Schunk}
\begin{Soutput}
       [,1]
[1,] 30.666
\end{Soutput}
\end{Schunk}

and we compare this with an $F$ distribution having $p$ and $(n_{1} + n_{2} - p - 1)$ d.f.

And we can check this as follows:

\begin{Schunk}
\begin{Sinput}
> pf(test, p, N[1]+N[2]-p-1,lower.tail = FALSE )
\end{Sinput}
\end{Schunk}

which gives us the area under the curve from our test statistic ($30.666$) to $\infty$.   Clearly in this case, we have reject H$_{0}$, i.e. there is evidence that the mean vectors, $\bar{\boldsymbol{x}}_{oleracea} = (194.4737, 267.0526, 137.3684, 185.9474)$, $\bar{\boldsymbol{x}}_{carduorum} = (179.55, 290.80, 157.20, 209.25)$,
 for the two species differ.   This is perhaps no surprise if you consider the data - do look at the scatterplot suggested by the helpfile.

\begin{itemize}
\item How would you modify the code to carry out a one sample T$^{2}$ test?
\item You may wish to repeat this exercise with the \texttt{turtles} data.
\end{itemize}

You also should note that in practice this calculation is done my means of the QR decomposition, details are given in Seber (1984).   There is no in-built \textbf{R} function for doing these calculations - a nice little project for someone.   The \texttt{manova()} function can be persuaded to carry out a two sample Hotelling's T$^{2}$ test as follows:

\begin{Schunk}
\begin{Sinput}
> hotel.test <- manova(as.matrix(flea.beetles[,-1]) ~ flea.beetles[,1])
> summary(hotel.test, test = "Hotelling")
\end{Sinput}
\end{Schunk}

\section{Drawing the ellipses}

This illustration is based on, but differs from code provided by Marco Bee to accompany Flury (1997).   Firstly, we need a function to draw ellipses:

\begin{Schunk}
\begin{Sinput}
> ellipse <- function(covmat, centroid, csquare, resolution, plot = TRUE) {
+ angles <- seq(0, by = (2 * pi)/resolution, length = resolution)
+   sd <- covmat[1,2] / sqrt(covmat[1,1] * covmat[2,2])
+     projmat <- matrix(0,2,2)
+     projmat[1,1] <- sqrt(covmat[1,1] %*% (1+sd)/2)
+     projmat[1,2] <- -sqrt(covmat[1,1] %*% (1-sd)/2)
+     projmat[2,1] <- sqrt(covmat[2,2] %*% (1+sd)/2)
+     projmat[2,2] <- sqrt(covmat[2,2] %*% (1-sd)/2)
+ circle <- cbind(cos(angles), sin(angles))
+ ellipse <- t(centroid + sqrt(csquare) * projmat %*% t(circle))
+ if (plot == TRUE) {lines(ellipse)}
+ return(ellipse)
+ }
\end{Sinput}
\end{Schunk}

It is possible to define a function which calculates $c^{2}$ and calls the ellipse routine (I'm not completely convinced this is doing the calculation correctly yet, in particular I'm not sure I'm using the correct tail).

\begin{Schunk}
\begin{Sinput}
> mean.ellipse <- function (data, alpha=0.05, resolution=500)
+ {
+ xbar <- colMeans(data)
+ n <- dim(data)[1]
+ p <- dim(data)[2]
+ f <- qf(1-alpha, p, n-p)
+ csquare <- ((n-1)/n) * (p / (n-p)) * f
+ cat(csquare)
+ ellipse <- ellipse(cov(data), xbar, csquare, resolution)
+ }
\end{Sinput}
\end{Schunk}

%# call procedure ellips

%X <- ellips(A, m, const, k)

%# graph the results

For illustrative purposes, we'll create a $n \times 2$ data object from our flea beetles.   Do note here that we are \emph{only} using two variables!

Given the above functions, it is quite straightforward to plot the centroids and constant density ellipses:

\begin{Schunk}
\begin{Sinput}
> X <- cbind(flea.beetles[,2], flea.beetles[,3])
> plot(X)
> points(t(colMeans(X)), pch = 16, col = "red")
> mean.ellipse(X, alpha = 0.01)
> mean.ellipse(X, alpha = 0.05)
\end{Sinput}
\end{Schunk}
\includegraphics{STAT3401Week7HotellingLab-plotellipse}

\begin{itemize}
\item Can you estimate the elongation of the ellipse? (Well, yes you can, but how?)
\end{itemize}

You may be interested in contrasting these with the univariate confidence intervals.   As an aide memoire, some code for doing this is given below:

\begin{Schunk}
\begin{Sinput}
> abline(v = confint(lm(X[,1]~1)))
> abline(h = confint(lm(X[,2]~1)))
\end{Sinput}
\end{Schunk}

In this case you shouldn't see much difference.   But repeat this exercise withthe turtles data and you should see a very different picture.

\begin{itemize}
\item Can you plot the simultaneous confidence ellipses (see Johnson and Wichern for details)?
\end{itemize}

%X <- cbind(turtles[,2], turtles[,3])

\end{document}


\section{Interactive Plotting}

\begin{frame}[fragile]
  \frametitle{First Plot}
  \begin{itemize}
  \item Start IPython with \texttt{-pylab}
  \end{itemize}
  \begin{lstlisting}
    $ ipython -pylab
  \end{lstlisting}  % $
  \begin{lstlisting}
   In[]: p = linspace(-pi,pi,100)
   In[]: plot(p, cos(p))
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{\texttt{linspace}}
  \begin{itemize}
  \item \texttt{p} has a hundred points in the range -pi to pi
    \begin{lstlisting}
     In[]: print p[0], p[-1], len(p)
    \end{lstlisting}
  \item Look at the doc-string of \texttt{linspace} for more details
    \begin{lstlisting}
     In[]: linspace?
    \end{lstlisting}
  \end{itemize}
  \begin{itemize}
  \item \texttt{plot} simply plots the two arguments with default
    properties
  \end{itemize}
\end{frame}

\section{Embellishing Plots}

\begin{frame}[fragile]
  \frametitle{Plot color and thickness}
  \begin{lstlisting}
   In[]: clf()
   In[]: plot(p, sin(p), 'r')
  \end{lstlisting}
  \begin{itemize}
  \item Gives a sine curve in Red.
  \end{itemize}
  \begin{lstlisting}
   In[]: plot(p, cos(p), linewidth=2)
  \end{lstlisting}
  \begin{itemize}
  \item Sets line thickness to 2
  \end{itemize}
  \begin{lstlisting}
   In[]: clf()
   In[]: plot(p, sin(p), '.')
  \end{lstlisting}
  \begin{itemize}
  \item Produces a plot with only points
  \end{itemize}
  \begin{lstlisting}
   In[]: plot?
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{\texttt{title}}
  \begin{lstlisting}
   In[]: x = linspace(-2, 4, 50)
   In[]: plot(x, -x*x + 4*x - 5, 'r',
         linewidth=2)
   In[]: title("Parabolic function -x^2+4x-5")
  \end{lstlisting}
  \begin{itemize}
  \item We can set title using \LaTeX~
  \end{itemize}
  \begin{lstlisting}
   In[]: title("Parabolic function $-x^2+4x-5$")
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Axes labels}
  \begin{lstlisting}
   In[]: xlabel("x")
   In[]: ylabel("f(x)")
  \end{lstlisting}
  \begin{itemize}
  \item We could, if required use \LaTeX~
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Annotate}
  \begin{lstlisting}
   In[]: annotate("local maxima", xy=(2, -1))
  \end{lstlisting}
  \begin{itemize}
  \item First argument is the annotation text
  \item The argument to \texttt{xy} is a tuple that gives the location
    of the text.
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Limits of Plot area}
  \begin{lstlisting}
   In[]: xlim()
   In[]: ylim()
  \end{lstlisting}
  \begin{itemize}
  \item With no arguments, \texttt{xlim} \& \texttt{ylim} get the
    current limits
  \item New limits are set, when arguments are passed to them
  \end{itemize}
  \begin{lstlisting}
   In[]: xlim(-4, 5)
  \end{lstlisting}
  \begin{lstlisting}
   In[]: ylim(-15, 2)
  \end{lstlisting}
\end{frame}

\begin{frame}
\frametitle{Plot}
\includegraphics[scale=0.45]{../advanced_python/images/plot.png}\\
\end{frame}

\section{Saving to Scripts}

\begin{frame}[fragile]
  \frametitle{Command history}
  \begin{itemize}
  \item To see the history of commands, we typed
    \begin{lstlisting}
     In[]: %hist
    \end{lstlisting}
  \item All commands, valid or invalid, appear in the history
  \item \texttt{\%hist} is a magic command, available only in IPython
  \end{itemize}
  \begin{lstlisting}
   In[]: %hist 5
    # last 5 commands
  \end{lstlisting}
  \begin{lstlisting}
   In[]: %hist 5 10
    # commands between 5 and 10
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Saving to a script}
  \begin{itemize}
  \item We wish to save commands for reproducing the parabola
  \item Look at the history and identify the commands that will
    reproduce the parabolic function along with all embellishment
  \item \texttt{\%save} magic command to save the commands to a file
  \end{itemize}
  \begin{lstlisting}
   In[]: %save plot_script.py 1 3-6 8
  \end{lstlisting}
  \begin{itemize}
  \item File name must have a \texttt{.py} extension
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Running the script}
  \begin{lstlisting}
   In[]: %run -i plot_script.py
  \end{lstlisting}
  \begin{itemize}
  \item There were no errors in the plot, but we don't see it!
  \item Running the script means, we are not in interactive mode
  \item We need to explicitly ask for the image to be shown
  \end{itemize}
  \begin{lstlisting}
   In[]: show()
  \end{lstlisting}
  \begin{itemize}
  \item \texttt{-i} asks the interpreter to check for names,
    unavailable in the script, in the interpreter
  \item \texttt{sin}, \texttt{plot}, etc. are taken from the
    interpreter
  \end{itemize}
\end{frame}

\section{Saving Plots}

\begin{frame}[fragile]
  \frametitle{\texttt{savefig}}
  \begin{lstlisting}
   In[]: x = linspace(-3*pi,3*pi,100)
   In[]: plot(x,sin(x))
   In[]: savefig('sine.png')
  \end{lstlisting}
  \begin{itemize}
  \item \texttt{savefig} takes one argument
  \item The file-type is decided based on the extension
  \item \texttt{savefig} can save as png, pdf, ps, eps, svg
  \end{itemize}
\end{frame}

\section{Multiple Plots}

\begin{frame}[fragile]
  \frametitle{Overlaid plots}
  \begin{lstlisting}
   In[]: x = linspace(0, 50, 10)
   In[]: plot(x, sin(x))
  \end{lstlisting}
  \begin{itemize}
  \item The curve isn't as smooth as we expected
  \item We chose too few points in the interval
  \end{itemize}
  \begin{lstlisting}
   In[]: y = linspace(0, 50, 500)
   In[]: plot(y, sin(y))
  \end{lstlisting}
  \begin{itemize}
  \item The plots are overlaid
  \item It is the default behaviour of \texttt{pylab}
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Legend}
  \begin{lstlisting}
   In[]: legend(['sine-10 points',
          'sine-500 points'])
  \end{lstlisting}
  \begin{itemize}
  \item Placed in the location, \texttt{pylab} thinks is `best'
  \item \texttt{loc} parameter allows to change the location
  \end{itemize}
  \begin{lstlisting}
   In[]: legend(['sine-10 points',
         'sine-500 points'],
            loc='center')
  \end{lstlisting}
\end{frame}

\begin{frame}
\frametitle{Overlaid Plots}
\includegraphics[scale=0.45]{../advanced_python/images/overlaid.png}\\
\end{frame}

\begin{frame}[fragile]
  \frametitle{Plotting in separate figures}
  \begin{lstlisting}
   In[]: clf()
   In[]: x = linspace(0, 50, 500)
   In[]: figure(1)
   In[]: plot(x, sin(x), 'b')
   In[]: figure(2)
   In[]: plot(x, cos(x), 'g')
  \end{lstlisting}
  \begin{itemize}
  \item \texttt{figure} command allows us to have plots separately
  \item It is also used to switch context between the plots
  \end{itemize}
  \begin{lstlisting}
   In[]: savefig('cosine.png')
   In[]: figure(1)
   In[]: title('sin(y)')
   In[]: savefig('sine.png')
   In[]: close()
   In[]: close()
  \end{lstlisting}
  \begin{itemize}
  \item \texttt{close('all')} closes all the figures
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Subplots}
  \begin{lstlisting}
   In[]: subplot(2, 1, 1)
  \end{lstlisting}
  \begin{itemize}
  \item number of rows
  \item number of columns
  \item plot number, in serial order, to access or create
  \end{itemize}
  \begin{lstlisting}
   In[]: subplot(2, 1, 2)
   In[]: x = linspace(0, 50, 500)
   In[]: plot(x, cos(x))

   In[]: subplot(2, 1, 1)
   In[]: y = linspace(0, 5, 100)
   In[]: plot(y, y ** 2)
  \end{lstlisting}
\end{frame}

\begin{frame}
\frametitle{Subplots}
\includegraphics[scale=0.45]{../advanced_python/images/subplot.png}\\
\end{frame}

\section{Plotting Data}

\begin{frame}[fragile]
  \frametitle{Loading data}
  \begin{itemize}
  \item \texttt{primes.txt} contains a list of primes listed
    column-wise
  \item We read the data using \texttt{loadtxt}
  \end{itemize}
  \begin{lstlisting}
  In[]: primes = loadtxt('primes.txt')
  In[]: print primes
  \end{lstlisting}
  \begin{itemize}
  \item \texttt{primes} is a sequence of floats
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Reading two column data}
  \begin{itemize}
  \item \texttt{pendulum.txt} has two columns of data
  \item Length of pendulum in the first column
  \item Corresponding time period in second column
  \item \texttt{loadtxt} requires both columns to be of same length
  \end{itemize}
  \begin{lstlisting}
   In[]: pend = loadtxt('pendulum.txt')
   In[]: print pend
  \end{lstlisting}
  \begin{itemize}
  \item \texttt{pend} is not a simple sequence like \texttt{primes}
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Unpacking with \texttt{loadtxt}}
  \begin{lstlisting}
   In[]: L, T = loadtxt('pendulum.txt',
                 unpack=True)
   In[]: print L
   In[]: print T
  \end{lstlisting}
  \begin{itemize}
  \item We wish to plot L vs. $T^2$
  \item \texttt{square} function gives us the squares
  \item (We could instead iterate over T and calculate)
  \end{itemize}
  \begin{lstlisting}
   In[]: Tsq = square(T)

   In[]: plot(L, Tsq, '.')
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{\texttt{errorbar}}
  \begin{itemize}
  \item Experimental data always has errors
  \item \texttt{pendulum\_error.txt} contains errors in L and T
  \item Read the values and make an error bar plot
  \end{itemize}
  \begin{lstlisting}
  In[]:  L, T, L_err, T_err = \
         loadtxt('pendulum_error.txt',
         unpack=True)
  In[]:  Tsq = square(T)

  In[]:  errorbar(L, Tsq , xerr=L_err,
         yerr=T_err, fmt='b.')
  \end{lstlisting}
\end{frame}

\begin{frame}
\frametitle{Errorbar}
\includegraphics[scale=0.45]{../advanced_python/images/L-Tsq.png}\\
\end{frame}

\section{Other kinds of Plots}

\begin{frame}[fragile]
  \frametitle{Scatter Plot}
  \begin{itemize}
  \item The data is displayed as a collection of points
  \item Value of one variable determines position along x-axis
  \item Value of other variable determines position along y-axis
  \item Let's plot the data of profits of a company
  \end{itemize}
  \begin{lstlisting}
   In[]:  year, profit = loadtxt(
                         'company-a-data.txt',
                         dtype=type(int()))

   In[]: scatter(year, profit)
  \end{lstlisting}
  \begin{itemize}
  \item \alert{\texttt{dtype=int}; default is float}
  \end{itemize}
\end{frame}

\begin{frame}
\frametitle{Scatter Plot}
\includegraphics[scale=0.45]{../advanced_python/images/scatter.png}\\
\end{frame}

\begin{frame}[fragile]
  \frametitle{Pie Chart}
  \begin{lstlisting}
   In[]: pie(profit, labels=year)
  \end{lstlisting}
\includegraphics[scale=0.35]{../advanced_python/images/pie.png}\\
\end{frame}

\begin{frame}[fragile]
  \frametitle{Bar Chart}
  \begin{lstlisting}
   In[]: bar(year, profit)
  \end{lstlisting}
\includegraphics[scale=0.35]{../advanced_python/images/bar.png}\\
\end{frame}

\begin{frame}[fragile]
  \frametitle{Log-log plot}
  \begin{itemize}
  \item Plot a \texttt{log-log} chart of $y=5x^3$ for x from 1 to 20
  \end{itemize}
  \begin{lstlisting}
   In[]: x = linspace(1,20,100)
   In[]: y = 5*x**3

   In[]: loglog(x, y)
   In[]: plot(x, y)
  \end{lstlisting}
  \begin{itemize}
  \item Look at \url{http://matplotlib.sourceforge.net/contents.html}
    for more!
  \end{itemize}
\end{frame}

\begin{frame}
\frametitle{Log-log plot Plot}
\includegraphics[scale=0.45]{../advanced_python/images/loglog.png}\\
\end{frame}


\section{Abstract}

At the end of the 2nd millennia, the Clay institute of mathematics put together a list of seven problems that would be the most important and hard problems in mathematics to solve for the next millennia.
The problems on that list is now the seven most famous problems in the world of mathematics, and for good reasons.
Three years later one of the problems, the Poincaré conjecture, was solved, and this problem will be the focus of this talk.

The Poincaré conjecture (really the Poincaré theorem, but the name stuck for historical reasons) is a very fundamental problem in the field of topology, and was the last piece in the puzzle of understanding which spaces are actually spheres in any dimension.
We will cover some basic introductory differential topology, the statement of the conjecture, the general outline of the proof and finally its generalizations and related problems.