Machine learning has become an extremely powerful tool in the realm of computer vision, particularly through the use of convolutional neural networks (CNNs) and the increased availability of data and computational resources. This chapter introduces the fundamentals of CNNs, as well as their application to computer vision and robotics.

\notessection{Modern Computer Vision}
Modern computer vision techniques\cite{ForsythPonce2011} rely heavily on deep learning and convolutional neural network architectures\cite{GoodfellowBengioEtAl2016}. A convolutional neural network is a type of neural network with additional structure that is beneficial for image processing tasks. In fact, CNNs can be said to be ``regularized'' neural networks since the additional structure reduces the ability of the network to overfit to data.
This chapter will introduce each component\footnote{Convolutional layers, nonlinear activations, pooling layers, and fully-connected layers.} in the architecture of a CNN, and then discuss how CNNs can be applied to problems in robotics.

\begin{figure}[ht]
\begin{center}
\includegraphics[width=\textwidth]{tex/figs/ch12_figs/lecun.png}
\caption{Example convolutional neural network architecture from LeCun et al. (1998)\nocite{LecunBottouEtAl1998}.}
\label{fig:activations}
\end{center}
\end{figure}

\subsection{Convolutional Neural Networks}

\subsubsection{Convolution Layers}
\begin{marginfigure}
\begin{center}
\includegraphics[width=0.75\textwidth]{tex/figs/ch12_figs/convolution.png}
\caption{A convolution filter being applied to a 3-channel RGB image.}
\label{fig:convfilter}
\end{center}
\end{marginfigure}
One of the main structural concepts that is unique to the architecture of a CNN is the use of convolution layers.
These layers exploit the underlying \textit{spatial locality} structure in images by using sliding ``learned'' filters, which are often much smaller than the image itself. Mathematically these filters perform operations in a similar way as other linear filters that have been used in image processing, such as Gaussian smoothing filters, and can be expressed as affine functions:
\begin{equation*}
    f(x) = w^\top x + b,
\end{equation*}
where $w$ is a vectorized representation of the weight parameters that define the filter, $x$ is a vectorized version of the image pixels covered by the filter, and $b$ is a scalar bias term.
For example in Figure \ref{fig:convfilter} a filter is applied over an image with three color channels (red, green, blue). In this case the filter may have dimension $m \times n \times 3$, which could be vectorized to a weight vector $w$ with $3mn$ elements. Additionally, the \textit{stride} of the filter describes how many positions it shifts by when sliding over the input. The output of the filter is also passed through a nonlinear activation, typically a ReLU function.

Once the filter has been applied to the entire image, the collection of outputs from the activation function will create a new ``filtered image'' typically referred to as an \textit{activation map.}
\begin{figure}[ht]
\begin{center}
\includegraphics[width=0.45\textwidth]{tex/figs/ch12_figs/convolution2.png}
\caption{The outputs of a convolution filter and activation function applied across an image make up a new image, called an \textit{activation} map.}
\label{fig:activations}
\end{center}
\end{figure}
In practice a number of different filters are usually learned in each convolution layer, which would simply produce a corresponding number of activation maps as the output\footnote{Besides the number of filters applied to the input, the width and height of the filter, the amount of padding on the input, and the stride of the filter are other hyperparameters.}. This is crucial such that each filter can focus on learning one specific relevant feature. Examples of different filters that might be learned in different convolution layers of a CNN are shown in Figure \ref{fig:convfeatures}\cite{ZeilerFergus2014}. Notice that the low-level features which are learned in earlier convolution layers look a lot like edge detectors (i.e. are more basic/fundamental) while later convolution layers have filters that look more like actual objects.
\begin{figure}[ht]
\begin{center}
\includegraphics[width=0.65\textwidth]{tex/figs/ch12_figs/features.png}
\caption{Low-level, mid-level, and high-level feature visualizations in a convolutional neural network from Zeiler and Fergus (2014)\nocite{ZeilerFergus2014}.}
\label{fig:convfeatures}
\end{center}
\end{figure}

In general, the use of convolution layers to exploit the spatial locality of images provides several benefits including:
\begin{enumerate}
    \item \textit{Parameter sharing}: the (small) filter's parameters are applied at all points on the image. Therefore the total number of learned parameters in the model is much smaller than if a fully-connected layer was used.
    \item \textit{Sparse interactions}: having the filter be smaller than the image allows for better detection of smaller, more meaningful features and improves computation time by requiring fewer mathematical operations to evaluate.
    \item \textit{Equivariant representation:} the convolution layer is equivariant to translation, meaning that the convolution of a shifted image is equivalent to the shifted convolution of the original image\footnote{However, convolution is not equivariant to changes in scale or rotation.}.
    \item The ability to work with images of \textit{varying size} if needed.
\end{enumerate}

\subsubsection{Pooling Layers}
Pooling is the second major structural component in CNNs. Pooling layers typically come after convolution layers (and their nonlinear activation functions). Their primary function is to replace the output of the convolution layer's activation map at particular locations with a ``summary statistic'' from other spatially local outputs. This helps make the network more robust against small translations in the input, helps improve computational efficiency by reducing the size of the input (i.e. it lowers the resolution), and is useful in enabling the input images to vary in size\footnote[][-2\baselineskip]{The size of the pooling can be modified to keep the size of the pooling layer output constant.}. The most common type of pooling is \textit{max} pooling, but other types also exist (such as \textit{mean} pooling).

\begin{marginfigure}
\begin{center}
\includegraphics[width=0.95\textwidth]{tex/figs/ch12_figs/maxpool.png}
\caption{Max pooling example with $2 \times 2$ filter and stride of $2$.}
\label{fig:pooling}
\end{center}
\end{marginfigure}
Computationally, both max and mean pooling layers operate with the same filtering idea as in the convolution layers. Specifically, a filter of width $m$ and height $n$ slides around the layer's input with a particular stride. The difference between the two comes from the mathematical operation performed by the filter, which as their names suggest are either a maximum element or the mean over the filter. If the output of the convolution layer has $N$ activation maps, the output of the pooling layer will also have $N$ ``images'', since the pooling filter is only applied across the spatial dimensions.

\subsubsection{Fully Connected Layers}
Downstream of the convolution and pooling layers are fully connected layers. These layers make up what is essentially just a standard neural network, which is appended to the end of the network. The function of these layers is to take the output of the convolution and pooling layers (which can be thought of as a highly condensed representation of the image) and actually perform a classification or regression. Generally the total number of fully connected layers at the end of the CNN will only make up a fraction of the total number of layers.

\subsubsection{CNN Performance}
A CNN can be said to learn how to process images \textit{end-to-end} because it essentially learns how to perform two steps simultaneously: feature extraction and classification or regression (i.e. it learns the entire process from image input to the desired output). In contrast, classical approaches to image processing use hand-engineered feature extractors. Since 2012, the performance of end-to-end learning approaches to image processing have dominated and continue to improve\footnote{Of course in some specific applications hand-engineered features may still be better! For example if engineering insight can identify a structure to the problem that a CNN could not.}. This continuous improvement has generally been realized with the use of deeper networks.

\subsection{CNNs for Object Detection and Localization}
Modern computer vision techniques such as convolutional neural networks have a large variety of applications in robotic perception, including object localization and detection.

In object localization problems, the goal is to identify the position of an object in the image. This is usually accomplished by specifying four numbers that define a \textit{bounding box} for the object in the image\footnote{Box coordinates are usually the $(x,y)$ position of the top-left corner and the width $w$ and height $h$ of the box.} (see Figure \ref{fig:boundingbox}). To solve object localization problems with a CNN, the standard approach is to have the output of the network be both the bounding box coordinates and an object class. This can be accomplished by reusing the convolution and pooling layers of the CNN but then have two separate branches of the fully connected layers: one trained for classification and the other for localization\footnote{Since the output of the localization branch is four real numbers $(x,y,w,h)$, this would be considered a regression problem and it is common to use and $l_2$ loss function.}. To train a network to simultaneously perform classification and localization, a multi-task loss function can be defined by adding together the loss functions of each branch.
\begin{figure}[ht]
\begin{center}
\includegraphics[width=0.5\textwidth]{tex/figs/ch12_figs/boundingbox.png}
\caption{Bounding box prediction for several objects in an image from Ren, He, et al. (2017)\nocite{RenHeEtAl2017}}
\label{fig:boundingbox}
\end{center}
\end{figure}

If multiple objects exist within a single image the object localization and classification problem becomes more difficult. First, the number of outputs of the network may change! For example, outputting a bounding box for $n$ objects would require $4n$ outputs. A practical solution to handling the problem of varying outputs is to simply apply the CNN to a series of cropped images produced from the original image, where the network can also classify the image as ``background'' (i.e. not an object)\footnote{This could be thought of as applying the entire CNN as a filter that slides across the image.}.
However, a naive implementation of this idea would likely result in an excessive number of CNN evaluations. Instead, different approaches have been developed for making this idea efficient by reducing the number of areas in the image that need to be evaluated. For example this has been accomplished by identifying ``regions of interest'' in the image through some other approach, or even partitioning the image into a grid.


\section{Local Polynomials}
In practical situations, a statistician is rarely blessed with simple
linear relationship between the predictor $X$ and the observed output
$Y$. That is, as a description of the regression function $f$, the
model
\[
g(x;\bg{\theta}) = \theta_1 + \theta_2 x, x \in I
\]
typically ignores obvious features in the data. This is certainly the
case for the values of $^{87} Sr$.

The Strontium data set was collected to test several hypotheses
about the catastrophic events that occurred approximately 65 million
years ago. The data contains Age in million of years and the ratios
described here. There is a division between
two geological time periods, the Cretaceous (from 66.4 to 144 million
years ago) and the
Tertiary (spanning from about 1.6 to 66.4 million years ago). Earth
scientist believe that the boundary between these periods is
distinguished by tremendous changes in climate that accompanied a mass
extension of over half of the species inhabiting the planet at the
time. Recently, the compositions of Strontium (Sr) isotopes in sea
water has been used to evaluate several hypotheses about the cause of
these extreme events. The dependent variable of the data-set is related to
the isotopic make up of Sr measured for the shells of marine
organisms. The Cretaceous-Tertiary boundary is referred to as
KTB. There data shows a
peak is at this time and this is used as
evidence that a meteor collided with earth.

The data presented in the Figure \ref{4.1} represents standardized ratio of
strontium-87 isotopes ($^{87}$Sr) to strontium-86 isotopes ($^{86}$Sr)
contained in the shells of foraminifera fossils taken form cores
collected by deep sea drilling. For each sample its time in history is
computed and  the standardized ratio is computed:
\[
^{87}\delta \mbox{Sr} = \left( \frac{ ^{87}\mbox{Sr}/^{86}\mbox{Sr sample}}{^{87}\mbox{Sr}/^{86}\mbox{Sr sea
    water}} - 1\right) \times 10^5.
\]
Earth scientist expect that $^{87}\delta \mbox{Sr}$ is a smooth-varying
function of time and that deviations from smoothness are  mostly
measurement error.

\begin{figure}[h]
\centerline{\epsfig{figure=Plots/plot-04-01.ps,angle=270,width=.8\textwidth}}
\caption{\label{f4.1}  $^{87}\delta \mbox{Sr}$ data.}
\end{figure}

To overcome this deficiency, we might
consider a more flexible polynomial model. Let ${\cal P}_k$ denote the
linear space of polynomials in $x$ of order at most $k$ defined as
\[
g(x;\bg{\theta}) = \theta_1 + \theta_2 x + \dots + \theta_k x^{k-1},
x\in I
\]
for the parameter vector $\bg{\theta} = (\theta_1,\dots,\theta_k) \in
{\mathbb R}^k$. Note that the space ${\cal P}_k$ consists of
polynomials having degree at most $k-1$.

In exceptional cases, we have reasons to believe that the regression
function $f$ is in fact a high-order polynomial. This parametric
assumption could be based on physical or physiological models
describing how the data were generated.

For historical values of $^{87}
\delta Sr$ we consider polynomials simply because our scientific
intuition tells us that $f$ should be
smooth.

Recall Taylor's theorem: polynomials are good at approximating
well-behaved functions in reasonably tight neighborhoods. If all we can
say about $f$ is that it is smooth in some sense, then either
implicitly or explicitly we consider high-order polynomials because
of their favorable approximation properties.

If $f$ is not in ${\cal P}_k$ then our
estimates will be biased by an amount that
reflects the approximation error incurred by a polynomial model.

% Given $(X_i,Y_i), i=1,\dots,n$ a unique OLS estimate
% $\hat{\bg{\theta}}$ exists provided there are at least $k$ unique
% values of $\{X_i\}$

% HOMEWORK: prove

Computational Issue: The basis of monomials
\[
B_j(x) = x^{j-1} \mbox{ for } j=1,\dots,k
\]
 is not well suited for numerical calculations ($x^8$ can be VERY BIG
 compared to $x$). While convenient for analytical manipulations
 (differentiation,
integration), this basis is {\it ill-conditioned} for $k$ larger than
$8$ or $9$. Most statistical packages use the orthogonal Chebyshev
polynomials (used by the R command {\tt poly()}).

An alternative to polynomials is to consider the space ${\cal
  PP}_k(\bt)$ of piecewise polynomials with break points
  $\bt=(t_0,\dots,t_{m+1})'$.
Given a sequence $a = t_0 < t_1 < \dots < t_m < t_{m+1} = b$,
  construct $m+1$ (disjoint) intervals
\[
I_l = [t_{l-1},t_l), 1 \leq l \leq m \mbox{ and } I_{m+1} =
[t_m,t_{m+1}],
\]
whose union is $I=[a,b]$. Define the piecewise polynomials of order
$k$
\[
g(x) = \left\{   \begin{array}{cc}
    g_1(x) = \theta_{1,1} + \theta_{1,2} x + \dots + \theta_{1,k}
    x^{k-1},&x \in I_1\\
    \vdots&\vdots\\
    g_{m+1}(x) = \theta_{m+1,1} + \theta_{m+1,2} x + \dots + \theta_{m+1,k}
    x^{k-1},&x \in I_{k+1}.
\end{array}
\right.
\]

In homework 2, we saw or will see that piecewise polynomials are a
linear space that present an
alternative to polynomials. However, it is hard to justify the breaks
in the function $g(x;\hat{\bg{\theta}})$.


\documentclass{article}
\usepackage[english]{babel}
\usepackage{amsmath,amssymb}

%%%%%%%%%% Start TeXmacs macros
\newcommand{\assign}{:=}
\newcommand{\nobracket}{}
\newcommand{\tmaffiliation}[1]{\\ #1}
\newcommand{\tmop}[1]{\ensuremath{\operatorname{#1}}}
%%%%%%%%%% End TeXmacs macros

\begin{document}

\title{Mathematical Statistics Assignment 2}

\author{
  yuejian mo
  \tmaffiliation{11510511}
}

\maketitle

\section*{Part II}

Q 2.9: Find the MLE of the unknown parameter $\theta$ when $X_1, X_2, \ldots,
X_n$ is a sample from the distribution whose density function is:
\[ f_X (x) = \frac{1}{2} e^{- | x - \theta |}, - \infty < x < \infty \]
Suggest Solution:
\[ f_X (x) = \frac{1}{2} e^{- | x - \theta |}, - \infty < x < \infty \]
is a special case of the Laplace distribution given as follows:
\[ f_X (x | \mu, \sigma \nobracket) = \frac{1}{\sqrt{2} \sigma} e^{-
   \frac{\sqrt{2} | x - \mu |}{\sigma}}, x \in \Re \]
for $\sigma = \sqrt{2}$ and $\mu \assign \theta$. To be more general, lets
consider the Laplace distribution with parameter $(\mu, \sigma) .$

Consider the likelihood function for $N$ data samples:
\[ L (\mu, \sigma ; x) = \prod_{t = 1}^N \frac{1}{\sqrt{2} \sigma} e^{-
   \frac{\sqrt{2} | x_t - \mu |}{\sigma}} = \left( \sqrt{2} \sigma \right)^{-
   N} e^{\frac{- \sqrt{2}}{\sigma} \sum_{t = 1}^N | x_t - \mu |} \]
Take the log likelihood function as $l (\mu, \sigma ; x) = \log (L (\mu,
\sigma ; x))$ and we get
\[ l (\mu, \sigma ; x) = - N \tmop{In} \left( \sqrt{2} \sigma \right) -
   \frac{\sqrt{2}}{\sigma} \sum_{t = 1}^N | x_t - \mu | \]
Take the derivate with respect to the parameter $\mu$
\[ \frac{\partial l}{\partial \mu} = - \frac{\sqrt{2}}{\sigma} \sum_{t = 1}^N
   \frac{\partial | x_t - \mu |}{\partial \mu} \]
which is equal to
\[ = \frac{\sqrt{2}}{\sigma} \sum_{t = 1}^N \tmop{sgn} (x_t - \mu) \]
using the identity
\[ \frac{\partial | x |}{\partial x} = \frac{\partial \sqrt{x^2}}{\partial x}
   = x (x^2)^{- 1 / 2} = \frac{x}{| x |} = \tmop{sgn} (x) \]
To maximize the likelihood function we need to solve
\begin{equation}
  = \frac{\sqrt{2}}{\sigma} \sum_{t = 1}^N \tmop{sgn} (x_t - \mu) = 0
\end{equation}
For which we have two cases: $N$ is even or odd.

If $N$ is odd and we choose $\hat{\mu} = \tmop{median} (x_1, \cdots x_N)$,
then there are $\frac{N - 1}{2}$ cases where $x_t < \mu$ and for the other
$\frac{N - 1}{2}$ cases $x_t > \mu$, therefore $\hat{\mu}$ satisfies (1) and
is the Maximum likelihood estimator for the parameter $\mu$.

If $N$ is even, we can not simply choose one $x_t$ which will satisfy(1),
however we can still minimize it through ranking the observations as $x_1
\leqslant x_2 \leqslant \ldots, x_N$ \ and then choosing either $x_{N / 2}$ or
$x_{(N + 1) / 2}$.

In summary $\hat{\mu} = \tmop{median} (x_1, \ldots, x_N)$ is the maximum
likelihood estimator for any $N$.

\section*{Sources}

https://math.stackexchange.com/questions/240496/finding-the-maximum-likelihood-estimator

\

\end{document}

Assignments and Suggested Solutions/Chapter2-11510511-YuejianMo.tex


\documentclass{article}
%\usepackage{fullpage}
\usepackage{amsmath,amsfonts,amssymb}

\newcommand \TODO {\fbox{\textbf{TODO}}}

\newcommand \Like {{\mathbb{L}}}
\newcommand \Prob {{\mathbb{P}}}

\begin{document}

\section{Matching via belief propagation}

Consider a general problem of associating individual observations into
extended collections,
in the presence of false alarms (spurrious singleton observations)
and missed detections (aspects in which a detection is missed).
Let $P$ be a vertex signature,
$V$ be the set of observations,
$p:P$,
let $E_i \subseteq \forall v:V. 1+P$ be a set of feasible edges.
We disallow vertices from appearing in an edge via multiple ports,
to avoid situations where vertex $\leftrightarrow$ edge messages are double-counted.

Augmenting this combinatorial structure,
let $F:V \to \Like$ be false alarm likelihoods,
$Ai : Ei \to \Like$ be association likelihoods, and
$Mi : 1 \times V + V \times 1 \to \Like$ be missed detection likeihoods.
We then wish to solve the problem

\newcommand \tab {\hspace{1cm}}

\begin{align*}
&\textbf{Given:}
\\&\tab P                     &&\text{set of ports}
\\&\tab V                     &&\text{set of vertices}
\\&\tab E : V \to (1 + P)     &&\text{set of feasible edges}
\\&\tab F : V \to \Like       &&\text{prior false alarm likelihoods}
\\&\tab M : ( 1 \times V
            + V \times 1 )
            \to \Like         &&\text{prior missed detection likelihoods}
\\&\textbf{Vary:}
\\&\tab A' : E \to \Prob      &&\text{posterior association probabilities}
\\&\tab F' : V \to \Prob      &&\text{posterior false alarm probabilities}
\\&\tab M' : ( 1 \times V
             + V \times 1 )
             \to \Prob        &&\text{posterior missed detection probabilities}
\\&\textbf{To Minimize:}
\\&\tab \TODO
\\&\textbf{Subject To:}
\\&\tab \forall v:V.\, \TODO
\end{align*}

\subsection{Example: Factored multi-frame assignment}

\subsection{Example: Bi-directional grid matching}

\end{document}


\documentclass[11pt]{amsbook}
\usepackage{../HBSuerDemir}

\begin{document}
    \hPage{b1p2/266}

    \begin{hEnumerateArabic}
        \setcounter{enumi}{46}
        \item Prove that the system
            \begin{align*}
                x - y + z &= 0\\
                2x + y - z &= 0\\
                x + 5y - 5z &= 0
            \end{align*}

        \noindent is consistent and solve it.\\

        \item Solve
            \begin{align*}
                3x + 5y - 7z &= 13\\
                4x + y - 12z &= 6\\
                2x + 9y - 3z &= 20\\
            \end{align*}

        \item Solve
            \begin{align*}
                x_{1} + 2x_{2} + 3x_{3}+ 4x_{4} &= 5\\
                2x_{1} + x_{2} + 4x_{3}+ x_{4} &= 2\\
                3x_{1} + 4x_{2} + x_{3}+ 5x_{4} &= 6\\
                2x_{1} + 3x_{2} + 5x_{3}+ 2x_{4} &= 3\\
            \end{align*}

        \item Solve
            \begin{align*}
                x_{1} + x_{2} + x_{3}+ x_{4}+ x_{5} &= 1\\
                x_{1} - x_{2} + x_{3}+ x_{4}+ x_{5} &= 1\\
                2x_{1} + 3x_{2} + x_{3}- x_{4}+ 2x_{5} &= -4\\
                x_{1} + 3x_{3}+ x_{5} &= 6\\
                x_{2} + 2x_{3}+ 3x_{4}- x_{5} &= 10\\
            \end{align*}

    \end{hEnumerateArabic}

    \begin{center}
        \section*{ANSWERS TO EVEN NUMBERED EXERCISES}
    \end{center}

    \begin{hEnumerateArabic}
        \item[36.] \hPairingParan{x+1}\hPairingParan{x^2+2x+8}
        \item[46.]
            D = \hPairingParan{a+b+c+d}\hPairingParan{a-b+c-d} \hPairingBraket{\hPairingParan{a-c}^2+         \hPairingParan{b-d}^2} = 0\\
            a + c = b + d   (circumscibed);   a=c, b=d (parallelogram)
        \item[48.]\hPairingParan{1, 2, 0}
        \item[50.]\hPairingParan{-2, 0, 3, 1, -1}
    \end{hEnumerateArabic}

\end{document}

hw1/non-merged/BERKAY KOZAN_35883_assignsubmission_file_/Berkay KOZAN- suerdemir 4/pages/b1p2-266.tex


\documentclass[11pt, fleqn]{article}

\input{../../utils/header.tex}

% \crefname{figure}{Figure}{Figures}
% \crefname{section}{Section}{Sections}
% \crefname{table}{Table}{Tables}
% \crefname{lstlisting}{Listing}{Listings}

\setlength{\parskip}{12pt} % Sets a blank line in between paragraphs
\setlength\parindent{0pt} % Sets the indent for each paragraph to zero

\begin{document}

\title{Machine Learning (41204-01)\\HW \#2}
\author{Will Clark $\vert$ Matthew DeLio \\
\texttt{will.clark@chicagobooth.edu} $\vert$ \texttt{mdelio@chicagobooth.edu} \\
University of Chicago Booth School of Business}
\date{\today}
\maketitle

\section{kNN with 1 Attribute}

In the previous assignment, we chose a k-nearest neighbors algorithm with a value of $k=40$. In this exercise, we will use 5-fold cross validation to chose $k$ in a more systematic manner. In \cref{fig:1p_k}, we show the average root mean square error (RMSE) for each of five folds over values of $k \in (2,300)$. The thick red line shows the average RMSE for each run with the minimum value highlighted at $k=28$.

Because the training and test samples are being chosen randomly and because the number of folds is small, there is some run-to-run variation in the minimum $k$ selected with this method. To account for this, we can repeat 5-fold cross validation. Each trail will yield an optimal value of $k$, and we can look at the mean of this sequence of optimal $k$ values to determine the most optimal $k$ value.

We can see the effect of the number of folds on the \textit{distribution} of optimal $k$ values in \cref{fig:nfolds_k_dist}. We run 100 trials of cross validation for 5, 10, and 20 folds and show the distribution of optimal $k$ values as box plots. We can see that for 5-fold cross validation, the spread of optimal $k$ values is fairly high (as is the presence of outliers), but that the spread falls as the number of folds rises.

\begin{figure}[!htb]
  \centering
  \caption{Distribution of Optimal k by Number of CV Folds}
  \includegraphics[scale=.5]{nfolds_k_dist.pdf}
  \label{fig:nfolds_k_dist}
\end{figure}

As it turns out, the mean optimal $k$ chosen over 100 trials of 5-fold cross validation is $k=27$, so our value of $k$ chosen over 1 run above is close to optimal. For the duration of the exercise we will use $k=28$ since the difference between the two model fits is negligible.

The difference between our ``eyeballed'' value of $k$ and the optimal value of $k$ discussed above is shown in \cref{fig:1p_fit}. The difference between the two is very small; the eyeballed value of $k=40$ provides a slightly smoother fit of the data, but by and large there is an insignificant difference between the fits.

As an aside, we wanted to test the value in using 5-fold cross validation instead of 10- or 20-fold, so we ran the following experiment. For varying values of fold numbers from 5-200, run a trial of n-fold cross validation and take the mean and the variance of the optimal $k$ selected across all trials. As it turns out, there is no systematic pattern in the mean across number of folds, but the variance of estimates of $k$ falls significantly as the number of folds increases from 5 to 20 (see the results in \cref{tab:cv_vars}). However, after 20 folds, the variance of the optimal $k$'s is roughly constant, suggesting that there is no additional benefit to adding folds. This is a useful cut-off point, since computing cross validation for more than 20 folds becomes computationally expensive.

\begin{figure}[!htb]
  \centering
  \caption{5-fold kNN CV for 1 Attribute}
  \includegraphics[scale=.5]{1p_cv_k.pdf}
  \label{fig:1p_k}
\end{figure}

\begin{figure}[!htb]
  \centering
  \caption{Comparison of Fit Between ``Eyeball'' and CV Chosen $k$'s}
  \includegraphics[scale=.5]{1p_fit_eye_ev.pdf}
  \label{fig:1p_fit}
\end{figure}

\input{cv_vars.tex}

\subsection{Price Predictions}

Using the value of $k=28$ discussed above, our prediction for the price of a car with 100 thousand miles is \$16,676.18 (see \cref{tab:1p_predict}). We can see the predicted point in \cref{fig:1p_fit} outlined in black; it appears that the predicted value is well within the range of observed values for similar cars.

\input{1p_predict.tex}

\section{kNN with 2 Attributes}\label{sec:2p}

In this section we explore the benefits of introducing a second attribute to our k-nearest neighbors algorithm. We will now use the car's mileage and its model year to predict is resale price.  To apply kNN to the data set we first examine the data-set and note that the two attributes used to predict price are on seemingly different scales.  In \cref{tab:data_scale} we see that the means are completely different and the standard deviations are also different. To avoid over-weighting one parameter with the other during model selection and prediction, these two attributes are independently scaled to ensure that $\mu_i=0$ and $\sigma_i^2=1$ using \cref{eq:scale}.  These scaled values are then used as inputs when training kNN.  When predicting prices, the covariates in the test set are put on the same scale by adjusting with the mean and standard deviation of the training values (to simulate out-of-sample prediction).

\begin{equation}\label{eq:scale}
x_{i,scaled} = \frac{x_i-\mu_i}{\sigma_i}
\end{equation}

\input{data_scale.tex}

With the resulting scaled inputs we run 5-fold cross-validated kNN, sweeping k from 1 to 300.  To save computation time, we focused our search by increasing the $k$ step-size in the range $k\in[14,50]$ reducing the step-size outside of this range. \Cref{fig:2p_k} shows the mean and per-fold RMSE a single run of the cross-validated kNN; here we find that RMSE is minimized with a choice of $k=23$.

\begin{figure}[!htb]
  \centering
  \caption{5-fold kNN CV for 2 Attributes}
  \includegraphics[scale=.5]{2p_cv_k.pdf}
  \label{fig:2p_k}
\end{figure}

To test the stability of this choice in $k$, 20 runs of the 5-fold CV kNN algorithm were run on a further focused range of $k\in[10,100]$.  The resulting RSMEs were averaged together and the minimum RMSE was found to occur at $k=26$.  \Cref{fig:2p_k_multi} shows a large amount of run-to-run variation in the minimal $k$  ($\sigma_{k_{min}}=2.81$).  Therefore, for predictive purposes we will use this value of $k$.

\begin{figure}[!htb]
  \centering
  \caption{20-run 5-fold kNN CV average for 2 Attributes}
  \includegraphics[scale=.5]{2p_cv_multi_k.pdf}
  \label{fig:2p_k_multi}
\end{figure}

\subsection{Price Predictions}
Using the kNN model with $k=26$ found in \cref{sec:2p} we now turn to predicting the price of a 2008 car with 75,000 miles.  To predict the value of the car we first scale the year and mileage using the scale values used in the original data set (see \cref{tab:data_scale}) and then apply kNN to predict the price.  The predicted price of such a car is found to be $\$31,646.65$ (see \cref{tab:2p_predict}).

\input{2p_predict.tex}

\section{Comparison of kNN for 1 \& 2 Attributes}
In this section we compare the models produced with 1 \& 2 attributes.  Here we sweep $k$ while performing a 5-fold CV kNN train/test sequence calculating the average RMSE at each value of $k$.  \Cref{fig:p_compare} shows the difference between having 1 attribute (mileage) and having 2 attributes (mileage + year).  There is a noticeable difference between the two with the second attribute lowering the RMSE quite dramatically.  Indeed, the average per-fold correlation between $price$ and $\widehat{price}$ is 86.3\% and 95.5\% for 1 and 2 attributes respectively.  Therefore it appears that having both $mileage$ \& $year$ yields a better model than just $mileage$ alone.

\begin{figure}[!htb]
  \centering
  \caption{RMSE for 1 \& 2 Attributes}
  \includegraphics[scale=.5]{1p_2p_cv_compare.pdf}
  \label{fig:p_compare}
\end{figure}

\end{document}

% \input{.tex}

% \begin{figure}
%   \centering
%   \begin{subfigure}[b]{0.49\textwidth}
%     \caption{}
%     \includegraphics[width=\textwidth]{.pdf}
%     \label{fig:}
%   \end{subfigure}
%   \hfill
%   \begin{subfigure}[b]{0.49\textwidth}
%     \caption{}
%     \includegraphics[width=\textwidth]{.pdf}
%     \label{fig:}
%   \end{subfigure}
%   \caption{}
% \end{figure}

% \begin{figure}[!htb]
%   \centering
%   \caption{}
%   \includegraphics[scale=.5]{.pdf}
%   \label{fig:}
% \end{figure}


\section{Cost-Optimal MAPF to ASP Given a Makespan}
In this section we describe two cost-optimal encodings for MAPF to ASP for a given makespan $m$. As such, when given to an ASP solver, they will generate a cost-optimal solution which does not exceed makespan $m$. In the next section, we show that these solutions are not necessarily truly cost-optimal. For now however, we focus on this simpler case to ease our presentation.

The first is a straightforward encoding that states that each time the agent performs an action away from the goal state, then $penalty(R,t,1)$ holds true.

\begin{align*}
penalty(R,t,1) \leftarrow &at(R,X,Y,t), not\: goal(R,X,Y). \\
penalty(R,t,1) \leftarrow &at(R,X,Y,t), goal(R,X,Y), exec(R,A,t),A!=wait. \\
penalty(R,t,1) \leftarrow &goal(R,X,Y), at(R,X,Y,t), exec(R,wait,t), not\: at\_goal\_back(R,t). \\
at\_goal\_back(R,m) \leftarrow &robot(R). \\
at\_goal\_back(R,T-1) \leftarrow &tiempo(t),at\_goal\_back(R,t),exec(R,wait,t-1).
\end{align*}


% Compiled and edited by Brian Bi
\documentclass[9pt]{extarticle}
\setlength{\parindent}{0.0in}
\usepackage{amsmath}
\usepackage{multicol}
\usepackage[landscape,letterpaper,twoside=false,top=15mm,bottom=15mm,left=10mm,right=10mm]{geometry}
\pagestyle{myheadings}
\markright{}
\usepackage{listings}
\usepackage{color}
\lstset{
	tabsize=4,
    basicstyle=\ttfamily\scriptsize,
    %upquote=true,
    aboveskip={1.5\baselineskip},
    columns=fixed,
    showstringspaces=false,
    extendedchars=true,
    breaklines=true,
    prebreak = \raisebox{0ex}[0ex][0ex]{\ensuremath{\hookleftarrow}},
    frame=single,
    rulecolor=\color[rgb]{0.75,0.75,0.75},
    showtabs=false,
    showspaces=false,
    showstringspaces=false,
    keywordstyle=\color[rgb]{0,0,1},
    commentstyle=\color[rgb]{0.133,0.545,0.133},
    stringstyle=\color[rgb]{0.627,0.126,0.941},
}

\begin{document}
% Disable balancing of columns on last page, which is ugly
\begin{multicols*}{3}
% Want compact table of contents, but normal spacing between paragraphs later on
\setlength{\parskip}{0.0in}
\tableofcontents
\setlength{\parskip}{0.1in}
% New codebook
\section{Flow and matching}

\subsection{Max flow (Dini\'c)} % Stanford
\lstinputlisting[language=c++]{dinic.cpp}

\subsection{Min-cost max-flow (successive shortest paths)}
% Frank Chu and Igor Naverniouk (modified by Brian Bi)
\lstinputlisting[language=c++]{mcmf4.cpp}

\subsection{Min-cost matching} % Stanford
\lstinputlisting[language=c++]{bipartite-mincost.cpp}

\subsection{Max bipartite matching} % Stanford
\lstinputlisting[language=c++]{bipartite.cpp}

\subsection{Global min cut (Stoer--Wagner)} % Stanford
\lstinputlisting[language=c++]{mincut.cpp}

\subsection{K\"onig's Theorem (Text)} % Jacob Plachta and Brian Bi
In any bipartite graph, the number of edges in a maximum matching equals the
number of vertices in a minimum vertex cover. To exhibit the vertex cover:
\begin{enumerate}
\item Find a maximum matching
\item Change each edge \textbf{used} in the matching into a directed edge from
\textbf{right to left}
\item Change each edge \textbf{not used} in the matching into a directed edge
from \textbf{left to right}
\item Compute the set $T$ of all vertices reachable from unmatched vertices on
the left (including themselves)
\item The vertex cover consists of all vertices on the right that are
\textbf{in} $T$, and all vertices on the left that are \textbf{not in} $T$
\end{enumerate}

\subsection{General Unweighted Maximum Matching (Edmonds' algorithm)}
% Ilya Grebnov (modified by Brian Bi)
\lstinputlisting[language=c++]{general-matching.cpp}

\subsection{Minimum Edge Cover (Text)} % Brian Bi
If a minimum edge cover contains $C$ edges, and a maximum matching contains $M$
edges, then $C + M = |V|$. To obtain the edge cover, start with a maximum
matching, and then, for every vertex not matched, just select some edge
incident upon it and add it to the edge set.

\subsection{Stable Marriage Problem (Gale--Shapley algorithm)} % Brian Bi
\lstinputlisting[language=c++]{stablemp.cpp}

\section{Geometry}

\subsection{Miscellaneous Geometry} % Stanford
\lstinputlisting[language=c++]{geom-2d.cpp}

\subsection{3D Geometry}
% Stanford (translated, modified, and expanded by Wesley May and Brian Bi)
\lstinputlisting[language=c++]{geom-3d.cpp}

\subsection{Convex hull} % Brian Bi
\lstinputlisting[language=c++]{monotone.cpp}

% TODO: Add faster one
\subsection{Slow Delaunay triangulation} % Stanford
\lstinputlisting[language=c++]{delaunay.cpp}

\subsection{Minimum Enclosing Disk (Welzl's Algorithm)} % Brian Bi
\lstinputlisting[language=c++]{welzl.cpp}

\subsection{Pick's Theorem (Text)} % Wesley May
For a polygon with all vertices on lattice points, $A = i + b/2 - 1$, where $A$
is the area, $i$ is the number of lattice points strictly within the polygon,
and $b$ is the number of lattice points on the boundary of the polygon. (Note,
there is no generalization to higher dimensions)

\section{Math Algorithms}

\subsection{Sieve of Eratosthenes} % Jimmy Mårdell (Yarin)
\lstinputlisting[language=c++]{yarin.cpp}

\subsection{Modular arithmetic and linear Diophantine solver} % Stanford
\lstinputlisting[language=c++]{modular.cpp}

\subsection{Gaussian elimination for square matrices of full rank; finds
inverses and determinants} % Stanford
\lstinputlisting[language=c++]{gaussian.cpp}

\subsection{Reduced row echelon form (RREF), matrix rank} % Stanford
\lstinputlisting[language=c++]{rref.cpp}

\subsection{Solving linear systems (Text)} % Brian Bi
To solve a general system of linear equations, put it into matrix form and
compute the reduced row echelon form. For example,
\begin{align*}2x + y &= 5 \\ 3x + 2y &= 6\end{align*}
corresponds to the matrix
\[ \left[ \begin{array}{cc|c} 2 & 1 & 5 \\ 3 & 2 & 6 \end{array} \right] \]
with RREF
\[ \left[ \begin{array}{cc|c} 1 & 0 & 4 \\ 0 & 1 & -3 \end{array} \right] \]
After row reduction, if any row has a 1 in the rightmost column and 0
everywhere else, then the system is inconsistent and has no solution.
Otherwise, to find a solution, set the variable corresponding to the leftmost 1
in each column equal to the corresponding value in the rightmost column, and
set all other variables to 0. Ignore rows consisting entirely of 0. The
solution is unique iff the rank of the matrix equals the number of variables.

\subsection{Fast Fourier transform (FFT)} % Stanford
\lstinputlisting[language=c++]{fft.cpp}

\subsection{Simplex algorithm} % Stanford
\lstinputlisting[language=c++]{simplex.cpp}

\subsection{Fast factorization (Pollard rho) and primality testing
(Rabin--Miller)} % Qiyu Zhu
\lstinputlisting[language=c++]{pollard-rho.cpp}

\subsection{Euler's Totient} % Andre Hahn Pereira
\lstinputlisting[language=c++]{totient.cpp}

\section{Graphs}

\subsection{Strongly connected components} % Stanford (modified by Brian Bi)
\lstinputlisting[language=c++]{scc.cpp}

\subsection{Bridges} % Andre Hahn Pereira
\lstinputlisting[language=c++]{bridges.cpp}

\subsection{Eulerian path} % Andre Hahn Pereira
\lstinputlisting[language=c++]{eulerian.cpp}

\subsection{Lowest Common Ancestor (Pseudocode)} % Jacob Plachta
Tarjan's offline algorithm (requires $O(N)$ disjoint set operations).
\begin{lstlisting}
function TarjanOLCA(u)
    MakeSet(u);
    u.ancestor := u;
    for each v in u.children do
        TarjanOLCA(v);
        Union(u,v);
        Find(u).ancestor := u;
    u.colour := black;
    for each v such that {u,v} in P do
        if v.colour == black
            print "Tarjan's Least Common Ancestor of " + u +
                " and " + v + " is " + Find(v).ancestor + ".";
\end{lstlisting}
This function is called on the root of the tree. The set P of pairs of nodes to
query must be specified in advance. Each node is initially white, and is
colored black after it and all its children have been visited. The lowest
common ancestor of the pair {u,v} is available as Find(v).ancestor immediately
(and only immediately) after u is colored black, provided v is already black.
Otherwise, it will be available later as Find(u).ancestor, immediately after v
is colored black.

\section{Data Structures}

\subsection{Suffix arrays} % Stanford
\lstinputlisting[language=c++]{suffix-array.cpp}

\subsection{Binary Indexed Tree (BIT) / Fenwick Tree} % Brian Bi
\lstinputlisting[language=c++]{BIT.cpp}

\subsection{BIT with range updates} % Brian Bi
\lstinputlisting[language=c++]{BIT-range.cpp}

\subsection{Segment Tree with Lazy Propagation} % Brian Bi
\lstinputlisting[language=c++]{segtree.cpp}

\subsection{Size-Balanced Tree}
% Tom Yubing Dong (tomtung) (modified by Brian Bi)
\lstinputlisting[language=c++]{SBT.cpp}

\section{Number Theory Reference}
\subsection{Polynomial Coefficients (Text)} % Brian Bi
$(x_1 + x_2 + ... + x_k)^n = \sum_{c_1 + c_2 + ... + c_k = n}
\frac{n!}{c_1! c_2! ... c_k!} x_1^{c_1} x_2^{c_2} ... x_k^{c_k}$

\subsection{M\"obius Function (Text)} % Brian Bi
$\mu(n) = \begin{cases}
0 & \text{$n$ not squarefree} \\
1 & \text{$n$ squarefree w/ even no. of prime factors} \\
1 & \text{$n$ squarefree w/ odd no. of prime factors} \\
\end{cases}$
Note that $\mu(a) \mu(b) = \mu(ab)$ for $a, b$ relatively prime
Also $\sum_{d \mid n} \mu(d) = \begin{cases} 1 & \text{if $n = 1$} \\
0 & \text{otherwise} \end{cases}$

\textbf{M\"obius Inversion}
If $g(n) = \sum_{d|n} f(d)$ for all $n \ge 1$, then
$f(n) = \sum_{d|n} \mu(d)g(n/d)$ for all $n \ge 1$.

\subsection{Burnside's Lemma (Text)} % Wesley May and Brian Bi
The number of orbits of a set $X$ under the group action $G$ equals the average
number of elements of $X$ fixed by the elements of $G$.

Here's an example. Consider a square of $2n$ times $2n$ cells. How many ways
are there to color it into $X$ colors, up to rotations and/or reflections?
Here, the group has only 8 elements (rotations by 0, 90, 180 and 270 degrees,
reflections over two diagonals, over a vertical line and over a horizontal
line). Every coloring stays itself after rotating by 0 degrees, so that
rotation has $X^{4n^2}$ fixed points. Rotation by 180 degrees and reflections
over a horizonal/vertical line split all cells in pairs that must be of the
same color for a coloring to be unaffected by such rotation/reflection, thus
there exist $X^{2n^2}$ such colorings for each of them. Rotations by 90 and 270
degrees split cells in groups of four, thus yielding $X^{n^2}$ fixed colorings.
Reflections over diagonals split cells into $2n$ groups of 1 (the diagonal
itself) and $2n^2-n$ groups of 2 (all remaining cells), thus yielding
$X^{2n^2-n+2n}=X^{2n^2+n}$ unaffected colorings.  So, the answer is
$(X^{4n^2}+3X^{2n^2}+2X^{n^2}+2X^{2n^2+n})/8$.

\section{Miscellaneous}

\subsection{Knuth--Morris--Pratt (KMP)} % Stanford
\lstinputlisting[language=c++]{KMP.cpp}

\subsection{2-SAT} % Brian Bi
\lstinputlisting[language=c++]{2sat.cpp}

\subsection{Shunting Yard (Pseudocode)} % Wesley May
\begin{lstlisting}
// Add '(' to start of expression, and ')' to end.
O = empty vector of tokens (values or operators)
S = empty stack of tokens (brackets or operators)
for each token:
  if token == value:
    O.push(token)
  else if token == '(':
    S.push(token)
  else if token == ')':
    while S.top() != '(':
      O.push(S.top())
      S.pop()
    S.pop()
  else:
    // Note: If token is a right-associative operator (^), this should be <=.
	// priority('(') < priority('+') < priority('*').
    while priority(S.top()) < priority(token):
      O.push(S.top())
      S.pop()
    S.push(token)
// Finally, evaluate O as a postfix expression.
\end{lstlisting}

\subsection{Convex hull trick} % Brian Bi
\lstinputlisting[language=c++]{chtrick.cpp}

\subsection{Binary search} % Brian Bi
\lstinputlisting[language=c++]{bsearch.c}

\subsection{Bignums (C++, slow)} % Brian Bi
\lstinputlisting[language=c++]{bignum.cpp}

\subsection{All nearest smaller values} % Brian Bi
\lstinputlisting[language=c++]{ansv.cpp}

\subsection{Longest palindromic substring} % Brian Bi
\lstinputlisting[language=c++]{manacher.cpp}
\end{multicols*}
\end{document}


\subsection{The special case of Newton's linearization of~(\ref{eq:toto1}) with FirstOrderType2R~(\ref{first-DS2})}

Let us now proceed with the time discretization of~(\ref{eq:toto1}) with FirstOrderType2R~(\ref{first-DS2})  by a fully implicit scheme :
\begin{equation}
  \begin{array}{l}
    \label{eq:mlcp2-toto1-DS2}
     M x_{k+1} = M x_{k} +h\theta f(x_{k+1},t_{k+1})+h(1-\theta) f(x_k,t_k) + h \gamma r(t_{k+1})
     + h(1-\gamma)r(t_k)  \\[2mm]
     y_{k+1} =  h(t_{k+1},x_{k+1},\lambda _{k+1}) \\[2mm]
     r_{k+1} = g(t_{k+1},\lambda_{k+1})\\[2mm]
  \end{array}
\end{equation}

 \paragraph{Newton's linearization of the first line of~(\ref{eq:mlcp2-toto1-DS2})} The linearization of the first line of the  problem~(\ref{eq:mlcp2-toto1-DS2}) is similar to the previous case so that (\ref{eq:rfree-2}) is still valid.

 \paragraph{Newton's linearization of the second  line of~(\ref{eq:mlcp2-toto1-DS2})} The linearization of the second line of the  problem~(\ref{eq:mlcp2-toto1-DS2}) is similar to the previous case so that (\ref{eq:NL11y}) is still valid.

 \paragraph{Newton's linearization of the third  line of~(\ref{eq:mlcp2-toto1-DS2})}
Since $ K^{\alpha}_{k+1} = \nabla_xg(t_{k+1},\lambda ^{\alpha}_{k+1}) = 0 $, the linearization of the third line of (\ref{eq:mlcp2-toto1-DS2}) reads as
\begin{equation}
  \label{eq:mlcp2-rrL}
  \begin{array}{l}
    \boxed{r^{\alpha+1}_{k+1} = g(t_{k+1},\lambda ^{\alpha}_{k+1})     + B^{\alpha}_{k+1} ( \lambda^{\alpha+1}-  \lambda^{\alpha}_{k+1} )}
  \end{array}
\end{equation}

\paragraph{Reduction to a linear relation between  $x^{\alpha+1}_{k+1}$ and
$\lambda^{\alpha+1}_{k+1}$}

Inserting (\ref{eq:mlcp2-rrL}) into~(\ref{eq:rfree-11}), we get the following linear relation between $x^{\alpha+1}_{k+1}$ and
$\lambda^{\alpha+1}_{k+1}$, we get the linear relation
\begin{equation}
  \label{eq:mlcp2-rfree-13}
  \begin{array}{l}
 \boxed{   x^{\alpha+1}_{k+1}\stackrel{\Delta}{=} x^\alpha_p + \left[ h \gamma (W^{\alpha}_{k+1})^{-1}    B^{\alpha}_{k+1} \lambda^{\alpha+1}_{k+1}\right]}
   \end{array}
\end{equation}
with
\begin{equation}
  \boxed{x^\alpha_p \stackrel{\Delta}{=}  h\gamma(W^{\alpha}_{k+1} )^{-1}\left[g(t_{k+1},\lambda^{\alpha}_{k+1})
    -B^{\alpha}_{k+1} (\lambda^{\alpha}_{k+1}) \right ] +x^\alpha_{\free}}
\end{equation}
and
\begin{equation}
  \label{eq:mlcp2-NL9}
  \begin{array}{l}
    W^{\alpha}_{k+1} \stackrel{\Delta}{=} M-h\theta A^{\alpha}_{k+1}\\
  \end{array}
\end{equation}

\paragraph{Reduction to a linear relation between  $y^{\alpha+1}_{k+1}$ and
$\lambda^{\alpha+1}_{k+1}$}

Inserting (\ref{eq:mlcp2-rfree-13}) into (\ref{eq:NL11y}), we get the following linear relation between $y^{\alpha+1}_{k+1}$ and $\lambda^{\alpha+1}_{k+1}$,
\begin{equation}
   \begin{array}{l}
 y^{\alpha+1}_{k+1} = y_p + \left[ h \gamma C^{\alpha}_{k+1} ( W^{\alpha}_{k+1})^{-1}  B^{\alpha}_{k+1} + D^{\alpha}_{k+1} \right]\lambda^{\alpha+1}_{k+1}
   \end{array}
\end{equation}
with
\begin{equation}\boxed{
y_p = y^{\alpha}_{k+1} -\mathcal R^{\alpha}_{yk+1} + C^{\alpha}_{k+1}(x_q) -
D^{\alpha}_{k+1} \lambda^{\alpha}_{k+1} }
\end{equation}
\textcolor{red}{
  \begin{equation}
    \boxed{ x^\alpha_q= x^\alpha_p - x^{\alpha}_{k+1}\label{eq:mlcp2-xqq}}
  \end{equation}
}

\subsection{The special case of Newton's linearization of~(\ref{eq:toto1}) with FirstOrderType1R~(\ref{first-DS1})}

Let us now proceed with the time discretization of~(\ref{eq:toto1}) with FirstOrderType1R~(\ref{first-DS1})  by a fully implicit scheme :
\begin{equation}
  \begin{array}{l}
    \label{eq:mlcp3-toto1-DS1}
     M x_{k+1} = M x_{k} +h\theta f(x_{k+1},t_{k+1})+h(1-\theta) f(x_k,t_k) + h \gamma r(t_{k+1})
     + h(1-\gamma)r(t_k)  \\[2mm]
     y_{k+1} =  h(t_{k+1},x_{k+1}) \\[2mm]
     r_{k+1} = g(t_{k+1}\lambda_{k+1})\\[2mm]
  \end{array}
\end{equation}

The previous derivation is valid with $ D^{\alpha}_{k+1} =0$.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "DevNotes"
%%% End:


\chapter{Stochastic Processes and Time Series Analysis}

\begin{multicols}{2}[\subsubsection*{Contents of this chapter}]
   \printcontents{}{1}{\setcounter{tocdepth}{2}}
\end{multicols}

In the discrete case, time series $X$ is a set of random variables $X = \{X_1, X_2, X_3,...\}$, or alternatively $X = \{X_t: t\in T\}$ where $T$ is the index set.

\section{Branching Processes}

\section{Markov Chains}
Markov Chains are time series in which the probability of a state depends only on the state preceding it. That is:

\begin{equation}
p(X_{t+1}|X_t, X{t-1},...,X_{1},X_0) = p(X_{t+1}|X_t)
\end{equation}

\section{Martingales}
Martingales are time series in which the expected value of a state is identical to the expected value of the state preceding it. That is:

\begin{equation}
\mathbb{E}X_{t+1} = \mathbb{E}X_{t}
\end{equation}

\subsection{Martingale Convergence Theorem}

\section{Hidden Markov Models}

\section{Ito Calculus}

\section{Chaos Analysis}

\section{Noise}

\subsection{White Noise}
White noise contains all frequencies with the same intensity. That is, it has constant power spectral density. Infinite bandwidth white noise is a purely theoretical concept, but if noise has a flat power spectrum across the frequencies of interest, it is referred to as white noise.

In discrete time, white noise is a series of uncorrelated random variables with zero mean and finite variance. When the variables all have normal distribution, the noise is referred to as Gaussian white noise.

\subsection{Gaussian Noise}
Gaussian noise is typically understood to be white noise in which the shocks have a normal distribution.

\subsection{Brownian Noise}
Brownian noise is noise corresponding to random motion (i.e. Brownian motion). The power spectrum is proportional to $\frac{1}{f^2}$.

\section{Empirical Processes}

\section{Mean Field Theory, Fluid Approximation, Deterministic Approximation}
In modeling phenomena with Markov Chain's, the number of elements constituting a system and the possible local states of that system may grow very large. This so-called \textit{state space explosion} is sometimes approached by approximating the system with a continuous state space. Those approaches include \textit{fluid approximation, mean field approximation} or sometimes the \textit{deterministic approximation}.

\section{Backtesting}


\documentclass[twoside,11pt]{article}\usepackage{amsmath,amsfonts,amsthm,fullpage}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{listings}
\setlength{\parindent}{0pt}
\usepackage{graphicx}
\usepackage{bm}
% Use the standard article template.
%
% The geometry package allows for easy page formatting.
\usepackage{geometry}
\geometry{letterpaper}
% Load up special logo commands.
\usepackage{doc}
% Package for formatting URLs.
\usepackage{url}
% Packages and definitions for graphics files.
\usepackage{epstopdf}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}

\def\argmin{\operatornamewithlimits{arg\, min}}
\newcommand{\rbr}[1]{\left(#1\right)}
\newcommand{\cbr}[1]{\left\{#1\right\}}
\newcommand{\Ncal}{\mathcal{N}}

%
% Set the title, author, and date.
%
\title{
Parallel Algorithm - Solve system of linear equations with Jacobi method with MPI
}
\author{
  D'Souza, Ajay\\
  \texttt{ajaydsouza@gatech.edu}
}
\date{\today}

\begin{document}

\maketitle

% Add an abstract.
\begin{abstract}
Implement Parallel Algorithm to solve a system of linear equations us the Jacobi method
\end{abstract}

% Add various lists on new pages.
\pagebreak
\tableofcontents

\pagebreak
\listoffigures

% Start the paper on a new page.
\pagebreak

%
% Body text.
%
\section{Algorithm Description}
\label{algorithm}
\begin{enumerate}
\item
We are given a square matrix of size $n$ and $p$ the available processors where $p$ is a perfect square. We will embed a logical square matrix of size  $\sqrt{p}\times\sqrt{p}$ on to the hypercubic processors $p$ using $\verb|MPI_Cart_Create|$.
\item
A property of this embedding is that the processors in a row or a column of the logical $\sqrt{p}\times\sqrt{p}$ matrix are hypercubic connected processors
\item
The $(0,0)$ processor or the rank $0$ processors reads the input matrix A,b and has to block distribute it to the other relevant processors
\item
The processors in the first column of the matrix have to get the vector $b$ block distributed to them from the rank $0$ processor in that column. This can be done by a $\verb|MPI_Scatterv|$ along the first column with the count and displacement arrays set to the appropriate $\lceil \frac{n}{\sqrt{p}} \rceil$ or $\lfloor \frac{n}{\sqrt{p}} \rfloor$ offset and count depending on if the rank of the processor is $< (n \mod \sqrt{p})$  or not. The complexity of this operation is $ \mathcal{O}(\tau \log \sqrt{p} + \mu n)$
\item
Next we have to distribute the $n\times n$ matrix A from processor of rank $0$ to every processor in the matrix so each processor has a local A block matrix of size $\frac{n}{\sqrt{p}} \times \frac{n}{\sqrt{p}}$.
\item
This is implemented using $\verb|MPI_Scatterv|$. The count and displacement arrays let us provide varying size of rows and columns that need to be block distributed to a processor based on its rank and of $n$ not being a multiple of $\sqrt{p}$. Matrix A is stored as an one dimensional array and non contiguous elements from the array would need to be scattered. This can be accomplished with creating a $\verb|MPI_Type_vector|$ which takes as input a count,block length and stride
\item
Since this involved scattering the elements along column and then along rows of processors which are hypercubic the run time complexity is
\begin{align}
&= \mathcal{O}(\tau \log \sqrt{p} + \mu n^2) + \mathcal{O}(\tau \log \sqrt{p} + \mu \frac{n^2}{\sqrt{p}})\nonumber\\
&= \mathcal{O}(\tau \log \sqrt{p} + \mu n^2) \label{l_1}
\end{align}
\item
Once the processors in the first column receive the block distributed rows of matrix A, those processors can keep an array local $local_D$ of $\frac{n}{\sqrt{p}}$ elements from the block distributed $local_A$, which correspond to the diagonal elements of the $n\times n$ matrix A.
\item
Or these diagonal elements of the $n\times n$ matrix A, could also be got by the processors in the first column, from the local block matrix A $local_A$ in the diagonal processors in the row of each column $(i,i)$. The sending and receiving can be performed by $\verb|MPI_Send|$ and $\verb|MPI_Recv|$ for a complexity of $\mathcal{O}(1)$
\item
Finally to perform the matrix multiplication $Ax$, each processor in column $i$ would need to have the bock distributed $local_x$ vector of size $\frac{n}{\sqrt{p}}$ from the $i^{th}$ processor in the first column
\item
This transpose broadcast of local $x$ can be performed using the following steps
\begin{enumerate}
\item
Each $i{th}$ processor in the left column will send its local $local_x$ array of size $\frac{n}{\sqrt{p}}$ to the diagonal processor ($i,i)$ in that row using $\verb|MPI_Send|$ and $\verb|MPI_Recv|$, this is of complexity $\mathcal{O}(1)$
\item
The diagonal processor $(i,i)$ in every row will then broadcast these $\frac{n}{\sqrt{p}}$ local $local_x$ elements to each processor in its column using $\verb|MPI_Bcast|$, this is of complexity $\mathcal{O}(\tau+\mu \frac{n}{\sqrt{p}}) \log \sqrt{p}$
\end{enumerate}
\item
Now each processor in the processor matrix of size $\sqrt{p}\times \sqrt{p}$ has a block allocated local A matrix $local_A$ and a local x vector $local_x$ and can perform a matrix vector multiplication
\item
Matrix multiplication is performed in each processor with a complexity of $\mathcal{O}({(\frac{n}{\sqrt{p}})}^2)$
\item
The multiplication results in a local y $local_y$ vector of size  $\frac{n}{\sqrt{p}}$ on each processor.
\item
Now the corresponding elements of local y, in every processor in a row are added and stored in the local $y$ vector of size $\frac{n}{\sqrt{p}}$ of the processor in the first column in each row using. This operation can be performed with $\verb|MPI_Reduce|$ with a complexity of $\mathcal{O}(\tau  + \mu \frac{n}{\sqrt{p}} ) \log \sqrt{p}$
\item
The processors in the first column now have the local $local_y$,$local_D$ and $local_b$ arrays of size $\frac{n}{\sqrt{p}}$. Then can now compute the the new x as $x_i = \frac{(local_{b_i} - (local{y_i} -local_{D_i}*local_{x_i}))}{local_{D_i}}$
\item
The new x computed in the processors in the first column in each row is stored in the $local_x$ vector on that processor
\item
Now this new $local_x$ is transpose distributed back from the $i^{th}$ processor in the first row to the processors in the $i^{th}$ column using transpose broadcast as described earlier
\item
With the new $\frac{n}{\sqrt{p}}$ local $local_x$ values at each processor will perform another matrix multiplication $local\_y= local\_A local_x$ as described above.
\item
The results from local y from the processors in a row are summed up back to the first processors in that row as described earlier
\item
Each processor in the first column can then compute the 2 norm of $|local_b-local_y|$
\item
We can then do a $\verb|MPI_AllReduce|$ to sum the local 2 norm of all processors in the grid to get the 2 norm of $|b-AX_{new}|$. The value of the 2 norm for the whole matrix will now be available on each processor in the matrix.
\item
This way every processor can check if the 2 norm is within the tolerance specified and exit the loop in the same way as other processors. This step of computing the norm using  $\verb|MPI_AllReduce|$ should be of complexity
\begin{align}
&= \mathcal{O}(\tau+\mu)\log \sqrt{p} + \mathcal{O}(\tau+\mu)\log \sqrt{p}\nonumber\\
&= \mathcal{O}(\tau+\mu)\log \sqrt{p} \label{l_2}
\end{align}
\item
If the 2 norm of $b-AX_{new}$ is beyond the tolerance specified and if we have still not reached the cut off iteration then each processor will continue the iteration
\item
On continuing the iteration the first column processors already have the value of the new $local_y$ from the last $AX_{new}$.So the first column processors can compute the new $local_{x_{new}}$ and then transpose broadcast it as described earlier and continue till convergence
\item
On termination of the iteration, only the processors in the first column need to send their $local_x$ values to the rank $0$ processor. This can be done using $\verb|MPI_Gather|$. This takes $\log \sqrt{p}$ steps and since we start by sending message of size $\frac{n}{\sqrt{p}}$ till we reach $\frac{n}{2}$. So the complexity of this operation will be $\mathcal{O}(\tau \log \sqrt{p} + \mu n)$
\end{enumerate}

\pagebreak
\section{Performance Plots}
The algorithm was executed for the following parameters
\begin{itemize}
\item
size n = 500,1000,2000,4000,8000,10000,12000,14000,16000,16000,20000,25000
\item
Difficulty d = 0.5,0.75,0.9,0.95
\item
Number of parallel processors p=4,9,16,25
\end{itemize}

\subsection{Runtime Performance Plots}

The following are the Run Time plots

\subsubsection{Runtime Performance for Serial Algorithm P=1}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_n_d_p_1_serial}
\caption{Serial Runtime Vs N}
\label{Serial Runtime Vs N}
\end{figure}

\pagebreak
\subsubsection{Runtime Performance - Plot of Run Time Vs N for different levels of difficulty for fixed No of Processors P}
The following is the performance plot of Runtime Vs N the matrix dimension for different levels of difficulty for a fixed number of processors.

\begin{enumerate}
\item
P=4 $\eqref{Run Time Vs N for P=4}$
\item
P=9 $\eqref{Run Time Vs N for P=9}$
\item
P=16 $\eqref{Run Time Vs N for P=16}$
\item
P=25 $\eqref{Run Time Vs N for P==25}$
\end{enumerate}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_n_d_p_4}
\caption{Run Time Vs N for P=4}
\label{Run Time Vs N for P=4}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_n_d_p_9}
\caption{Run Time Vs N for P=9}
\label{Run Time Vs N for P=9}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_n_d_p_16}
\caption{Run Time Vs N for P=16}
\label{Run Time Vs N for P=16}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_n_d_p_25}
\caption{Run Time Vs N for P==25}
\label{Run Time Vs N for P==25}
\end{figure}

\pagebreak
\subsubsection{Runtime Performance - Plot of Run Time Vs Number of Processors P for different levels of difficulty for fixed size N}
The following is the performance plot of Run Time Vs Number of Processors P for different levels of difficulty for fixed matrix size N

\begin{enumerate}
\item
N=1000 $\eqref{Run Time Vs P for N=1000}$
\item
N=8000 $\eqref{Run Time Vs P for N=8000}$
\item
N=16000 $\eqref{Run Time Vs P for N=16000}$
\item
N=20000 $\eqref{Run Time Vs P for N=20000}$
\item
N=25000 $\eqref{Run Time Vs P for N=25000}$
\end{enumerate}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_d_n_1000}
\caption{Run Time Vs P for N=1000}
\label{Run Time Vs P for N=1000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_d_n_8000}
\caption{Run Time Vs P for N=8000}
\label{Run Time Vs P for N=8000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_d_n_16000}
\caption{Run Time Vs P for N=16000}
\label{Run Time Vs P for N=16000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_d_n_20000}
\caption{Run Time Vs P for N=20000}
\label{Run Time Vs P for N=20000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_d_n_25000}
\caption{Run Time Vs P for N=25000}
\label{Run Time Vs P for N=25000}
\end{figure}

\pagebreak
\subsubsection{Runtime Performance - Plot of Run Time Vs Number of Processors P for different N for fixed Difficulty d}
The following is the performance plot of Run Time Vs Number of Processors P for different N for fixed Difficulty d

\begin{enumerate}
\item
d=0.5 $\eqref{Run Time Vs P for d=0.50}$
\item
d=0.75 $\eqref{Run Time Vs P for d=0.75}$
\item
d=0.9 $\eqref{Run Time Vs P for d=0.90}$
\item
d=0.95 $\eqref{Run Time Vs P for d=0.95}$
\end{enumerate}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_n_d_50}
\caption{Run Time Vs P for d=0.50}
\label{Run Time Vs P for d=0.50}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_n_d_75}
\caption{Run Time Vs P for d=0.75}
\label{Run Time Vs P for d=0.75}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_n_d_90}
\caption{Run Time Vs P for d=0.90}
\label{Run Time Vs P for d=0.90}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/runtime_p_n_d_95}
\caption{Run Time Vs P for d=0.95}
\label{Run Time Vs P for d=0.95}
\end{figure}

\subsection{SpeedUp Plots}

The following are the SpeedUp Ratio plots

\pagebreak
\subsubsection{SpeedUp - Plot of SpeedUp Vs N for different levels of difficulty for fixed No of Processors P}
The following is the performance plot of SpeedUp Vs N the matrix dimension for different levels of difficulty for a fixed number of processors.

\begin{enumerate}
\item
P=4 $\eqref{SpeedUp Vs N for P=4}$
\item
P=9 $\eqref{SpeedUp Vs N for P=9}$
\item
P=16 $\eqref{SpeedUp Vs N for P=16}$
\item
P=25 $\eqref{SpeedUp Vs N for P==25}$
\end{enumerate}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_n_d_p_4}
\caption{SpeedUp Vs N for P=4}
\label{SpeedUp Vs N for P=4}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_n_d_p_9}
\caption{SpeedUp Vs N for P=9}
\label{SpeedUp Vs N for P=9}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_n_d_p_16}
\caption{SpeedUp Vs N for P=16}
\label{SpeedUp Vs N for P=16}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_n_d_p_25}
\caption{SpeedUp Vs N for P==25}
\label{SpeedUp Vs N for P==25}
\end{figure}

\pagebreak
\subsubsection{Runtime Performance - Plot of SpeedUp Vs Number of Processors P for different levels of difficulty for fixed size N}
The following is the performance plot of SpeedUp Vs Number of Processors P for different levels of difficulty for fixed matrix size N

\begin{enumerate}
\item
N=1000 $\eqref{SpeedUp Vs P for N=1000}$
\item
N=8000 $\eqref{SpeedUp Vs P for N=8000}$
\item
N=16000 $\eqref{SpeedUp Vs P for N=16000}$
\item
N=20000 $\eqref{SpeedUp Vs P for N=20000}$
\item
N=25000 $\eqref{SpeedUp Vs P for N=25000}$
\end{enumerate}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_d_n_1000}
\caption{SpeedUp Vs P for N=1000}
\label{SpeedUp Vs P for N=1000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_d_n_8000}
\caption{SpeedUp Vs P for N=8000}
\label{SpeedUp Vs P for N=8000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_d_n_16000}
\caption{SpeedUp Vs P for N=16000}
\label{SpeedUp Vs P for N=16000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_d_n_20000}
\caption{SpeedUp Vs P for N=20000}
\label{SpeedUp Vs P for N=20000}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_d_n_25000}
\caption{SpeedUp Vs P for N=25000}
\label{SpeedUp Vs P for N=25000}
\end{figure}

\pagebreak
\subsubsection{Runtime Performance - Plot of SpeedUp Vs Number of Processors P for different N for fixed Difficulty d}
The following is the performance plot of SpeedUp Vs Number of Processors P for different N for fixed Difficulty d

\begin{enumerate}
\item
d=0.5 $\eqref{SpeedUp Vs P for d=0.50}$
\item
d=0.75 $\eqref{SpeedUp Vs P for d=0.75}$
\item
d=0.9 $\eqref{SpeedUp Vs P for d=0.90}$
\item
d=0.95 $\eqref{SpeedUp Vs P for d=0.95}$
\end{enumerate}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_n_d_50}
\caption{SpeedUp Vs P for d=0.50}
\label{SpeedUp Vs P for d=0.50}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_n_d_75}
\caption{SpeedUp Vs P for d=0.75}
\label{SpeedUp Vs P for d=0.75}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_n_d_90}
\caption{SpeedUp Vs P for d=0.90}
\label{SpeedUp Vs P for d=0.90}
\end{figure}

\begin{figure}[!htbp]
\centering
\includegraphics[scale=.46]{charts/speedup_p_n_d_95}
\caption{SpeedUp Vs P for d=0.95}
\label{SpeedUp Vs P for d=0.95}
\end{figure}

\pagebreak
\section{Conclusions}
\begin{itemize}
\item
The serial runtime of the algorithm is of the order of
\begin{align}
T(n,1) &= iterations * \mathcal{O}(n^2)\label{p1_runtime_1}
\end{align}
\item
The parallel algorithm has a complexity of
\begin{align}
T(n,p) &= iterations*[\mathcal{O}(\frac{n^2}{p})] + iterations*[\mathcal{O}(\tau \log \sqrt{p} + \mu \frac{n}{\sqrt{p}})] + \mathcal{O}(\tau \log \sqrt{p} + \mu n^2) \label{p1_runtime_2}
\end{align}
\item
At very small values of $n$, the communication overhead given by $iterations*[\mathcal{O}(\tau \log \sqrt{p} + \mu \frac{n}{\sqrt{p}})] + \mathcal{O}(\tau \log \sqrt{p} + \mu n^2)$, especially $\tau \log \sqrt{p}$ would dominate over computation costs. So at very low values of $n$ it might make no sense to try to parallel algorithm as it has no speed up over the serial algorithm. This is borne out by the graphs at values of $n<1000$
\item
For the Serial algorithm the run time increases with $N$ and is higher for higher difficulty levels
\item
For the Parallel algorithm for a given number of fixed processors, the run time
\begin{enumerate}
\item
Increases as $N$ is increased
\item
Is higher with increased difficulty
\item
Increases at a higher rate at higher values on $N$
\end{enumerate}
\item
For the Parallel algorithm if $N$ is fixed then
\begin{enumerate}
\item
Initially for low values of $p$, the SpeedUp increases as we increase the number of processors $p$
\item
We have a point of inflexion in the curve where run time is lowest at a certain value of $P$ and then increases again as P is increased. This indicates that for a given problem size of $N$ there is a optimum number of processors $P$ which give the best run time. The optimum value of $P$ for a fixed problem size of $N$ is the value at which the computation cost of the parallel algorithm $iterations*[\mathcal{O}(\frac{n^2}{p})]$ still dominates over the communication cost $iterations*[\mathcal{O}(\tau \log \sqrt{p} + \mu \frac{n}{\sqrt{p}})] + \mathcal{O}(\tau \log \sqrt{p} + \mu n^2)$
\item
It can also seen that the value of the point of inflexion , ie $P$ the number of processors at which runtime is the least, gets higher as the fixed value of $N$ is increased. This indicates that as $N$ is increased the computation cost of the parallel algorithm $iterations*[\mathcal{O}(\frac{n^2}{p})]$ is higher and thus can absorb the larger communication cost $iterations*[\mathcal{O}(\tau \log \sqrt{p} + \mu \frac{n}{\sqrt{p}})] + \mathcal{O}(\tau \log \sqrt{p} + \mu n^2)$ of a larger $N$
\end{enumerate}
\item
Higher the difficulty level higher is the run time. The spread of the run time is higher at higher values of $N$
\item
As seen with Run Time, SpeedUp is also fairly stable in a small range but falls off gradually as either $N$ decreases below a threshold or is increased beyond a threshold for a fixed number of processors
\begin{enumerate}
\item
SpeedUp decreases as $N$ is increased beyond a certain threshold for a given fixed number of processors, showing that for a given fixed number of processors as $N$ is increased communication ( specifically the brandwidth overhead in  $iterations*[\mathcal{O}(\tau \log \sqrt{p} + \mu \frac{n}{\sqrt{p}})] + \mathcal{O}(\tau \log \sqrt{p} + \mu n^2)$) begins to dominate over computation cost for the Parallel Algorithm.
\end{enumerate}
\item
It can be seen from the graph that SpeedUp $<1$ for values of $N < 1000 $. This is again as expected as communication costs of the parallel algorithm will dominate over computation costs at low values of $N$
\item
Thus we can conclude that
\begin{enumerate}
\item
At low values of $N$, parallel algorithm does not have a utility because of communication overheads
\item
For a given fixed value of $N$, SpeedUp of the parallel algorithm increases initially as $P$ is increased from a low value of $P$. However on reaching a optimum value of SpeedUp for a certain value of $P$, the SpeedUp then begins to fall gradually as $P$ is increased further
\item
So for a given problem size of $N$ there is a optimum value of $P$ to get the best SpeedUp
\end{enumerate}
\end{itemize}

% Generate the bibliography.
\bibliography{latex-sample}
\bibliographystyle{unsrt}

\end{document}


\documentclass[letterpaper]{article}

\usepackage[utf8]{inputenc}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{January 26, 2015}
\maketitle
\section*{non measurable set example}
$x,y\in [0,1)$

$x\oplus y:\begin{cases}x+y&\text{if }x+y<1\\x+y-1&\text{if }x+y\ge 1\end{cases}$

\subsection*{facts}
\begin{enumerate}
\item
$x\oplus y=y\oplus x$
\item
$(x\oplus y)\oplus z=x\oplus(y\oplus z)$
\end{enumerate}

if $E\subseteq[0,1)$ and $x\in [0,1)$ then define $E\oplus x:\{z:y\oplus x=z \text{ with }y\in E\}$

\subsection*{lemma}
if $E\subseteq[0,1)$ is measurable and $x\in[0,1)$ then $E\oplus x$ is measurable and $m*(E\oplus x)=m*(E)$
\subsubsection*{proof}
$E_1=E\cap[0,1-x), E_2=E\cap[1-x,1)$ where $E_1$ is regular addition, and $E_2$ is overflow.

notice $E_1$ and $E_2$ are measurable. Now $E_1\cup E_2=E$ and because they are disjoint and measurable then $m*(E_1)+m*(E_2)=m*(E)$

$E_1\oplus x=E_1+ x$. Now $m*(E_1\oplus x)=m*(E_1+x)=m*(E_1)$ (from homework)

on the other hand $E_2\oplus x=E_2+x-1$ and so $m*(E_2\oplus x)=m*(E_2+x-1)=m*(E_2)$.

\begin{align*}
  m*(E)=m*(E_1)+m*(E_2)&=m*(E_1\oplus x)+m*(E_2\oplus x)\\
  &=m*((E_1\oplus x)\cup(E_2\oplus x))\\
  y&\in E_1\oplus x\\
  y&\in E_2\oplus x\\
  y&=x+z_1, z_1\in [0,1-x)\\
  y&=x+z_2-1, z_2\in [1-x,1)\\
  z_1&=z_2-1\\
  (E_1\oplus x)\cap (E_2\oplus x)&=\emptyset
  &=m*(E\oplus x)
\end{align*}

take $x,y\in [0,1)$ where $x\sim y$ if $x=y+q$ where $q$ is rational. claim is that this is equivalence relation.
\begin{enumerate}
\item
symmetry

$x=y+q\to y=x+(-q)$
\item
reflexivity

$x=x+0$ and $0\in \mathbb{Q}$
\item
transitivity

$x=y+q_1, y=z+q_2\to x=(z+q_2)+q_1=z+(q_1+q_2)$
\end{enumerate}

$[0,1)/\sim=$ equivalence classes (space modulo equivalence classes)

$[0,1)=\cup[x]_\sim\leftarrow$ disjoint

\section*{axiom of choice}
if you have a collection of sets, then you can choose something from each set.

now for each equivalence class, choose one element. the collection of these choices is $P$

every rational number is equivalent to every other rational and so $P$ contains one and only one rational number.

one can show that for example $\pi/4$ will create it's own equivalence class. and $e/3$ will also have it's own class. It is difficult to show that $e$ and $\pi$ are in their own equivalence classes. $P$ has at least 3 elements. Actually it has an uncountable number of elements.

$\{r_i\}_{i=0}^\infty$ is an enumeration of $\mathbb{Q}\cap[0,1)$ with $r_0=0$.This list is the rationals in any order, but $0$ is at the beginning

let $P_i=P\oplus r_i$
\begin{enumerate}
\item
$m*(P_i)=m*(P)$
\item
$P_0=P$
\item
$P_i\cap P_j=\emptyset$ if $i\ne j$.

if $x\in P_i\cap P_j$ then $x=x_1\oplus r_i=x_2\oplus r_j$ where $x_1,x_2\in P$.

$x_1+r_1=x_2+r_2\to x_1=x_2 +(r_2-r_1)$

$x_1+r_1-1=x_2+r_2\to x_1=x_2+(r_2-r_1+1)$

$x_1+r_1=x_2+r_2-1\to x_1=x_2+(r_2-1-r_1)$

$x_1+r_1-1=x_2+r_2-1\to x_1=x_2+(r_2-r_1)$

and so $x_1\sim x_2$ and so $x$ is not in both and we have a contradiction
\item
$\bigcap_{i=1}^\infty P_i=[0,1)$

$x\in [0,1)$ and $x\sim y$ with $y\in P$ and $x=y+q$ with $q\in \mathbb{Q}$

$q\in \mathbb{Q}$ implies $q=r_j$ for some $j$ so $x\in P_j$.
\end{enumerate}

if $P$ is measurable then so is $P_i$ for all $i$ and so $1=m*([0,1))=m*(\bigcup_{i=1}^infty P_i=\sum\limits_{i=1}^\infty{m*(P_i)}=\sum\limits_{i=1}^\infty{m*(P)}$. Now either the sum is 0 and we can't get to one, or it's not 0 and we can get bigger than one. contradiction

\end{document}


% !TEX root = ./Basilisk-avsLibrary20170812.tex

\section{Test Description and Success Criteria}
The AVS support libraries undergo a series of unit test within C-functions that are called by the Swig'd interface {\tt avsLibrarySelfCheck}.  The automated unit test is performed via {\tt pytest} through the script:
\begin{verbatim}
     simulation/utilitiesSelfCheck/_UnitTest/test_avsLibrarySelfCheck.py
\end{verbatim}
In each unit test the C-code library function output is compared to a externally hand-computed solution.  The success criteria is when the pre-computed solution and the C-code computed solution match to within a certain numerical tolerance.  The success criteria absolute value tolerances are shown in Table~\ref{tbl:tolerance}.

\begin{table}[htbp]
	\caption{Unit Test Absolute Difference Tolerances}
	\label{tbl:tolerance}
	\centering \fontsize{10}{10}\selectfont
	\begin{tabular}{c | c} % Column formatting,
		\hline
		\hline
		Test    & Absolute Tolerance \\
		\hline
		  Linear Algebra    & $10^{-10}$ \\
		  Rigid Body Kinematics    & $10^{-10}$ \\
		  Orbital Motion: Anomalies    & $10^{-10}$ \\
		  Orbital Motion: Element conversion    & $10^{-10}$ \\
		  Orbital Motion: Environment    & $10^{-10}$ \\
		\hline
		\hline
	\end{tabular}
\end{table}

\section{Test Parameters}

\subsection{Linear Algebra}
In these unit tests each function is provided non-trivial inputs, such as {\tt v1 = [1,2,3]} to evaluate the associate matrix math.  For specific information on the values used, look at the source code for the function {\tt testLinearAlgebra()} in the file {\tt avsLibrarySelfCheck.c}.

\subsection{Rigid Body Kinematics}
The attitude kinematics sub-routines are provided with non-trivial inputs to test the functions.   For specific information on the values used, look at the source code for the function {\tt testRigidBodyKinematics()} in the file {\tt avsLibrarySelfCheck.c}.

\subsection{Orbital Motion}
The orbital anomaly sub-routines are provided with non-trivial inputs to test the functions.   For specific information on the values used, look at the source code for the function {\tt testOrbitalAnomalies()} in the file {\tt avsLibrarySelfCheck.c}.

Regarding the orbit element to Cartesian coordinate conversion routines, an Earth orbit is simulated that is either non-equatorial and non-circular (case 1), non-circular equatorial (case 2), circular and inclined (case 3) and circular equatorial (case 4).  The resulting orbit elements used are shown in Table~\ref{tbl:orbitValues}.

\begin{table}[htbp]
	\caption{Orbit Elements Used in Orbit Coordinate Conversion Routine Checking}
	\label{tbl:orbitValues}
	\centering \fontsize{10}{10}\selectfont
	\begin{tabular}{c || c | c | c | c | c | c} % Column formatting,
		\hline
		\hline
		Case    & SMA  [km]& $e$ & $i$[\dg] & $\Omega$[\dg\ & $\omega$[\dg\ & $f$[\dg] \\
		\hline
		  1   &  7500 & 0.5 & 40 & 133 & 113 & 123 \\
		  2   &  7500 & 0.5 & 0 & 133 & 113 & 123 \\
		  3   &  7500 & 0.0 & 40 & 133 & 113 & 123 \\
		  4   &  7500 & 0.0 & 0 & 133 & 113 & 123 \\
		\hline
		\hline
	\end{tabular}
\end{table}

\section{Test Results}
An automated suite of tests are run to perform unit tests on all the AVS support library functions.  The results are shown in Table~\ref{tbl:avsLibraryResults}.

\begin{table}[h]
	\caption{Integration test results.}
	\label{tbl:avsLibraryResults}
	\centering \fontsize{10}{10}\selectfont
	\begin{tabular}{c | c | l } % Column formatting,
		\hline\hline
		\textbf{Test} 			& \textbf{Pass/Fail} 	 & \textbf{BSK Error Notes}
		\\ \hline
		{\tt rigidBodyKinematics}		  	&
		\input{AutoTex/test_avsLibrarySelfCheckTestMsg-testRigidBodyKinematics}      	  &
		\input{AutoTex/test_avsLibrarySelfCheckMsg-testRigidBodyKinematics}
		\\
		{\tt orbitalElements}		  	&
		\input{AutoTex/test_avsLibrarySelfCheckTestMsg-testOrbitalElements}      	  &
		\input{AutoTex/test_avsLibrarySelfCheckMsg-testOrbitalElements}
		\\
		{\tt orbitalAnomalies}		  	&
		\input{AutoTex/test_avsLibrarySelfCheckTestMsg-testOrbitalAnomalies}      	  &
		\input{AutoTex/test_avsLibrarySelfCheckMsg-testOrbitalAnomalies}
		\\
		{\tt linearAlgebra}		  	&
		\input{AutoTex/test_avsLibrarySelfCheckTestMsg-testLinearAlgebra}      	  &
		\input{AutoTex/test_avsLibrarySelfCheckMsg-testLinearAlgebra}
		\\
		{\tt environment}		  	&
		\input{AutoTex/test_avsLibrarySelfCheckTestMsg-testEnvironment}      	  &
		\input{AutoTex/test_avsLibrarySelfCheckMsg-testEnvironment}
		\\
		\hline\hline
	\end{tabular}
\end{table}


\documentclass[aps,ams,prl,twocolumn,superscriptaddress,10pt]{revtex4-1}

\usepackage{graphicx}  % this is the up-to-date package for all figures
\usepackage{amssymb}   % for math
\usepackage{verbatim}  % for the comment environment
\usepackage{color}
\usepackage[hidelinks]{hyperref}
\usepackage{listings}	% for code snippets
\usepackage{gensymb}

% Settings for code appendices
\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=C++,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

%\bibliographystyle{apsrev}

% these are some custom control of the page size and margins
% \topmargin= 0.2in  % these 1st two may be needed for some computers
\textheight=9in
\textwidth=6.5in
% these next two lines give us centered text
\oddsidemargin=0cm
\evensidemargin=0cm

\begin{document}

\pagestyle{headings}

\title{Polar Stereographic\\ Heightfield Intersection}

\author{Lukas R. Hertrich}
\affiliation{Student Researcher at University of Hawaii at Manoa}

\begin{abstract}
In order to simulate neutrino and RF traversal of the Antarctic firn, raycasting may be used. This is accomplished by mapping bedrock elevation and ice sheet thickness
data, provided by projects such as BEDMAP 2, to the WGS84 ellipsoid.
\end{abstract}

\maketitle

\section{Data Format}
Antarctic elevation and ice sheet data are provided by BEDMAP 2 in Antarctic polar stereographic projection with a 1 km resolution.
The projection plane is located at 71$\degree$ south latitude and rays are cast down from the north pole to intersect the projection plane and ellipsoid.
The angle is geodetic latitude determined by WGS84.
\section{Raycast}
The 3D raycast is parameterized with an initial position $\vec{r_0}$, a direction $\hat{d}$, and a parameter $\tau$, yielding
\begin{equation}\label{eqn:r_of_tau}
 \vec{r}(\tau) = \vec{r_0} + \tau \hat{d}
\end{equation}
This ray traverses flat, unmodified 3D cartesian space, meaning that distance traversed by the ray is equal to $\tau$.
The same ray may be parameterized in terms of normalized cartesian coordinates,
meaning that the $\vec{r_0}$ terms $r_{0x}$ and $r_{0y}$ are divided by the equatorial radius $R_{eq}$ of the WGS84 ellipsoid,
and the $r_{0z}$ term is divided by the polar radius $R_{po}$.
This yields a magnitude of radius relative to the radius of the ellipsoid in the given direction, and a ray that intersects the ellipsoid's surface
where $r_{el}(\tau) = 1$.
The magnitude of $r_{el}$ may be determined as follows:
\begin{eqnarray*}\label{eqn:r_el}
 r_{el} & = & \left[\frac{r_{0x}^2+r_{0y}^2}{R_{eq}^2}+\frac{r_{0z}^2}{R_{po}^2}\right. \\
 & & {} + 2\tau\left(\frac{r_{0x} d_x+r_{0y} d_y}{R_{eq}^2}+\frac{r_{0z} d_z}{R_{po}^2}\right) \\
 & & {} +\left. \tau^2 \left( \frac{d_x^2+d_y^2}{R_{eq}^2} + \frac{d_z^2}{R_{po}^2} \right)\right]^{\frac{1}{2}}
\end{eqnarray}
\section{Heightfield Intersection}
The ray intersects the heightfield when the normalized surface radius $r_{surf}(\tau)$ is equal to the normalized ellipsoidal radius $r_{el}(\tau)$.
Their difference is then zero. Likewise the difference of their squares is zero.
\begin{equation}\label{eqn:intersection}
0 = r_{surf}(\tau) - r_{el}(\tau)
\end{equation}\\
$r_{el}(\tau)$ may be determined by Eqn. (\ref{eqn:r_el}).
$r_{surf}(\tau)$ is unity added to the quotient of the interpolated height $h(\tau)$ and the unnormalized magnitude of the ellipsoid's radius $R(\tau)$ at the given latitude.
\begin{equation}\label{eqn:r_surf}
 r_{surf}(\tau) = 1 + \frac{h(\tau)}{R(\tau)}
\end{equation}
The interpolated height $h(\tau)$ is determined by generating the vector $\vec{v}_{proj}$ from the north pole of the ellipsoid $(0, 0, R_{po})$
to the point of ray traversal $\vec{r}(\tau)$ from Eqn. (\ref{eqn:r_of_tau}),
then using the point of intersection of this vector with the projection plane to calculate (u,v) coordinates with which to poll the heightfield data.
The point of intersection with the projection plane is found by determining how far the direction of $\vec{v}_{proj}$ must be traversed, in terms of its $\hat{z}$ component,
in order to hit the constant $\hat{z}$ position of the plane $z_{plane}$. This is accomplished by scaling $\vec{v}_{proj}$ by $\alpha_{proj}$ to form $\vec{v'}_{proj}$.
\begin{equation}\label{eqn:v_proj}
 \vec{v'}_{proj} = \alpha_{proj} \vec{v}_{proj}
\end{equation}
The scalar $\alpha_{proj}$ is determined as follows, with $z(\tau)$ being the $z$ component of $\vec{r}(\tau)$ in Eqn. (\ref{eqn:r_of_tau}):
\begin{equation}\label{eqn:alpha_proj}
 \alpha_{proj} = \frac{-R_{po} - z_{plane}}{-R_{po} + z(\tau)}
\end{equation}
This yields the $(x'_{proj},y'_{proj})$ coordinates that may be converted to $(u,v)$ coordinates to poll and interpolate data.
\begin{equation}\label{eqn:x'y'_proj}
 (x'_{proj},y'_{proj}) = \alpha_{proj} (x_{proj},y_{proj})
\end{equation}
$R(\tau)$ may be determined from the geodetic latitude $\varphi$, measured from the equator.

\begin{equation}\label{eqn:phi}
 \varphi = \arcsin{\frac{z_{el}(\tau)}{r_{el}(\tau)}}
\end{equation}

\begin{equation}\label{eqn:R_of_phi}
 R(\varphi) = \sqrt{\frac{R_{eq}^4\cos^2{\varphi} + R_{po}^4\sin^2{\varphi}}{R_{eq}^2\cos^2{\varphi} + R_{po}^2\sin^2{\varphi}}}
\end{equation}
$R(\tau)$ may be expressed by substituting the expressions for $\cos^2{\varphi}$ and $\sin^2{\varphi}$ into Eqn. (\ref{eqn:R_of_phi}).
\begin{eqnarray*}\label{eqn:cos_sqr_phi}
 \cos^2{\varphi} & = & \frac{x_{el}^2 + y_{el}^2}{r_{el}^2} \\
 && = \frac{(r_{0x} + \tau d_x)^2 + (r_{0y} + \tau d_y)^2}{R_{eq}^2 r_{el}^2}
\end{eqnarray}
\begin{equation}\label{eqn:sin_sqr_phi}
 \sin^2{\varphi} = \frac{z_{el}^2}{r_{el}^2} = \frac{\left(r_{0z} + \tau d_z\right)^2}{R_{po}^2 r_{el}^2}
\end{equation}
Simplifying the substituted terms reduces $R(\tau)$ to:
\begin{eqnarray*}\label{eqn:R_of_tau}
 R(\tau) & = & \left[\frac{R_{eq}^2 \left[ (r_{0x} + \tau d_x)^2 + (r_{0y} + \tau d_y)^2 \right]}{r^2(\tau)}\right. \\
 && {} +\left.\frac{R_{po}^2 (r_{0z} + \tau d_{z})^2}{r^2(\tau)}\right]^{\frac{1}{2}}
\end{eqnarray}

\section{Interpolation of Height Data}
Precomputing the values of $r_{surf}^2$ at data grid points allows Eqn. (\ref{eqn:intersection}) to be solved analytically for a given data cell.
There is a negligible error introduced by interpolating the square of the surface radius rather than the radius itself.
The hypothetical error introduced by considering the unrealistic situation of adjacent maxima and minima of Antarctic elevation in a data cell yields an error on the order of one meter.
This is well within the range of uncertainty in the BEDMAP 2 measurements themselves, which document uncertainties of as much as 1 km for ice thickness data in certain regions.

Normalized $(u,v)$ coordinates for interpolation in the range $[0,1]$ are generated using the results of Eqn. (\ref{eqn:x'y'_proj}) and the coordinates at the corners of a given data cell.
\begin{equation}\label{eqn:u}
 u = \frac{x'_{proj} - x_{0,cell}}{x_{1,cell} - x_{0,cell}}
\end{equation}
\begin{equation}\label{eqn:v}
 v = \frac{y'_{proj} - y_{0,cell}}{y_{1,cell} - y_{0,cell}}
\end{equation}
Interpolation of the $r_{surf}^2$ values at the cell corners is accomplished as follows:
\begin{eqnarray*}\label{r_surf_sqr}
 r_{surf}^2 & = & r_{00}^2(1-u)(1-v) \\
 && {} + r_{10}^2(u)(1-v) \\
 && {} + r_{01}^2(1-u)(v) \\
 && {} + r_{11}^2uv
\end{eqnarray}
This results in a quadratic form, in terms of dependence on $\tau$, which can be shown by substituting terms successively and redefining constant expressions:
\begin{eqnarray*}\label{eqn:r_surf_quadratic}
 r_{surf}^2 & = & r_{00}^2 + u(r_{10}^2 - r_{00}^2) + v(r_{01}^2 - r_{00}^2) \\
 && {} + uv(r_{11}^2 - r_{01}^2 - r_{10}^2 - r_{00}^2)
\end{eqnarray}
\begin{eqnarray*}
 r_{surf}^2 & = & A + uB + vC + uvD
\end{eqnarray}
with $A = r_{00}^2$, $B = r_{10}^2 - r_{00}^2$, $C = r_{01}^2 - r_{00}^2$, and $D = r_{11}^2 - r_{01}^2 - r_{10}^2 - r_{00}^2$.
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{x'_{proj} - x_{0,cell}}{x_{1,cell} - x_{0,cell}}B \\
 && {} + \frac{y'_{proj} - y_{0,cell}}{y_{1,cell} - y_{0,cell}}C \\
 && {} + \left(\frac{x'_{proj} - x_{0,cell}}{x_{1,cell} - x_{0,cell}}\right)\left(\frac{y'_{proj} - y_{0,cell}}{y_{1,cell} - y_{0,cell}}\right) \\
 && {} \cdot D
\end{eqnarray}
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{x'_{proj} - x_{0,cell}}{E}B \\
 && {} + \frac{y'_{proj} - y_{0,cell}}{F}C \\
 && {} + \left(\frac{x'_{proj} - x_{0,cell}}{E}\right)\left(\frac{y'_{proj} - y_{0,cell}}{F}\right) \\
 && {} \cdot D
\end{eqnarray}
with $E = x_{1,cell} - x_{0,cell}$ and $F = y_{1,cell} - y_{0,cell}$.
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{\alpha_{proj}x(\tau) - x_{0,cell}}{E}B \\
 && {} + \frac{\alpha_{proj}y(\tau) - y_{0,cell}}{F}C \\
 && {} + \left(\frac{\alpha_{proj}x(\tau) - x_{0,cell}}{E}\right)\left(\frac{\alpha_{proj}y(\tau) - y_{0,cell}}{F}\right) \\
 && {} \cdot D
\end{eqnarray}
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{\frac{-R_{po} - z_{plane}}{-R_{po} + z(\tau)}x(\tau) - x_{0,cell}}{E}B \\
 && {} + \frac{\frac{-R_{po} - z_{plane}}{-R_{po} + z(\tau)}y(\tau) - y_{0,cell}}{F}C \\
 && {} + \left(\frac{\frac{-R_{po} - z_{plane}}{-R_{po} + z(\tau)}x(\tau) - x_{0,cell}}{E}\right) \\
 && {} \cdot \left(\frac{\frac{-R_{po} - z_{plane}}{-R_{po} + z(\tau)}y(\tau) - y_{0,cell}}{F}\right) \\
 && {} \cdot D
\end{eqnarray}
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{\frac{G (r_{0x} + d_x \tau)}{-R_{po} + r_{0z} + d_z \tau} - x_{0,cell}}{E}B \\
 && {} + \frac{\frac{G (r_{0y} + d_y \tau)}{-R_{po} + r_{0z} + d_z \tau} - y_{0,cell}}{F}C \\
 && {} + \left(\frac{\frac{G (r_{0x} + d_x \tau)}{-R_{po} + r_{0z} + d_z \tau} - x_{0,cell}}{E}\right) \\
 && {} \cdot \left(\frac{\frac{G (r_{0y} + d_y \tau)}{-R_{po} + r_{0z} + d_z \tau} - y_{0,cell}}{F}\right) \\
 && {} \cdot D
\end{eqnarray}
with $G = -R_{po} - z_{plane}$.
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{G (r_{0x} + d_x \tau) - x_{0,cell}(H + d_z \tau)}{H + d_z \tau}\cdot \frac{B}{E} \\
 && {} + \frac{G (r_{0y} + d_y \tau) - y_{0,cell}(H + d_z \tau)}{H + d_z \tau}\cdot \frac{C}{F} \\
 && {} + \left(\frac{G (r_{0x} + d_x \tau) - x_{0,cell}(H + d_z \tau)}{H + d_z \tau}\right) \\
 && {} \cdot \left(\frac{G (r_{0y} + d_y \tau) - y_{0,cell}(H + d_z \tau)}{H + d_z \tau}\right) \\
 && {} \cdot \frac{D}{EF}
\end{eqnarray}
with $H = -R_{po} + r_{0z}$.
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{I + J\tau}{H + d_z \tau}\cdot \frac{B}{E} \\
 && {} + \frac{K + L\tau}{H + d_z \tau}\cdot \frac{C}{F} \\
 && {} + \frac{I + J\tau}{H + d_z \tau} \\
 && {} \cdot \frac{K + L\tau}{H + d_z \tau} \\
 && {} \cdot \frac{D}{EF}
\end{eqnarray}
with $I = G r_{0x} - x_{0,cell}H$, $J = G d_x - x_{0,cell}d_z$, $K = G r_{0y} - y_{0,cell}H$, and $L = G d_y - y_{0,cell}d_z$.
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{M + N\tau}{H + d_z \tau} \\
 && {} + \frac{O + P\tau}{H + d_z \tau} \\
 && {} + \frac{I + J\tau}{H + d_z \tau} \\
 && {} \cdot \frac{K + L\tau}{H + d_z \tau} \\
 && {} \cdot \frac{D}{EF}
\end{eqnarray}
with $M = \frac{BI}{E}$, $N = \frac{BJ}{E}$, $O = \frac{CK}{F}$, and $P = \frac{CL}{F}$.
\begin{eqnarray*}
 r_{surf}^2 & = & A \\
 && {} + \frac{M + O + (N + P)\tau}{H + d_z \tau} \\
 && {} + \frac{IK + (IL + JK)\tau + JL\tau^2}{H^2 + 2Hd_z\tau + d_{z}^2 \tau^2} \\
 && {} \cdot \frac{D}{EF}
\end{eqnarray}
\begin{equation}\label{eqn:reduced}
 r_{surf}^2 & = & \frac{Q + R\tau + S\tau^2}{H^2 + 2Hd_z\tau + d_{z}^2 \tau^2}
\end{equation}
with $Q = AH^2 + H(M+O)+\frac{IKD}{EF}$, $R = 2AHd_z + H(N+P) + d_z(M+O) + \frac{D(IL+JK)}{EF}$, and $S = Ad_{z}^2 + d_z(N+P) + \frac{JLD}{EF}$.
\\
\\
Manipulating $r_{el}^2(\tau)$ to share the denominator of Eqn. (\ref{eqn:reduced}) yields a quartic equation over a quadratic equation.
Real roots of the quartic determine the values of $\tau$ that result in an intersection of the parameterized ray in
Eqn. (\ref{eqn:r_of_tau}) with the surface in the tested data cell.
Real roots of the quadratic in the denominator represent rays that pass tangentially through the north pole, yielding an undefined projection.
This method of interpolating the square of normalized radii yields analytical solutions that are valid within a data cell. If no solution exists,
the entry of the ray into an adjacent cell must be evaluated.
\section{Piecewise Quadratic Interpolation}
Another, perhaps more robust method, is the use of a quadratic approximation to the behavior of distance to surface
along segments of the ray.
\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[a4paper,top=2cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}
\geometry{a4paper}
\usepackage{amsmath}
\usepackage{physics}
\usepackage{color,soul}
\DeclareMathOperator{\di}{d\!}
\newcommand*\Eval[3]{\left.#1\right\rvert_{#2}^{#3}}
\newcommand{\inlist}[1]{\texttt{#1}}

\author{J L Kaplan}

\begin{document}

\section*{SICP Exercise 1.13}

\subsection*{Real number Fibonacci sequence}

Let $\tilde{f}(n) = \phi^n / \sqrt{5}$. Let $R(x)$ be a rounding function. First, let's check the base cases.

\begin{equation}
  R(\tilde{f}(0)) = R \left( \frac{1}{\sqrt{5}} \right) = 0,
\end{equation}

\begin{equation}
  R(\tilde{f}(1)) = R \left( \frac{1 + \sqrt{5}}{2 \sqrt{5}} \right) = 1.
\end{equation}

Now we can test whether the recurrence relation holds.

\begin{equation}
  \tilde{f}(n) = \tilde{f}(n-1) \tilde{f}(n-2),
\end{equation}

\begin{equation}
  \frac{(1 + \sqrt{5})^n}{2^n \sqrt{5}} = \frac{(1 + \sqrt{5})^{n-1}}{2^{n-1} \sqrt{5}} + \frac{(1 + \sqrt{5})^{n-2}}{2^{n-2} \sqrt{5}},
\end{equation}

\begin{equation}
  (1 + \sqrt{5})^2 = 2 (1 + \sqrt{5}) + 4,
\end{equation}

\begin{equation}
  6 + 2 \sqrt{5} = 6 + 2 \sqrt{5}.
\end{equation}

Thus the recurrence relation holds for the real number recurrence relation.

\subsection*{Integer Fibonacci sequence}

Introducing the rounding correction:

\begin{equation}
  \hat{f}(n) = \psi^n = \frac{(1 - \sqrt{5})^n}{2^n \sqrt{5}},
\end{equation}

in combination with the real number Fibonacci term

\begin{equation}
  F(n) = \tilde{f}(n) - \hat{f}(n) = \frac{\phi^n - \psi^n}{\sqrt{5}},
\end{equation}

we can check the base cases

\begin{equation}
  F(0) = \frac{1 - 1}{\sqrt{5}} = 0,
\end{equation}

\begin{equation}
  F(1) = \frac{1 + \sqrt{5} - (1 - \sqrt{5})}{2 \sqrt{5}} = 1.
\end{equation}

Again, test the recurrence relation holds.

\begin{equation}
  \frac{(1 + \sqrt{5})^{n} - (1 - \sqrt{5})^{n}}{2^n \sqrt{5}} = \frac{(1 + \sqrt{5})^{n-1} - (1 - \sqrt{5})^{n-1}}{2^{n-1} \sqrt{5}} + \frac{(1 + \sqrt{5})^{n-2} - (1 - \sqrt{5})^{n-2}}{2^{n-2} \sqrt{5}}
\end{equation}

\begin{equation}
  (1 + \sqrt{5})^{n} - (1 - \sqrt{5})^{n} = 2 \left[ (1 + \sqrt{5})^{n-1} - (1 - \sqrt{5})^{n-1} \right] + 4 \left[ (1 + \sqrt{5})^{n-2} - (1 - \sqrt{5})^{n-2} \right],
\end{equation}

which, after some algebra leads to

\begin{equation}
  (1 + \sqrt{5})^{2}) \frac{(1 + \sqrt{5})^{n-2}}{(1 - \sqrt{5})^{n-2}} = \frac{(1 + \sqrt{5})^{2} (1 + \sqrt{5})^{2n-4}}{(-4)^{n-2}},
\end{equation}

\begin{equation}
  \left[ (1 + \sqrt{5})^{2} - 2(1 + \sqrt{5}) - 4 \right] \frac{(1 + \sqrt{5})^{2n-4}}{(-4)^{n-2}} = (1 + \sqrt{5})^{2} - 2(1 + \sqrt{5}) - 4,
\end{equation}

from which it can be seen that both sides are equal to 0.

\subsection*{Boundedness of correction term}

As we can show that the correction term is always less than 1/2, we know that the nearest integer to a $\tilde{\phi}(n)$ term must be the corresponding Fibonacci number. Proof of boundedness is below.

\begin{equation}
  4 < 5 < 9,
\end{equation}

\begin{equation}
  2 < \sqrt{5} < 3,
\end{equation}

\begin{equation}
  -2 > - \sqrt{5} > -3,
\end{equation}

\begin{equation}
  -1 > 1 - \sqrt{5} > -2,
\end{equation}

\begin{equation}
  -\frac{1}{2} > \frac{1 - \sqrt{5}}{2} > -1,
\end{equation}

\begin{equation}
  0 > -\frac{-1}{2 \sqrt{5}} > \left( \frac{1 - \sqrt{5}}{2} \right) \frac{1}{\sqrt{5}} > -\frac{1}{\sqrt{5}} > - \frac{1}{2},
\end{equation}

or simplified we can see that

\begin{equation}
  0 > \left( \frac{1 - \sqrt{5}}{2} \right) \frac{1}{\sqrt{5}} > - \frac{1}{2},
\end{equation}

and as the term in the centre decreases in size as it is exponentiated, the inequality holds for all values of n.

\end{document}