\documentclass[main.tex]{subfiles}
\begin{document}

\marginpar{Wednesday\\ 2020-3-25, \\ compiled \\ \today}

Today we are going to examine the third case of wave equations: we discussed the KG equation for a scalar particle, then we moved to a vector particle, and now we are going to consider a spin-1/2 particle.

One has to prove that the equation \(\qty(i \slashed{\partial} - m)  \varphi (x )=0\) is Lorentz-covariant.
Also, solutions of the Dirac equation are also solutions of the Klein-Gordon equation, therefore they satisfy \(E^2=p^2+m^2\).
The latter is relatively easy to prove: we apply the operator \(i \slashed{\partial} + m\) to the Dirac equation and find
%
\begin{align}
\qty(i \slashed{\partial} + m) \qty(i \slashed{\partial} - m) \psi (x) &= 0  \\
\qty(- \gamma^{\mu } \gamma^{\nu } \partial_{\mu } \partial_{\nu } - m^2 ) \psi (x) &= 0  \\
(\square + m^2 )\psi (x) &= 0
\,,
\end{align}
%
where we used the fact that, since the derivatives \(\partial_{\mu } \partial_{\nu } \) commute, we can substitute \(\gamma^{\mu } \gamma^{\nu }\) with half of their anticommutator, which equals \(\eta^{\mu \nu }\).

Since we have proven the equivalence, we expect the Dirac equation to also have negative energy solutions.

Proving the covariance is harder, even though the equation looks covariant, since \(\gamma^{\mu }\) does not transform as a vector \emph{a priori}.

The object \(\psi (x)\) is a spinor: it is neither a scalar, nor a vector. A priori we can say that its transformation law will look like
%
\begin{align}
\psi (x) \rightarrow \psi^{\prime } (x) = S(\Lambda ) \psi (x)
\,,
\end{align}
%
where \(S(\Lambda )\) is a unitary transformation associated with the Lorentz transformation \(\Lambda \), and by imposing the covariance of the Dirac equation we find that we must have
%
\begin{align}
S (\Lambda ) \gamma^{\mu } S^{-1} (\Lambda ) = \qty(\Lambda^{-1})^{\mu }_{\nu } \gamma^{\nu }
\,.
\end{align}

So, in order to find the explicit form of \(S(\Lambda )\) we write an infinitesimal Lorentz transformation: it can be shown that we can write it as
%
\begin{align}
\tensor{\Lambda }{^{\mu }_{\nu }} = \tensor{\eta }{^{\mu }_{\nu }} + \delta \tensor{\omega}{^{\mu }_{\nu }}
\,,
\end{align}
%
where \(\delta \tensor{\omega }{^{\mu }_{\nu }}\) is an antisymmetric matrix, if it has nonzero \(0i\) components it gives boosts, if it has nonzero \(ij\) components it gives rotations.
The result for the form of \(S\) is:
%
\begin{align}
S = \mathbb{1} + \frac{1}{8 } \qty[\gamma_{\mu }, \gamma_{\nu }] \delta \tensor{\omega }{^{\mu \nu }}
\,.
\end{align}

If we make a rotation, for example, we get
%
\begin{align}
\psi (x) \rightarrow \exp(i \frac{i}{2} \qty[\gamma_{i}, \gamma_{j}] \delta \tensor{\omega }{^{ij}}) \psi (x)
\,,
\end{align}
%
and typically one uses the shorthand rotation
%
\begin{align}
\frac{i}{2} \qty[\gamma_{i}, \gamma_{j}] \overset{\text{def}}{=} \sigma_{ij}
\,.
\end{align}

If, for example, we want to perform a rotation by an angle \(\varphi \) around the \(z\) axis we get
%
\begin{align}
\psi' (x') = S(\Lambda) \psi (x) = \exp( \frac{i}{2} \varphi \sigma_{12}) \psi (x)
\,,
\end{align}
%
where
%
\begin{align}
\sigma_{12} = \frac{i}{2} \qty[\gamma^{1}, \gamma^{2}]
= \left[\begin{array}{cc}
\sigma_3 & 0 \\
0 & \sigma_3
\end{array}\right]
\,,
\end{align}
%
where we used the fact that \(\qty[\sigma_i, \sigma_{j}] = i \epsilon_{ijk}\sigma_{k}\).

This means that a spinor \(\psi (x)\) reacts in a peculiar way to rotations: it rotates by an angle \( \varphi /2 \) if we perform a Lorentz rotation of an angle \(\varphi \); its periodicity is \(4\pi \).

We introduce the \emph{chiral representation} of the gamma matrices:
%
\begin{subequations}
\begin{align}
\gamma^{0} = \left[\begin{array}{cc}
0 & -\mathbb{1} \\
- \mathbb{1} & 0
\end{array}\right]
\qquad \text{and} \qquad
\vec{\gamma} = \left[\begin{array}{cc}
0 & \vec{\sigma}  \\
-\vec{\sigma} & 0
\end{array}\right]
\,.
\end{align}
\end{subequations}

Then, for Lorentz boosts we have
%
\begin{subequations}
\begin{align}
\sigma_{0i} = \frac{1}{2} \qty[\gamma_0, \gamma_{i}] = -i \left[\begin{array}{cc}
\sigma_{i} & 0 \\
0 & - \sigma_{i}
\end{array}\right]
\,,
\end{align}
\end{subequations}
%
while for rotations we have
%
\begin{subequations}
\begin{align}
\sigma_{ij} = \frac{i}{2} \qty[\gamma_{i}, \gamma_{j}] = \epsilon_{ijk} \left[\begin{array}{cc}
\sigma_{k} & 0 \\
0 & \sigma_{k}
\end{array}\right]
\,,
\end{align}
\end{subequations}
%
which is useful since it gives us block-diagonal matrices.
So, we can interpret the spinor as being made up of two components:
%
\begin{subequations}
\begin{align}
\psi (x) = \left[\begin{array}{c}
\psi_1  \\
\psi_2  \\
\psi_3   \\
\psi_4
\end{array}\right]
= \left[\begin{array}{c}
\eta  \\
\omega
\end{array}\right]
\,,
\end{align}
\end{subequations}
%
on which Lorentz transformations act independently.
This is relevant since, for example, if we deal with an electron, we  will describe it with a 4-component spinor, however we will be able to divide it into two components \(e_L\) and \(e_R\), which are two component spinors on which we can act independently.
We have effectively divided our representation of the Lorentz group into the sum of two irreps, of dimension \((1/2, 0)\) and \((0, 1/2)\) respectively.

This will become very concrete when we will discuss how many degrees of freedom were present in the original plasma.

Since a spinor \(\psi \) also solves the KG equation, it will be able to be written as
%
\begin{subequations}
\begin{align}
\psi = \left[\begin{array}{c}
a \\
b \\
c \\
d
\end{array}\right] e^{-i px}
\,,
\end{align}
\end{subequations}
%
but what are the relations between the coefficients?
We consider a simple case, that of \(\vec{p} =0\), so that we are in the rest frame of the particle.

Then, the Dirac equation reads:
%
\begin{subequations}
\begin{align}
\qty(\gamma^{0}E - m ) \left[\begin{array}{c}
a \\
b \\
c \\
d
\end{array}\right]=0
\,,
\end{align}
\end{subequations}
%
so we need to choose a representation for the \(\gamma^{0}\) in order to write this explicitly. We choose the Dirac representation, in which
%
\begin{subequations}
\begin{align}
\gamma^{0} = \left[\begin{array}{cc}
\mathbb{1} & 0 \\
0 & -\mathbb{1}
\end{array}\right]
\,.
\end{align}
\end{subequations}

Then, the equation reads
%
\begin{subequations}
\begin{align}
\left[\begin{array}{cccc}
E-m & 0 & 0 & 0 \\
0 & E-m & 0 & 0 \\
0 & 0 & -E-m & 0 \\
0 & 0 & 0 & -E-m
\end{array}\right]
\left[\begin{array}{c}
a \\
b \\
c \\
d
\end{array}\right] e^{-ipx}
= 0
\,,
\end{align}
\end{subequations}
%
so we get a solution in which \(E=m\), and a solution in which \(E = -m \). So, in general we write the two linearly independent  solutions, respectively with positive and negative energy, as
%
\begin{subequations}
\begin{align}
\psi = \left[\begin{array}{c}
\xi  \\
0
\end{array}\right] e^{-imt}
\qquad \text{and} \qquad
\Psi = \left[\begin{array}{c}
0 \\
\eta
\end{array}\right] e^{+imt}
\,.
\end{align}
\end{subequations}

The negative energy solution, as we will see, represents the antiparticle of the Dirac fermion.

The assumption we made, \(\vec{p} = 0\), does not actually mean we lose generality: we can simply boost into the rest frame of the particle.
If we do this, we get the general
%
\begin{subequations}
\begin{align}
\left[\begin{array}{cc}
E-m & - \vec{\sigma} \cdot \vec{p} \\
\vec{\sigma} \cdot \vec{p} & -E-m
\end{array}\right]
\left[\begin{array}{c}
\left[\begin{array}{c}
a \\
b
\end{array}\right] \\
\left[\begin{array}{c}
c \\
d
\end{array}\right]
\end{array}\right]
e^{-imt + i \vec{p} \cdot \vec{x}}
= 0
\,,
\end{align}
\end{subequations}
%
so we can decompose our solution into
%
\begin{align}
\psi &= u^{s} e^{-i p  x} & E&>0 \\
\psi &= v^{s} e^{i p x} & E&<0
\,,
\end{align}
%
where \(s\) is an index denoting which 2D unit vector we are considering, that is, \(s = 1, 2\).
So, now we come to the interpretation: we introduce the existence of an antifermion, which corresponds to the solution to the Dirac equation with negative energy, but it has the opposed momentum: so, it has positive energy.
Then, both of our solutions have positive energy and evolve forward in time, and both have the same mass.

Now, we make the jump to second quantization: we start interpreting \(\psi (x) \) as an operator, which can destroy a one-state particle or create a particle starting from the vacuum.

We will have
%
\begin{align}
\bra{0} \psi (x) \ket{e^{-}(p,s)} = u^{s} (p) e^{-ipx}
\,.
\end{align}

Now, the one-particle state \(\ket{e^{-}(p, s)}\) is promoted to a spinor.
On the other hand, we have the creation operator \(\psi ^\dag\):
%
\begin{align}
\bra{e^{-}(p,s)} \psi ^\dag (x) \ket{0} = u^{s\dag} e^{-ipx}
\,.
\end{align}

Now, the tricky question is to introduce the negative-energy solution.
The \(\psi ^\dag\) operator will destroy this state, while \(\psi \) will create it.
So we will write an equation like
%
\begin{align}
\bra{0} \psi ^\dag (x) \ket{e^{+} (p, s)}
= v^{s \dag} (p) e^{-ipx}
\,,
\end{align}
%
where we would write \(v^{s}(p)\) if we were considering the negative energy particle, instead we are looking at the antiparticle.

Now, we will be able to operate with \(\psi (x)\) on the vacuum, to find
%
\begin{align}
\bra{e^{+}(p, s)} \psi (x) \ket{0} = v^{s} (p) e^{ipx}
\,.
\end{align}

\subsection{Photon-fermion coupling}

Yesterday we discussed the Lagrangian of the free photon field,
%
\begin{align}
\mathscr{L} \propto F^{\mu \nu } F_{\mu \nu }
\,,
\end{align}
%
but as we said there can also be coupling to external currents, which we did not quantize.
However, now we quantized the electron: so, can we construct the external current \(j_{\mu }\) in the coupling term \(j_{\mu }A^{\mu }\)?

The first attempt would be to write something like
%
\begin{align}
j^{\mu } \sim \psi ^\dag \gamma^{\mu } \psi
\sim e^{+} \gamma^{\mu } e^{-}
\,,
\end{align}
%
but we would need to check whether it is a vector: in fact, it does not transform correctly.

Is this at least a Hermitian operator? Well, its adjoint is
%
\begin{align}
\qty(\psi ^\dag \gamma^{\mu } \psi ) ^\dag = \psi ^\dag \qty(\gamma^{\mu })^\dag \psi
\,,
\end{align}
%
but
%
\begin{align}
\qty(\gamma^{\mu })^\dag = \qty(\gamma^{0}, - \gamma^{i}) \neq \gamma^{\mu }
\,.
\end{align}

One finds that the correct definition is to have
%
\begin{align}
\overline{\psi} \overset{\text{def}}{=}
\psi ^\dag \gamma^{0}
\,,
\end{align}
%
and then
%
\begin{align}
j^{\mu } = \overline{\psi} \gamma^{\mu } \psi
\,
\end{align}
%
is the correct definition. This, then, transforms as a 4-vector; it can also be shown starting from Dirac's equation (and its conjugate) that this is a conserved current: \(\partial_{\mu } j^{\mu } = 0\).

Now, in order to couple the EM field to the electron we will use minimal coupling:
%
\begin{align}
\partial_{\mu } \rightarrow \DD_{\mu } = \partial_{\mu } + ie A_{\mu }
\,,
\end{align}
%

so the Dirac equation will read
%
\begin{subequations}
\begin{align}
\qty( i \slashed{\DD} -m ) \psi &= 0  \\
\qty(i \gamma^{\mu } \qty(\partial_{\mu } + ie A_{\mu }) -m ) \psi &= 0
\,.
\end{align}
\end{subequations}

From which density Lagrangian can we derive the Dirac equation? It turns out to be
%
\begin{align}
\mathscr{L} = \overline{\psi} \qty(i \gamma^{\mu } \DD_{\mu } -m ) \psi
\,,
\end{align}
%
so if we want to describe both the EM field, the electron and their interaction, we have
%
\begin{align}
\mathscr{L} (e, A_{\mu }) &=
\underbrace{- \frac{1}{4} F^{\mu \nu } F_{\mu \nu }}_{\text{free EM field}} + \underbrace{\overline{\psi} \qty(i \slashed{\partial} - m) \psi}_{\text{electron}}
- \underbrace{e\overline{\psi} \slashed{A} \psi}_{\text{electron-EM coupling}}  \\
&= - \frac{1}{4} F^{\mu \nu }F_{\mu \nu } + \overline{\psi} \qty(i \slashed{\DD} - m )\psi
\,.
\end{align}

This is the density Lagrangian of Quantum ElectroDynamics.
This is the first interacting QFT which was constructed, and it was extraordinarily successful.

Its predictions for the anomalous magnetic moment of the electrons were exceptional: we can solve it perturbatively to different orders in
%
\begin{align}
\alpha = \frac{e^2}{4 \pi } \approx \frac{1}{137}
\,.
\end{align}

\end{document}


\section{Facts: reasoning building blocks}
\label{sec-reas}

Reasoning in {\GF} consists of a sequence of reasoning steps, each step being
performed on {\bf facts}.
A fact is a data structure containing a {\em hook} identifying it and some extra
information characterizing the the fact itself.
A fact can be a {\bf proof line}, an {\bf axiom}, a {\bf theory} or a {\bf
definition}.
These four categories are disjoint.
The hooks of proof lines, axioms, theories and definitions are called labels
({\em label}s), axiom labels ({\em axlabel}s), theory labels ({\em thlabel}s)
and definition labels ({\em deflabel}s) respectively.
We will simply call them {\em label}s whenever the distinction is not relevant.

\subsection{Proof lines: reasoning steps}
\label{sec-proofline}

Proof lines are the basic reasoning steps in {\GF}.
A proof line is a 4-tuple of the form:

\begin{center}
$<$ {\em label} \ , \ {\em wff} \ , \ {\em reason} \ , \ {\em deplist} $>$
\end{center}

where
{\em label} is a ``hook" identifying the proof line itself; {\em wff} is the
well formed formula of the proof line; {\em reason} records how the proof line
has been asserted; {\em dep} records the assumptions the proof lines depends on.
For instance, the proof line
%
\begin{center}
	{\tt < 3 , B , impe 1 2 , (1 2) >}
\end{center}
%
tells us that: the proof line {\tt 3} whose formula is ``{\tt B}'' has been
deduced by modus ponens ({\tt impe}) applied to the previously deduced proof
lines whose labels are {\tt 1} and {\tt 2} and that depends on the proof lines
{\tt 1} and {\tt 2}.
Analogously, the proof line
%
\begin{center}
	{\tt < 4 , s(0) > 0 , alle smonotonic x 0 ,  >}
\end{center}
%
tells us that the proof line {\tt 4} whose formula is ``{\tt s(0) > 0}'' has
been deduced by instantiating the variable ``{\tt x}'' in the {\em fact} whose
{\em hook} is {\tt smonotonic} (probably an axiom asserting that
$\forall x \ (\ s(x) > x\ )$).

A {\GF} {\bf proof} is a sequence of proof lines.
The proof line labels are generated by {\GF} as increasing natural numbers
starting from $1$.
The user can ``name'' proof line labels with desired {\GF} symbols (see the
command {\tt label} in section \ref{sec-adm}) or to refer to the previous {\em
n-th} proof line in the current proof by typing \verb+^n+.

\subsection{Axioms and theories}

A {\GF} axiom is the mechanization of the notion of axiom in logic, a {\GF}
theory is the mechanization of the notion of theory, defined as a set of axioms,
in logic.

In {\GF} you can define the axioms of your own theory.
No check is performed on their consistency.
You are thus free to define an unsound theory.
However, the {\GF} logic is complete without need of any axioms.
An axiom can be used to generate proof lines.
A {\GF} axiom is a 2-tuple of the form:

\begin{center}
	$<$ {\em axlabel} \ , \ {\em wff} $>$\\
\end{center}

where {\em axlabel} is the ``hook'' and {\em wff} is the well formed formula of
the axiom.

A theory is a set of axioms.
A theory allows you to use all the axioms of the theory as a whole.


\section*{REGRESSION}\label{regression}
The ship power data is split into a train and a test part. In the time
series the samples are not independent, which means that splitting the
data into a train and test set should not be done randomly! With a time
series, training should be done on historic data to predict the future.
This means that the training set must happen in time prior to the test
set. Fig.\ref{fig:train_test} shows how the train-test-split
should been done for the present data, where the last 20\% of the data
is used for testing.
In Fig.\ref{fig:train_test_random} this train/test approach
(time split) has been compared with the corresponding random train/test
split (random split). A decision tree model has also been trained and
tested to show that the random split overestimates the accuracy by far.
To ensure that this is not just a coincidence for a 20\% test, the test
size has also been varied between 20\% and 70\% as seen in
Fig.\ref{fig:test_size_variation}, where the random split gets
very low accuracy for the entire range, while the random split gets very
good accuracy for all test sizes.
Due to the same reasoning, cross-validation on a rolling basis is used
for parameter tuning as seen in Fig.\ref{fig:rolling_basis}.
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/train_test.pdf}\end{center}
\vspace{-0.7cm}
\caption{Data is split into a train and test part}
\label{fig:train_test}
\end{figure}
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/train_test_random.pdf}\end{center}
\vspace{-0.7cm}
\caption{Comparison of using random or rolling base test set with a decision tree model }
\label{fig:train_test_random}
\end{figure}
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/test_size_variation.pdf}\end{center}
\vspace{-0.7cm}
\caption{Accuracy with a decision tree model for varying test sizes, using random test or rolling base test}
\label{fig:test_size_variation}
\end{figure}
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/rolling_basis.pdf}\end{center}
\vspace{-0.7cm}
\caption{Cross-validation on a rolling basis}
\label{fig:rolling_basis}
\end{figure}


\chapter{Theoretical Foundations of Geometrical Computer Vision}\label{chap:theoretic_found}

% In your case, it could for instance contain the math of camera models, the theory of feature point detection and matching, motion field models, in your case in particular the model of a moving plane, parameterization of motion in terms of rotation matrices and translation vectors, and how they are computed from a set of feature point correspondences

\section{Coordinate Frames}\label{sec:coord_frames}
We are working with camera images, which are 2-D representations of points in a 3-D world. We also have different points of reference: in the real world, according to the camera etc. It is thus of utmost importance to define the coordinate frames we will be using to avoid confusion.\bigskip

First of all, there is the image itself, the \newterm{image coordinate frame ICF}. This is a 2-D frame with x-axis and y-axis along the horizontal axis and vertical axis of the image respectively. The unit of these coordinates is measured in pixels and the center of the frame is at the top left of the image. The image frame is shown in \autoref{fig:im_frame}\bigskip

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/image_frame.png}
    \caption{Image frame}
    \label{fig:im_frame}
\end{figure}

Secondly, there is the \newterm{vehicle coordinate frame VCF}. This is a right-handed 3-D coordinate system. The center of the frame is located at the bottom center of the car. The x-axis is pointing right, the y-axis is pointing down and the z-axis is leaving the car perpendicular to the car in the direction of the front of the car. \bigskip

Closely related, we have the \newterm{camera coordinate frame CCF}. This is a similar 3-D coordinate system, but as I stated before, the camera is mounted slightly tilted at the side of the car. With respect to the camera, the x-axis is once again pointing right, the y-axis pointing down and the z-axis leaving the camera perpendicular to the camera in the direction of the view. These two coordinate frames are shown in \autoref{fig:cam_frame}, with the cube being an abstract way of showing the car.\bigskip

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/camera_frame.png}
    \caption{Vehicle and camera coordinate frame}
    \label{fig:cam_frame}
\end{figure}

The fourth and last coordinate frame is the \newterm{world coordinate frame WCF}. This is also a right-handed 3-D coordinate system and to keep things simple, the center is located at center of the VCF at frame 1. The difference is that the frame is rotated around the z-axis so the y-axis is pointing upwards and the x-axis is pointing left. See \autoref{fig:world_frame} where the cube is once again the car in its initial position.\bigskip

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/world_frame.png}
    \caption{World coordinate frame}
    \label{fig:world_frame}
\end{figure}

As we are working with three different coordinate frames, it's important to know how to transfer points from one frame to another. We'll discuss this at length in a later section.

\subsection{Relation of different coordinate frames}
Working with different coordinate frames can be tricky, to avoid any confusion I will define the transformation equations between the different coordinate frames.

\subsubsection{VCF to CCF}
The first thing we have to do is define how the center of the VCF is translated to the center of the CCF, therefore we define $\vt_{VC}$ as the translation from the VCF to the CCF. Looking at \autoref{fig:cam_frame} from the point of view of the VCF, the center is translated along the x-axis to the right thus being positive, along the y-axis to the left thus being negative. This is of course all depending on where the camera is mounted exactly, but from \autoref{fig:footage_example} it is clear that in our case the camera is mounted on the upper right side of the car. Whether the camera is moved along the z-axis and in which direction is unknown. This gives the following definition:
\begin{equation}
    \vt_{VC} = \begin{pmatrix}
         t_{VC,1} \\ -t_{VC,2} \\ t_{VC,3}
     \end{pmatrix}
\end{equation}

Secondly, we need to have a look at the rotation, how a rotation matrix works is explained in \autoref{ssec:rotmat}. The rotation from VCF to CCF is defined like this (for a detailed explanation on how rotation matrices work, see \autoref{ssec:rotmat}):

\begin{equation}
    \MR_{VC} = \MR_z(\phi_{VC})\cdot\MR_y(\psi_{VC})\cdot\MR_x(\theta_{VC})
\end{equation}

Thus, to transform a point $P_V = (P_{V1}, P_{V2}, P_{V3})^T$ in the VCF to it's corresponding coordinates $P_C = (P_{C1}, P_{C2}, P_{C3})^T$ in the CCF, the following transformation is done:
\begin{equation}
    \begin{pmatrix}
        P_{C1} \\ P_{C2}\\ P_{C3}
    \end{pmatrix} = \MR_{VC}\begin{pmatrix}
        P_{V1} \\ P_{V2}\\ P_{V3}
    \end{pmatrix} + \vt_{VC}
\end{equation}

To go from the CCF to the VCF, it is only logical to do the reverse transformation: first subtracting $\vt_{VC}$ followed by a rotation inverse to $\MR_{VC}$.

\subsubsection{VCF to WCF}
When switching between the VCF and the WCF there are different scenario's. Unlike in the previous section, the VCF and WCF have no fixed transformation. The VCF will move relative to the WCF (which is what we will be trying to track).\bigskip

Before the car has moved, the VCF and WCF will be located at the same point by definition of the WCF. Thus the translation vector $\vt_{VW}$ equals $(0, 0, 0)^T$. The WCF equals the VCF at frame 0 rotated around the z-axis over 180 degrees or $\pi$. This gives us the following rotation matrix:
\begin{equation}
    \MR_{VW} = \MR_z(\pi)\cdot\MR_y(0)\cdot\MR_x(0)
\end{equation}
or
\begin{equation}
    \MR_{VW} = \begin{pmatrix}
        -1 & 0 & 0 \\
        0 & -1 & 0 \\
        0 & 0 & 1
    \end{pmatrix}
\end{equation}

Resulting in the following conversion from $P_V = (P_{V1}, P_{V2}, P_{V3})^T$ to $P_W = (P_{W1}, P_{W2}, P_{W3})^T$:

\begin{equation}\label{eq:VW}
    \begin{pmatrix}
        P_{W1} \\ P_{W2}\\ P_{W3}
    \end{pmatrix} = \MR_{VW}\begin{pmatrix}
        P_{V1} \\ P_{V2}\\ P_{V3}
    \end{pmatrix}
\end{equation}

When the vehicle has already moved from its original starting point, this transformation must be accounted for. To go from the VCF to the WCF, first do the inverse transformation from $VCF_n$ to $VCF_0$ and then rotate following \autoref{VW}.

\subsubsection{CCF to WCF}
To go from the CCF to the WCF, it suffices to concatenate the two previous discussed transformation: going from the CCF to the VCF and then going to the WCF.

\section{Motion Parameters}
Every 3-D rigid motion can be decomposed into two components, each of which have three degrees of freedom. The displacement in this plane is called the translation and can be expressed by the translation vector $\vt$. The second parameter is the rotation, this can be expressed with the rotation matrix $\MR$. Additionally, when a motion is happening on a flat surface we can define this plane uniquely, the normal vector $\vn$ does this.\bigskip

\subsection{Rotation Matrix}\label{ssec:rotmat}
A 3-D rotation can always be described by a sequence of 3 partial rotations, a yaw, a pitch and a roll. It is important to note that the decomposition from rotation matrix to its components is done according to a convention. This convention states the order in which the rotations are done, if these are switched, the decomposition will be wrong. The convention I will use is the Tait-Bryan convention as follows: pitch, yaw, roll, \autoref{eq:rotation} shows this.\bigskip
\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/yaw_pitch_roll.png}
    \caption{Yaw, pitch and roll}
    \label{fig:rotations}
\end{figure}

\autoref{fig:rotations} shows these rotation components. $\psi$, $\theta$ and $\phi$ are yaw, pitch and roll respectively. As it is a right handed system, we have the following conventions:
\begin{itemize}
    \item When $\psi$ is positive, there is a rotation to the left.
    \item When $\theta$ is positive, there is an upward rotation.
    \item When $\phi$ is positive, there is a clockwise rotation.
\end{itemize}
This is all when looking forward from the center alongside the Z-axis.\bigskip

Each of these rotation components can be defined by a $3\times3$ matrix:
\begin{equation}
    \MR_y(\psi) = \begin{pmatrix}
        \cos\psi & 0 & \sin\psi \\
        0 & 1 & 0 \\
        -\sin\psi & 0 & \cos\psi
    \end{pmatrix}
\end{equation}
\begin{equation}
    \MR_x(\theta) = \begin{pmatrix}
        1 & 0 & 0 \\
        0 & \cos\theta & -\sin\theta \\
        0 & \sin\theta & \cos\theta \\
    \end{pmatrix}
\end{equation}
\begin{equation}
    \MR_z(\phi) = \begin{pmatrix}
        \cos\phi & -\sin\phi & 0 \\
        \sin\phi & \cos\phi & 0 \\
        0 & 0 & 1
    \end{pmatrix}
\end{equation}
To get the rotation matrix $\MR$, we multiply the yawn, pitch and roll like this:
\begin{equation} \label{eq:rotation}
    \MR = \MR_z(\phi)\cdot\MR_y(\psi)\cdot\MR_x(\theta)
\end{equation}
\begin{equation*}
    = \begin{pmatrix}
    \cos\psi\cos\phi & \sin\theta\sin\psi\cos\phi - \cos\theta\sin\phi & \cos\theta\sin\psi\cos\phi + \sin\theta\sin\phi \\
    \cos\psi\sin\phi & \sin\theta\sin\psi\sin\phi + \cos\theta\cos\phi & \cos\theta\sin\psi\sin\phi - \sin\theta\cos\phi \\
    -\sin\psi          & \sin\theta\cos\psi                                  & \cos\theta\cos\psi \\
  \end{pmatrix}
\end{equation*}

\subsection{Translation Vector}
A translation vector is a $3\times1$ vector of the form $\vt = (t_1, t_2, t_3)^T$. The vector describes the translation along the three different axes. When point $P = (P_1, P_2, P_3)^T$ undergoes a translation described by $\vt$ you get the following:
\begin{equation}
    P' = P + \vt = \begin{pmatrix}
        P_1 + t_1 \\ P_2 + t_2 \\ P_3 + t_3
    \end{pmatrix}
\end{equation}

\section{Motion Description}
\subsection{Motion equation}
We have talked about motion and the motion parameters. Each frame-to-frame motion will be described using these parameters in a motion equation.\bigskip

To define this equation, take a point with the following coordinates in the CCF: $X = (X_1, X_2, X_3)^T$. After the camera moved, this same point $X$ has now the following coordinates in the CCF $X' = (X'_1, X'_2, x'_3)^T$. The transformation from any $X$ to its corresponding $X'$ can be described by the same equation:
\begin{equation}\label{eq:motion}
    \begin{pmatrix}
        X_1\\ X_2\\ X_3
    \end{pmatrix} =  \MR \begin{pmatrix}
        X'_1\\ X'_2\\ X'_3
    \end{pmatrix}+ \vt
\end{equation}

\subsubsection{Homogeneous coordinates}
In what follows, we'll use homogeneous coordinates instead of Cartesian coordinates. The use of Homogeneous coordinates makes some formulas easier to work with and lets us represent projective transformations by a matrix. \bigskip

To write the motion equation from \autoref{eq:motion} in Homogeneous coordinates, we take $P_1 = (X_1, Y_1, Z_1, 1)^T$ and $P_2 = (X_2, Y_2, Z_2, 1)^T$ and come to the following equation:
\begin{equation}
    \begin{pmatrix}
        X_2 \\ Y_2 \\ Z_2 \\ 1
    \end{pmatrix} = \begin{pmatrix}
        \MR & 0 \\
        0 & 0
    \end{pmatrix} \begin{pmatrix}
        X_1 \\ Y_1 \\ Z_1 \\ 1
    \end{pmatrix} + \begin{pmatrix}
        t_1 \\ t_2 \\ t_3 \\ 1
    \end{pmatrix}
\end{equation}
which can be rewritten as
\begin{equation}
    \begin{pmatrix}
        X_2 \\ Y_2 \\ Z_2 \\ 1
    \end{pmatrix} = \begin{pmatrix}
        \MR & \vt \\
        0 & 1
    \end{pmatrix} \begin{pmatrix}
        X_1 \\ Y_1 \\ Z_1 \\ 1
    \end{pmatrix}
\end{equation}
This matrix characterises the transformation or motion that occurred between two frames. Based on these transformation matrices, the position of the camera (and thus the car) can be determined by concatenating the transformations.

\section{Camera model}\label{sec:cammodel}\footnote{Knowledge of this section was primarily gained by the lecture slides of R. Mester \cite{computer_vision}.}
As we work with 2-D images that are a representation of a 3-D world, there are some things to be said about the projection from 3-D to 2-D. The camera used and its properties are important factors in determining motion.

\subsection{Pinhole camera model}
\autoref{fig:cammodel} shows the pinhole camera model. $C$ is the perspective center of the camera, the plane $E$ is the plane on which the 3-D world is projected upon. $H$ is called the principal point, it is the perpendicular projection of $C$ onto $E$, this axis is called the optical axis. $f$ is a constant based on the properties of the camera.\bigskip

Take point $P$ in the 3-D world, its coordinates can be described with respect to camera coordinate frame with $C$ as its center. When a picture is taken, the $P$ is projected on $E$ resulting in point $p$. $P$ with coordinates $X$, $Y$ and $Z$ is transformed to $p$ with coordinates $x$, $y$ and $z$. $z$ always equals $f$ as axis $Y_3$ is perpendicular to $E$.\bigskip

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/camera_model.jpg}
    \caption{Pinhole camera model}
    \label{fig:cammodel}
\end{figure}

We need a way to describe the relationship between $P$ and $p$, the transformation from real world coordinates to camera coordinates. (It is important to note that the next equations are only valid when all the pairs of entities are measured in the same length entities). Given point $P$ with coordinates $(X, Y, Z)^T$ and point $p$ with coordinate $(x, y, z)$ the following relation can be found using the law of similar triangles:
\begin{equation}
    \frac{x}{X} = \frac{z}{Z} = \frac{f}{Z}
\end{equation}
\begin{equation}
    \frac{y}{Y} = \frac{z}{Z} = \frac{f}{Z}
\end{equation}
Rewriting the equations above, we get the following coordinates for point $\vp$:
\begin{equation}\label{eq:im_proj1}
    x = f\frac{X}{Z}
\end{equation}
\begin{equation}\label{eq:im_proj2}
    y = f\frac{Y}{Z}
\end{equation}
However, these are not linear equations of projection. To keep working with linear equations as long as possible for simplicity, we us an extended vector:
\begin{equation}\label{eq:exvec}
    \begin{pmatrix}
        x \\ y
    \end{pmatrix} \rightarrow
    \begin{pmatrix}
        X \\ Y \\ Z/f
    \end{pmatrix}
\end{equation}
To get the original vector $(x, y)^T$ from $(X, Y, Z/f)^T$ we introduce the projection operator 'Proj':
\begin{equation}
    Proj\begin{pmatrix}
        a \\ b \\ c
    \end{pmatrix} :=
    \begin{pmatrix}
        a/c \\ b/c
    \end{pmatrix}
\end{equation}
So the projection from real word coordinates to camera coordinates can be written as follows:
\begin{equation}\label{eq:projection}
    \begin{pmatrix}
        x \\ y
    \end{pmatrix} = Proj
    \begin{pmatrix}
        X \\ Y \\ Z/f
    \end{pmatrix} =
    \begin{pmatrix}
        f\frac{X}{Z} \\[0.25cm] f\frac{Y}{Z}
    \end{pmatrix}
\end{equation}
This projection operator has an important property: Vectors upon which the projection operator Proj(.) is to be applied may be multiplied by arbitrary factors $\alpha\neq 0$ without changing the result of the projection operation. Proof can be found in the lecture slides of R. Mester \cite{computer_vision}.\bigskip

The vector on the right hand side of \autoref{eq:exvec} can be rewritten as follows:
\begin{equation}
    \begin{pmatrix}
        X \\ Y \\ Z/f
    \end{pmatrix} =
    \begin{pmatrix}
        1 & 0 & 0\\
        0 & 1 & 0\\
        0 & 0 & 1/f
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z
    \end{pmatrix}
\end{equation}
Using the property previously discussed of the projection operator which we will use on the vector $(X, Y, Z/f)^T$ we can multiply the vector with factor $f$:
\begin{equation}
    f\begin{pmatrix}
        X \\ Y \\ Z/f
    \end{pmatrix} = f
    \begin{pmatrix}
        1 & 0 & 0\\
        0 & 1 & 0\\
        0 & 0 & 1/f
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z
    \end{pmatrix}
\end{equation}
\begin{equation}
    = \begin{pmatrix}
        f & 0 & 0\\
        0 & f & 0\\
        0 & 0 & 1
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z
    \end{pmatrix}
\end{equation}
So \autoref{eq:projection} can be written as
\begin{equation}\label{eq:projfull}
    \begin{pmatrix}
        x \\ y
    \end{pmatrix} = Proj\left[
    \begin{pmatrix}
        f & 0 & 0\\
        0 & f & 0\\
        0 & 0 & 1
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z
    \end{pmatrix} \right]
\end{equation}
To avoid the use of the projection operator, we could rewrite \autoref{eq:projfull} can be rewritten like this:
\begin{equation}
    k
    \begin{pmatrix}
        x \\ y \\ 1
    \end{pmatrix} =
    \begin{pmatrix}
        f & 0 & 0\\
        0 & f & 0\\
        0 & 0 & 1
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z
    \end{pmatrix}
\end{equation}
By expressing the point $P$ as a homogeneous vector $(X, Y, Z, 1)^T$ we get the following:
\begin{equation}\label{eq:homproj}
    k
    \begin{pmatrix}
        x \\ y \\ 1
    \end{pmatrix} =
    \begin{pmatrix}
        f & 0 & 0 & 0\\
        0 & f & 0 & 0\\
        0 & 0 & 1 & 0
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z \\ 1
    \end{pmatrix}
\end{equation}
\autoref{eq:homproj} shows how to go from the homogeneous real world coordinates, where the origin is the perspective center of the camera $C$, to the camera coordinates, where the origin is the principal point $H$.\bigskip

When working with pictures however, points are pixels on a grid with the origin in the top left corner of the image. So the the points in camera coordinates need to be transformed to image coordinates. Since these coordinate systems are in the same plane and have the same directions, this transformation is a simple translation (assuming both coordinates are given in pixels), whether or not with a minus sign. This means that \autoref{eq:homproj} can be rewritten, adding the translation like this:

\begin{equation}\label{eq:improj}
    k
    \begin{pmatrix}
        x \\ y \\ 1
    \end{pmatrix} =
    \begin{pmatrix}
        f & 0 & h_x & 0\\
        0 & f & h_y & 0\\
        0 & 0 & 1 & 0
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z \\ 1
    \end{pmatrix}
\end{equation}
As stated before, when working with these equations, all entities should be measured in the same length entities. For now, this is not the case, $f$ is a value commonly measured in millimeters and image coordinates are measured in pixels. This means that \autoref{eq:improj} is not valid using the values we currently have for $f$ and $H$.\bigskip

To solve this problem, we should transform one of the units to the other unit's length entity. As we are primarily working with images, converting everything to pixels is the best way.\bigskip

To do this, we look at the side view of the camera model, as seen in \autoref{fig:fov}. The Field of View $F$ is a value directly related to $f$ and can be found in the data sheet from the camera that is used (the subscript H and V denote the horizontal and vertical Field of View respectively). To calculate the height of the image, we can use basic trigonometry. In a right-angled triangle the tangent of a corner is equal to the length of the opposite side divided by the length of the adjacent side, in our case this means:
\begin{equation}
    \tan(F_V/2) = \frac{h/2}{f}
\end{equation}\label{eq:fovv}
solving for $h$:
\begin{equation}
    h = 2f*\tan(F_V/2)
\end{equation}
Analogous, we can find the width $w$ of the image like this:
\begin{equation}\label{eq:fovh}
    w = 2f*\tan(F_H/2)
\end{equation}
\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/field_of_view.jpg}
    \caption{field of view (vertical)}
    \label{fig:fov}
\end{figure}
We know the dimensions of the image we capture in pixels. Solving \autoref{eq:fovv} and \autoref{eq:fovh} for $f$ we find the following equations:
\begin{equation}
    f_x = \frac{h}{2\tan(F_V/2)}
\end{equation}
\begin{equation}
    f_y = \frac{w}{2\tan(F_V/2)}
\end{equation}
Using these values, we define the following $3\times3$ matrix $\MK$ as the \textit{intrinsic parameters}:
\begin{equation}
    \MK = \begin{pmatrix}
        f_x & 0 & h_x \\
        0 & f_y & h_y \\
        0 & 0 & 1
    \end{pmatrix}
\end{equation}
These intrinsic parameters are related to a certain camera used. The projection from real world coordinates to the image coordinates (in homogeneous coordinates) can then be expressed as
\begin{equation}
    k\begin{pmatrix}
        x_{img}\\y_{img}\\1
    \end{pmatrix} = \begin{pmatrix}
        f_x & 0 & h_x & 0\\
        0 & f_y & h_y & 0\\
        0 & 0 & 1 & 0
    \end{pmatrix}\begin{pmatrix}
        X \\ Y \\ Z \\ 1
    \end{pmatrix}
\end{equation}
\begin{equation*}
    = \begin{pmatrix}
        f_x & 0 & h_x \\
        0 & f_y & h_y \\
        0 & 0 & 1
    \end{pmatrix}
    \begin{pmatrix}
        1 & 0 & 0 & 0\\
        0 & 1 & 0 & 0\\
        0 & 0 & 1 & 0\\
    \end{pmatrix}
    \begin{pmatrix}
        X \\ Y \\ Z \\ 1
    \end{pmatrix}
\end{equation*}
written more compactly:
\begin{equation}
    k\vy = \MK \biggl(\MI \lvert \vnull\biggr)\vq_c
\end{equation}

The point $\vq_c$ is expressed in the coordinate system with the principal axis of the camera pointing straight down the $X_z$-axis. At the origin of this Euclidean coordinate system, the camera is located.

\section{The general case: two views of a 3D point set}\label{sec:gen_case}
In this section I will use point-to-point correspondences between two frames. I will discuss how to get these correspondences in \autoref{chap:found_mot_est}, for now let us assume we have found these correspondences already.\bigskip

In general, we have a perspective model because we work with a camera that can be simplified to the pinhole camera model as described in \autoref{sec:cammodel}. The equations that relate motion to the image coordinates are non-linear in the motion parameters \cite{tekalp}. A two-step linear algorithm has been developed by T. S. Huang \cite{improc} in order to get the motion parameters from (at least eight) point correspondences. Doing this, an intermediate essential matrix $\ME$, consisting of the so called essential parameters is estimated. The way this algorithm works will be discussed in detail later on.

\subsubsection{Essential matrix}
The essential matrix encodes the epipolar geometry of two images. Thus, to break down the essential matrix, we need to have a look at epipolar geometry. \autoref{fig:epigeo} shows a camera in two positions $O_L$ and $O_R$. Point $X$ is projected onto the left image plane resulting in $X_L$ and onto the right image plain resulting in $X_R$.\bigskip

The points $e_L$ and $e_R$ are called the epipoles or epipolar points. They are the projection of the other camera position (its perspective center) onto the image plane. They lay on one line together with the perspective centers $O_L$ and $O_R$.\bigskip

From the perspective of $O_L$, the line $X-O_L$ is a single point $X_L$ while from the perspective of $O_R$ this is a line. This line $e_R-X_R$, shown as a red line in \autoref{fig:epigeo} is called an epipolar line.\bigskip

When the projection $X_L$ onto the left image plane is known as well as the epipolar line $e_R-X_R$ and point $X$ is projected onto the right image plane: $X_R$ evidently laying on the epipolar line, then each point observed in one image must be observed in the other image on a known epipolar line. This gives us an epipolar constraint, that for each set of points it is possible to test whether they correspond to the same 3-D point. This epipolar constraint can be described by an essential matrix.\bigskip

The essential matrix maps a point in one image to an epipolar line in the other image. If you take a point in one image and multiply it by the essential matrix, you will get the epipolar line on the other image. This essential matrix can then be decomposed into the rotation matrix and translation vector, in \autoref{ssec:essentialmat} I will discuss in detail how to estimate and then decompose the essential matrix.\bigskip

\begin{figure}
    \centering
    \includegraphics[width=1\textwidth]{figures/epipolar_geometry.png}
    \caption{Epipolar geometry (source: \textit{en.wikipedia.org/wiki/Epipolar\_geometry})}
    \label{fig:epigeo}
\end{figure}

Suppose we have keypoint on the image, then this keypoint corresponds with a point in the real world $\MX = (X_1, X_2, X_3)^T$ that has been matched to another keypoint corresponding to another real world point $\MX' = (X_1', X_2', X_3')^T$. The transformation of $X$ to $X'$ can be described as follows:

\begin{equation} \label{eq:1}
    \begin{pmatrix}
        X_1' \\
        X_2' \\
        X_3'
    \end{pmatrix}
    = \MR
    \begin{pmatrix}
        X_1 \\
        X_2 \\
        X_3
    \end{pmatrix}
    + \vt
\end{equation}
where $\MR$ is the $3 \times 3$ rotation matrix and $\vt$ the translation vector.

\subsection{General case: Essential matrix}\label{ssec:essentialmat}
The OpenCV method \textit{findEssentialMat} can be used to estimate the essential matrix based on matched keypoints between two frames. In the following we will explain the way this works.\bigskip

In \cite{improc} T. S. Huang proposed that with at least eight keypoint matches a two-step linear algorithm can be developed. To do this, intermediate values called essential parameters need to be estimated first. These essential parameters form a matrix $\ME$ called the essential matrix:
\begin{equation}
    \ME =
    \begin{pmatrix}
        e_1&e_2&e_3 \\
        e_4&e_5&e_6 \\
        e_7&e_8&e_9
    \end{pmatrix}
    = \begin{pmatrix}
        0&-T_3&T_2 \\
        T_3&0&-T_1 \\
        -T_2&T_1&0
    \end{pmatrix}
     \MR
\end{equation}

These nine essential elements are not all independent due to the fact that  $\ME$ is a product of a skew-symmetric matrix and a rotation matrix.\bigskip

\cite{tekalp} states that vectors $\MX'$, $\vt$ and $\MR\MX$ are coplanar, since $\MX' = \MR\MX + \vt$ from (\ref{eq:1}). So now we have
\begin{equation}
    \MX \cdot (\vt \times \MR\MX) = 0
\end{equation}
because $\vt \times \MR\MX$ is orthogonal to this plane. From this, it follows that
\begin{equation} \label{eq:2}
    \MX'^T\ME\MX = 0
\end{equation}

We want to obtain a relationship in terms of the image plane coordinates. To do this, we divide both sides of \ref{eq:2} by $X_3 X'_3$:
\begin{equation} \label{eq:essential}
    \begin{pmatrix}
        x_1'&x_2&1
    \end{pmatrix}
    \ME
    \begin{pmatrix}
        x_1 \\
        x_2 \\
        1
    \end{pmatrix}
    = 0
\end{equation}

This linear homogeneous equation, in terms of the nine essential parameters, has either no solution or infinitely many solutions. By setting one of these nine essential parameters to one in the equation above, we can solve it for the remaining eight essential parameters. The essential parameter set to one is used as a scale factor. It is important to note that because of scale ambiguity in perspective projection, translation can only be estimated up to a scale factor.

\subsection{Estimation of the Essential Parameters}
To estimate these essential parameters, \cite{tekalp} proposes three methods: a linear lest squares method, an optimisation method requiring at least eight keypoint matches and a nonlinear method where only five, six or seven keypoint matches are needed. We'll discuss the Linear Least Squares Method.\bigskip

As stated before, one of the essential parameters will be set to one. Take $e_9 = 1$, then equation (\ref{eq:essential}) can be written as:
\begin{equation}
    \begin{pmatrix}
        x_1'x_1 &
        x_1'x_2 &
        x_1' &
        x_2'x_1 &
        x_2'x_2 &
        x_2' &
        x_1 &
        x_2
    \end{pmatrix}
    \begin{pmatrix}
        e_1 \\
        e_2 \\
        e_3 \\
        e_4 \\
        e_5 \\
        e_6 \\
        e_7 \\
        e_8 \\
    \end{pmatrix}
    = -1
\end{equation}

This linear equation in eight essential parameters can be set up as a system of $N$ linear equations in 8 unknowns. To solve this, we need at least $N \geq 8$ keypoint matches. The solution of this equation gives the essential matrix $\ME$ up to a scale factor (as discussed before). We took $e_9$ as the scale factor, but all of the other essential parameters could have been chosen as well. When using this method, $e_i, i = 1,...,9$ will be chosen based on the condition number, the $e_i$ that yields the smallest condition number will be chosen as scale factor.\bigskip

To have a solution for this equation, the coefficient matrix needs to have full rank, \cite{tekalp} states that this is the case if $\vt \neq 0$ and the shape of the 3-D surface satisfies certain conditions, called the surface constraint.

Other methods to estimate the essential parameters can be found in \cite{tekalp}.

\subsection{Decomposition of the Essential Matrix}
Now that we have the essential parameters that make up the essential matrix $\ME$, we can use this to compute rotation matrix $\MR$ and translation vector $\vt$. To do this, we rewrite $\ME$ as

\begin{equation} \label{eq:essential2}
    \ME =
    \begin{pmatrix}
        e1 \mid e2 \mid e3
    \end{pmatrix} =
    \begin{pmatrix}
        k \hat{t} \times \boldsymbol{r_1} \mid
        k \hat{t} \times \boldsymbol{r_2} \mid
        k \hat{t} \times \boldsymbol{r_3}
    \end{pmatrix}
\end{equation}

with $\boldsymbol{r_i}, i = 1, 2, 3$ are the columns of rotation matrix $\MR$, $\hat{t}$ and $k$ are the unit vector along and the length of $\vt$ respectively. \cite{tekalp} proposes a method to calculate $\MR$ and  $\hat{t}$ from noise-free as well as a method for noisy point correspondence data. As we strive to have noise-free data, we'll discuss only the first method. For the full step by step mathematical explanation, we refer to \cite{tekalp}. Here we will limit ourselves to the essential steps.\bigskip

By looking at \autoref{eq:essential2}, it is clear that each column of $\ME$ is orthogonal to $\vt$. By taking the cross product of two of the three columns, the unit vector along $\vt$ can be calculated:
\begin{equation}
    \hat{t} = \pm \frac{\boldsymbol{e_i \times e_j}}{\mid\mid e_i \times e_j\mid\mid}
    \quad i \neq j
\end{equation}

The sign of the unit vector can't be calculated with the equation above. Zhuang \cite{ZHUANG1989175} proposed the following to do so:
\begin{equation}
    \sum\begin{pmatrix}
        \hat{t} \times \begin{pmatrix}
            x_1' & x_2' & 1
        \end{pmatrix}^T
    \end{pmatrix}^T
    \cdot \ME\begin{pmatrix}
        x_1 & x_2 & 1
    \end{pmatrix}^T
    > 0
\end{equation}
the summation is taken over all observed keypoint matches.\bigskip

T. S. Huang \cite{improc} showed that
\begin{equation}
    k^2 = \frac{1}{2}(\boldsymbol{e_1}\cdot\boldsymbol{e_1} + \boldsymbol{e_2}\cdot\boldsymbol{e_2} + \boldsymbol{e_3}\cdot\boldsymbol{e_3})
\end{equation}
Calculating $k$ this way doesn't solve the problem we had with the scale as $\boldsymbol{e_3}$ contains an arbitrary parameter but to determine the rotation parameters, we need this value for $k$.\bigskip

To calculate $\boldsymbol{r_1}$ we first calculate $\boldsymbol{e_2 \times e_3}$:
\begin{align*}
    \boldsymbol{e_2} \times \boldsymbol{e_3} = [(k\hat{t}\times \boldsymbol{r_2})\cdot\boldsymbol{r_3}]k\hat{t} - [(k \hat{t} \times \boldsymbol{r_2})k \hat{t}]\boldsymbol{r_3}
\end{align*}
The first term can be simplified as
\begin{align}
    [(\boldsymbol{r_2}\times\boldsymbol{r_3})\cdot k\hat{t}]k\hat{t} = (\boldsymbol{r_1}\cdot k\hat{t})k\hat{t}
\end{align}
because $\boldsymbol{r_1}$, $\boldsymbol{r_2}$ and $\boldsymbol{r_3}$ are mutually orthogonal and have unity length. Since $(k\hat{t}\times\boldsymbol{r_2})$ is orthogonal to $k\hat{t}$, the second term is zero. So we have the following equations (\autoref{eq:r2} and \autoref{eq:r3} can be calculated in an analogous way):
\begin{equation}
    \boldsymbol{e_2} \times \boldsymbol{e_3} = k\hat{t}(k\hat{t}\cdot\boldsymbol{r_1})
\end{equation}
\begin{equation}\label{eq:r2}
    \boldsymbol{e_3} \times \boldsymbol{e_1} = k\hat{t}(k\hat{t}\cdot\boldsymbol{r_2})
\end{equation}
\begin{equation}\label{eq:r3}
    \boldsymbol{e_1} \times \boldsymbol{e_2} = k\hat{t}(k\hat{t}\cdot\boldsymbol{r_3})
\end{equation}

By solving these equations to $\boldsymbol{r_1}$, $\boldsymbol{r_2}$ and $\boldsymbol{r_3}$ we get the following equations:
\begin{equation}
    \boldsymbol{r_1} = \biggl[\frac{1}{k^2}\hat{t}\cdot(\boldsymbol{e_2}\times\boldsymbol{e_3})\biggr]\hat{t}+\frac{1}{k}(\boldsymbol{e_1}\times\hat{t})
\end{equation}
\begin{equation}
    \boldsymbol{r_2} = \biggl[\frac{1}{k^2}\hat{t}\cdot(\boldsymbol{e_3}\times\boldsymbol{e_1})\biggr]\hat{t}+\frac{1}{k}(\boldsymbol{e_2}\times\hat{t})
\end{equation}
\begin{equation}
    \boldsymbol{r_2} = \biggl[\frac{1}{k^2}\hat{t}\cdot(\boldsymbol{e_1}\times\boldsymbol{e_2})\biggr]\hat{t}+\frac{1}{k}(\boldsymbol{e_3}\times\hat{t})
\end{equation}\bigskip

\section{The planar case: two views of points on a plane}
Once again, as in \autoref{sec:gen_case} we will assume we have found point-to-point correspondences between two frames.\bigskip

When all points lay in one plane, the general case with the essential matrix does not work. The Essential matrix cannot be determined from such a set of points. Fortunately, there is another way to deal with this case. To do this, I need to introduce the term \newterm{homography}. \bigskip

Take a certain planar surface, when you have two images of this surface, a homography will describe the relation between that planar surface in these two images (when using the pin hole camera model). A homography is a projection between two such images.\bigskip

Such a planar surface in 3-D is defined like this:
\begin{equation}\label{eq:plane}
    n_1X_1 + n_2X_2 + n_3X_3 = 1
\end{equation}
Where $\vn = (n_1, n_2, n_3)^T$ is the normal vector of that given plane.\bigskip

A point $P = (P_1, P_2, P_3)^T$ is a point of this plane, assuming it fits in \autoref{eq:plane}. When taking an image, point $P$ will be projected onto the image plane following the pinhole camera model projection that is described by \autoref{eq:im_proj1} and \autoref{eq:im_proj2}. Accounting for the camera parameters, using \autoref{eq:improj}, each image will have its own projection of this point $P$, let us call them $P^*_1$ and $P^*_2$:
\begin{equation}
    P^*_1 = \begin{pmatrix}
        f_1 & 0 & h_{1,x} & 0\\
        0 & f_1 & h_{1,y} & 0\\
        0 & 0 & 1 & 0
    \end{pmatrix}
    \begin{pmatrix}
        P_1 \\ P_2 \\ P_3 \\ 1
    \end{pmatrix}
\end{equation}
\begin{equation}
    P^*_2 = \begin{pmatrix}
        f_2 & 0 & h_{2,x} & 0\\
        0 & f_2 & h_{2,y} & 0\\
        0 & 0 & 1 & 0
    \end{pmatrix}
    \begin{pmatrix}
        P_1 \\ P_2 \\ P_3 \\ 1
    \end{pmatrix}
\end{equation}
Note that each of these points is written in homogeneous coordinates.

The relation between these two points can be described like this:
\begin{equation}
    sP^*_1 =   \MK(\MR + \vt\vn^T)\MK^{-1}P^*_2
\end{equation}
and we define $\MH$ like this:
\begin{equation} \label{eq:homography}
    \MH = (\MR + \vt\vn^T)
\end{equation}

It is possible to estimate a homography matrix if we have multiple corresponding points on both planar surfaces. It has been shown that this homography can then be decomposed into the rotation matrix, translation vector and normal vector by Huang et al. \cite{506592e3b5484e57928e215df49a83cb}. How to estimate the homography and how to decompose it will be discussed in detail in in \autoref{ssec:est_homography}.

\subsection{Planar case: Homography}\label{ssec:est_homography}

When all keypoints lie on a planar surface, we have to treat it differently as the essential matrix can not be estimated when all points lie on a single plane. When all points lie on a planar surface, we can use that fact that a homography $\MH$ describes the transformation of that planar surface. For this we only look at the points below the horizon on the road surface. Then all keypoints lie on a plane that can be described as
\begin{equation}
    aX_1 + bX_2 + cX_3 = 1
\end{equation}
where $(a, b, c)^T$ is the normal vector of this plane. \autoref{eq:1} can then be rewritten as
\begin{equation}
    \begin{pmatrix}
        X_1' \\
        X_2' \\
        X_3'
    \end{pmatrix} = \MR
    \begin{pmatrix}
        X_1 \\
        X_2 \\
        X_3
    \end{pmatrix} + \vt
    \begin{pmatrix}
        a & b & c
    \end{pmatrix}
    \begin{pmatrix}
        X_1 \\
        X_2 \\
        X_3
    \end{pmatrix}
\end{equation}
or
\begin{equation}
    \begin{pmatrix}
        X_1' \\
        X_2' \\
        X_3'
    \end{pmatrix} =
    \begin{pmatrix}
        a_1 & a_2 & a_3 \\
        a_4 & a_5 & a_6 \\
        a_7 & a_8 & a_9 \\
    \end{pmatrix}
    \begin{pmatrix}
        X_1 \\
        X_2 \\
        X_3
    \end{pmatrix} = \MA
    \begin{pmatrix}
        X_1 \\
        X_2 \\
        X_3
    \end{pmatrix}
\end{equation}
with
\begin{equation} \label{eq:matrixA}
    \MA = \MR + \vt
    \begin{pmatrix}
        a & b & c
    \end{pmatrix}
\end{equation}

$a_1,...,a_9$ are called the pure parameters \cite{506592e3b5484e57928e215df49a83cb}. As with the computation of the essential parameters, the scale ambiguity problem is also present here so we normalise $a_9 = 1$. The 3-D displacement is projected onto a 2-D plane using the perspective transformation. The mapping of 3-D displacement $t$ to 2-D image plane $t'$ is given by
\begin{equation*}
    x_1' = \frac{a_1x_1 + a_2x_2 + a_3}{a_7x_1 + a_8x_2 + 1}
\end{equation*}
\begin{equation}\label{eq:pure}
     x_2' = \frac{a_4x_1 + a_5x_2 + a_6}{a_7x_1 + a_8x_2 + 1}
\end{equation}

\subsection{Estimation of the pure parameters}
\autoref{eq:pure} can be rearranged by cross-multiplying the equations for each keypoint match:
\begin{equation}
    \begin{pmatrix}
        x_1 & x_2 & 1 & 0 & 0 & 0 & -x_1x_1' & -x_2x_1' \\
        0 & 0 & 0 & x_1 & x_2 & 1 & -x_1x_2' & -x_2x_2'
    \end{pmatrix}
    \begin{pmatrix}
        a_1 \\ a_2 \\ a_3 \\ a_4 \\ a_5 \\ a_6 \\ a_7 \\ a_8 \\
    \end{pmatrix} =
    \begin{pmatrix}
        x_1' \\ x_2'
    \end{pmatrix}
\end{equation}
So if we have at least four keypoint matches, eight or more linear equations can be set up to solve for the pure parameters. \cite{improc} showed that the rank of the matrix is 8 if and only if no three of the four observed points are collinear in three dimensions.

\subsection{Estimation of the motion and structure parameters}
We now have found the pure parameters and thus matrix $\MA$. As \autoref{eq:matrixA} states, motion parameters $\MR$ and $\vt$ can be derived from $\MA$, as well as the normal vector $(a,b,c)^T$. To do this, we need to decompose $\MA$ using singular value decomposition (SVD). \cite{Huang:86} describes describes a method to do this:

First of all, we write matrix $\MA$ as
\begin{equation}
    \MA = \MU
    \begin{pmatrix}
        \lambda_1 & 0 & 0 \\
        0 & \lambda_2 & 0 \\
        0 & 0 & \lambda_3
    \end{pmatrix}\MV^T
\end{equation}

where $\MU = (\boldsymbol{u_1}\mid\boldsymbol{u_2}\mid\boldsymbol{u_3})$ and $\MV = (\boldsymbol{v_1}\mid\boldsymbol{v_2}\mid\boldsymbol{v_3})$ are $3\times 3$ orthogonal matrices and $\lambda_1\geq\lambda_2\geq\lambda_3\geq 0$ are the singular values of $\MA$. Depending on the multiplicity of the singular values there are 3 different cases.

\subsubsection{Multiplicity of singular values is 1: $\lambda_1 > \lambda_2 > \lambda_3$}

This indicates that the motion can be decomposed into a rotation around an axis through the origin and a translation in a direction, not along the normal vector. In this case, there are two possible solutions for the pure parameters:
\begin{equation*}
    \MR = \MU
    \begin{pmatrix}
        \alpha & 0 & \beta \\
        0 & 1 & 0 \\
        -s\beta & 0 & s\alpha
    \end{pmatrix}
    \MV^T
\end{equation*}
\begin{equation*}
    \vt = k\biggl(-\beta\boldsymbol{u_1} + (\frac{\lambda_3}{\lambda_2}-s\alpha)\boldsymbol{u_3}\biggr)
\end{equation*}
\begin{equation*}
    \begin{pmatrix}
        a & b & c
    \end{pmatrix}^T = \frac{1}{k}(\delta\boldsymbol{v_1} + \boldsymbol{v_3})
\end{equation*}
with
\begin{equation*}
    \alpha = \frac{\lambda_1 + s\lambda_3\delta^2}{\lambda_2(1+\delta^2)}
\end{equation*}
\begin{equation*}
    \beta = \frac{1}{\delta}\biggl(\alpha-\frac{\lambda_1}{\lambda_2}\biggr)
\end{equation*}
\begin{equation*}
    \delta = \pm\biggl(\frac{\lambda_1^2-\lambda_2^2}{\lambda_2^2-\lambda_3^2}\biggr)^\frac{1}{2}
\end{equation*}
\begin{equation*}
    s = \det{\MU} \det{\MV}
\end{equation*}
and $k$ is an arbitrary scale factor (either positive or negative). To resolve the scale ambiguity, $1/X_3 > 0$ is required for all points.

\subsubsection{Multiplicity of singular values is 2: $\lambda_1 = \lambda_2 \neq \lambda_3$}
This indicates that the motion can be decomposed into a rotation around an axis through the origin and a translation along the normal vector. Now a unique solution for the motion parameters exist:
\begin{equation*}
    \MR = \frac{1}{\lambda_1}\MA-\biggl(\frac{\lambda_3}{\lambda_1}-s\biggr)\boldsymbol{u_3}\boldsymbol{v_3^T}
\end{equation*}
\begin{equation*}
    \vt = k\biggl(\frac{\lambda_3}{\lambda_1}-s\biggr)
\end{equation*}
\begin{equation*}
    \begin{pmatrix}
        a & b & c
    \end{pmatrix}^T = \frac{1}{k}\boldsymbol{v_3}
\end{equation*}
\begin{equation*}
    s = \det{\MU} \det{\MV}
\end{equation*}
Once again, $k$ is an arbitrary scale factor.
\subsubsection{Multiplicity of singular values is 2: $\lambda_1 = \lambda_2 = \lambda_3$}
This indicates the rotation is a pure rotation around an axis through the origin. Once again, there's a unique solution for the rotation matrix:
\begin{equation*}
    \MR = \left(\frac{1}{\lambda_1}\right)\MA
\end{equation*}
In this case, it is not possible to determine translation vector $\vt$ and the normal vector $(a, b, c)^T$ as there is no translation.

\section{Combining the use of homography and essential matrix}
Using one of these does not have to exclude using the other. As the footage we will be using will have both points laying on a fixed planar surface (being the road surface) and points spread out in space, we can use the essential matrix and homography matrix complementary.


\subsection{Literature review on graphical causal inference, graph learning and variable selection}

\subsubsection*{Variable selection and corresponding issues in linear graph learning}

In empirical researches, a graph learning task typically starts from linear graph learning (e.g., \citet{bollen1989structural, geiger1994learning, spirtes2000causation, friedman2008sparse}). Compared with nonlinear graphs on the same dataset, linear graphs requires much less computation load and is easier for inference. Also, linear graph is deeply related to linear modelling methods like linear regression, best subset variable selection, shrinkage and lasso. Only if linearity cannot approximate the dependency in the data, researchers apply nonlinear dependence measure like Hilbert Schmidt independence criterion \citep{gretton2005measuring} and mutual information.

As a major issue in linear graph learning, multicollinearity is frequently observed among variables with complicated causal structures, which will cause several problems for the parameter estimation in both linear modelling and linear graph learning. First, since linear model estimation is based on error minimization, multicollinearity will reduce the magnitude of the minimal eigenvalue in the linear space, causing numerical convergence problems (e.g., Cholesky decomposition or gradient descent) when applying maximum liklihood or maximum a posteriori. Second, severe multicollinearity amplifies parameter estimate instability across samples, making it difficult to interpret the coefficients reliably and accurately. Third, multicollinearity causes problems for statistical tests that rely on the sample covariance (e.g., the post-OLS t-test or the lasso covariance test \citep{lockhart2014significance}. The conditional correlation tests of in constraint-based learning \citep{farrar1967multicollinearity}). Last but not least, multicollinearity may also reduce the algorithmic stability of the model \citep{elisseeff2003leave}, which reduces the generalization ability and the out-of-sample prediction of the estimated model.

Moreover, multicollinearity also affects the reliability of variable selection algorithms in linear modelling and linear graph learning. \citet{zou2005regularization, jia2010model} find that, if a group of variables are highly correlated with one another, lasso-type estimators may randomly select one variable from the group and drop the others out of the regression, referred to as \textbf{the grouping effect}. Since all linear modelling techniques make the variable selection decision based on the conditional correlation between $\mathbf{x}_j$ and $\mathbf{y}$, the grouping effect may well apply to all variable selection methods in linear models. Multicollinearity also affect linear graph learning. \citet{heckerman95, chickering04} shows that learning a linear graph is NP-hard on data with large $p$. As a result, graph learning algorithms typically work well on data with large $n$ and very sparse $p$. In many graph learning applications, variable selection algorithms (e.g., SCAD \citep{fan2001variable}, ISIS \citep{fan2008sure} or different lasso-type estimators \citep{fan2009network}) are used to filter out redundant variables before graph learning. As a result, with grouping effect, variable selection methods may randomly drop some of the highly correlated variables, resulting in omissions of important variables in the linear graph learning.

Many attempts have been made to reduce the effects of multicollinearity. For more stable regression coefficient estimates, \citet{hoerlkennard70} apply Tikhonov regularization to OLS, resulting in the ridge regression. However, it complicates statistical tests and post-estimation inference. To reduce the grouping effect and obtain stable variable-selection results, cross-validated group lasso and cross-validated elastic net (CV-en) have been introduced \citep{zou2005regularization, friedman10}. However, group lasso relies on manual grouping of variables, which depends heavily on accurate field knowledge. On the other hand, while \citet{zou2005regularization} and \citet{jia2010model} show that CV-en may improve the stability of variable selection, \citet{jia2010model} counter that the improvement is marginal and that ``when the lasso does not select the true model, it is more likely that the elastic net does not select the true model either.''


The proposed autoscaling system scales a web application in respose to change in throughput at fixed intervals, which we denote by reconfiguration intervals of 15 minutes. This architecture has the following key components:

\noindent\textbf{Predictor: } Predictor component takes the monitoring data as input and uses different time-series analysis techniques to predict the future service demand for the next time interval.

\noindent\textbf{Profiler: } This component measures the current and maximum capacity of each type of provisioned vm instance. To do that, the Profiler uses online profiling techniques without any server instrumentation.

\noindent\textbf{Dynamic load balancer: } In order to adapt our system to the heterogeneity of cloud platforms, this component dynamically assign weights to the backend servers to proportionally distribute the incomming traffic across them taking into account the heterogeneity of their performance characteristics.

\noindent\textbf{Scaler: }  This component contains the central intelligence of our autoscaling system. Scaler uses the Predictor and Profiler components to find the optimal provisioning strategy that fulfills a pre-established SLO (Service Level Objective). Furthermore, this component constantly analyzes the behaviour of the provisioned VMes to trigger the Dynamic load balancer component.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=.85\linewidth]{images/monitoringSchema}
  \end{center}
\vspace{-5mm}
  \caption{Auto-scaling system.}
  \label{autoScalingSys}
\end{figure}

\subsection{Predicting future service demand}
There is a vast literature related to resource provisioning systems that utilizes time-series analysis to predict the future service demand. However, it is not clear which prediction technique to employ when having a very heterogeneous workload mix.

As a consequence, we designed a Predictor component that first analyzes the monitoring data using five different prediction techniques (six very soon), and then uses the model that exhibits the lowest cumulative error measure at any given moment (previous iteration). To handle different types of workload, the Predictor component utilizes the following statistical  and learning-based techniques for estimating the future service demand:

\begin{itemize}
\item Linear Regression (LR): It is a statistical method whose objective consists to find a polynomial function on which the distance from each of the points to a polynomial curve is the smallest as possible. This model is applied to time-series with a linear trend. Equation ...

\begin{equation}\label{xx}
\begin{split}
Y = {\beta}_1 +  \sum_{i=1}^n {\beta}_i x_i
\end{split}
\end{equation}

\item Exponential Smoothing (ES): Regarding the different variety of exponential smoothing methods, we decided to use the \emph{Holt-Winters'} model that can be used for time-series with an existing trend and seasonality. This model fits to daily and seasonal behavior of web applications by assigning weigths over time based on the influence of a smotthing factor \emph{$\alpha$} to the historic data. Equation ..

\begin{equation}\label{xx}
\begin{split}
Y_{t+m} = (2y_t - y_t) + m(y_t - y_t) \alpha / (1 - \alpha)
\end{split}
\end{equation}

\item Autoregression (AR): Based on the linear regression method, this model defines a polynomial function that first calculates and then uses the auto-correlation coefficients of the time-series similar to finally apply the linear regression model. Equation ...

\begin{equation}\label{xx}
\begin{split}
Y_t =a_1 Y_t-1 + ... + a_p Y_t-p
\end{split}
\end{equation}

\item Vector Autoregression (VAR): This model is a natural extension of the  autoregressive model to dynamic multivariate time series. The VAR model has proven to be especially useful for describing the dynamic behavior of economic and financial time series and for forecasting. Equation ...

\begin{equation}\label{xx}
\begin{split}
Y_{t} =a_1 Y_{t-1} + ... + a_p Y_{t-p} + {\epsilon}_p
\end{split}
\end{equation}

\item Auto Regression Moving Average (ARMA): This averaging method combines both autoregression and moving average. Equation ...

\begin{equation}\label{xx}
\begin{split}
Y_{t+1} =a_1 Y_{t-1} + a_2 Y_{t-2} + ... + b_o X_t + b_1 X_{t-1} + ...
\end{split}
\end{equation}

\item Neural networks (NN): ...
\end{itemize}

In this system, the Predictor component forecasts the future response time values (5min ahead) of an application answering to the question "When to provision?". These prediction models are fed using the monitoring data collected from the web proxy servers. Furthermore, this component also predicts the future values (1h ahead) of monitoring metrics such as request rate and CPU usage to  identify the resource requirements for the next time intervals. This last prediction will contribute to answer the question "How much to provision?", as mentioned in Section~\ref{profiling}.
When using the Predictor for this purpose, a reduction in the finnacial cost is caused, as offline training phases are not needed to provide accurate predictions to handle a future service demand.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=.85\linewidth]{images/prediction_models_6min.eps}
  \end{center}
\vspace{-5mm}
  \caption{Prediction results, response time 5min ahead.}
  \label{threshold}
\end{figure}

To calculate the cumulative error, we use MAPE method  (Moving Average Percentage Errors).

\subsection{Online profiling \label{profiling}}

Regarding the resource heterogeneity of cloud infrastructures, online profiling-based techniques~\cite{kaviani_profiling-as--service:_2011} have recently emerged as a possible solution to the estimation of the resource throughput under certain workload. This technique replicates at runtime a server hosting an application, with a new server with profiling instrumentation. Despite the use of the online-profiling is still under study, the configuration of a parallel environment and the limited accuracy of its scaling decisions are the major drawbacks for its adoption.

In our system, we designed a novel online profiling technique that gives an estimation of the maximum throughput of each allocated vm instance without need for additional resources. To do that, the Profiler component stores the latest hour of monitoring data from each instance's type allocated in one application. The profiling data is composed of monitoring metrics such as the request rate, cpu usage and response time.

Once the Scaler component decides to trigger a scaling action, the Profiler component processes the profiling data to estimate the resource throughput according the following steps:

\begin{enumerate}
\item Collect the profiling data of each instance's type.
\item Perform a smoothing technique over the profiling data to remove the noise generated by traffic spikes. As illustrated in Figure~\ref{fig:vm_performance}, the Profiler extracts the smoothed 75th and 25th percentiles to identify the maximum-ideal  throughput of each instance while enforcing the performance requirements and avoiding CPU saturation (based on the Amazon EC2 recommendation 30 < CPU usage < 75  ).

\begin{figure*}[htb]
	\begin{minipage}[b]{0.45\linewidth}
		\includegraphics[height=4.5cm]{images/vm_performance_resp.eps}
		\vspace{-4mm}
	\end{minipage}
	\hfill
	\begin{minipage}[b]{0.45\linewidth}
		\includegraphics[height=4.5cm]{images/vm_performance_cpu.eps}
		\vspace{-4mm}
	\end{minipage}
\caption{Profiling data and percentiles for the Response time and CPU usage.}
\label{fig:vm_performance}
\end{figure*}

\item Classify the different instance types depending on the performance capacity. The Predictor calculates a factor, named \emph{CPU speed}, as the number of clocks required to process a request (clocks/request) by using the next Equation~\ref{cpu_speed}:

\begin{equation}\label{cpu_speed}
\begin{split}
CPU_{vm} = Num\_requests_{vm} * CPU speed_{vm}
\end{split}
\end{equation}

The \emph{CPU speed} gives an estimation of the performance behavior of one instance when processing the current workload. Based on the value of this factor, the Profiler creates a classification of the different instances when running an application. This classification gives an interesting feedback to the Scaler component, thus allowing it to take more accurate decisions when deciding the optimal scaling strategy to use.

\end{enumerate}

Thank to the Profiler component, the accuracy of the scaling decisions is improved independtly of factors such as VM sharing or request heterogeneity.

\subsection{The scaling decision maker}

The Scaler component is the central governance of this autoscaling system, and thereby is the responsible of triggering any scaling decision.

%% Threshold fixed by the user

Initially, the user defines several performance requirements that will be utilized to trigger scaling actions. When hosting web applications, these performance requirements are usually related to the response time value that determines the maximum processing time needed to serve any request.

\subsubsection{Time-series smoothing}

When minimazing the SLA violations, a precise analysis of the monitoring data represents a crucial step to improve the accuracy of the scaling decisions. This step has even more importance when hosting web application like Wikipedia. Its workload heterogeneity requires a meticulous analysis of every monitoring data-point to reduce at the maximum the number of SLO violations.

To obtain a precise response time value that represents the performance behavior of an application, we decided to extend a known smoothing technique called Moving Weight Average (MVA). This technique is widely used in resource provisiong systems instead of others such as the median, average or moving average. Using our extension of MVA, it first associated weights in an increassing order giving more importance to the latest monitoring data; and second it doubles the original weight value of each data-point that exceeds the performance requirements (response time). By doing so, we analyze the time-series giving special importance to the data points on which SLO violations have occurred.

\subsubsection{Decision making}

Based on the performance requirements, the Scaler defines a reactive threshold that will enable  in advance to react against any SLO violation or under-utilization of the provisioned resources. This reactive threshold specifies one upper and lower boundaries creating two head-rooms between them and the fixed thresholds (pre-defined by the user), as illustrated in Figure~\ref{threshold}. This reactive threshold is defined by the response time and CPU usage (we could also use the EC2 recommendations for the CPU usage). In the future, these two boundaries could be adjusted depending on the hardware configuration of each provisioned VM, as introduced in~\cite{beloglazov_adaptive_2010}.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=.85\linewidth]{images/thresholdGraphic.jpg}
  \end{center}
\vspace{-5mm}
  \caption{Reactive Threshold.}
  \label{threshold}
\end{figure}

The scaling process starts by removing the noise from the monitoring data. Next the Scaler evaluates whether the current performance behavior requires to trigger any of the following scaling actions:

\begin{itemize}

\item \textbf{Scale out or up:} Additional resources are provisioned if the performance behavior exceeds the upper boundary of the reactive threshold, and the Predictor confirms that such traffic spike will remain at least during the next 5min.

\item \textbf{Scale back or down:} Resources are released if the performance behaviour exceeds the lower boundary of the reactive threshold, and the Predictor confirms that such traffic oscillation will remain at least during the next 5min.

\end{itemize}

Note that, short-term (5min) forecasts operations are triggered to minimize at the maximum the number of SLA violations, as well as to keep a high level of efficiency in the predictions.

\begin{figure*}[htb]
	\begin{minipage}[b]{0.3\linewidth}
		\includegraphics[height=4.5cm]{images/prediction_conpaas_6min.eps}
		\vspace{-4mm}
	\end{minipage}
	\hfill
	\begin{minipage}[b]{0.3\linewidth}
		\includegraphics[height=4.5cm]{images/prediction_conpaas_10min}
		\vspace{-4mm}
	\end{minipage}
	\hfill
	\begin{minipage}[b]{0.3\linewidth}
		\includegraphics[height=4.5cm]{images/prediction_conpaas_30min}
		\vspace{-4mm}
	\end{minipage}
\caption{ConPaaS Predictor, response times for 5min, 10min and 30min ahead.}
\label{fig:vm_performance}
\end{figure*}

To decide which type of VM instances to release or add, the Scaler uses an optimal decision tree that calculates all the possible scaling strategies. In this decision tree, each node represents each type of VM instance provided by the cloud providers. The links indicate the percentage of CPU usage consumed to process the current workload when using a specific combination of nodes. To calculate the different combinations of VM instances that enforce the performance requirements, we used the next formula:

{\scriptsize
\begin{equation}\label{cpu_usage}
\begin{split}
CPU_{total} \geqslant \dfrac{ \sum_{i=1}^n ( \dfrac{ N\_reqs_{total} * Workload_{complex} }  {Num\_machines}  ) * CPU speed_{i} }  {Num\_machines}
\end{split}
\end{equation}
}

In Equation~\ref{cpu_usage}, the \emph{$Workload_{complex}$} factor represents the complexity of the incoming traffic  which englobes effects such as those caused by the \emph{i.e.} operative system activities, vm sharing or request heterogeneity. $CPU_{total}$ and $N\_reqs_{total}$ are calculated by the Predictor component that estimates the future service demand for the next 30min (request rate and cpu usage). As illustrated in Figure~\ref{}, forecast operations of the resource requirements for the next 30min offer an aceptable level of accuracy. $CPU_{speed}$ is calculated using the Profiler component and will variate depending on the type of vm instance used in a strategy. The Equation~\ref{cpu_usage} answers to the question "How much and which type of vm instances to provision?".

Once the Scaler obtains a list of possible scaling strategies, they are classified based on their infrastructure cost and degree of slo fulfillment, as defined in the Equation~\ref{strategy_cost}. The infrastructure cost specifies the price required to provision extra cloud resources. While the slo fulfillment (slo\_cost in the Equation~\ref{strategy_cost}) represents the degree of vulnerability of a strategy to experience SLO violations. It is calculated based on the percentage of CPU usage of a strategy in comparison with the maximum CPU usage (based on the Amazon EC2 recomendations 75\%), and multiply by a SLO penalty (user pre-defined value). Obviously, higher values in the percentage of CPU usage imply an increment in the probabilty of having SLO violations, so slo\_cost will increase as well.

\begin{equation}\label{strategy_cost}
\begin{split}
slo\_cost =  ( \dfrac{ capacity } {max capacity} ) * slo penalty \\
cost\_strategy = \dfrac{  slo\_fulfillment  } {infra\_cost}
\end{split}
\end{equation}

Note that, the search of an optimal scaling strategy can represent a NP hard problem, so that we again used the CPU fixed threshold to limit the maximum or minimum CPU percentage value offered for a strategy. This assumption avoids to trigger new scaling actions in the next time interval saving costs.  Moreover, a cost policy was also included for avoiding to choose wasteful scaling strategies. This policy reject strategies releasing VM instances which have been recently started ( 5 < time to the end of its hour < 20).

As explained in the Equation~\ref{strategy_cost}, the cost of a strategy is based on the cost incurred by its slo fulfillment degree and the infrastructure cost. To refine the search process with the goal of selecting the most appropriate strategy. Scaler provides three type classes of SLA agreements in function of the type of customer. The three classes of customers namely, gold, silver and bronze minimize the SLA violations with a different cost. Accordingly, a gold customer pays more in order to get the best service at the cost of some extra over-provisioning (lower slo cost). A silver customer gets good availability while a bronze customer obtains a reduced,
but acceptable, SLA fulfillment but with very little over-provisioning.

A complete point of view of the whole scaling flow is shown in Figure~\ref{autoScalingFlow}.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=\linewidth]{images/NewAutoScalingFlow}
  \end{center}
\vspace{-5mm}
  \caption{Auto scaling flow.}
  \label{autoScalingFlow}
\end{figure}

\subsection{Dynamic load balancing weights: }

The problem we consider here is the heterogeneity of cloud platforms.
Different virtual machines from the same cloud might have different performance
characteristics, even when their specifications from the cloud vendor are
the same~\cite{ec2Performance}. This issue can be addressed through various
load balancing techniques, like assigning weights to the backend servers or
taking into account the current number of connections that each server
handles. Furthermore, the performance behavior of the virtual servers may
also fluctuate, either due to changes in the application's usage
patterns, or due to changes related to the hosting of the virtual servers
(e.g., VM migration).

In order to address these issues in ConPaaS we implemented a weighted
load balancing system in which the weights of the servers are
periodically re-adjusted automatically, based on the monitoring data.
This method assigns the same weight to each backend server at the
beginning of the process. The weights are then periodically
adjusted (in our experiments, every $\sim$ 15min) proportionally
with the difference among the average cpu\_usage and request rate of the servers
during this time interval. By adding this technique to the feedback-based
algorithm, we noticed a performance improvement when running the
benchmarks, as discussed in the following.


\subsection{Logistic Regression}

A key subroutine in logistic regression \cite{Hastie2009} is improving the
estimates of the
coefficients, \(\beta\), using the Newton-Raphson algorithm.
In the course of this, we need to compute
\(\frac{1}{1 + e^{-x}}\) and
\(\frac{1}{(1 + e^{-x})^2}\).
We now discuss a sequence of increasingly sophisticated way of implementing
these operators in Q.

\subsubsection{The Lua implementation}
\label{Logit_Lua}
The ``quick and dirty'' way is to implement it in Lua. An example is
in Figure~\ref{logit_in_lua}. We take advantage of {\tt ffi.cast}
to directly access C data from Lua.
\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{logit.lua}
%% TODO HEAVILY EDIT THIS FOR FINAL PAPER
\caption{Using LuaJIT FFI to map C data to Lua}
\label{logit_in_lua}
\end{minipage}
}
\end{figure}

\subsubsection{The templatized implementation}
\label{Logit_template}
Figure~\ref{logit_in_lua} indicates that precisely one line is specific to the
operator in question, the rest is necessary boiler plate. Therefore, the design
of the run time has been around providing machinery to move data and leaving
the actual operator as a template that can be provided by the user. All the operator
% NOTE ACTUALLY this is f1s1opf2 not v1s1opv2
writer needs to (1) pick a template --- in this case it is {\tt v1s1opv2}, indicating that a
vector v1 and an optional scalar s1 are used to produce another vector v2.
(2) provide the code fragment {\tt  v2 = 1.0 / ( 1.0 + exp(-1.0 * v1)); )} (3)
guard against improper usage e.g, we wouldn't apply this function to a string.

In our experiments,
JIT compilation makes the Lua code 25\% faster than single-threaded C code,
although 3 times slower than the multi-threaded C code which shows almost linear
speedup.

\subsubsection{Eliminating common sub-expressions}
However, one quickly recognizes that creating separate operators for those 2
functions would lead to
redundant computations, which can be eliminated as in Figure~\ref{sub_expr}.
This cuts the time by half.

\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{sub_expr.lua}
\caption{Common sub-expression elimination}
\label{sub_expr}
\end{minipage}
}
\end{figure}

\subsubsection{Lock step evaluation}
However, the code in Figure~\ref{sub_expr} has a subtle but
critical bug when the number of elements in \(x\) exceeds the chunk
size.  If we were to call {\tt eval()} on \(y\), then we would
end up consuming sucessive chunks of \(t_3\). Now, if we were to call
{\tt eval()} on \(z\), we would fail when requesting the first
chunk of \(t_3\), since it has {\bf not} been memo-ized. One solution
is to ensure that \(y\) and \(z\) are evaluated in lock-step, after they have
been created, as in Figure~\ref{lock_step}.
\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{lock_step.lua}
\caption{Lock step evaluation}
\label{lock_step}
\end{minipage}
}
\end{figure}

The problem with this solution is that the burden of lock-step evaluation falls
on the Q programmer. This is remedied by the {\tt conjoin} function
(Figure~\ref{conjoin})

\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{conjoin.lua}
\caption{Conjoined Vectors}
\label{conjoin}
\end{minipage}
}
\end{figure}

%% TODO Normalizing the
%% TODO \be
%% TODO \item Lua versus C (single  threaded)
%% TODO \item Lua versus C (multi threaded)
%% TODO \item separate operators versus eliminating common sub expr
%% TODO \ee

\subsubsection{Custom Operators}

As part of the \(\beta\) calculation step, we need to compute an \(m \times m\)
matrix \(A\) as  \(x^T W x\) where (1)
\(x\) is a \(n \times m\) matrix and (2) \(W\) is a \(n \times n\) matrix where
non-diagonal elements are 0 and can be represented as a vector, \(w\) of legnth
\(n\).
The straight-forward Q code is in Figure~\ref{compute_A_Lua_basic}
\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{compute_A_Lua_basic.lua}
\caption{Q code for \(x^T W x\)}
\label{compute_A_Lua_basic}
\end{minipage}
}
\end{figure}

However, with a little more effort, we created custom operator,
a simplified sketch of which is provided in Figure~\ref{sum_prod_c}. The actual code
uses blocking to avoid writing out to the entire temp array before reading it
back in. Setting \(m=32\), we varied \(n\) from \(2^{16}\) to \(2^{24}\). With
blocking, performance increases as we increase \(n\) but saturates after
\(2^{20}\). Without blocking, performance peaks at \(n=2^{18}\) and steadily
declines after that, to about a third of peak performance.
What we hope to illustrate is that a
simple data model allows us to move along the performance versus simplicity
continuum as desired.

\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{sum_prod3.c}
\caption{Custom operator for \(x^T W x\)}
\label{sum_prod_c}
\end{minipage}
}
\end{figure}

This in turn raised the question: could we have ``chunked'' the computation
without resorting to a custom operator? To do so, (1)
we first set up the reducers --- no actual
computation happens. This is exactly as in Figure~\ref{compute_A_Lua_basic}, except that there is no call to
{\tt eval()}. (2) We evaluate the reducer in chunks, stopping
when the input vectors have no more data as in
Figure~\ref{compute_A_Lua_chunked}.
\begin{figure}
\centering
\fbox{
\begin{minipage}{14 cm}
\centering
\verbatiminput{compute_A_Lua_chunked.lua}
\caption{Chunked Q code for \(x^T W x\)}
\label{compute_A_Lua_chunked}
\end{minipage}
}
\end{figure}

%% \begin{table}
%%   \centering
%%   \begin{tabular}{|l|l|l|} \hline \hline
%%     {\bf Chunking used?}    & {\bf Q} & {\bf Custom Operator} \\ \hline \hline
%%     {\bf Yes} & TODO & TODO \\ \hline
%%     {\bf No} & TODO & TODO \\ \hline
%%     \hline
%%   \end{tabular}
%%   \caption{Rlative times for different computational strategies}
%% \end{table}