\documentclass[12pt]{ClasseMatematicamente}

\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{verbatim}
\usepackage{subfigure}
\usepackage{amsfonts}
%\usepackage{tikz}

\begin{document}

\pagestyle{fancy} \pagenumbering{arabic}

% --------------------------------------------------------------------

%---------  input heading  ------------
 \HeadingNumero{Numero 23}
 \HeadingData{Settembre 2014 - update \today ~- v0.0.1}
% %--------- input titolo -----------
 \NumeroArticolo{112}
 \Titolo{Hungarian algorithm: \\ a Short Presentation using Sage}

 \AutoreA{Sebastiano Ferraris}
 \EmailA{sebastiano.ferraris@gmail.com}

%%% ---  decommentare qui e nel .class le righe opportune per aggiungere altri autori (versione beta)

% \AutoreB{altroautore}
% \EmailB{altroautore@gmail.com}

% \AutoreC{altroautore}
% \EmailC{altroautore@gmail.com}

% \AutoreD{altroautore}
% \EmailD{altroautore@gmail.com}

% \AutoreE{altroautore}
% \EmailE{altroautore@gmail.com}

\Dedica{Dedicated to Federica Narciso, \\ to her courageous sweetness in life \\
and her tenacious talent in Mathematics.}
% %--------- input footing: numero di pagina di partenza ------------
\setcounter{page}{5}
% %----------------------------------------

\Issue

%\section*{Sommario}
%Qui mettere eventuale sommario ed introduzione

%------------ indice eventuale:
 %\tableofcontents
%____________________________

\noindent
The following pages are the result of sincere but disconcerted passion, some hours of spare time and the help of Luca Lussardi, Nicola De Nitti and Giuliano \lq\lq Nuanda\rq\rq \phantom{x}De Rossi. \\
This is to be intended as an introduction to the Hungarian Algorithm as well as a set of examples of recursive algorithms in Python. Any contribution, as suggestion, correction or improvement is welcome, so please do not hesitate to leave a comment or send me an email!

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Introduction}\label{se:introduction}

Almost every article about Hungarian starts talking about associating worker and tasks. According to the tradition, imagine to be the manager of a manufacturing company and to have to decide the allocation of $4$ workers in $4$ different workstations.
Every worker has different experiences and expresses a preference on the station in which he would like to work. The preference is expressed by giving a score from 1 to 20 (where 1 is the job that they prefer and 20 is the job they would like to avoid) \\
Your scope is to assign workers to workstations/tasks by making sure that everyone has a job close to its preference.\\
A way to find a solution to this problem, without comparing manually all the possibilities, is called \emph{Hungarian algorithm}. It was published by Harold Kuhn in $1955$, grounded on previous works made by two Hungarian mathematicians, Denes Konig and Jeno Egervary \cite{wiki}. In a more sophisticated perspective, not presented here, this problem can be seen as a particular application of the Hitchcock algorithm \cite{papadmitriou}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Bipartite Matching}

A way to represent input data is using dots and connectors. Dots are used to represent both workers (Abigail, Baldwin, Cade, Diane) and tasks ($t1$, $t2$, $t3$, $t4$). Connectors to represent possible decisions between workers and tasks.
\begin{figure}[!ht]
  \centering
  \includegraphics[height=7cm, width=4cm, angle=0,
   keepaspectratio]{figures/bipartite_graph_1.pdf}
  %\caption{Newspaper page with nodes and links}
  \label{fig:bipartite_graph}
\end{figure}
\noindent
We assign to each connector a value corresponding to the score given by each worker to the connected task: if Abigail gives a score to the task $t1$ equals to $15$, then the value of the connector from $A$ to $t1$ will be $15$. \\
Models made by dots and connectors are in general called \emph{graphs}, dots are called \emph{nodes} and connectors are called \emph{edges}. Graph used in this situation are called \emph{bipartite complete graph}. In the first partition we have workers and in the second partition we have tasks \cite{grafi}.
The number associated to each edge is called \emph{weight of the edge}. \\
In general, given two sets $A$ and $B$ with the same number of elements $n = \vert A \vert = \vert B \vert$, for which is defined a \emph{weight function} $w$,

\begin{align*}
  w : A \times B  &\longrightarrow  \mathbb{R}_{+} \\
   (a,b) &\longmapsto w(a,b)
\end{align*}
such that for every couple $(a,b)$ it associates a positive value,
it is always possible to draw a corresponding bipartite complete graph
\footnote{You may have to take your time to unpack this sentence. If $X$ is a set, the number of its elements is written as $\vert X \vert$ and it is called cardinality of $X$. $A \times B$ is the cartesian product of the set $A$ times the set $B$ and it is the domain of the function $w$. The range of the function is the set of positive real numbers $ \mathbb{R}_{+}$, the arrow $\mapsto$ means that the element $(a,b)$ of the domain is mapped into $w(a,b)$.}.
A subset of edges with no common nodes is called \emph{matching}. A matching in a bipartite complete graph with $n$ nodes in each partition is \emph{maximal} if you cannot add an edge to the matching while keeping the condition of not having common nodes for each edge. To each maximal matching corresponds a possible assignment workers-tasks. \\
In the figure below the maximal matching is highlighted in blue with the weight of the choose arch.

\noindent
Abigail is assigned to the job $t2$ (score $16$), Baldwin to the $t3$ (score $2$), Cade to the $t1$ (score $17$) and Diane to $t4$ (score $14$).
The \emph{weight of the matching} is the sum of the weights of the edges involved (in this case it is 49).
Our aim is to find the maximal matching with minimal weight, i.e. to solve the so called \emph{matching problem}.

\begin{figure}[!ht]
	\centering
	\includegraphics[height=7cm, width=4cm, angle=0,
	keepaspectratio]{figures/bipartite_graph_2.pdf}
	\label{fig:bipartite_graph_blue}
\end{figure}
\begin{figure}[!ht]
	\centering
	\includegraphics[height=10cm, width=10cm, angle=0,
	keepaspectratio]{figures/dia_main_1.png}
	\caption{Hungarian Algorithm Flowchart}
	\label{fig:main_flow}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Cost Matrix}
The representation of data with bipartite graph could be enough to reach the solution with a straight strategy, involving a device called \emph{augmenting path}\footnote{This way is well presented in \cite{mordecai}.}. In the solution to the matching problem that is presented here, the bipartite graph is not enough, and a second tool is necessary: the \emph{cost matrix}. \\
Consider the set of employees as a vector in alphabetical order $\mathbf{e} = (A,B,C,D)$ and the vector of tasks ordered as $\mathbf{t}=(t1,t2,t3,t4)$; the \emph{cost matrix} is defined as follows
\begin{align*}
  M[i,j] = w(\mathbf{e}[i],\mathbf{t}[j])
\end{align*}
Where $w(\mathbf{e}[i],\mathbf{t}[j])$ is the score that the worker $\mathbf{e}[i]$ gives to $\mathbf{t}[j]$.\\
In our leading example we will consider:
\begin{equation}\label{matrix_M}
M =
\left(\begin{array}{rrrr}
15 & 16 & 19 & 4 \\
6 & 1 & 2 & 6 \\
17 & 3 & 5 & 12 \\
13 & 6 & 16 & 14
\end{array}\right)
\end{equation}
At the position $[1,0]$ of $M$ we find\footnote{According to Python language, indexes start from zero: $M[1,0]$ is the elment in the second row and in the firs column of $M$.} the score of Baldwin for the job $t1$, which is equal to $6$.
A set of $n$ of elements of the cost matrix, such that none of them are on the same row or in the same vector corresponds to a matching. The association presented in the above figure can be represented in the cost matrix with bold numbers:
$$
M =
\left(\begin{array}{rrrr}
15 & \mathbf{16} & 19 & 4 \\
6 & 1 & \mathbf{2} & 6 \\
\mathbf{17} & 3 & 5 & 12 \\
13 & 6 & 16 & \mathbf{14}
\end{array}\right)
$$
It is easy to see that the set of the feasible solutions, i.e. the possible associations, has $4!$ elements ($4$ possible on the first row, $3$ on the second, $2$ on the third). Each feasible solution can be represented in the most economical way with a vector $\mathbf{s}$ of dimension $4$, such that in the first position we find the index of the column of the choose element in the first row of $M$. In the second position, the index of the column of the element in the second row of $M$ and so on.
In the previous case the vector corresponding to the matching is:
\begin{align*}
 \mathbf{s} = (1,2,0,3)
\end{align*}
The \emph{space of the feasible solution} $\mathcal{F}$ is the set of the vectors of dimension $4$ containing numbers in the set $\lbrace 0,1,2,3\rbrace $ with no repetitions.\\
The \emph{cost} of a vector $\mathbf{s}$ in $\mathcal{F}$ is defined as
\begin{align*}
 w(\mathbf{s}) = \sum_{i=0}^{3} M[i,\mathbf{s}[i]]
\end{align*}
and it corresponds to the weight of the matching.\\
The solution we are searching for is a vector in the set $\mathcal{F}$ with minimal cost (it is called \emph{solution vector} and it may be not unique). Since $\mathcal{F}$ has $n!$ elements, it is inconvenient to employ a brute force algorithm to find it.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Step by Step Algorithm}\label{se:stepbystep}
We propose a version of the algorithm which uses 4 main functions (\emph{row reduction}, \emph{column reduction}, \emph{percolation finder}, \emph{shaker}, \emph{resolvability test}), and 2 auxiliary function (\emph{covering segments searcher} and \emph{redundance filter}).
The \emph{input} of the algorithm is the cost matrix $M$ (it contains all information we need to solve the problem) and the \emph{output} is the solution vector.
In the following subsections each step is presented separately with Sage implementation and examples.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Step 1: Row Reduction}
The step 1 consists of the application of a function which finds the minimal element to each row and subtract it to all the elements of the row.

\begin{small}
\begin{lstlisting}
def row_reduction(m):
	"""
	step 1: row reduction
	:param m input matrix (type np.matrix)
	:return: m after row reduction: for each row the minimal element
	         is subtracted to each element of the row
	"""
	assert isinstance(m, np.matrixlib.defmatrix.matrix)
	return m - np.min(m, axis=1)
\end{lstlisting}
\end{small}

%%%% Esempio step 1
\noindent
The first line of the code asserts that m is an instance of the class np.matrix. If not the behaviour of np.min would change.
The matrix \ref{matrix_M} is provided to an ipython session as follows:
\begin{small}
\begin{lstlisting}
import numpy as np  # importing the library numpy
M = np.matrix([[15,16,19,4],[6,1,2,6],[17,3,5,12],[13,6,16,14]]); M
\end{lstlisting}
\end{small}
$$
\left(\begin{array}{rrrr}
15 & 16 & 19 & 4 \\
6 & 1 & 2 & 6 \\
17 & 3 & 5 & 12 \\
13 & 6 & 16 & 14
\end{array}\right)
$$
\begin{small}
 \begin{verbatim}
M1 = row_reduction(M); M1
\end{verbatim}
\end{small}
$$
\left(\begin{array}{rrrr}
11 & 12 & 15 & 0 \\
5 & 0 & 1 & 5 \\
14 & 0 & 2 & 9 \\
7 & 0 & 10 & 8
\end{array}\right)
$$

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Step 2: Column Reduction}
Step 2 refers to the columns: for each column it finds the minimal element and subtract it to all of the elements of the column.

\begin{small}
\begin{lstlisting}
def col_reduction(m):
	"""
	step 2: row reduction
	:param m input matrix (type np.matrix)
	:return: m after col reduction: for each column the minimal element
	is subtracted to each element of the column
	"""
	assert isinstance(m, np.matrixlib.defmatrix.matrix)
	return m - np.min(m, axis=0)
\end{lstlisting}
\end{small}

%%%% Esempio step 1
\noindent
Applying this in our leading example we get:
\begin{small}
 \begin{verbatim}
M2 = col_reduction(M1); M2
\end{verbatim}
\end{small}
\begin{equation}\label{matrix_M2}
\left(\begin{array}{rrrr}
6 & 12 & 14 & 0 \\
0 & 0 & 0 & 5 \\
9 & 0 & 1 & 9 \\
2 & 0 & 9 & 8
\end{array}\right)
\end{equation}
The solution vector does not change if we consider $M$ or $M$ after having applied row or column reduction.
\begin{lemma}
 Let $\mathbf{s}_1$ and $\mathbf{s}_2$ two vectors in the space of feasible solutions, $w$ the weight function defining $M$ and $\hat{w}$ the weight function defining $row\textunderscore reduction(M)$. If $w(\mathbf{s}_1) \leq w(\mathbf{s}_2)$ then $\hat{w}(\mathbf{s}_1) \leq \hat{w}(\mathbf{s}_2)$.
\end{lemma}
\begin{proof}
 Let $m_{i}$ the min in the $i$-th row. Subtract $m_{i}$ to the $i$-th row is the same as subtract it to each edge with a common node of the first partition in the related bipartite complete graph. If $w_{i}$ is the weight function defined by the matrix where we have subtracted the $m_{i}$ only in the $i$-th row we have $w_{i}(\mathbf{s}) = w(\mathbf{s}) - m_{i}$.\\
 It follows that $\hat{w}(\mathbf{s}) = w(\mathbf{s}) - \sum_{i=0}^{n-1} m_{i}$ for each $\mathbf{s}$ in the feasible space.
\end{proof}
It can be shown that the lemma is still valid if $\hat{w}$ is the weight function of $col\textunderscore reduction(M)$. It follows then:
\begin{theor}
 The solution vector for the matrix $M$ is the same as for the matrix \\
 $col\textunderscore reduction(row\textunderscore reduction(M))$ as well as for the matrix
 $row\textunderscore reduction(col\textunderscore reduction(M))$.
\end{theor}
Note that in general $col\textunderscore reduction(row\textunderscore reduction(M)) \neq row\textunderscore reduction(col\textunderscore reduction(M))$. Which means that you may not get the same matrix if the order of application of step 1 and step 2 is inverted.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Step 3: Percolation Finder}

After step 1 and 2 we have got a matrix with some zeros (at least one for each row and one for each column). If it is possible to \lq\lq walk\rq\rq\phantom{z}the matrix from the first row to the last just stepping on zeros and never passing twice on the same column, our walk will be the solution vector we are searching for, and we are done with the Hungarian Algorithm!\\

\noindent
For example in the following matrix
$$
N =
\left(\begin{array}{rrrr}
6 & 11 & 0 & \mathbf{0} \\
12 & \mathbf{0} & 0 & 4 \\
\mathbf{0} & 15 & 5 & 1 \\
8 & 14 & \mathbf{0} & 1
\end{array}\right)
$$
we can see such kind of walk and we can notice that the solution is $\mathbf{s} = (3,1,0,2)$. In the matrix (\ref{matrix_M2}) of the main example, a possible walk through zeros corresponds to the vector $(3,2,1,1)$ which is not a feasible solution. \\

\noindent
This rough idea of walk can be formalized under the name of percolation:
\begin{definition}
Given a costs matrix $M$, a {\bf percolation} over $M$ is a subset of $n$ element of the matrix $M$, one and only one for each row. If all the elements of a percolation are zeros, it is called $0$-{\bf percolation}.
\end{definition}
As just seen, not all the matrices admits $0$-percolation, but it easy to prove that every output of step 1 and step 2 admits at least one. \\
As made in our example we can identify every $0$-percolation with a vector $\mathbf{v} = (j_1, j_2, \dots , j_n)$ of dimension $n$ made of integer in the set $\lbrace 0,1,2, \dots, n-1 \rbrace$. With this nomenclature if $\mathbf{v}[i] = j_i$ then the element $M[i,j_i]$ belongs to the percolation.
For example in the matrix $N$, $4$ $0$-percolations are possible:
$$
\left(\begin{array}{rrrr}
6 & 11 & \mathbf{0} & 0 \\
12 & \mathbf{0} & 0 & 4 \\
\mathbf{0} & 15 & 5 & 1 \\
8 & 14 & \mathbf{0} & 1
\end{array}\right)
\qquad
\left(\begin{array}{rrrr}
6 & 11 & 0 & \mathbf{0} \\
12 & \mathbf{0} & 0 & 4 \\
\mathbf{0} & 15 & 5 & 1 \\
8 & 14 & \mathbf{0} & 1
\end{array}\right)
\qquad
\left(\begin{array}{rrrr}
6 & 11 & \mathbf{0} & 0 \\
12 & 0 & \mathbf{0} & 4 \\
\mathbf{0} & 15 & 5 & 1 \\
8 & 14 & \mathbf{0} & 1
\end{array}\right)
\qquad
\left(\begin{array}{rrrr}
6 & 11 & 0 & \mathbf{0} \\
12 & 0 & \mathbf{0} & 4 \\
\mathbf{0} & 15 & 5 & 1 \\
8 & 14 & \mathbf{0} & 1
\end{array}\right)
$$
The first, third and fourth do not represent a feasible solution as two zeros involved in the 0-percolation are in the same column.
It is interesting to count how many zeros of the same $0$-percolation are in the same column, since the solution we are searching for will be the $0$-percolation having exactly $1$ zero in each column: the second 0-percolation emphasised in the previous example.

\noindent
The mathematical tool that states how far is our percolation from the one that we are searching lies in the following definition:
\begin{definition}
 Given a $0$-percolation $\mathbf{v}$ over $M$ of length $n$, the index of redundancy of $\mathbf{v}$ is a real number defined as the maximal number of zeros of the percolation in the same column minus 1, plus the number of columns with no elements in the 0-percolation divided by the length of the percolation.
\end{definition}
Sometimes a formula can make a definition easier to be understand. This is one of this cases:
\begin{align*}
 \text{index\textunderscore of\textunderscore redundancy}(\mathbf{v})
 =&
 \text{~max (zeros of the percolation in the same column)} -1 +\\
 &+ |\{ \text{columns with no elements in the 0-percolation} \} | /n
\end{align*}
where max is the maximal number of zeros of the percolation in the same column.\\
In the previous example we have
$$
\begin{array}{l l l l }
 (2,1,0,2)~1.25\qquad\qquad & (3,1,0,2)~0 \qquad\qquad & (2,2,0,2)~2.5 \qquad\qquad & (3,2,0,2)~1.25
\end{array}
$$
where each index of redundancy is shown on the right of its percolation.
Given the percolation $V$ a function to calculate the redundancy index is:

\begin{small}
\begin{lstlisting}
def redundace_index(v):
	"""
	step 3: redundacy index
	"""
	n = v.size
	occurrence_vector = [v.count(i) for i in range(n)]
	return np.max(occurrence_vector) - 1 + occurrence_vector.count(0) / float(n)
\end{lstlisting}
\end{small}

\noindent
where \emph{occurrence vector} at the position $i$ gives the number of occurrence of the number $i$ in the vector $V$.
If $M$ admits at leas one $0$-percolation with redundancy index equals to $0$, a recursive\footnote{An introduction to recursive algorithm can be found in \cite{sedgewick}.} algorithm to find it, can be:
\begin{small}
\begin{lstlisting}
ans = []

def percolation_finder_0(M):
	"""
	percolation finder initial version
	:param M: cost matrix after col and row reduction.
	:return: True if the matrix admits a 0-percolation with redundancy
	index equals to 0, False otherwise.
	In the global variable ans is recorded the first 0-percolation found if it  exists.
	"""
	def find_walk(M, row, sol):
		global ans
		print(row)
		for i in range(M.shape[1]):
			if M[row, i] == 0 and i not in sol:
				sol = sol + [i]
				ans = sol
				#print(ans)
				if row >= M.shape[1] - 1 or find_walk(M, row+1, sol):
					return True
				else:
				sol = sol[0:-1]
		return False
	return find_walk(M, 0, [])
\end{lstlisting}
\end{small}

\noindent
Uncommenting the line \emph{print ans} allows to see the recursive algorithm at work. Note that to use this algorithm twice in the same session, you must initialize the global variable to the empty list.
If the matrix has more than one solution, the previous algorithm provides us only the first that he finds, stored in the variable $ans$. In general the matrix could admit no solution at all. In this case all the $0$-percolation have redundancy index grater then $0$.\\
This means that we have identified the following two cases:
\begin{enumerate}
 \item $row\textunderscore reduction(col\textunderscore reduction(M))$ admits $0$-percolation with redundancy index equals to $0$.
 \item $row\textunderscore reduction(col\textunderscore reduction(M))$ does not admit any $0$-percolation with redundancy index equals to $0$.
\end{enumerate}
We need an algorithm that provides {\bf all} the $0$-percolation and tells if some of them has index of redundancy equals to zero or not, as this will be our solution.
We use again recursion:

\begin{small}
\begin{lstlisting}
	def percolation_finder_1(m, max_num_percolation=6):
		"""
		percolation finder initial version 1
		:param m: cost matrix m.
		:param max_num_percolation: maximal number of 0-percolations returned.
		:return: list of all of the 0-percolation, up to max_num_percolation, followed by redundancy index.
		"""
		walks = { 'walk': [] }  # Name binding

		def scout(m, breadcrumbs):
			ncols = m.shape[1]
			crumbs_number = len(breadcrumbs)

			#print(breadcrumbs)
			if crumbs_number < ncols:
				for j in range(ncols):
					if m[crumbs_number,j] == 0:
						scout(m,breadcrumbs + [j])
			elif crumbs_number == ncols:
				walks['walk'] = walks['walk'] + [breadcrumbs]

		for j in range(m.shape[1]):
			if m[0,j] == 0:
				scout(m, [j])
		return walks['walk']
\end{lstlisting}
\end{small}

\noindent
As before, uncommenting the line \emph{print breadcrumbs}, we can see recursion at work. You may have noticed that a matrix with a lots of zeros makes the walks list too big to be manageable.
Since we are not interested in all the percolation but only in those with minimal redundancy, we add a strategy to decide if we may want to stop after having collected \lq\lq enough\rq\rq\phantom{z}percolations in the vector \emph{walks}:

\begin{small}
\begin{lstlisting}
def percolation_finder(m, max_num_percolation=6):
	"""
	Step 3 percolation finder
	:param m: cost matrix m.
	:param max_num_percolation: maximal number of 0-percolations returned.
	:return: list of all of the 0-percolation, up to max_num_percolation,
	         followed by redundancy index.
	"""
	walks = { 'walk': [] }  # Name binding

	def redundancy_index(v):
		n = len(v)
		occurrence_vector = [v.count(i) for i in range(n)]
		return max(occurrence_vector) - 1 + occurrence_vector.count(0) * 1.0 / n

	def scout(m, breadcrumbs):
		n_cols = m.shape[0]
		crumbs_number = len(breadcrumbs)
		# print(breadcrumbs)
		if crumbs_number < n_cols:
			col_index_arranged = list(set(range(n_cols)) - set(breadcrumbs)) + \
			                     list(set(breadcrumbs))
			for j in col_index_arranged:
				if m[crumbs_number, j] == 0:
					# More scouts are launched only if some place in the walks'
					# vector is available. Rearranging indexes may guarantee
					# that a percolation with 0 redundancy index is stored in the
					# first explorations, when available.
					if len(walks['walk']) < 2 * max_num_percolation:
						scout(m, breadcrumbs + [j])
		elif crumbs_number == n_cols:
			walks_recorder(breadcrumbs)

	def walks_recorder(v):
		ri = redundancy_index(v)
		walks['walk'] = walks['walk'] + [v] + [ri]

	first_zero = True
	for j in range(m.shape[1]):
		if m[0, j] == 0:
			if first_zero:
				# reset to [] the list of walks
				walks['walk'] = []
				first_zero = False
			#print('A scout goes in mission')
			scout(m, [j])

	if len(walks['walk']) == 0:
		raise TypeError('Input matrix has no 0-percolations.')

	return [m, walks['walk']]
\end{lstlisting}
\end{small}

\noindent
The function \emph{walks\textunderscore recorder} adds to the global variable \emph{walks} each new $0$-percolation followed by its redundance index \emph{ri}.
If the variable \emph{find\textunderscore all}, in the input value, is set to \emph{False} then the vector walks will collect only a number of $0$-percolation equals to \emph{max\textunderscore percolation}.\footnote{Note that this just stop the recording, and do not stops the recursive steps. It is also possible to stop the recursion when the \emph{max \textunderscore percolation} is reached. This option is not proposed here for clarity.} It does make sense to stop the recursion before all $0$-percolation are collected, just because the indexes of the first cycle-for had been rearranged:
\begin{small}
\begin{lstlisting}
col_index_arranged = list(set(range(ncols))-set(breadcrumbs)) + list(set(breadcrumbs))
\end{lstlisting}
\end{small}
This means that when the algorithm is arrived at the row \emph{crumbs\textunderscore number} it searches for a new zero before in columns that have not been used already. For example, if the input matrix is like the following (the best case to find a solution as well as one of the worst case for a recursive algorithm)
$$
N =
\left(\begin{array}{rrrr}
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{array}\right)
$$
the first $0$-percolation that the algorithm finds is the solution $(0,1,2,3)$. So it makes sense to limit the solution, even thought if the number \emph{max\textunderscore percolation} is too little we may lose some multiple solution.\\
Applying this to our main example (matrix \ref{matrix_M2}) we get:
\begin{small}
\begin{lstlisting}
walks
[[3, 0, 1, 1], 1.25, [3, 1, 1, 1], 2.5, [3, 2, 1, 1], 1.25]
\end{lstlisting}
\end{small}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Step 4: Resolvability Query}
At this stage we have got the matrix reduced and the list of $0$-percolation (complete or not) with related redundancy indexes.
After the step $4$ we will have the list of $0$-percolation with minimal redundancy index, and a flag which states if the $0$-percolation (or percolations) found is already our solution or not.

\begin{small}
\begin{lstlisting}
def resolvability_query(m, walks_):
	"""
	:param m: cost matrix
	:param walks_: list of 0-percolation followed by its index of redundancy
	as returned by percolation_finder
	:return: M again untouched, followed by the list of $0$-percolation with
	minimal index of redundancy, and with a flag, True if the minimal index
	is 0 and so we have already our solution, False otherwise.
	"""
	min_redundancy = np.min(walks_[1::2])
	filtered_walks = [walks_[i] for i in list(range(len(walks_)))[::2] \
			          if walks_[i + 1] == min_redundancy]
	if min_redundancy == 0:
		flag = True
	else:
		flag = False
	return [m, filtered_walks, flag]

\end{lstlisting}
\end{small}

The iterator (or list in python 2) $range(len(walks))[::2]$ is the list of the redundancy index in the vector walks (all the odd position in the vector).\\
Lets see it at work on our main example:
\begin{small}
 \begin{lstlisting}
A,B = percolation_finder(M2); A,B
resolvability_query(A,B)
\end{lstlisting}
\end{small}
\begin{equation}\label{matrix_M4}
\left[\left(\begin{array}{rrrr}
6 & 12 & 14 & 0 \\
0 & 0 & 0 & 5 \\
9 & 0 & 1 & 9 \\
2 & 0 & 9 & 8
\end{array}\right), \left[3, 0, 1, 1\right], \left[3, 2, 1, 1\right],
\mathrm{False}\right]
\end{equation}
When the flag is True the job is done. If it is False we must do some manipulation on the matrix in order to add some zeros without change the results. This tricky passage is here the step number $5$, called shaker.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Step 5: Shaker}

To perform this passage, first of all we must find the \emph{covering segments} of the input matrix $M$. A covering of $M$ is the minimal set of horizontal or vertical segments that covers all the zeros in the matrix. In \cite{sito_1}, it is suggested to actually draw horizontal or vertical lines over the matrix, covering all the zeros such that the number of lines is minimal.
Here, instead of using covering segments, we use a couple of vectors called \emph{covered\textunderscore col} and \emph{covered\textunderscore row}:
\begin{definition}
 Let $M$ be a square matrix with real entries of dimension $n\times n$. We define two vectors of dimension $n$ called \emph{covered\textunderscore col} and \emph{covered\textunderscore row} with elements in the set $\lbrace 0,1 \rbrace$
 such that if $M[i,j]$ is a zero then or \emph{covered\textunderscore row}$[i] = 1$ or \emph{covered\textunderscore col}$[j] = 1$ and the number of $1$ in both vectors is minimal. \\
 The non-zero elements in the vector covered\textunderscore row  are called \emph{horizontal covering segments}. The non-zero elements in the vector covered\textunderscore col are called \emph{vertical covering segments}.
\end{definition}
%% example auxiliary fucntion step 4
For the matrix \ref{matrix_M2} those vectors are:
$$
covered\textunderscore row = \left[0, 1, 0, 0\right]\qquad covered\textunderscore col = \left[0, 1, 0, 1\right]
$$

$$
\bordermatrix{\text{} & \mathbf{0} & \mathbf{1} & \mathbf{0} & \mathbf{1} \cr
                \mathbf{0} &    6 & 12 & 14 & 0  \cr
                \mathbf{1} &    0 & 0 & 0 & 5  \cr
                \mathbf{0} &    9 & 0 & 1 & 9  \cr
                \mathbf{0} &    2 & 0 & 9 & 8   }
$$

%%%
\noindent
The algorithm that we use to find covering segments requires the matrix as well as one $0$-percolation with minimal redundancy index. It is called \emph{covering\textunderscore segments \textunderscore searcher} and will be used as auxiliary function of the shaker:

\begin{small}
\begin{lstlisting}
def covering_segments_searcher(m, min_redundancy_percolation):
	"""
	step 5, auxiliary function.
	Returns the positions of horizontal and vertical covering segments
	"""
	# (A)
	n_rows = m.shape[0]
	n_cols = m.shape[1]
	marked_row = [0] * n_rows
	marked_col = [0] * n_cols
	# (B)
	occurrence_vector = [min_redundancy_percolation.count(i) for i in range(n_rows)]
	for pos in range(n_rows):
		if occurrence_vector[pos] > 1:
			duplicates_pos = [k for k in range(n_rows) if min_redundancy_percolation[k] == pos][1:]
			for j in duplicates_pos:
				marked_row[j] = 1
	# (C)
	flag_mark = 1
	while flag_mark != 0:
		flag_mark = 0
		# (C-1)
		for i in range(n_rows):
			if marked_row[i] == 1:
				for j in range(n_cols):
					if m[i, j] == 0 and marked_col[j] != 1:
						marked_col[j] = 1
						flag_mark += flag_mark
		# (C-2)
		for j in range(n_cols):
			if marked_col[j] == 1:
				for i in range(n_rows):
					if m[i, j] == 0 and marked_row[i] != 1 \
					   and min_redundancy_percolation[i] == j:
						marked_row[i] = 1
						flag_mark += 1
	# (D)
	covered_row = [(i + 1) % 2 for i in marked_row]
	covered_col = marked_col

	return covered_row, covered_col

\end{lstlisting}
\end{small}

\noindent
It performs the following tasks:
\begin{enumerate}
 \item[(A)] Initialize marked\textunderscore row and marked\textunderscore col as zero vectors.
 \item[(B)] Mark all the rows having a zero that makes the input $0$-percolation redundant (i.e. if $(M)_{h,j}$ and $(M)_{k,j}$ are zeros in the same column we mark the row $k$). The vector duplicates\textunderscore pos gives the position of all the duplicate zeros in the same columns.
 \item[(C-1)] Mark all the column having a zero in the row marked in the previous step.
 \item[(C-2)] Mark all the row having zeros in marked cols, only if those zeros are in the $0$-percolation with minimal redundancy (so if their cols are in the min\textunderscore redundant\textunderscore percolation vector).
 \item[(C)] Perform (C-1) and (C-2) in sequence until it is possible to mark new positions.
 \item[(D)] Vertical covering segments are the segments marked in previous step in the column, whereas horizontal covering segments are all the unmarked row.
\end{enumerate}
%%% theory auxiliary function step 4
It can be proved that covering\textunderscore segments\textunderscore searcher finds the covering segments for the input matrix. Moreover the number of covering segments is always equals or less than the dimension of the matrix. In case of equality the vector min\textunderscore redundant\textunderscore percolation has redundancy index equals to $0$.
%%%
The concept of covering segments leads us to the crucial definition of \emph{uncovered}, \emph{covered} and \emph{bi-covered} elements in the matrix $M$.
\begin{definition}
 Let $M$ be a square matrix with real entries of dimension $n\times n$ considered with \emph{covered\textunderscore col} and \emph{covered\textunderscore row} vectors. \\
 If at the element $M[i,j]$ corresponds covered\textunderscore row$[i] = 0 $ and covered\textunderscore col$[j] = 0$ then the element $M[i,j]$ is called \emph{uncovered} (and certainly is not a zero). \\
 If only one between covered\textunderscore row$[i]$ and covered\textunderscore col$[j]$ are zero then the element $M[i,j]$ is called \emph{covered}. \\
 If both covered\textunderscore row$[i]$ and covered\textunderscore col$[j]$ are $1$ then the element $M[i,j]$ is called \emph{bi-covered}.\\
 The total number of non-zero entries in \emph{covered\textunderscore col} and \emph{covered\textunderscore row} is called \emph{covering number}\footnote{It can be defined a discrete monotonous function g, such that (covering number) + g(redundancy index) = $n$.}.
\end{definition}
%%%
%%% step 4 main funcion
%%%
Now we have all that we need to ''shake'' our matrix, which means perform what the following theorem states:
\begin{theor}\label{th:shaker}
 Let $M$ be a matrix considered with \emph{covered\textunderscore col} and \emph{covered\textunderscore row} vectors and with a solution vector $\mathbf{s}$ (element of the space of feasible solution that minimize the cost of the association). Let $m$ be the minimal element of the set of the uncovered elements. If $m$ is subtracted to all the uncovered elements and $2m$ is added to all the bi-covered elements, the new matrix obtained in this way still admits $\mathbf{s}$ as solution vector.
\end{theor}
The matrix that results applying the previous theorem is called here \emph{shaked matrix}.
Our shaker algorithm is as follows:\\

\begin{small}
\begin{lstlisting}
def shaker(m, filtered_walks):
	"""
	:param m : cost matrix m,
	:param filtered_walks: set of walks with minimal redundance index.
	:return: shaked matrix m
	"""
	n_rows = m.shape[0]
	n_cols = m.shape[1]
	min_redundancy_percolation = filtered_walks[0]
	# (1)
	[cov_row, cov_col] = covering_segments_searcher(m, min_redundancy_percolation)
	# (2)
	zero_pos_in_cov_row = [i for i in range(n_rows) if cov_row[i] == 0]
	zero_pos_in_cov_col = [j for j in range(n_cols) if cov_col[j] == 0]
	uncovered_elements = [m[i, j] for i in zero_pos_in_cov_row
	                              for j in zero_pos_in_cov_col]
	if not uncovered_elements:  # uncovered_elements == []
		raise EnvironmentError()

	min_val = min(uncovered_elements)
	# (3)
	for i in range(n_rows):
		for j in range(n_cols):
	        if cov_row[i] == 0 == cov_col[j]:
				m[i, j] -= min_val
			elif cov_row[i] == 1 == cov_col[j]:
				m[i, j] += 2 * min_val
	return m
\end{lstlisting}
\end{small}

\begin{enumerate}
 \item[(A)] Uses auxiliary function to find the covering segments
 \item[(B)] Finds $m$ the minimal uncovered element value.
 \item[(C)] $m$ is subtracted to every uncovered element and $2m$ is added to each bicovered element.
\end{enumerate}

\noindent
This step is performed after percolation finder and resolvability test until the resolvability test is not passed (see again figure \ref{fig:main_flow}). \\
%%%
%%%
%%% Main example step 5
Back to the main example:
\begin{small}
 \begin{verbatim}
 A,B = percolation_finder(M2); A,B
 N = shaker(A,B); N
\end{verbatim}
\end{small}
\begin{equation}\label{matrix_M5}
N=
\left(\begin{array}{rrrr}
6 & 14 & 14 & 0 \\
0 & 2 & 0 & 5 \\
8 & 0 & 0 & 8 \\
1 & 0 & 8 & 7
\end{array}\right)
\end{equation}
Which admits a $0$-percolation with redundancy index equals to $0$ (it is our solution!). Applying again percolation finder and resolvability test we get
\begin{small}
 \begin{verbatim}
S = percolation_finder(N);
T = resolvability_test(S[0],S[1]); T
\end{verbatim}
\end{small}
\begin{equation}
\left[\left(\begin{array}{rrrr}
6 & 14 & 14 & 0 \\
0 & 2 & 0 & 5 \\
8 & 0 & 0 & 8 \\
1 & 0 & 8 & 7
\end{array}\right), \left[\left[3, 0, 2, 1\right]\right],
\mathrm{True}\right]
\end{equation}
An so the solution to the main problem (that is the association of workers to different tasks that maximizes satisfaction and productivity) is $[3, 0, 2, 1]$.

%%%%%%%%%%%%%%%% TEORIA STEP 4
\subsubsection*{Why does it work?}
In this section we want to answer the following questions:
\begin{enumerate}
 \item Why does the solution vector remain unchanged after the shaker is applied to $M$?
 \item Why does the shaker take us away from trouble in a finite number of steps?
\end{enumerate}
The answer to the first question consists is the proof of theorem \ref{th:shaker}. We approach the proof via some lemmas:
\begin{lemma}
 Given a cost matrix $M$ of dimension $n\times n$, if $p$ is a percolation with redundancy index equals to $0$, it is uniquely defined by a bijection $\sigma$ between the set of indexes $\lbrace 0,1, \dots , n-1 \rbrace$.
\end{lemma}
\begin{proof}
 A percolation $\mathbf{p} = (j_0, j_1, \dots , j_{n-1})$ with redundancy index equals to $0$ is an vector that associate to each position (row in the corresponding matrix) one and only one integer number between $0$ and $n-1$ (column in the corresponding matrix) without repetitions:
 \begin{equation*}
(j_0, j_1, \dots , j_{n-1})
\qquad \longmapsto \qquad
\left(\begin{array}{ccccc}
0 & 1 & 2 & \dots & n-1 \\
j_0 & j_1 & j_2 & \dots & j_{n-1}
\end{array}\right) = \sigma
\end{equation*}
Each percolation with redundancy index equals to $0$ can be represented by the set of elements of the matrix $\lbrace M[i, \sigma(i)]\rbrace_{i=0}^{n-1}$.
\end{proof}
\begin{lemma}
 Let $M$ be a cost matrix of dimension $n\times n$ and \emph{covering\textunderscore row} and \emph{covering\textunderscore col} having $n-1$ non-zero elements (covering segments are n-1), and let $\sigma$ a bijection of $n$ elements. It must always exists an integer $a\in \lbrace 0,1, \dots , n-1 \rbrace$ such that $M[a, \sigma(a)]$ is uncovered.  Moreover if for some $k$, $M[k, \sigma(k)]$ is a bi-covered element then it must exists $b$ and $c$ such that $M[b, \sigma(b)]$ and $M[c, \sigma(c)]$ are uncovered.
\end{lemma}
\begin{proof}
 Such $k$ do not always exists: all covering segments can be parallel. In this case either \emph{covering\textunderscore row} has only one zero and \emph{covering\textunderscore col} is all made by zeros or \emph{covering\textunderscore col} has only one zero and \emph{covering\textunderscore row} is all made by zeros. If we consider \emph{covering\textunderscore row} with only one zero at the position $t$ then our uncovered element will be $M[t, \sigma(t)]$ and so we have proved the first part of the thesis for $a=t$.

 Let $M[k, \sigma(k)]$ a bi-covered element, then we must have at least one non-zero element in \emph{covering\textunderscore col} in the position $k$ and at least two zeros in the vector \emph{covering\textunderscore col}, in the position $a \neq k$ and $b \neq k$ (or vice versa one non-zero element in \emph{covering\textunderscore col} in the position $\sigma^{-1}(k)$ and at least two zeros in the vector \emph{covering\textunderscore row}).

 If the percolation defined by $\sigma$, passes trough the bi-covered element (and therefore through $M[k, \sigma(k)]$), when it passes through the rows of $M$, indicated with $a$ and $b$, it will necessarily encounter two uncovered elements, since $ \sigma(a) \neq \sigma(k)$ and $ \sigma(b) \neq \sigma(k)$.
\end{proof}
As a consequence of the previous lemma we have:
\begin{lemma}\label{le:per_shaker}
 Let $\mathbf{s}_1$ and $\mathbf{s}_2$ be two vector in the space of the feasible solution, $w$ the weight function of $M$ and $\hat{w}$ the weight function defined by the shaked matrix of $M$. If $w(\mathbf{s}_1) \leq w(\mathbf{s}_2)$ then $\hat{w}(\mathbf{s}_1) \leq \hat{w}(\mathbf{s}_2)$.
\end{lemma}
And now, considering the fact that The thesis of theorem \ref{th:shaker} is equivalent to the lemma \ref{le:per_shaker}, theorem \ref{th:shaker} has been proved.

The answer to the second questions is a consequence of the following results:
\begin{lemma}
 Let $M[i,j] = 0$ a bi-covered element in the cost matrix $M$. It follows that
 \begin{align*}
  z\textunderscore cols[j] > 1 \\
  z\textunderscore rows[i] > 1
 \end{align*}
 where $z\textunderscore cols[j]$ is the number of zeros in the col $j$ of $M$ and $z\textunderscore rows[i]$ is the number of zeros in the row $i$.
\end{lemma}
\begin{proof}
 By contradiction, if $z\textunderscore rows[i] =  1$, there is only one zero on the $i$-th row. This must be $M[i,j]$, but it is already covered by the covering segment $z\textunderscore cols[j]$. According to the minimality of the covering segments we should have $z\textunderscore rows[i] = 0$ and $M[i,j] = 0$ should not be a bi-covered element.
\end{proof}
Every time step $4$ is performed, we add a zero to the set of the uncovered elements, and we remove (in the worst case) a number of zero equals to the number of bi-covered elements. However according to the previous lemma this step do not reduces the non-zero element on the vectors $z\textunderscore rows$ and $z\textunderscore cols$. \\
Since after step $1$ and step $2$ we have at least $n+1$ non-zero elements in both $z\textunderscore rows$ and $z\textunderscore cols$, after a finite number of application of the steps $4$, in the worst case given by
\begin{align*}
\sum_{i \mid z\textunderscore rows[i]>1} z\textunderscore rows[i]
+
\sum_{j \mid z\textunderscore cols[j]>1} z\textunderscore cols[j]
+n-1
\end{align*}
all the zeros in vectors $z\textunderscore rows$ and $z\textunderscore cols$ we will be eliminated .\\
If there is no more zeros means that for each row and col we can always find a percolation with redundancy index equals to $0$ and so we can be sure that step\textunderscore4 take us away from trouble in a finite number of steps.
%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Hungarian Algorithm}
Now it is possible to join all the previous steps in one single function, according to the flowchart presented at the beginning of the section.

\begin{small}
 \begin{lstlisting}
#-------------------------
# Hungarian Algorithm
#-------------------------
def hungarian(M, find_all = True, max_percolation = 6):
	"""
	Hungarian algorithm.
	Note that too many zeros in the initial cost matrix may need an high max_num_percolation,
	Set the weight away from zero, if this is a common value.
	Parameters:
	------------
	:param m: cost matrix
	:param max_num_percolation:
	:return: solution to the hungarian algorithm with cost matrix m
	"""
    walks = []
    cont = 0
    max_loop = max(M.nrows(), M.ncols())
    S = row_reduction(col_reduction(M))
    [S, walks] = percolation_finder(S, find_all, max_percolation)
    [S, filtered_walks, flag] = resolvability_test(S, walks)
    while flag == False and cont < max_loop:
        S = shaker(S, filtered_walks)
        walks = []
        [S, walks] = percolation_finder(S, find_all, max_percolation)
        [S, filtered_walks, flag] = resolvability_test(S, walks)
        cont = cont + 1
    walks = []
    return filtered_walks
\end{lstlisting}
\end{small}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Conclusion}\label{se:conclusion}
A tool for finding the solution vector of any square matrix has been presented.
Saving all functions here presented in a single file (hungarian.py) we can load it into an ipython session with \emph{run}:
\begin{small}
 \begin{lstlisting}
 run hungarian.py
\end{lstlisting}
\end{small}
We can test it on a random matrix $A$
\begin{small}
 \begin{lstlisting}
A = np.matrix(np.random.randint(0, 30, size=(10, 10)))
\end{lstlisting}
$$
\left(\begin{array}{rrrrrrrrrr}
18 & 3 & 39 & 27 & 2 & 4 & 34 & 8 & 0
& 5 \\
11 & 0 & 11 & 17 & 23 & 25 & 32 & 22 &
14 & 0 \\
0 & 13 & 0 & 12 & 16 & 12 & 0 & 11 & 14
& 1 \\
8 & 28 & 10 & 0 & 24 & 0 & 19 & 3 & 17
& 18 \\
16 & 17 & 36 & 0 & 0 & 26 & 32 & 5 & 3
& 20 \\
7 & 11 & 11 & 7 & 21 & 4 & 0 & 18 & 8
& 13 \\
23 & 14 & 26 & 8 & 2 & 15 & 26 & 0 & 24
& 10 \\
11 & 3 & 12 & 0 & 25 & 0 & 24 & 0 & 8
& 1 \\
3 & 1 & 0 & 7 & 5 & 12 & 22 & 10 & 13
& 10 \\
0 & 0 & 7 & 7 & 0 & 23 & 25 & 17 & 15
& 14
\end{array}\right)
$$
\begin{lstlisting}
s = hungarian(A); s
\end{lstlisting}
$$
\left[\left[8, 9, 0, 3, 4, 6, 7, 5, 2, 1\right],
\left[8, 9, 0, 5, 4, 6, 7, 3, 2, 1\right]\right]
$$
\end{small}
If there are more employees than workstations, we can still use the Hungarian algorithm, adding dummy nodes to the second partition of the biparted complete graph.
Each worker will give them a score from $1$ to $20$. $1$ if they wish to find new job and be moved to another department or workplace; $20$ if they prefer not to be moved.\\
Coming back to our leading example, we suppose that $t4 =$ \lq\lq relocate to new plant\lq\lq  and the workers $(A,B,C,D)$ are very loyal to their place so they give all a score equals to $20$ for the node $t4$. The cost matrix will be:
\begin{equation}
M =
\left(\begin{array}{rrrr}
15 & 16 & 19 & 20 \\
6 & 1 & 2 & 20 \\
17 & 3 & 5 & 20 \\
13 & 6 & 16 & 20
\end{array}\right)
\end{equation}
The solution is to move Abigail since
\begin{small}
 \begin{lstlisting}
s = hungarian(M); s
\end{lstlisting}
\end{small}
$$
\left[3, 0, 2, 1\right]
$$
%

\vspace{0.1cm}

%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%
%% BIBLIOGRAFIA
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%

\begin{thebibliography}{5}

\bibitem{grafi}
J.A. Bondy, U. S. R. Murty, \emph{Graph theory with applications}, Elevier Science Publishing, 1982.

\bibitem{sedgewick}
R. Sedgewick, P. Flajolet, \emph{An introduction to Analysis of Algorithm}, Pearson, 1996.

\bibitem{papadmitriou}
C. Papadmitriou, K. Steiglitz, \emph{Combinatorial Optimization}, Drover Books, 1998.

\bibitem{sito_1}
\begin{small}
\begin{verbatim}
http://www.hungarianalgorithm.com/
\end{verbatim}
\end{small}

\bibitem{wiki}
\begin{small}
\begin{verbatim}
http://en.wikipedia.org/wiki/Hungarian_algorithm
\end{verbatim}
\end{small}

\bibitem{mordecai}
Mordecai J. Golin, \emph{Bipartite Matching and the Hungarian Method}, Course Notes, Hong Kong University of Science and Technology.

\bibitem{lutz}
Mark Lutz, \emph{Programming Python}, 3rd edition, Oreilly And Associates 2007.

\end{thebibliography}

\end{document}


\chapter{Results}
In this chapter we examine the rendered output results of our implementation of our BRDF models applied to different input patches such as the Blaze grating and the Elaphe $\ref{fig:elpahespecies}$ and Xenopeltis $\ref{fig:xenospeicies}$ snake skins nanostructures. We are discussing and comparing both, their BRDF maps $\ref{fig:brdfmapexplanation}$ and the corresponding renderings on a snake geometry as shown in section $\ref{sec:snakegeomrenderings}$, for various input parameters. Last we also show a real experimental image showing the effect of diffraction for setting readers perception about expected results.

\section{BRDF maps}
A BRDF map shows a shader's output for all possible viewing directions for a given fixed, incident light direction. We assume that each viewing direction is expressed in spherical coordinates (See appendix $\ref{sec:sphericalcoordinates}$) $(\theta_v, \phi_v)$ and is represented in the map at a point

\begin{align}
(x,y) = (sin(\theta_v)cos(\phi_v), sin(\theta_v)sin(\phi_v))
\end{align}

with its origin at the center of the map. The light direction for normal incidence $(\theta_i, \phi_i)$ has been fixed to $(0,0)$ for our rendered results.

\begin{figure}[H]
  \centering
  \subfigure[BRDF map schema]{
    \includegraphics[scale=0.3]{resultsnew/brdfmapschema.png}
    \label{fig:brdfmapschema}
  }
~
  \subfigure[Light reflection geometrical setup]{
    \includegraphics[scale=1.05]{results/Lightreflectiongeometry.png}
    \label{fig:lightreflectiongeometry}
  }
~

\caption[BRDF Map]{BRDF maps$\footnotemark$ for different patches: $\Theta=(\theta_i,\phi_i)$ is the direction of light propagation}
\label{fig:brdfmapexplanation}
\end{figure}
\footnotetext{image source of figure:
\begin{itemize}
  \item \ref{fig:brdfmapschema}: Taken from D.S.Dhillon et al's Paper $\cite{daljitpaper}$
  \item \ref{fig:lightreflectiongeometry}: Taken from \texttt{http://math.nist.gov/\textasciitilde FHunt/appearance/brdf.html}
  \end{itemize}
}

\begin{figure}[H]
  \centering
  \subfigure[Blaze grating with scale of 2.500 $\mu m$]{
    \includegraphics[scale=0.5]{evaluation/blaze_res.png}
    \label{fig:blazegratingpatch}
  }
~
  \subfigure[Elaphe patch with scale of 3.270 $\mu m$]{
    \includegraphics[scale=0.5]{evaluation/elaphe_res.png}
    \label{fig:elpahegratingpatch}
  }
~
  \subfigure[Xenopeltis patch with scale of 3.210 $\mu m$]{
    \includegraphics[scale=0.5]{evaluation/xeno_res.png}
    \label{fig:xenogratingpatch}
  }

\caption[Our Diffraction Gratings]{Cutouts of our nano-scaled surface gratings used for rendering with actual our shader with a scale indicator (red line) for each patch. Note that for actual rendering, we use larger patches.}
\label{fig:gratingpatches}
\end{figure}

Figure $\ref{fig:brdfmapsdiffpatches}$ shows the BRDF maps of the full lambda space sampling approach (\textbf{FLSS}) as described in section $\ref{sec:fragmentshader}$ applied on different nanoscale surface gratings shown in figure $\ref{fig:gratingpatches}$. In subfigure $\ref{fig:brdfmapBlaze}$ we see the BRDF map for the Blazed grating, showing high relative brightness for its first order diffraction, i.e. for the Blazed gratings most of the diffracted spectral energy lies in its first order. Notice that the surface of blazed grating is forming a step structure for which the angle between the step normal and the grating normal is denoted by $\emph{blaze angle}$. Every blazed grating is manufactured in the Littrow$\footnote{For further information please see \texttt{http://en.wikipedia.org/wiki/Blazed\textunderscore grating}.}$ configuration. This means that the blaze angle is chosen such that for a chosen mode m and a chosen wavelength $\lambda$ with maximum intensity and the incidence angle are identical. Thus a blazed grating has its maximal efficiency for the chosen wavelength of the light used. Higher diffraction modes are still perceivable (second and higher diffraction orders) but with a much lower relative brightness. The asymmetry in the brightness of the pattern is due to the asymmetric geometry of the grating $\ref{fig:blazegratingpatch}$. \\

The finger-like structures contained in the Elaphe surface grating $\ref{fig:elpahegratingpatch}$ are considerably regularly aligned and hence diffraction occurs along the horizontal axis for the BRDF map as shown in figure $\ref{fig:brdfmapElaphe}$. The reason for not seeing any strong diffraction color contribution along other directions in the BRDF map is due to the fact that these ‘nano-fingers’ overlap across layers and thus do not exhibit any well-formed periodicity along finger direction. \\

For the Xenopeltis surface grating shown in figure $\ref{fig:xenogratingpatch}$, we observe diffraction along many different, almost vertical directions in the BRDF map $\ref{fig:brdfmapXeno}$ since the layers of the finger-like structures do not overlap and are shifted significantly along their length but still exhibit some local consistency. A similar argument holds true for diffraction across locally periodic finger patches with slightly different orientations.

\begin{figure}[H]
  \centering
  \subfigure[Blazed grating]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_gratings/blazed.png}
    \label{fig:brdfmapBlaze}
  }
~
  \subfigure[Elaphe grating]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_gratings/elaphe.png}
    \label{fig:brdfmapElaphe}
  }
~
  \subfigure[Xenopeltis grating]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_gratings/xeno.png}
    \label{fig:brdfmapXeno}
  }

\caption[BRDF Map: FLSS Approach applied on various Gratings]{BRDF maps for different patches}
\label{fig:brdfmapsdiffpatches}
\end{figure}

%%TODO eplain what gound truth is refering to, what it is supposed to denote.

Figure $\ref{fig:brdfmapsdiffrenderingapproaches}$ shows BRDF maps of all our BRDF models applied on the Blaze grating. Figure $\ref{fig:brdfmapblazeallLambda}$ shows the FLSS shading approach result for our blazed grating and it is used in order to compare with our other rendering approaches. \\

Figure $\ref{fig:brdfmapblazeonlyreq}$ shows the BRDF map for the NMM approach, introduced in section $\ref{sec:nmmapproach}$, which is close to the FLSS approach as verified in section $\ref{sec:approachesverifications}$ (see figure $\ref{fig:blazneval}$). Nevertheless there is a small, noticeable difference: For the NMM approach we see a white, circular spot around the map center. Nevertheless, apart from this white spot, the NMM approach resembles the FLSS approach. The reason for this differences is due to the fact that the NMM approach treats the center of a BRDF map as a special case, as described in section $\ref{sec:nmmapproach}$. Technically, every location around a small $\epsilon$-circumference from the map center gets white color assigned. \\

Figure $\ref{fig:brdfmapblazepq}$ shows the BRDF map for the PQ approach which relies on sinc-interpolation. The PQ BRDF map and the FLSS results are visual alike. In contrast to the evaluation plots in figure $\ref{fig:evaluationdiffshaderpq}$, the BRDF map for the PQ approach matches well with the one for the FLSS approach. Compared to FLSS, one difference we notice is that the first order of diffraction is a little spread for the PQ approach. \\

Last, let us consider figure $\ref{fig:brdfmapblazegem}$ which shows the BRDF map produced by using Nvidia Gem's implementation $\cite{cpugems}$ of Stam's BRDF model corresponding to periodic like structures with regularly repeated bumps (along y-axis of the BRDF map). This corresponds to a one dimensional diffraction grating, along the x-axis. This model only uses the spacing $d$ of a given grating. It also always produces highly symmetric results and fails to confirm with the Littrow configuration.

% brdf maps patches
\begin{figure}[H]
  \centering
  \subfigure[FLSS]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/approaches_blazed/flss.png}
    \label{fig:brdfmapblazeallLambda}
  }
~
  \subfigure[NMMS]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/approaches_blazed/nmm.png}
    \label{fig:brdfmapblazeonlyreq}
  }

  \subfigure[PQ]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/approaches_blazed/pqsinc.png}
    \label{fig:brdfmapblazepq}
  }
~
  \subfigure[Nvidia Gem]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/approaches_blazed/gem.png}
    \label{fig:brdfmapblazegem}
  }

\caption[BRDF Map: Our Approaches applied on a Blazed Grating]{BRDF maps for Blazed grating comparing our different rendering approaches}
\label{fig:brdfmapsdiffrenderingapproaches}
\end{figure}

Figure $\ref{fig:brdfmapsdifflambdastepsblaze}$ and figure $\ref{fig:brdfmapsdifflambdastepselaphe65}$ show the BRDF maps for different wavelength step sizes used in the fragment shader for the FLSS approach applied to the blazed grating and the Elaphe snake shed, respectively. Within our fragment shaders the outermost loop iterates over the range $[380nm, 780nm]$ for a given step size $\lambda_{step}$ to integrate over the wavelength spectrum as illustrated in algorithm $\ref{alg:fragmentshaderall}$. Having bigger step sizes implies having fewer $\lambda$-samples which will reduce the overall runtime of a shader but it will also introduce artifacts and therefore lower the overall shading quality. For an Elaphe surface grating, artifacts are perceivable when $\lambda_{step} \geq 10nm$. Results produced by using $5nm$ step sizes do not differ from those produced by using $\lambda_{step}= 1nm$. This allows us to set $\lambda_{step}$ at $5nm$. For a Blazed grating we may chose even bigger step sized without losing any rendering quality(see figure $\ref{fig:brdfmapsdifflambdastepsblaze}$).

\begin{figure}[H]
  \centering
  \subfigure[$\lambda_{step=1 nm}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_lambdasteps/1.png}
    \label{fig:brdfmapsDiffLambdaStepsL1Blaze}
  }
~
  \subfigure[$\lambda_{step=5 nm}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_lambdasteps/5.png}
    \label{fig:brdfmapsDiffLambdaStepsL5Blaze}
  }
~
  \subfigure[$\lambda_{step=10 nm}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_lambdasteps/10.png}
    \label{fig:brdfmapsDiffLambdaStepsL10Blaze}
  }

  \subfigure[$\lambda_{step=25 nm}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_lambdasteps/25.png}
    \label{fig:brdfmapsDiffLambdaStepsL25Blaze}
  }
~
  \subfigure[$\lambda_{step=50 nm}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_lambdasteps/50.png}
    \label{fig:brdfmapsDiffLambdaStepsL50Blaze}
  }
~
  \subfigure[$\lambda_{step=100 nm}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_lambdasteps/100.png}
    \label{fig:brdfmapsDiffLambdaStepsL100Blaze}
  }

\caption[BRDF Map: Varying step sizes FLSS Blazed Grating]{Blazed grating at $2.5 \mu m$: Different $\lambda$ step sizes}
\label{fig:brdfmapsdifflambdastepsblaze}
\end{figure}

% elpahe steps
\begin{figure}[H]
  \centering
  \subfigure[$\lambda_{step=1 nm}$]{
    \includegraphics[scale=0.43]{nresults/brdfmaps/flss_elaphe_lambdasteps/1.png}
    \label{fig:brdfmapsDiffLambdaStepsL1Elaphe65}
  }
~
  \subfigure[$\lambda_{step=5 nm}$]{
    \includegraphics[scale=0.43]{nresults/brdfmaps/flss_elaphe_lambdasteps/5.png}
    \label{fig:brdfmapsDiffLambdaStepsL5Elaphe65}
  }
~
  \subfigure[$\lambda_{step=10 nm}$]{
    \includegraphics[scale=0.43]{nresults/brdfmaps/flss_elaphe_lambdasteps/10.png}
    \label{fig:brdfmapsDiffLambdaStepsL10Elaphe65}
  }

  \subfigure[$\lambda_{step=25 nm}$]{
    \includegraphics[scale=0.43]{nresults/brdfmaps/flss_elaphe_lambdasteps/25.png}
    \label{fig:brdfmapsDiffLambdaStepsL25Elaphe65}
  }
~
  \subfigure[$\lambda_{step=50 nm}$]{
    \includegraphics[scale=0.43]{nresults/brdfmaps/flss_elaphe_lambdasteps/50.png}
    \label{fig:brdfmapsDiffLambdaStepsL50Elaphe65}
  }
~

\caption[BRDF Map: Varying step sizes FLSS Elaphe Grating]{Elaphe grating at $65 \mu m$: Different $\lambda$ step sizes}
\label{fig:brdfmapsdifflambdastepselaphe65}
\end{figure}

The figures $\ref{fig:pqblaze}$ and $\ref{fig:pqelaphe}$ show a comparison of the BRDF maps produced by the FLSS approach (on the left) and the PQ shading approach (on the right) applied on all our patches. For Blazed grating, as already mentioned, we notice that both approaches, FLSS and PQ, look similar but have notable differences. In the PQ map, the first order diffraction color contribution is spread. In general, a Blazed Grating is manufactured in a way that a large fraction of the incident light is diffracted preferentially into the first order. Therefore, most of the energy in its BRDF map lies in the first order of diffraction at its blaze angle. This implies that largest portion of the color contribution, visible on the corresponding BRDF map, lies at that angle. In figure $\ref{fig:pqblaze}$, in contrast to the results produced by the FLSS approach, we see color fringes at the first order modes in the BRDF map produced by our PQ approach. This implies that the PQ approach does not produce reliable results which also affirms our evaluation plots shown in figure $\ref{fig:evaluationdiffshaderpq}$. The BRDF resulting BRDF map for the Elaphe and Xenopeltis gratings using the PQ approach are similar to those resulting by the FLSS approach. Nevertheless they also exhibit some artifacts, similar to these discussed for the Blazed grating.

\begin{figure}[H]
  \centering
  \subfigure[FLSS Approach: Blazed grating]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/approaches_blazed/flss.png}
    \label{fig:fullLambdaBlaze}
  }
~
  \subfigure[PQ Sinc Interpolation Approach: Blazed grating]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/approaches_blazed/pqsinc.png}
    \label{fig:pqBlaze}
  }
\caption[BRDF Map: PQ vs FLSS Approach on Blazed Grating]{A comparison between the PQ- and the FLSS approach applied on an Blazed grating.}
\label{fig:pqblaze}
\end{figure}

\begin{figure}[H]
  \centering
  \subfigure[FLSS Approach: Elaphe grating]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/flss_pq_elaphe/flss.png}
    \label{fig:fullLambdaElaphe}
  }
~
  \subfigure[PQ Approach: Elaphe grating]{
    \includegraphics[scale=0.25]{nresults/brdfmaps/flss_pq_elaphe/pq.png}
    \label{fig:pqElaphe}
  }

\caption[BRDF Map: PQ vs FLSS Approach on Elaphe Grating]{A comparison between the PQ- and the FLSS approach applied on an Elaphe grating.}
\label{fig:pqelaphe}
\end{figure}

In figure $\ref{fig:pqxeno}$ we are also comparing the PQ approach with the FLSS applied on a Xenopeltis grating as shown in figure $\ref{fig:xenogratingpatch}$. For the subfigures $\ref{fig:pqxeno}$(a), $\ref{fig:pqxeno}$(b), $\ref{fig:pqxeno}$(c) and $\ref{fig:pqxeno}$(d) we were using the full size of the Xenopeltis grating, which is equal to 65$\mu m$ $\times$ 65$\mu m$ as our input patch. This implies that we were using 1 period for the PQ approach for producing the renderings shown in subfigures $\ref{fig:pqXenoti0}$ and $\ref{fig:pqXenoti20}$. On the other hand, figure $\ref{fig:fullLambdaXenoti10}$ shows a PQ rendering using a smaller patch of the Xenopeltis grating as shown in figure $\ref{fig:pqXenoti10}$. The this patch is only 3.25$\mu m$ $\times$ 3.25$\mu m$ which corresponds to 20 periods.

\begin{figure}[H]
  \centering
  \subfigure[FLSS Approach: Xenopeltis grating $\theta_i = 0 \degree$]{
    \includegraphics[scale=0.19]{nresults/brdfmaps/flss_xeno_thetai/0.png}
    \label{fig:fullLambdaXenoti0}
  }
~
  \subfigure[PQ Approach: Xenopeltis grating $\theta_i = 0 \degree$]{
    \includegraphics[scale=0.19]{nresults/brdfmaps/pq_xeno_thetai/0.png}
    \label{fig:pqXenoti0}
  }

  \subfigure[FLSS Approach: Xenopeltis grating $\theta_i = 20 \degree $]{
    \includegraphics[scale=0.19]{nresults/brdfmaps/flss_xeno_thetai/20.png}
    \label{fig:fullLambdaXenoti20}
  }
~
  \subfigure[PQ Approach: Xenopeltis grating $\theta_i = 20 \degree $]{
    \includegraphics[scale=0.19]{nresults/brdfmaps/pq_xeno_thetai/20.png}
    \label{fig:pqXenoti20}
  }

    \subfigure[PQ Approach: Applied on the patch in figure $\ref{fig:pqXenoti10}$]{
      \includegraphics[scale=0.19]{nresults/brdfmaps/special/xeno_pq_20.png}
      \label{fig:fullLambdaXenoti10}
    }
  ~
    \subfigure[A 3.25$\mu m$ patch from our Xenopeltis grating]{
      \includegraphics[scale=5.0]{nresults/brdfmaps/special/xeno6x2.png}
      \label{fig:pqXenoti10}
    }

\caption[BRDF Map: PQ vs FLSS Approach on Xenopeltis Grating]{A comparison between the PQ- and the FLSS approach applied on an Xenopeltis grating.}
\label{fig:pqxeno}
\end{figure}

Figure $\ref{fig:brdfmapsdiffsigmasizeblaze}$ shows BRDF maps for the FLSS approach when applied to the Blazed grating, while varying the value for the spatial variance $\sigma_s$ for the coherence window. This is akin to changing the coherence length for the incident light. We see that the lower the coherence length gets, the fewer of the interacting grating periods are involved. Having fewer periods cause the production of overlapping blurry diffraction bands (i.e. blobs, see e.g. figure $\ref{fig:brdfmapsDiffSigmaStepsL5Blaze}$) for different wavelengths $\lambda$. Thus, when reducing the number of periods this will produces blurry artifacts which ends in having poorly resolved colors.

\begin{figure}[H]
  \centering
  \subfigure[$\sigma_{s=3.25 \mu m}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_sigmas/3_25.png}
    \label{fig:brdfmapsDiffSigmaStepsL1Blaze}
  }
~
  \subfigure[$\sigma_{s=6.5 \mu m}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_sigmas/6_5.png}
    \label{fig:brdfmapsDiffSigmaStepsL5Blaze}
  }
~
  \subfigure[$\sigma_{s=15 \mu m}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_sigmas/15.png}
    \label{fig:brdfmapsDiffSigmaStepsL10Blaze}
  }

  \subfigure[$\sigma_{s=30 \mu m}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_sigmas/30.png}
    \label{fig:brdfmapsDiffSigmaStepsL25Blaze}
  }
~
  \subfigure[$\sigma_{s=45 \mu m}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_sigmas/45.png}
    \label{fig:brdfmapsDiffSigmaStepsL50Blaze}
  }
~
  \subfigure[$\sigma_{s=65 \mu m}$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_blazed_sigmas/65.png}
    \label{fig:brdfmapsDiffSigmaStepsL100Blaze}
  }

\caption[BRDF Map: Different spatial variance $\sigma_s$ values]{Blazed grating with periodicity of $2.5 \mu m$: Different $\sigma_s$}
\label{fig:brdfmapsdiffsigmasizeblaze}
\end{figure}

Figures $\ref{fig:brdfmapstayloriterationsblaze}$ and $\ref{fig:brdfmapstayloriterationselaphe65}$ show the BRDF maps for the FLSS approach using a different number of terms $N$ used in the Taylor series approximation. For both input patches we clearly, visually observe the convergence of the taylor series for higher values of N. We visually observe convergence of the Taylor series for all our patches for a very large value of $N$$\footnote{Using N equal to 40 lead to visual convergence for all our used gratings.}$. \\

As discussed in section $\ref{sec:taylorapproximation}$ there exists a certain value of $N$ for which our approach converges. For all our shading approaches, applied on our gratings, we visually observed a convergence of their BRDF maps when using $N \geq 39$ DFT terms. Furthermore, for a Blazed grating it suffices to use only $N \geq 7$ - and for an Elaphe grating only $N \geq 9$ DFT terms. Notice, that these numbers of required DFT terms were empirically determined with a trial and error strategy. \\

However, by making use of Taylor error term estimates, as introduced in the appendix section $\ref{chap:taylorseriesapproxappendix}$, we can derive an upper bound for $N$. This computation is dependent on many aspects, such as on the grating spacing, the sampling rate $dH$ and the used wavelength spectrum. Thus, it is usually simpler to empirically determine an actual value for $N$ to be used. \\

In algorithm $\ref{alg:matlabprecomp}$ we compute different powers of DFT terms of our height fields $h$ by evaluating the expression $DFT(h)^n \cdot i^n$, where $i$ denotes the imaginary number $i$ and $n$ is a natural number. Note that when applying the DFT operator on a real or complex number (e.g. such as one particular value of our height field) we get a complex number. Using our Taylor series approximation has basically four possible convergence images, each having its own convergence radius. The reason for this is due to the fact we are multiplying the DFT terms by $i^n$. \\

In order to understand the reason behind this, let us consider the complex unit $i$. Raising $i$ to the power of a natural number $n$ leaves us four possible results as shown in equation $\ref{eq:thefourunits}$:

\begin{align}
  i^n = \left\{ \begin{array}{rl}
  1 &\mbox{ if $n \equiv_4 0$} \\
  i &\mbox{ if $n \equiv_4 1$} \\
  -1 &\mbox{ if $n \equiv_4 2$} \\
  -i &\mbox{ otherwise}
  \end{array} \right.
\label{eq:thefourunits}
\end{align}

Thus multiplying a complex number $c$, such as particular value of a DFT term, by $i$ raised to a certain power, will permute the the value of $c$.
Since we compute $\sum_{k=0}^N DFT(h)^k \cdot i^k$ we can split this summation into four partial summations, according to their applied power factor $i$. Each of these summands converges with a different convergence radius.

\begin{figure}[H]
  \centering
  \subfigure[$N=1$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_xeno_taylor/0.png}
    \label{fig:brdfmapsTaylorN0Blaze}
  }
~
  \subfigure[$N=2$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_xeno_taylor/1.png}
    \label{fig:brdfmapsTaylorN1Blaze}
  }
~
  \subfigure[$N=4$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_xeno_taylor/3.png}
    \label{fig:brdfmapsTaylorN2Blaze}
  }

  \subfigure[$N=24$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_xeno_taylor/23.png}
    \label{fig:brdfmapsTaylorN3Blaze}
  }
~
  \subfigure[$N=26$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_xeno_taylor/25.png}
    \label{fig:brdfmapsTaylorN4Blaze}
  }
~
  \subfigure[$N=40$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_xeno_taylor/39.png}
    \label{fig:brdfmapsTaylorN5Blaze}
  }

\caption[BRDF Map: Xenopeltis Grating Convergence]{Blazed grating at $65 \mu m$: $N$ Taylor Iterations}
\label{fig:brdfmapstayloriterationsblaze}
\end{figure}

%taylor var elaphe
\begin{figure}[H]
  \centering
  \subfigure[$N=1$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_elaphe_taylor/0.png}
    \label{fig:brdfmapsTaylorN0Elaphe65}
  }
~
  \subfigure[$N=2$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_elaphe_taylor/1.png}
    \label{fig:brdfmapsTaylorN1Elaphe65}
  }
~
  \subfigure[$N=5$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_elaphe_taylor/4.png}
    \label{fig:brdfmapsTaylorN2Elaphe65}
  }

  \subfigure[$N=10$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_elaphe_taylor/9.png}
    \label{fig:brdfmapsTaylorN3Elaphe65}
  }
~
  \subfigure[$N=11$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_elaphe_taylor/10.png}
    \label{fig:brdfmapsTaylorN4Elaphe65}
  }
~
  \subfigure[$N=40$]{
    \includegraphics[scale=0.16]{nresults/brdfmaps/flss_elaphe_taylor/39.png}
    \label{fig:brdfmapsTaylorN5Elaphe65}
  }

\caption[BRDF Map: Elaphe Grating Convergence]{Elaphe grating at $65 \mu m$: $N$ Taylor Iterations}
\label{fig:brdfmapstayloriterationselaphe65}
\end{figure}

Figure $\ref{fig:brdfmapsxenodiffthetaiangles}$ shows the BRDF maps of the FLSS approach applied on the Xenopeltis snake shed, using different $\theta_i$ incident angles. When slightly moving the incident angle $\theta_i$, we can observe how the BRDF map changes. For higher values of $\theta_i$ we start seeing additional diffraction color contributions on the right side of the BRDF map.

\begin{figure}[H]
  \centering
  \subfigure[Xenopeltis grating $\theta_i=0 \degree$]{
    \includegraphics[scale=0.22]{nresults/brdfmaps/flss_xeno_thetai/0o.png}
    \label{fig:brdfmapXenoti0}
  }
~
  \subfigure[Xenopeltis grating $\theta_i=10 \degree$]{
    \includegraphics[scale=0.15]{nresults/brdfmaps/flss_xeno_thetai/10.png}
    \label{fig:brdfmapXenoti10}
  }
~
  \subfigure[Xenopeltis grating $\theta_i=20 \degree$]{
    \includegraphics[scale=0.15]{nresults/brdfmaps/flss_xeno_thetai/20.png}
    \label{fig:brdfmapXenoti20}
  }

\caption[BRDF Map: Varying Viewing Angles]{BRDF maps for Xenopeltis grating: different $\theta_i$ angles}
\label{fig:brdfmapsxenodiffthetaiangles}
\end{figure}

\section{Rendering Surface Geometries}
\label{sec:snakegeomrenderings}
In this section we are going to present our actual renderings. These renderings are simulating the effects of diffraction produced when a directional light source encounters different nano-scaled surfaces on a given curved snake skin mesh. We will see that diffraction colors change dramatically with changes in the light direction, surface normals and the viewing direction. This is typical for diffraction colors observed in nature. Therefore, the aim of this section is tho provide a visual comparison between diffraction patterns occurring in nature as shown in figure $\ref{fig:snakespecies}$ and the renderings produced by our shader. Last we compare the results produced by our shader against those produced by Stam's approach. For this purpose we compare applied these methods on a synthetic grating (on a CD) and on a natural grating (on a Xenopeltis grating). \\

For rendering we are going to rely on our FLSS approach. Unfortunately, this approach is rather slow and can barely be considered as being interactively performing. The NMM approach on the other hand, has interactive runtime. In general, both approaches are valid according to to their evaluation plots shown in figure $\ref{fig:evaluationdiffshaderalllambda}$ and in figure $\ref{fig:evaluationdiffshadernminmax}$. Therefore, their rendered results may be considered as being accurate. However, the reason for choosing the FLSS approach instead of the NMM approach is that the renderings resulting from the NMM approach look purplish compared to those produced by the FLSS approach. This color-tone shift towards the purple color region for NMM renderings does not correspond to the reality and is a result of its non-uniform wavelength spectrum sampling. Further information about NMM's tone shift issue and how it can be addressed is discussed in the appendix section $\ref{chap:diffflssnmm}$. \\

The Laboratory of Artificial and Natural Evolution in Geneva provided us with a triangular mesh of a snake. This mesh was produced by a 3D scan for an Elaphe snake species and it consists of \emph{11696} vertices and \emph{22950} faces. Note that, for all our renderings, we used this snake mesh. \\

For the snake species under consideration, their macroscopic geometry is highly similar. Only the geometry of their nano-structures varies and is responsible for a snake's iridescence. Thus, we can use the same snake surface model to render diffraction for different species. Table $\ref{tab:hardwarespecifications}$ lists the system specifications of the machine I used in order to produce the rendered images.

\begin{table}[H]
  \centering
  \begin{tabular}{|r|l|}
    \hline
    Processor & Intel i7 CPU 970 @ 3.20 GHz (12 CPUs) \\
    Memory & 12288 MB RAM \\
    Graphics Card & GeForce GTX 770 \\
    Graphics Clock & 1150 MHz \\
    Graphics Memory & 4096 MB \\
    Graphics Memory Bandwidth & 230.4 $GB/s$ \\
    \hline
  \end{tabular}
\caption[Hardware Specifications]{Hardware specifications of the machine used to render snake surface. Statistics are provided using the tool $\emph{NVIDIA Geforce Experience}$.}
\label{tab:hardwarespecifications}
\end{table}

Figure $\ref{fig:renderingdifferentsnankegratings}$ shows renderings produced by the FLSS approach applied on our snake mesh for different, given input patches. Due do the fact that a Blazed grating has its maximum intensity for a certain direction and the geometry of the snake mesh is curved i.e. it is non-flat, we can expect rather less diffraction color contribution as shown in figure $\ref{fig:renderingelaphegrating}$. \\

%%TODO write something about similar configuration for blazed grating

In contrast, for both the renderings, we see colorful patterns on the skin of our snake species, Elaphe and Xenopeltis, due to the effect of diffraction. We see much less colorful patterns for Elaphe as shown in figure $\ref{fig:renderingelaphegrating}$ than for Xenopeltis as shown in figure $\ref{fig:renderingxenograting}$. This is consistent with the observations in the real world as shown in figure $\ref{fig:snakespecies}$. As observable in figure $\ref{fig:elpahegratingpatch}$, the substructures (the finger like structures) in the height field of a Elaphe snake skin overlap (i.e are not very regularly aligned) along the y-axis. This is why the Elaphe species is less iridescent than the other specie. The Xenopeltis snake has a brownish body with no pigmentation, which makes the iridescence more spectacular than on Elaphe as seen in figure $\ref{fig:snakespecies}$.

\begin{figure}[H]
  \centering
  \subfigure[Blazed grating]{
    \includegraphics[scale=0.23]{nresults/snakerenderings/flss_gratings/blazed1552.png}
    \label{fig:renderingblazegrating}
  }
~
  \subfigure[Elaphe grating]{
    \includegraphics[scale=0.23]{nresults/snakerenderings/flss_gratings/elaphe.png}
    \label{fig:renderingelaphegrating}
  }
~
  \subfigure[Xeno grating]{
    \includegraphics[scale=0.23]{nresults/snakerenderings/flss_gratings/xeno.png}
    \label{fig:renderingxenograting}
  }

\caption[Snake Renderings: Our Approaches applied on our Gratings]{Diffraction of different snake skin gratings rendered on a snake geometry}
\label{fig:renderingdifferentsnankegratings}
\end{figure}

Figure $\ref{fig:renderingelaphe65}$ shows a set of subfigures for rendering the effect of diffraction produced by the FLSS approach (used as our reference approach), applied on our snake mesh using the Elaphe nano structure. Figure $\ref{fig:renderingelaphe65dt}$ shows the final diffraction color contribution result with texture-blending. We note that the diffraction color contribution is not significant in this subfigure which resembles quite well to the reality as shown in figure $\ref{fig:elpahespecies}$. In subfigure $\ref{fig:renderingelaphe65ns}$ we see the light cone in order to show the direction of the light source besides the rendered results. Subfigure $\ref{fig:renderingelaphe65ft}$ is a sample visualization of a Fourier transformation for an Elaphe nano-scale surface structure as shown in figure $\ref{fig:renderingelaphe65ns}$.

\begin{figure}[H]
  \centering
  \subfigure[Diffraction Patten]{
    \includegraphics[scale=0.25]{results/snakerenderings/elaphe65/1.png}
    \label{fig:renderingelaphe65dp}
  }
~
  \subfigure[Diffraction + Texture]{
    \includegraphics[scale=0.25]{results/snakerenderings/elaphe65/2.png}
    \label{fig:renderingelaphe65dt}
  }

  \subfigure[Texture with Light Direction]{
    \includegraphics[scale=0.2]{nresults/viewingcone.png}
    \label{fig:renderingelaphe65tl}
  }
~
  \subfigure[Nanostructure]{
    \includegraphics[scale=0.11]{results/snakerenderings/elaphe65/4.png}
    \label{fig:renderingelaphe65ns}
  }
~
  \subfigure[8th DTF Term]{
    \includegraphics[scale=0.31]{nresults/elaphe8dft.png}
    \label{fig:renderingelaphe65ft}
  }

\caption[Snake Renderings: FLSS Approach for Elaphe Grating]{Diffraction for Elaphe snake skin produced by our reference approach.}
\label{fig:renderingelaphe65}
\end{figure}

Figure $\ref{fig:renderingxeno65}$ shows a set of subfigures for the effect of diffraction for the Xenopeltis snake surface. In our Xenopeltis rendering as shown in figure $\ref{fig:renderingXeno65FT}$ when using a grating as shown in figure $\ref{fig:renderingXeno65NS}$ we see a lot of iridescent color contribution. Comparing this to a real image $\ref{fig:renderingXeno65DP}$ we notice much resemblance regarding the reflectance strength and colorful pattern.

\begin{figure}[H]
  \centering
  \subfigure[Diffraction Patten]{
    \includegraphics[scale=0.5]{results/snakerenderings/xeno65/1.png}
    \label{fig:renderingXeno65DP}
  }

  \subfigure[Nanostructure]{
    \includegraphics[scale=0.15]{results/snakerenderings/xeno65/4.png}
    \label{fig:renderingXeno65NS}
  }
~
  \subfigure[3th DTF Term]{
    \includegraphics[scale=0.563]{nresults/xeno3dft.png}
    \label{fig:renderingXeno65FT}
  }
\caption[Snake Renderings: FLSS Approach for Xenopeltis Grating]{Diffraction for Xenopeltis snake skin produced by our reference approach.}
\label{fig:renderingxeno65}
\end{figure}

Figure $\ref{fig:renderingdifferentzoomlevelselaphe}$ shows the diffraction pattern for an Elaphe snake shed at different zoom levels for a fixed incident light and a fixed viewing direction. We changed the zoom-levels by adjusting the field of view angle of our camera. For each image in this figure, the one to its right side is a five times zoomed-in version of the region within its red box. The close up perspectives exhibit complex and colorful diffraction patterns.

\begin{figure}[H]
  \centering
  \subfigure[$zoom = 1 \times$]{
    \includegraphics[scale=0.5]{nresults/snakerenderings/flss_elaphe_zoom/10.png}
    \label{fig:renderingZoomElaphe05}
  }
~
  \subfigure[$zoom = 3 \times$]{
    \includegraphics[scale=0.28]{nresults/snakerenderings/flss_elaphe_zoom/5.png}
    \label{fig:renderingZoomElaphe02}
  }
~
  \subfigure[$zoom = 6 \times$]{
    \includegraphics[scale=0.24]{nresults/snakerenderings/flss_elaphe_zoom/1.png}
    \label{fig:renderingZoomElaphe01}
  }

\caption[Snake Renderings: FLSS Approach for a varying FoV]{Showing the diffraction pattern on an Elaphe grating for different camera zoom levels by varying the field of view.}
\label{fig:renderingdifferentzoomlevelselaphe}
\end{figure}

Figure $\ref{fig:renderingelaphelightrotations6}$ shows how the diffraction pattern changes when the incident light direction is moved slightly. This figure gives us an impression of what kind of complex, perspective-dependent pattern the diffraction phenomenon produces. The figure column from figure $\ref{fig:renderingElapheRotX2}$ to figure $\ref{fig:renderingElapheRotX6}$ models a scene in which a virtual sun (i.e. a directional light source) moves from south (S) to north (N). For each of this subfigure we can see in its inset (red box) how the diffraction color changes according to how the position of the sun changes. The figure column from figure $\ref{fig:renderingElapheRotY2}$ to figure $\ref{fig:renderingElapheRotY6}$ models a scenario when a sun moves from south-east (SE) to west (W). Similarly, the diffraction patterns changes according to the position of the sun.

\begin{figure}[H]
  \centering
  \subfigure[$(-3.3130, 0.0, -0.9999)$]{
    \includegraphics[scale=0.345]{nresults/snakerenderings/flss_xeno_lightdir/1/1.png}
    \label{fig:renderingElapheRotX2}
  }
  ~
  \subfigure[$(0.0995, 0.0993, -0.9900)$]{
    \includegraphics[scale=0.245]{nresults/snakerenderings/flss_xeno_lightdir/2/2.png}
    \label{fig:renderingElapheRotY2}
  }

  \subfigure[$(-0.1989, 0.0, -0.9799)$]{
    \includegraphics[scale=0.245]{nresults/snakerenderings/flss_xeno_lightdir/1/2.png}
    \label{fig:renderingElapheRotX4}
  }
  ~
  \subfigure[$(0.0995, 0.2940, -0.9505)$]{
    \includegraphics[scale=0.245]{nresults/snakerenderings/flss_xeno_lightdir/2/3.png}
    \label{fig:renderingElapheRotY4}
  }

  \subfigure[$(-0.3897, 0.0, -0.9208)$]{
    \includegraphics[scale=0.245]{nresults/snakerenderings/flss_xeno_lightdir/1/3.png}
    \label{fig:renderingElapheRotX6}
  }
  ~
  \subfigure[$(0.0995, 0.4770, -0.8731)$]{
    \includegraphics[scale=0.245]{nresults/snakerenderings/flss_xeno_lightdir/2/4.png}
    \label{fig:renderingElapheRotY6}
  }

\caption[Snake Renderings: FLSS Approach for varying Light Directions]{Showing the change of the diffraction pattern on an Elaphe grating for different light directions. Figure $\ref{fig:renderingElapheRotX2}$ shows the geographic direction in order to describe a local orientation. The first subfigure column the sun moves from south (S) to north (N). In the second column, the sun moves from south-east to west (W).}
\label{fig:renderingelaphelightrotations6}
\end{figure}

Figure $\ref{fig:experimentelaphe65}$ shows a photo of a real experiment which is demonstrating color production due to the effect of diffraction produced when a laser beam hits an Elaphe grating. The exact parameters for this experimental setup are unknown. However, an exemplary laboratory setup is shown in figure $\ref{fig:experimentalsetuppew}$. Nevertheless, this photo of a real experiment gives us an impression of how much our model resembles to the reality. When comparing our BRDF map renderings of the Elaphe grating in figure $\ref{fig:brdfmapElaphe}$ (when using the FLSS approach) with the photo from figure $\ref{fig:experimentelaphe65}$, we notice similarities in the diffraction patterns.

\begin{figure}[H]
  \centering
  \includegraphics[scale=0.9]{nresults/labsetup.png}
  \caption[Lab Setup]{Illustration of the laboratory setup for the diffraction experiments}
  \label{fig:experimentalsetuppew}
\end{figure}

\begin{figure}[H]
  \centering
  \includegraphics[scale=0.2]{results/experiment/elaphe/g2.png}
  \caption{Diffraction Elaphe: Pattern for a sample setup}
  \label{fig:experimentelaphe65}
\end{figure}

In the following we provide a brief comparison between Stam's$\footnote{A reference implementation of Stam's Diffraction Shader\cite{diffstam} is provided by Nvidia's GPU Gems at \texttt{http://http.developer.nvidia.com/GPUGems/gpugems\textunderscore 08.html}}$ and our FLSS method. For this purpose we use two different kinds of gratings, a synthetic, regularly aligned grating and a natural, complex structured grating. These gratings are shown in figure $\ref{fig:stameggratings}$.

\begin{figure}[H]
  \centering
  \includegraphics[scale=0.5]{background/gratingsstamreworked.png}
  \caption[Comparing Stam's Approach: Gratings]{Alignment of nano-structures in diffraction gratings. On the left a complex, natural grating of the Elaphe snake species and on the right a synthetic, very regularly aligned grating of a CD.}
  \label{fig:stameggratings}
\end{figure}

Figure $\ref{fig:stameggratingsgoodeg}$ shows an example of a case where Stam's approach performs well. Considering the red-line in the figure we notice that the nano-scaled structures of a compact disc are very regularly aligned along the surface. Tracks of a CD are uniformly spaced and bumps along a track are distributed according to a poisson distribution$\footnote{See \texttt{http://en.wikipedia.org/wiki/Poisson\textunderscore distribution}}$. We notice that the diffraction patterns in both approaches look similar.

\begin{figure}[H]
  \centering
  \includegraphics[scale=0.5]{background/stamgoodeg.png}
  \caption[Comparing Stam's approach: Good Example]{Comparison of our approach against a reference implementation of Stam's method provided by Nvidia Gem. For synthetic diffraction gratings, which have a very regular structure, Stam's approach is doing well. Both approaches produce similar diffraction patterns.}
  \label{fig:stameggratingsgoodeg}
\end{figure}

Finally, tried to reproduce a real Xenopeltis image (as shown in figure $\ref{fig:xenopeltisrealimage}$) for a unknown light and viewing direction. For this purpose I used our FLSS rendering approach and compared its results against Nvidia Gem's implementation. The results are shown in figure $\ref{fig:attemptxeno}$. Even the results of Stam's method look convincing, they have some issues. In some regions in his renderings close to specular regions, there are missing colors. The color distribution in renderings produces by his approach is rather discrete in Stam's. Also notice that Stam's approach always produces a symmetric reflectance map (i.e. BRDF map) exhibiting results in regions where it must not.

\begin{figure}[H]
  \centering
  \subfigure[Real Xenopeltis Photograph]{
    \includegraphics[scale=0.6]{nresults/snakerenderings/gem_flss_xeno/xeno.png}
    \label{fig:xenopeltisrealimage}
  }

  \subfigure[FLSS]{
    \includegraphics[scale=0.25]{nresults/snakerenderings/gem_flss_xeno/flss.png}
    \label{fig:flssattemptxeno}
  }
~
  \subfigure[Stam]{
    \includegraphics[scale=0.25]{nresults/snakerenderings/gem_flss_xeno/gem.png}
    \label{fig:gemattemptxeno}
  }

\caption[Snake Renderings: Stam's vs. FLSS Approach on Xenopeltis]{Comparison of our FLSS approach(see figure $\ref{fig:flssattemptxeno}$) against a reference implementation of Stam's method(see figure $\ref{fig:gemattemptxeno}$) provided by Nvidia Gem. We attempt to reproduce a real Xenopeltis skin coloration(see figure $\ref{fig:xenopeltisrealimage}$). For natural diffraction gratings, which have a rather complex structure, Stam's approach is qualitatively different from the real image.}
\label{fig:attemptxeno}
\end{figure}

In this chapter we have discussed the results produced by our rendering approaches. For the actual snake renderings, we were using the FLSS approach since the NMM approach produces renderings that exhibit an unpleasant purplish color-tone shift. A discussion how to address this issue is discussed in the appendix chapter $\ref{chap:diffflssnmm}$. Relying on the FLSS approach for producing the renderings of the section $\ref{sec:snakegeomrenderings}$ is appropriate since the aim of this section is to produce convincing renderings. However, when speed matters using the NMM approach is the way to go. Even we did not provide any runtime measurement analysis, we usually achieved interactive runtime using the NMM approach. In average, we get \emph{about 5-10 FPS} using the NMM approach relying on a hardware setup as listed in table $\ref{tab:hardwarespecifications}$.


\documentclass{article}

\begin{document}

\section*{1.}

\section*{a.}

\[ F = 9\times10^9 \frac{(8\times10^{-6}) \times (40\times10^{-6})}{0.001} \]

\section*{2.}

\section*{3.}

\section*{a.}

\[ I = \frac{Q}{t}\]

\[ Q = I \times t \]

\[ Q = (2 \times 10^{-8}) \times (2 \times 10^{-3}) \]

\section*{4.}

\section*{5.}

\end{document}


\section{Introduction}
\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{scene_with_block.png}
  \caption{Single-camera TagSLAM scene with dynamic ``rig'' and ``block''
    bodies, and a static ``lab'' body.}
  \label{fig:scene_with_block}
\end{figure}

Accurate and robust Simultaneous Localization And Mapping (SLAM)
algorithms are arguably one of the corner pieces for building
autonomous systems. For this reason, SLAM has been studied extensively
since the 1980s \cite{cadena2016}.
In a nutshell, SLAM methods take sensor data such as camera images as
input and extract easily recognizable features (``landmarks''). The
landmarks are entered into a map such that when the same landmark is
detected again later, the observer's pose can be determined by
triangulation. In graph based SLAM, a nonlinear optimizer such as
GTSAM \cite{kaess2011} is used to simultaneously optimize the pose of
the observer and the location of the landmarks, given the sensor
measurements.

Much research effort \cite{cadena2016} has gone into solving the
hard aspects of SLAM. One of them is recognizing previously seen
landmarks (''loop closure''). This is often difficult if landmarks are
observed from a different viewpoint, or under different lighting
conditions. The other is maintaining a map of landmarks. To limit memory
consumption and achieve fast retrieval, landmarks must be at some
point discarded, efficient retrieval databases must be updated and
queried, adding considerable complexity \cite{murartal2017}. If loop
closure fails, the estimated camera pose starts to drift, which is a
well-known problem with local-map-only algorithms such as
visual-inertial odometry (VIO).

In many situations, and in particular for laboratory experiments these
problems can be sidestepped with a mild intervention to the
environment. By placing visual fiducial markers such as the
popular AprilTags \cite{wang2016} in a scene, one can create a small
set of artifical landmarks, typically numbering less than 100. The
TagSLAM framework presented in this paper is designed for such a situation.
With only a few landmarks to consider, memory and retrieval speed are
no longer critical, and loop closure is
guaranteed once a tag is successfully decoded by the AprilTag
library. If one or more tags are visible at all times, then TagSLAM
can perform both tracking and loop closure.

But more importantly, tags permit the introduction of geometric
constraints that have been observed by other means. For instance, markers
can be placed inside a building at coordinates that are known because
a highly accurate building plan is available
\cite{pfrommer2017}. Another alternative is measuring the locations
of select tags with a laser distance measuring tool. When a measured
marker is later recognized by the moving camera, it will pull the
trajectory to the reference location. As we demonstrate in the
experimental section, this can augment existing SLAM or
VIO methods. By utilizing well-known tags in a few places, and
ubiquitous but anonymous feature points elsewhere, one can leverage
the advantages of both approaches.

Even though tag based SLAM is much simpler than general SLAM, some
remaining difficulties are addressed in this paper. Since TagSLAM
utilizes GTSAM  \cite{kaess2011}, a graph based non-convex optimizer, care must be taken
to properly initialize all starting values. A bad initial value can
cause the optimizer to fail or to converge to a local minimum. In the
present work we give a detailed description of how we achieve robust
initialization to make TagSLAM a tool that can be applied to a wide range of
real-world situations without parameter tuning.

It is worthwhile noting that SLAM is a rather general algorithm, and
therefore can solve several related, simpler problems as well. If for
example the camera pose is known, one can estimate the pose of the
landmarks. Likewise, if a map of the landmarks is known, the camera
pose can be inferred. TagSLAM inherits all of this flexibility. In
contrast to traditional SLAM packages however, where moving landmarks
are filtered out, TagSLAM can explicitly model and track multiple
objects that have tags attached to them. This means TagSLAM can also
be used for pose estimation of objects other than cameras, as shown in
Fig. \ref{fig:scene_with_block}. In fact, TagSLAM has been deployed for
the University of Pennsylvania's Smart Aviary project in a scenario
where {\em all} cameras are static, and only some tags are
moving. This is not directly possible with any SLAM package that we
are aware of.

TagSLAM's robustness and
availability as an open-source ROS \cite{quigley2009} package make it
particularly accessible for researchers new to robotics. It is now
being used at the University of Pennsylvania's GRASP lab for extrinsic
camera calibration with non-overlapping views, loop closure on
visual-inertial odometry benchmarks, tag mapping, object pose
estimation, and more. TagSLAM is also employed in a forthcoming
project by the University of Michigan's DASC laboratory.


\subsection{Ergodic processes}

Sample moments must converge to generating momements. Not guaranteed.

Eg process with path dependence. 50% of getting £100 a day v 50% of getting £0.

Generating average is £50, but sample will only convergen to £100 or £0