%begin-include

\section{The axiomatic method. Syntax and semantics}

\begin{para}[The breakdown of logicism]
In the early days of set theory, logicism was on the rise.
It was by then believed --- and rightly so --- that set theory could be used as a foundation for all mathematics, so, if set theory could be reduced to logic, mathematics would be reducible to logic too.
If you allow me to oversimplify a bit, the idea behind this endeavour was the following. How do you define a set? Well, they said, just take any predicate $\phi(x)$ and define the set $R = \{x\mid \phi(x)\}$ of all elements $x$ verifying $\phi$.
Thus, for instance, we could take $\phi(x)$ to be the predicate ``$x$ is a natural number'', and define the set of natural numbers as the set of elements verifying $\phi$.

It was all good and great until, one day, Bertrand Russell found a paradox that threatened to destroy set theory and the very foundations of mathematics.
Let us consider the set $R = \{x \mid x \not \in x\}$. By the definition of $R$, we would have $R\in R$ if and only if $R\not\in R$. Boom!
Many more paradoxes emerged that sentenced the logicist approach to set theory to death.
What these paradoxes showed was that logic was not enough to define mathematics: mathematics can of course be built on top of logic, but it needs its own formal structure.

Nevertheless, the idea of identifying predicates with sets stayed alive, but not in such a wild state as it originally did.
In general, there is no harm in identifying predicates with sets\ldots provided you do it with care.
What does it mean to do it carefully? Be patient, you will get a proper answer later.

If you are interested in the events that followed and preceded Russell's paradox and would like to get some more context, I invite you to read chapter II.7 of \cite{Companion}.
\label{logifail}
\end{para}

\begin{para}
Mathematics is all about proofs. When you are doing mathematics, you are working with certain abstract objects and your job is to deduce (that is, prove) some properties about them.
From an informal perspective, a proof could be defined as an argument that would convince any intelligent being that something is true; from a formal perspective, as we will soon see, things are a little bit more subtle.

If mathematics is all about proofs and proofs are all about deduction, we need an starting point.
Thus, when we are working with a mathematical object, we need to understand its defining properties: the properties that are so essential and intrinsic to that object, that not only are unquestionable, but can also be used to define the object itself.
Some of these defining properties properties are chosen to be \emph{axioms}, and they are the starting point for any proof concerning the mathematical entities they define.
That is what the axiomatic method is all about.

For example, let us consider the natural numbers. The statement
``if $x = y$, then $x+1 = y+1$''
could well be an axiom of the natural numbers.
It is unquestionably true according to our concept of natural number, and it can be used to define what a natural number is.

Now that we have all the intuition behind how mathematical reasoning should work, we will seek to formalise everything that we have done so far.
Firstly, we aim to provide an axiomatic framework for logic itself.
\end{para}

\begin{para}
Before diving into the world of mathematical logic, I would like to give you some vocabulary. In mathematics, we use the words \emph{proposition}, \emph{lemma}, \emph{theorem} and \emph{corollary} when we are referring to proven statements (also known as \emph{results}) in a particular theory. Specifically:
\begin{itemize}
\item A proposition is an ordinary result (and should not be confused with the ``propositions'' of propositional logic that we studied before),
\item a lemma is an auxiliary result that is only used as a means of proving something more important,
\item a theorem is an important result,
\item and a corollary is an immediate consequence of another result.
\end{itemize}

By the way, when a mathematician says that something is ``immediate'' from something, it means that it can be somewhat mechanically deduced without using any extravagant ideas.
If something is simply obvious, it is said to be trivial.
Lastly, if something is somewhere in between trivial and immediate, it is said to be ``direct''.

We will now be working not within any theory, but analysing theories themselves. These theories will be called \emph{object theories} and, in rigour, we should refer to the results we obtain about those theories as \emph{metatheorems} because they will not be results in the theory itself, but in the \emph{metatheory} that we will be using to analyse our object theory. Nevertheless, being aware of this fact, we will still use the usual terms (proposition, theorem\ldots) to refer to the different metatheorems.
\end{para}

\begin{para}
The first notion that we will formalise is that of a language.
I think we can both agree that any language needs two things: a collection of symbols and a collection of accepted constructions with those symbols.
For instance, the English language has a collection of symbols (the letters of the alphabet, empty space and punctuation marks) and a collection of appropriate combinations of those symbols (the collection of all sentences).

In the context of formal languages, we can take this classification one step further.
Among the allowed constructions with symbols, we can make a distinction between terms and formulas: a term would be a construction representing an object, whereas a formula would be a construction representing a well-formed statement about terms.
For example, if we were to define the language or arithmetic, we should define it in such a way that $x+1$ were a term and $x+1 = 0$ were a formula.

In an attempt to further connect these notions with human languages, we could identify terms with nominal phrases and formulas with sentences.
\end{para}

\begin{definition}
Given a set of symbols $\Sigma$, the set of words $\Sigma^*$ over $\Sigma$ is defined as the set of finite sequences of symbols of $\Sigma$.
Given any sequence of elements $a_1,a_2,\ldots,a_n$ of $\Sigma$, their corresponding element in $\Sigma^*$ is represented by concatenation: $a_1a_2\cdots a_n\in \Sigma^*$.

A \emph{formal language} --- or \emph{language}, for short --- is a set of symbols $\Sigma$ together with three subsets $T,F\subseteq \Sigma^*$ and $\Lambda \subseteq F$.
The set $\Sigma$ is said to be the \emph{alphabet} of the language, and $T$, $F$ and $\Lambda$ are said to be the sets of \emph{terms}, \emph{well-formed formulas} (or just \emph{formulas}) and \emph{sentences} of the language.
Such a formal language is represented by the tuple $(\Sigma, T, F, \Lambda)$.

For must practical purposes, the set of sentences is irrelevant.
Thus, we might sometimes refer to a language $(\Sigma, T, F, \Lambda)$ simply as $(\Sigma, T, F)$ if there is no need to know what its sentences are.
\end{definition}

\begin{para}
So that was our first formal definition.
It should have been clear enough, but there is a little thing I would like to clarify.
In mathematics, when we define an object that consists of several pieces, we often represent it with a tuple.
It is just a fancy way of representing things, do not try to find in it any sort of metaphysical nature.

As unnecessary as this may seem, it is done for a good reason.
Imagine that we are defining a formal language and we have already constructed the alphabet $A$, the set of terms $B$, the set of formulas $C$ and the set of sentences $D$.
Instead of writing ``the language having $A$ as alphabet, $B$ as set of terms, $C$ as set of formulas and $D$ as set of sentences'', we can just say ``the language $(A,B,C,D)$''.
It is more clean and convenient.
\end{para}

\begin{example}
\label{lp}
The first formal language that we will define is the formal language $L_P$ of propositional logic.
The alphabet of this language will consist of an infinite collection of symbols $p_1,p_2,\ldots$ meant to represent propositional variables, of parentheses, and of the $\limplies$ and $\lnot$ connectives.
We only include these connectives because, as we showed in \ref[prel]{adequateni}, they suffice to construct any propositional form.

Theoretically, we could add more connectives to our language, but that would simply be redundant.
The purpose of formal languages is to provide a precise framework in which to analyse languages theoretically and, therefore, they should be as simple and minimal as possible.
In our work as mathematicians, we can, of course, add as many connectives as we want --- regarding them as aliases for their equivalent constructions involving the $\limplies$ and $\lnot$ connectives.

Thus, the alphabet of the language of propositional logic will be
\[\Sigma_P = \{(,),\limplies,\lnot,p_1,p_2,\ldots,p_n,\ldots\}.\]
The set of terms $T_P$ of will simply be the set of words representing propositional variables: $T_P = \{p_1,p_2,\ldots\}\subseteq \Sigma_P^*$.
The terms of $L_P$ will be known as \emph{propositional symbols}.
Finally, the set of formulas $F_P$ is defined following what is known as an \emph{inductive procedure}:
\begin{enumerate}
\item All terms are well-formed formulas: $T_P \subseteq F_P$.
\item If $A$ and $B$ are well-formed formulas, so are $(\lnot A)$ and $(A\limplies B)$.
\end{enumerate}
Just to better understand what is going on here, how could we prove that $((\lnot p_1)\limplies p_2)$ is a formula of propositional logic?
By the first rule, we know that both $p_1$ and $p_2$ are formulas.
By the second one, we know that $(\lnot p_1)$ is also a formula; therefore, as both $(\lnot p_1)$ and $p_2$ are formulas, if we apply the second rule again, we know $( (\lnot p_1) \limplies p_2)$ to be a formula.

Just to conclude the definition of $L_P$, we will take any formula of this language to be a sentence.

The language of propositional logic is then $L_P = (\Sigma_P, T_P, F_P, F_P)$.
It is easy to see that the formulas of this language are, precisely, the propositional forms on the propositional variables $p_1,p_2,\ldots$.
That is why we will refer to the formulas in $F_P$ as \emph{propositional forms} too.

Informally, for any formulas $A,B\in F_P$, we will define the formula $(A\land B)$ as an abbreviation of $(\lnot(A\limplies (\lnot B)))$, the formula $A\lor B$ as an abbreviation of $((\lnot A )\limplies B)$, and the formula $(A\liff B)$ as an abbreviation of $((A\limplies B) \land (B\limplies A))$.
\end{example}

\begin{definition}
Let $S$ be a collection of symbols together with a \emph{signature} function $\sigma : S \longrightarrow \mathbb{Z}$. For any $s\in S$, we say that $s$ is an \emph{$n$-ary function symbol} if $\sigma(s) = n > 0$; we say that $s$ is an \emph{$n$-ary predicate symbol} if $\sigma(s) = -n < 0$, and we say that $s$ is a \emph{constant symbol} if $\sigma(s) = 0$.
Let $\{x_1,\ldots,x_n,\ldots\}$ be an arbitrary collection of symbols such that, for any $i\in\mathbb{N}$, $x_i\not\in S$.
The alphabet $\Sigma_{(S,\sigma)}$ associated to $(S,\sigma)$ is
\[\Sigma = S\cup \{\lnot,\limplies,\forall,(,)\} \cup \{,\} \cup \{x_1,x_2,\ldots,x_n,\ldots\}.\]
The set of terms $T_{(S,\sigma)}$ is defined inductively according to the following rules.
\begin{enumerate}
\item All the elements of the set $\{x_1,x_2,\ldots,x_n,\ldots\}$ of \emph{variables} are terms.
\item All constant symbols are terms.
\item Given any $n$-ary function symbol $f$ and given any $n$ terms $t_1,\ldots,t_n$, the construction $f(t_1,\ldots,t_n)$ is a term.
\end{enumerate}
The set of formulas $F_{(S,\sigma)}$ is defined inductively according to:
\begin{enumerate}
\item Given any $n$-ary predicate symbol $P$ and any $n$ terms $t_1,\ldots,t_n$, the \emph{atomic formula} $P(t_1,\ldots,t_n)$ is a formula.
\item If $A$ and $B$ are formulas and $i\in\mathbb{N}$, then $(\lnot A)$, $(A\limplies B)$ and $(\forall x_i) A$ are formulas.
\end{enumerate}

As in the informal treatment of predicate logic, given any formula of the form $(\forall x_i) A$, the formula $A$ is said to be the \emph{scope} of the quantifier $(\forall x_i)$.
Moreover, any occurrence of a variable $x_i$ within the scope of a quantifier $(\forall x_i)$ is said to be \emph{bound}.
If an appearance of a variable is not bound, we say that it is \emph{free}.

The set of sentences $\Lambda_{(S,\sigma)}$ is the set of all the \emph{closed formulas}: the collection of all formulas having no free occurrences of variables.

The language $L_{(S,\Sigma)} = (\Sigma_{(S,\sigma)}, T_{(S,\sigma)}, F_{(S,\sigma)}, \Lambda_{(S,\sigma)})$ is said to be the \emph{first-order language} associated to $(S,\sigma)$.

Just as we did in the language of propositional logic, given any formulas $A$ and $B$ in any first order language, we can define the formula $(A\land B)$ as an abbreviation of $(\lnot(A\limplies (\lnot B)))$, the formula $A\lor B$ as an abbreviation of $((\lnot A )\limplies B)$, and the formula $(A\liff B)$ as an abbreviation of $((A\limplies B) \land (B\limplies A)$.
In addition, for any variable $x_i$, we will define $(\exists x_i) A$ to be an alias for $(\lnot(\forall x_i)(\lnot A))$.
\end{definition}

\begin{para}
First-order languages will make it easy to define the language of any theory built in the framework of first-order logic.
If you have perfectly understood the definition I have given you, that's wonderful! Just move on.
Otherwise, if you feel a little bit overwhelmed and are wondering why on earth I have just done this to you\ldots let us have a small chat here.
One of the most important skills you will learn in your journey through the wonders of mathematics is being able to see beyond formal concepts.
I have given you a purely formal definition, and now you need to work on it to get the intuition behind it.
This might be the first time you are facing this kind of challenge, so let me guide you through.

The idea behind all of this is simple.
The alphabet of any first-order language always has an infinite amount of symbols representing variables ($x_1,\ldots,x_n,\ldots$), two connectives ($\lnot$ and $\limplies$), one quantifier ($\forall$) and some punctuation symbols (parentheses and a comma).
In addition, it may have some constants, some symbols representing predicates and some symbols representing functions.
These last objects are what can make first-order languages distinct, and they are encoded in $S$ and $\sigma$.
Instead of using  a set for the constants, a set for the predicate symbols and another set for the function symbols, I thought it would be better to put them all in one set and use the $\sigma$ function to distinguish them.
Why? Because we were going to need such a function anyway to specify the arity of each of the symbols for, as you have seen, knowing the arity of a function or predicate symbol is necessary when defining how terms and formulas can be constructed.
So, instead of having three sets and two arity functions (one for the set of predicates and another one for the set of function symbols), why not use just one set and a ``signature'' function? A signature of zero just means that symbol is a constant, a positive signature that a symbol is a function, and a negative signature that it is a predicate. Moreover, the absolute value indicates the arity. It is all more compact and simple to encode.
\end{para}

\begin{example}
\label{lfo}
\begin{parlist}
\item \label{lq} Any first-order language can be used as a language for predicate logic. Nonetheless, we will do it for the most general first-order language available: one that contains an infinite amount of constants, predicates and functions. Our first-order language $L_Q$ will be the one associated to the set $S$ of all the symbols of the form
\[P_i^j,\quad f_i^j,\quad c_i\]
with $i,j\in\mathbb{N}$, and to the signature function $\sigma$ defined, for every $i,j\in\mathbb{N}$ as
\[\sigma\left(P_i^j\right) = -j,\qquad\sigma\left(f_i^j\right) = j,\qquad\sigma\left(c_i\right) = 0.\]

\item \label{lpa} The language of first-order arithmetic is the first-order language associated to the set $S = \{=,1,s,+,\cdot\}$ and the signature $\sigma(=)=-2$, $\sigma(1) = 0$, $\sigma(s) = 1$, $\sigma(+) = 2$ and $\sigma(\cdot) = 2$. Instead of witting $=\!(x,y)$, $+(x,y)$ and $\cdot(x,y)$, we will use \emph{infix notation} and write $x=y$, $x+y$ and $x\cdot y$ respectively.
Furthermore, $x\cdot y$ will often be abbreviated as $xy$.

The symbol $=$ is meant to represent equality, $+$ is meant for addition, $\cdot$ for multiplication, $1$ is meant for the number one, and $s$ for the successor function.
I suppose that wasn't very surprising.

In order to simplify the notation and reduce the number of parentheses, by convention, $\cdot$ will have higher precedence than $+$ in the creation of terms. Thus $x + y \cdot  z$ will be parsed as $x + ( y \cdot z) $ and not as $(x + y ) \cdot z$.
\end{parlist}
\end{example}

\begin{para}[Syntax and semantics]
With formal languages, we now have a formal device enabling us to precisely express statements about a particular theory. What remains to be done is providing mechanical rules for transforming formulas in order to enable deductions, and providing ways of deciding the truth or falsity of such formulas and the validity of the methods of deduction.
These two different tasks are tackled by syntax and semantics respectively.

As we will soon see, the central object in syntax is that of a \emph{formal system}: a framework in which we can use a deductive machinery to operate with the formulas of a language.

The fundamental tools in semantics are \emph{interpretations}. An interpretation is nothing more than an assignment of meaning to the elements of a language. Thus, once a formula is considered under a particular interpretation, we are able to decide its truth or falsity.

The informal analysis of propositional and predicate logic that we have made has been purely semantic.
When we studied propositional logic, we developed a complete and systematic understanding of the truth of propositional forms for each possible assignment of truth values to its variables;
as we will soon formalise, those assignments of truth values are the possible interpretations of propositional logic.
In our study of predicate logic, we learned to identify the truth or falsity of formulas in a certain domain of discourse;
following the same ideas, an interpretation of any first-order language will be nothing more than a choice of a domain of discourse and an assignment of meaning to the symbols of the language under that domain.
\end{para}

\begin{definition}
A \emph{formal system} $\mathsf{S}$ is a language $L = (\Sigma_L, T_L, F_L)$ together with a subset $\Xi\subseteq F_L$ of formulas known as its set of \emph{axioms} and a set $R$ of \emph{rules of inference}.
We write $\mathsf{S} = (L,\Xi,R)$.

We say that a formula $P \in F_L$ can be \emph{deduced} from a set of formulas $\Gamma\subseteq F_L$ in the formal system $\mathsf{S}$ if there exists a sequence of formulas $P_1,\ldots,P_n$ with $P_n = P$ such that, for every $i\in \{1,\ldots,n\}$, $P_i$ belongs to $\Xi$ or $\Gamma$, or has been obtained by applying a rule of inference on previous formulas of the sequence. If $\Gamma = \emptyset$, $P$ is said to be a \emph{theorem} of $\mathsf{S}$ and the sequence $P_1,\ldots,P_n$ is said to be a \emph{proof} of $P$ in $\mathsf{S}$.

If $P$ can be deduced from $\Gamma$, we write $\Gamma \vdash_{\mathsf{S}} P$. Furthermore, if $\Gamma = \{H_1,\ldots,H_m\}$, we can also write $H_1,\ldots,H_m\vdash_{\mathsf{S}} P$. We can denote the fact that $P$ is a theorem of $S$ by $\vdash_{\mathsf{S}} P$ or by $\mathsf{S}\vdash P$. In addition, if the context makes it clear that the deduction or proof is taking place in $\mathsf{S}$, the symbol $\vdash_{\mathsf{S}}$ can be safely replaced by $\vdash$.
\end{definition}

\begin{para}
I would like to make a small remark about the definitions I have just given you.
What we have done is a mere formalisation.
Thus, if we are ever proving or deducing something in any particular formal system, we will probably not find ourselves just applying inference rules in such a mechanical way as that described in the formal definition of proofs.

Instead, in practice, we will use some methods that will enable us to reason in a more intuitive and human-like manner, but always with the assurance that a ``formal proof'' like the one we have defined can be constructed from that reasoning --- even if that would involve an unfeasible amount of mechanical work.
\end{para}

\begin{para}[Notation]
Instead of writing ``$P_1,\ldots,P_n$ with $P_n = P$'' as we did in the previous definition, it is customary and convenient to write $P_1,\ldots,P_n = P$.

This is quite a bit more than something specific of this context. In fact, one should always be flexible in the way they read notation. For instance, if I wanted to say ``this applies to the set $A$ that is a subset of $B$'', I could just say ``this applies to $A\subseteq B$''.
\end{para}

\begin{proposition}
Let $\mathsf{FS}$ be a formal system on a language $L = (\Sigma, T, F)$. If $\Gamma,\Gamma'\subseteq F$ are collections of formulas and $A$ and $B$ are two arbitrary formulas, then:
\begin{statements}
\item If $\Gamma \vdash A$, then $\Gamma \cup \{B\} \vdash A$.
\item If $\Gamma \vdash A$ and $\Gamma' \cup \{A\} \vdash B$, then $\Gamma \cup \Gamma' \vdash B$.
\item If $\Gamma \vdash A$, then $\Gamma \vdash B$ if and only if $\Gamma \cup \{A\} \vdash B$.
\end{statements}
\label{<+label+>}
\end{proposition}

\begin{definition}
Given a language $L = (\Sigma,T,F)$, an \emph{interpretation} of $L$ is an assignment of meaning to the elements of $L$. The elements of $L$ are nothing more than a bunch of meaningless symbols.
Under a certain interpretation, however, those symbols are given an abstract reference.
For example, in the formal language of arithmetic, the symbol $1$ is nothing more than a meaningless character, but we can construct an interpretation in which $1$ represents ``the number one''.
The way in which these interpretations can be constructed is different for each of the two ``flavours'' of logic that we have studied and will be described, with full precision, later.

An interpretation of $L$ will define two subsets of $F$: a subset of \emph{true} formulas, and a subset of \emph{false} formulas; we denote that a formula $A\in F$ is true in an interpretation $I$ by $I\vDash A$. Bear in mind, though, that there might be formulas that are neither true or false.

A formula $A\in F$ is said to be \emph{valid} in $L$ if it is true in any interpretation. That is denoted by $\vDash A$.

Given a formula $B\in F$ and a collection of formulas $\Gamma\subseteq F$, if whenever an interpretation $I$ satisfies $I\vDash A$ for every $A\in \Gamma$, we have $I\vDash B$, we say that $B$ is a \emph{semantic consequence} of $\Gamma$ or that $\Gamma$ \emph{semantically entails} $B$.
In this case, we write $\Gamma\vDash B$.
If $\Gamma = \{A_1,\ldots,A_n\}$, we may also write $A_1,\ldots,A_n\vDash B$.
Notice how $\emptyset \vDash B$ is the same as $\vDash B$.
\end{definition}

\begin{para}[Why bother?]
At this point, one question might be popping up in your mind. You should by now have a fully developed understanding of propositional and predicate logic --- an understanding that we have labelled as ``semantic''.
why isn't that everything you need? Why bother with the axiomatic method and with syntax altogether?

While what you already know is, probably, everything you will ever need to know about logic in your future career as a mathematician, syntactical approaches in general and the axiomatic method in particular have a purpose.

You see, logic is very intuitive, mechanical and simple; so much so that it sometimes seems just like a linguistic artefact.
This will not be the case with mathematics. When arguing about the truth or falsity of, let us say, a proposition (in propositional logic), we have systematic semantic methods that enable us to do so. In mathematical theories, such a thing does not exist.

How would you show me that there exists an infinite amount of prime numbers in a semantic way? You cannot --- at least unless you are able to write down an infinite list of prime numbers to prove it.
While it is semantically clear that there either is or there is not an infinite amount of prime numbers, there is no way to decide semantically.
We need to have some structure enabling deduction: that is why we need syntax.
The purpose of a formal system is capturing the ``essence'' of the entities it aims to add syntax to.
Syntax aims to model an abstract reality in such a way that we can work in it whereas semantics is concerned with the abstract realities themselves.

Of course, one should expect syntax and semantics --- that is, formal systems and interpretations --- to work well together. In an ideal situation, we should all expect that a formula be a theorem in a formal system if and only if it be true in any suitable interpretation of its language.
That is what should happen and, indeed, that is what we know happens in the formalisations of propositional and predicate logic. Unfortunately, that is everything we have.
As we will later discuss, G\"odel showed that this perfect harmony can never be reached for any formal system powerful enough to capture basic arithmetic. I know, that is a real pain in the neck: syntax can only do its job perfectly wherever it is not necessary.

As catastrophic as this may seem, this is not that bad; in fact, depending on your views on the philosophy of mathematics, it can be great news. Please, bear with me.
\label{whybother}
\end{para}


\chapter{Riordan arrays: a chronological development}
\label{ch:chronologial:development}

In this chapter we would like to introduce some important mathematical objects
necessary to tackle topics in later chapters. Such objects are the foundations
of the \emph{Riordan group theory} and we would like to describe them in our
words, after a quick review of fundamental concepts, such as sequences,
\ac{fps} and \ac{gf}.  We aim at descriptions that point out why a particular
concept is important and how it allows generalizations and further
developments. Because of this approach, our prose will be longer than
traditional ones.

%In the first part we enter into the playground, where we will recall concepts
%according to chronological and dependency order; in the last part we describe our
%motivations and ideas underlying this work.

So, start the journey!

\section{Basic concepts}

\input{Chapters/back-to-the-basics/basic-concepts}

\section{Understanding the playground}

We are now ready to show how the concept of \emph{Riordan array} has been
developed, starting from a triangle of numbers that counts
\emph{non-intersecting} pairs of paths to some alternative characterizations
for these arrays.

\input{Chapters/back-to-the-basics/shapiro}
\input{Chapters/back-to-the-basics/rogers}
\input{Chapters/back-to-the-basics/eplett}
\input{Chapters/back-to-the-basics/riordan-group}
\input{Chapters/back-to-the-basics/sprugnoli}
\input{Chapters/back-to-the-basics/sequences}


\input{../header_function}

%---------- start document ---------- %
 \section{factor.ecm -- ECM factorization}\linkedzero{factor.ecm}

 This module has curve type constants:
 \begin{description}\linkedone{factor.ecm}{curvetype}
   \item[S]\linkedone{factor.ecm}{S}: aka SUYAMA. Suyama's parameter selection strategy.
   \item[B]\linkedone{factor.ecm}{B}: aka BERNSTEIN. Bernstein's parameter selection strategy.
   \item[A1]\linkedone{factor.ecm}{A1}: aka ASUNCION1. Asuncion's parameter selection strategy variant 1.
   \item[A2]\linkedone{factor.ecm}{A2}: aka ASUNCION2.  ditto 2.
   \item[A3]\linkedone{factor.ecm}{A3}: aka ASUNCION3.  ditto 3.
   \item[A4]\linkedone{factor.ecm}{A4}: aka ASUNCION4.  ditto 4.
   \item[A5]\linkedone{factor.ecm}{A5}: aka ASUNCION5.  ditto 5.
 \end{description}
 See J.S.Asuncion's master thesis~\cite{Janice} for details of each family.

%
  \subsection{ecm -- elliptic curve method}\linkedone{factor.ecm}{ecm}
   \func{ecm}
   {%
     \hiki{n}{integer},\ %
     \hikiopt{curve\_type}{\linkingone{factor.ecm}{curvetype}}{A1},\ %
     \hikiopt{incs}{integer}{3},\ %
     \hikiopt{trials}{integer}{20},\ %
     \hikiopt{verbose}{bool}{False}%
   }{%
     \out{integer}
   }\\
   \spacing
   % document of basic document
   \quad Find a factor of \param{n} by elliptic curve method.\\
   \spacing
   % added document
   If it cannot find non-trivial factor of $n$, then it returns $1$.\\
   \spacing
   % input, output document
   \quad \param{curve\_type} should be chosen from \linkingone{factor.ecm}{curvetype} constants above.\\
   \quad The second optional argument \param{incs} specifies a number
   of changes of bounds. The function repeats factorization
   trials several times changing curves with a fixed bounds.\\
   \quad Optional argument \param{trials} can control how quickly move
   on to the next higher bounds.\\
   \quad \param{verbose} toggles verbosity.\\
\C

%---------- end document ---------- %

\input{../footer}


\subsection{Services}

This problem will have you write a service for the client-service structure.

\subsubsection{Creating a Service}\label{p:service1}

The work for this problem should be done in the file \texttt{draw\_server.py}.

\begin{figure}[h]
  \centering
  \includegraphics[width=150pt]{figures/p1/problem4a.png}
  \caption{For Problem \ref{p:service1}}
  \label{fig:3}
\end{figure}

\begin{enumerate}[(a)]
  \item In \texttt{draw}:
  \begin{enumerate}[i.]
    \item Initialize \texttt{vel\_msg}.
    \item Set $k$ to be the count parameter of \texttt{req}. %maybe this is too much hand holding idk
    \item To calculate constants, use \texttt{numpy} to perform the following operations on the
    following matrices:

    $$M_3 = \begin{bmatrix}
      1 & 1 & 2\\
      3 & 5 & 8 \\
      13 & 21 & 34
    \end{bmatrix}
    M_4 = \begin{bmatrix}
      1 & 8 & 7\\
      2 & 5 & 3 \\
      9 & 2 & 6
    \end{bmatrix}$$

    $$a = det(M_3) -1$$

    $$b = -((M_3*M_4)_{(2,0)}-(M_3*M_4)_{(2,2)}) +1$$

    $$A = (\langle M_3, M_4 \rangle^T_{(0,0)} - \langle M_3,M_4 \rangle^T_{(1,0)})/10$$

    $$B = ((M_3*M_4)^T_{(2,1)}-\langle M_3,M_4 \rangle^T_{(2,1)})/2.0 $$

    \item Each of the above computed constants should be set as rosparams \texttt{a}, \texttt{b},
    \texttt{A}, and \texttt{B} with the type \texttt{float}.

  \end{enumerate}

  \item In \texttt{if req.rotate} section:
  \begin{enumerate}[i.]
    \item If \texttt{req.rotate}, set angular z to -3 and linear x to 0.
    \item Otherwise use \texttt{numpy} to set angular z to $B*cos(b*count)$ and linear x to $A*sin(a*count)$.
  \end{enumerate}

  \item Also be sure to return your velocity message before the end of the \texttt{draw} function.
  \item In \texttt{draw\_server}: Create a service with the name \texttt{`draw'}, service\_class
  \texttt{Draw} and handler \texttt{draw}. Please see rospy documentation for further details.
  \item In \texttt{\_\_name\_\_ == `\_\_main\_\_'}: Call your function!
  \item In \texttt{draw\_server.launch}:
  \begin{enumerate}[i.]
    \item Launch the turtlesim node.
    \item Launch \texttt{draw\_service} node from \texttt{draw\_server.py}.
    \item  Launch \texttt{draw\_flower} node from \texttt{client.py}.
  \end{enumerate}

\end{enumerate}
Your resulting flower should resemble Figure~\ref{fig:3}.

\subsubsection{Integrating the pieces}\label{p:service2}

The work for this problem should be done in file \texttt{spawn\_flower.py}.

\begin{figure}[h]
  \centering
  \includegraphics[width=150pt]{figures/p1/problem4b_3turts.png}
  \caption{For Problem~\ref{p:service2}}\label{fig:4}
\end{figure}

\begin{enumerate}[(a)]
  \item In \texttt{draw\_flower}:
  \begin{enumerate}[i.]
    \item Initialize your \texttt{`draw\_turtle'} node
    .
    \item Block while waiting for the \texttt{`spawn'} service to become available. See rospy
    documentation for more information.

    \item In first \texttt{try}: Create a handle for calling the spawn service.
    \item In second \texttt{try}: Follow the instructions in the comments.
    \item Block while waiting for \texttt{`/draw'} service, and then set up the service handle for
    the draw service (in the same way as problem 3). %these instructions seem vague, come back and fix

    \item Set up velocity publisher for \texttt{`{}/cmd\_vel'.format(turtle\_name)} and set the rate to 5.
  \end{enumerate}

  \item In \texttt{while not rospy.is\_shutdown()} section of \texttt{draw\_flower}: Publish the
  velocity messages returned from draw. Then sleep for predetermined rate.

  \item In \texttt{\_\_name\_\_ == `\_\_main\_\_'}: Call your function!
  \item In \texttt{spawn\_flower.launch}:
  \begin{enumerate}[i.]
    \item Launch the turtlesim node.
    \item Launch \texttt{draw\_service} node from \texttt{draw\_server.py}.
    \item  Launch \texttt{launcher} node from \texttt{launcher.py}.
  \end{enumerate}

  Your resulting product should resemble Figure~\ref{fig:4}. There should be between 2 and 5 turtles
  spawned, in addition to the original turtle. The starting points of the turtles is random.

\end{enumerate}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../assessment"
%%% End:


\documentclass{article}
\usepackage[margin=1in]{geometry}
\usepackage{physics}
\usepackage{amsmath}
\usepackage{amssymb}

\title{Notes on Sequence Transduction with Recurrent Neural Networks}
\author{Mingkun Huang}
\date{\today}

\begin{document}
\maketitle

\def \x {\mathbf{x}}
\def \y {\mathbf{y}}
\def \a {\mathbf{a}}
\def \f {\mathbf{f}}
\def \g {\mathbf{g}}
\def \X {\mathcal{X}}
\def \Y {\mathcal{Y}}

\section{Recurrent Neural Network Transducer}
Let $\x = (x_1, x_2, \dots, x_T)$ be a length $T$ $input\ sequence$ of arbitrary length belonging to the set $\X^*$ of all sequences over some input space $\X$.
Let $\y = (y_1, y_2, \dots, y_U)$ be a length $U$ $output\ sequence$ belonging to the set $\Y^*$ of all sequences over some output space $\Y$.
Both the inputs vectors $x_t$ and the output vectors $y_u$ are represented by fixed-length real-valued vectors.

Define the extended output space $\bar{\Y}$ as $\Y \cup \varnothing$, where $\varnothing$ denotes the $null$ output.
We refer to the elements $\a \in \bar{\Y}^*$ as $alignments$, since the location of the null symbols determines an alignment between the input and output sequences.
Given $\x$, the RNN transducer defines a conditional distribution $\Pr(\a \in \bar{\Y}^* | \x)$.
This distribution is then collapsed onto the following distribution over $\Y^*$
\begin{equation}
    \Pr(\y \in \Y^* | \x) = \sum_{\a \in \mathcal{B}^{-1}(\y)} \Pr(\a | \x)
\end{equation}
where $\mathcal{B}: \bar{\Y}^* \rightarrow \Y^*$ is a function that removes the null symbols from the alignments in $\bar{\Y}^*$.

Two recurrent neural networks are used to determine $\Pr(\a \in \bar{\Y}^* | \x)$.
One network, referred to as the $transcription\ network\ \mathcal{F}$, scans the input sequence $\x$ and outputs the sequence $\f = (f_1, \dots, f_T)$ of transcription vectors\footnote{For simplicity we assume the transcription sequence to be the same length as the input sequence; however this may not be true, for example if the transcription network uses a pooling architecture to reduce the sequence length.}.
The other network, referred to as the $prediction\ network\ \mathcal{G}$, scans the output sequence $\y$ and outputs the prediction vector sequence $\g = (g_0, g_1, \dots, g_U)$.

\subsection{Prediction Network}
The prediction network $\mathcal{G}$ is a recurrent neural network consisting of an input layer, an output layer and a single hidden layer.
The length $U + 1$ input sequence $\y^* = (\varnothing, y_1, \dots, y_U)$ to $\mathcal{G}$ output sequence $\y$ with $\varnothing$ prepended.
If $\Y$ consists of $K$ labels, the input layer is therefore size $K$. $\varnothing$ is encoded as a length $K$ vector of zeros.
The output layer is size $K + 1$ (one unit for each element of $\bar{\Y}$) and hence the prediction vectors $g_u$ are also size $K + 1$.

The prediction network attempts to model each element of $\y$ given the previous ones; it is therefore similar to a standard next-step-prediction RNN, only with the added option of makeing $null$ predictions.
Since the input and output lengths are equal, we can apply attention onto it with the output of transcription network.

\subsection{Transcription Network}
The transcription network $\mathcal{F}$ is an arbitrary representation learning network.
Given a length $T$ input sequence $(x_1, \dots, x_T)$, it outputs the same length hidden vectors $(f_1, \dots, f_T)$ with each vector size $K + 1$.

The transcription network is similar to a CTC RNN, which also uses a null output to define a distribution over input-output alignments.

\subsection{Output Distribution}
Given the transcription vector $f_t$, where $1 \leq t \leq T$, the prediction vector $g_u$, where $0 \leq u \leq U$, and label $k \in \bar{\Y}$, define the \textit{log output density function}
\begin{equation} \label{eq:add}
    h(k, t, u) = f_t^k + g_u^k
\end{equation}
where superscript $k$ denotes the $k^{th}$ element of the vectors. The density can be softmaxed to yield the conditional $output\ distribution$
\begin{equation} \label{eq:softmax}
    \Pr(k \in \bar{\Y} | t, u) = \frac{e^{h(k, t, u)}}{\sum_{k' \in \bar{\Y}} e^{h(k', t, u)}}
\end{equation}
To simplify notation, define
\begin{equation}
    \begin{split}
        y(t, u) &:= \Pr(y_{u+1} | t, u) \\
        \varnothing(t, u) &:= \Pr(\varnothing | t, u)
    \end{split}
\end{equation}
$\Pr(k | t, u)$ is used to determine the transition probabilities in the lattice shown in~\cite{graves2012rnnt}. For any specific node, there are at most two transitions.

\subsection{Forward-Backward Algorithm}
Define the $forward\ variable\ \alpha(t, u)$ as the probability of outputting $\y_{[1:u]}$ during $\f_{[1:t]}$. The forward variables for all $1 \leq t \leq T$ and $0 \leq u \leq U$ can be calculated recursively using
\begin{equation} \label{eq:alpha}
    \alpha(t, u) = \alpha(t-1, u) \cdot \varnothing(t-1, u) + \alpha(t, u-1) \cdot y(t, u-1)
\end{equation}
with initial condition $\alpha(1, 0) = 1$. The total output sequence probability is equal to the forward variable at the terminal node:
\begin{equation}
    \Pr(\y | \x) = \alpha(T, U) \cdot \varnothing(T, U)
\end{equation}

Define the $backward\ variable\ \beta(t, u)$ as the probability of outputting $\y_{[u+1:U]}$ during $\f_{[t:T]}$. Then
\begin{equation} \label{eq:beta}
    \beta(t, u) = \beta(t+1, u) \cdot \varnothing(t, u) + \beta(t, u+1) \cdot y(t, u)
\end{equation}
with initial condition $\beta(T, U) = \varnothing(T, U)$.
From the definition of the forward and backward variables it follows that their product $\alpha(t, u) \cdot \beta(t, u)$ at any point $(t, u)$ in the output lattice is equal to the probability of emitting the complete output sequence
\textit{if $y_u$ is emitted during transcription step $t$}.

\subsection{Training}
Given an input sequence $\x$ and a \textit{target sequence} $\y^*$, the natural way to train the model is to minimize the negtive log likelihood $\mathcal{L} = -\ln \Pr(\y^* | \x)$ of the target sequence.
Analysing the diffusion of probability through the output lattice shows that $\Pr(\y^* | \x)$ is equal to the sum of $\alpha(t, u) \cdot \beta(t, u)$ over any top-left to bottom-right diagonal through the nodes. That is,
$\forall n : 1 \leq n \leq U + T$
\begin{equation} \label{eq:pyx}
    \Pr(\y^* | \x) = \sum_{(t,u): t+u=n} \alpha(t, u) \cdot \beta(t, u)
\end{equation}
From Eqs. (\ref{eq:alpha}), (\ref{eq:beta}) and (\ref{eq:pyx}) and the definition of $\mathcal{L}$ it follows that
\begin{equation} \label{eq:grad_pr}
    \frac{\partial \mathcal{L}}{\partial \Pr(k|t,u)} = -\frac{\alpha(t,u)}{\Pr(\y^*|\x)}
    \begin{cases}
        \beta(t, u+1) & \text{if } k = y_{u+1} \\
        \beta(t+1, u) & \text{if } k = \varnothing \\
        1 & \text{if } k = \varnothing, t = T, u = U \\
        0 & \text{otherwise}
    \end{cases}
\end{equation}
And therefore
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal L}{\partial h(k,t,u)} = \sum_{k' \in \bar{\Y}} \frac{\partial \mathcal L}{\Pr(k'|t,u)} \frac{\partial \Pr(k'|t,u)}{\partial h(k,t,u)}
    \end{split}
\end{equation}
where, from Eq. (\ref{eq:softmax})
\begin{equation}
    \frac{\partial \Pr(k'|t,u)}{\partial h(k,t,u)} = \Pr(k'|t,u)[\delta_{k,k'} - \Pr(k|t,u)]
\end{equation}
Then we have
\begin{equation} \label{eq:grad_acts}
    \begin{split}
        \frac{\partial \mathcal L}{\partial h(k,t,u)} &=
            -\frac{\alpha(t,u)}{\Pr(\y^*|\x)} \left \{ \beta(t, u+1) \cdot y(t,u) \cdot [\delta_{k,y_{u+1}} - \Pr(k|t,u)] +
                \beta(t+1, u) \cdot \varnothing(t,u) \cdot [\delta_{k,\varnothing} - \Pr(k|t,u)] \right \} \\
        &= -\frac{\alpha(t,u)}{\Pr(\y^*|\x)}
            \begin{cases}
                \beta(t, u+1) \cdot y(t,u) \cdot [1 - y(t,u)] + \beta(t+1, u) \cdot \varnothing(t,u) \cdot (-y(t,u)), & k = y_{u+1} \\
                \beta(t, u+1) \cdot y(t,u) \cdot (-\varnothing(t,u)) + \beta(t+1, u) \cdot \varnothing(t,u) \cdot (1 - \varnothing(t,u)), & k = \varnothing \\
                \varnothing(t,u) \cdot (1 - \beta(t,u)), & k = \varnothing, t = T, u = U \\
                \beta(t, u+1) \cdot y(t,u) \cdot (-\Pr(k|t,u)) + \beta(t+1, u) \cdot \varnothing(t,u) \cdot (-\Pr(k|t,u)), & \text{otherwise}
            \end{cases} \\
        &= -\frac{\alpha(t,u)}{\Pr(\y^*|\x)}
            \begin{cases}
                \beta(t, u+1) \cdot y(t,u) - \beta(t, u) \cdot y(t,u), & k = y_{u+1}, u < U \\
                \beta(t+1, u) \cdot \varnothing(t,u) - \beta(t, u) \cdot \varnothing(t,u), & k = \varnothing, t < T \\
                \varnothing(t,u) - \beta(t, u) \cdot \varnothing(t,u), & k = \varnothing, t = T, u = U \\
                -\beta(t, u) \cdot \Pr(k|t,u), & \text{otherwise} \\
            \end{cases} \\
        &= \frac{\alpha(t,u) \cdot \Pr(k|t,u)}{\Pr(\y^*|\x)} \cdot \beta(t,u) - \frac{\alpha(t,u) \cdot \Pr(k|t,u)}{\Pr(\y^*|\x)}
            \begin{cases}
                \beta(t, u+1), & k = y_{u+1}, u < U \\
                \beta(t+1, u), & k = \varnothing, t < T \\
                1, & k = \varnothing, t = T, u = U \\
                0, & \text{otherwise} \\
            \end{cases}
    \end{split}
\end{equation}
From Eq. (\ref{eq:add})
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal L}{\partial f_t^k} &= \sum_{u=0}^U \frac{\partial \mathcal L}{\partial h(k,t,u)} \\
        \frac{\partial \mathcal L}{\partial g_u^k} &= \sum_{t=0}^T \frac{\partial \mathcal L}{\partial h(k,t,u)} \\
    \end{split}
\end{equation}

There are two ways to calculate the gradient:
\begin{itemize}
    \item Gradient to softmax output (Eq. \ref{eq:grad_pr}). \\
    Easy to program. But consumes too much memory.
    \item Gradient to activations (Eq. \ref{eq:grad_acts}). \\
    Memory efficient, can be accelerated in GPU.
\end{itemize}
Note that in practical programming, all probabilities are in log scale. Since the calculation may overflow float point, we calculate the two terms in Eq. (\ref{eq:grad_acts}) seperately.

\begin{thebibliography}{1}
\bibitem{graves2012rnnt}
Alex Graves.
Sequence Transduction with Recurrent Neural Network,
arXiv preprint arXiv:1211.3711, 2012.
\end{thebibliography}

\end{document}


%    Angulo.  Measure angles and slopes with Android!
%    Copyright (C) 2011  Daniel Kraft <d@domob.eu>
%
%    This program is free software: you can redistribute it and/or modify
%    it under the terms of the GNU General Public License as published by
%    the Free Software Foundation, either version 3 of the License, or
%    (at your option) any later version.
%
%    This program is distributed in the hope that it will be useful,
%    but WITHOUT ANY WARRANTY; without even the implied warranty of
%    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%    GNU General Public License for more details.
%
%    You should have received a copy of the GNU General Public License
%    along with this program.  If not, see <http://www.gnu.org/licenses/>.

\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{url}
\usepackage{hyperref}

\newcommand{\abs}[1]{\ensuremath{\left| #1 \right|}}
\newcommand{\inner}[2]{\ensuremath{#1 \cdot #2}}
\newcommand{\cross}[2]{\ensuremath{#1 \times #2}}
\newcommand{\trace}[0]{\ensuremath{\text{tr} \;}}
\newcommand{\R}[0]{\ensuremath{\mathbb{R}}}
\newcommand{\Rthree}[0]{\ensuremath{\R^3}}
\newcommand{\SO}[1]{\ensuremath{SO\left(#1\right)}}

\renewcommand{\eqref}[1]{\hyperref[eq:#1]{Equation~\ref*{eq:#1}}}
\newcommand{\secref}[1]{\hyperref[sec:#1]{Section~\ref*{sec:#1}}}

\newcommand{\Angulo}[0]{\textbf{Angulo} }

\title{Angulo}
\author{Daniel Kraft}
\date{\today}

\begin{document}
\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}

\Angulo is a tool which utilizes the internal sensors (for acceleration and
magnetic field) of a mobile device to allow measuring angles with it.  We
assume that both acceleration (the Earth's gravitational field) and the
surrounding magnetic field are constant and fixed in space; thus if we
measure both at two times and compare the measured vectors (their directions
in particular), we can calculate how the device was rotated between the
measurements.

Thus you can set a ``reference point'' with the device pointing in one
direction, and rotate it into another one, and have the angle between both
directions calculated.  With this tool, one can for instance find out the
inclination of a slope or actually simply measure an angle between any two
given directions.

The code is free (released under the GPL), check it out at
\url{http://www.sourceforge.net/projects/angulo/}.  \Angulo runs on Google's
Android operating system (and obviously needs devices which actually
have the sensors used).  This document is not intended as an usage introduction,
for that see the built-in help text or the webpage at
\url{http://www.domob.eu/projects/angulo.php}.  Instead, I want to describe
the internal logic here (in particular, how the shown results are calculated).

\Angulo displays measured angles mainly in degrees; for small angles (and
in particular slopes) sometimes also percent are used --- where the percent
value corresponds to the tangens of the actual angle.  If the angle is small
(below 45 degrees), \Angulo as a utility feature displays the same angle
again in percent.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{With a Single Sensor}
\label{sec:singleValue}

Assume for a moment that we only consider data from a single sensor, and let
it without loss of generality be acceleration.  If we
measure twice, we get two vectors $a, b \in \Rthree$.  $a$ corresponds to
the acceleration at the first measurement (which \Angulo calls the
``reference point''), $b$ to that at the second time (which is the ``current''
value updated continuously during runtime).

Now, if $\phi$ denotes the angle in-between the vectors $a$ and $b$,
elementary vector analysis implies that
\begin{equation}
\label{eq:abCosine}
\cos \phi = \frac{\inner{a}b}{\abs{a} \abs{b}}.
\end{equation}
Hence, using the arccosine we can easily calculate the angle between both
directions.  However, since the inner product is symmetric, the same angle
will be reported if we rotate in the opposite direction --- which is somewhat
counter-intuitive.  On the other hand, the cross-product $\cross{a}b$ is
\emph{anti}symmetric.  Thus we can set the sign of $\phi$ depending on the
direction of $n = \cross{a}b$; since there's no ``absolute reference'' however,
\Angulo simply uses the sign of the sum of all component of $n$ as the
assumed sign of $\sin \phi$ and thus $\phi$.
That way, if $a$ and $b$ are flipped (which corresponds to rotating in the
opposite direction), also $n$ is flipped and the sign will be opposite.  This
is of course an arbitrary choice (and a lot of other options would be possible),
but it makes \Angulo display a useful sign in most circumstances.  Just note
to be careful in particular with the sign; but usually, the interesting
quantity is anyways the magnitude of the angle and not its direction (since that
is most of the time apparent).

In addition to \eqref{abCosine}, another relation is
\begin{equation}
\label{eq:abSine}
\abs{\sin \phi} = \frac{\abs{\cross{a}b}}{\abs{a} \abs{b}}.
\end{equation}
If we combine \eqref{abCosine} and \eqref{abSine}, we get (except for
the sign of $\sin \phi$, but the same solution as above applies for the
angle's sign) an even more (numerically) accurate result for the angle.

The problem with this approach is that it only measures the angle between
the two actually measured vectors --- which may not always correspond to the
angle the device was actually rotated!  For instance, if you rotate it exactly
around the direction of acceleration (which usually means horizontally), the
measured value \emph{does not change at all} and consequently, \Angulo will
display $\phi = 0$ despite there obviously being a nonzero rotation.  Since
\Angulo actually displays the two angles calculated independently from
the direction of acceleration and magnetic field, one can sometimes mitigiate
this problem by choosing one of them over the other, depending on the axis
of rotation; but it may well be that \emph{none} is correct on its own.
See \secref{combined} for a way to combine both values into a single
measurement that does not have that problem.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Combined Angle}
\label{sec:combined}

As mentioned above, one gets a better estimate on the actual angle if the
measurements of both sensors are combined into a single value --- this is
what \Angulo displays in large, green writing on the bottom of the screen and
the value is supposed to be the most accurate estimate available in most
cases.  This works well as long as the directions of both quantities used
are not linearly dependent --- but since the acceleration is usually pointing
straight through the floor and the magnetic field corresponds to Earth's
magnetic field,
those vectors being linearly dependent means that the magnetic field is
also pointing straight up or down, which in turn only happens at the
magnetic poles.  Thus this situation is quite unlikely, and usually the
the assumption of linear independence is fulfilled.

To make it more precise, let $a$ and $b$ be the values of acceleration and
magnetic field at the ``reference measurement'', respectively.  Let further
$a$ and $b$ be linearly independent.  After the
device is rotated, we assume that the new directions are $R a$ and $R b$, where
$R$ is the three-dimensional rotation matrix corresponding to the movement of
the device.  We assume that the ``current'' vectors can be represented in this
way and clearly $R \in \SO3$.

\subsection{Rotation Angle}
%%%%%%%%%%%%%%%%%%%%%%%%%%%

Assume now that $R$ is known.  If we choose a suitable orthonormal
coordinate system (with the axis of rotation being the third basis vector),
the matrix representation of $R$ will be
\begin{equation*}
R' = \left(\begin{array}{ccc}
       \cos \phi & -\sin \phi & 0 \\
       \sin \phi & \cos \phi & 0 \\
       0 & 0 & 1
     \end{array}\right).
\end{equation*}
Since the trace of a matrix is invariant under basis changes, also
\begin{equation}
\label{eq:rCosine}
\trace R = 1 + 2 \cos \phi.
\end{equation}
It can further be shown (see for instance the thread at
\url{http://www.mathworks.com/matlabcentral/newsreader/view_thread/160945})
that additionally
\begin{equation}
\label{eq:rSine}
\abs{\left(\begin{array}{c}
  R_{32} - R_{23} \\
  R_{13} - R_{31} \\
  R_{21} - R_{12}
\end{array}\right)} = 2 \abs {\sin \phi}
\end{equation}
holds.
Taking \eqref{rCosine} and \eqref{rSine} together, we can again nicely find
$\phi$ itself (up to the sign, but again this is not clearly defined).  For
the sign, the same rule of thumb is used as in \secref{singleValue} --- namely
all (in total six) components of $\cross{a}{Ra}$ and $\cross{b}{Rb}$
are summed up, and the sign of this sum is used to adapt the sign of the
resulting angle.

\subsection{Finding the Matrix}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

It remains now to actually \emph{find} the matrix $R$ from the four
vectors $a, b, Ra, Rb$ that are known.  Note that since $R$ is orthogonal
and in particular regular, $Ra$ and $Rb$ are also linearly independent.
As a first step, we can orthonormalize the vectors $a$ and $b$ and extend
them to a right-handed orthonormal basis $(a', b', c')$ of $\Rthree$:
\begin{equation}
\label{eq:abcBasis}
a' = \frac{a}{\abs{a}}, \;\;
\tilde{b} = b - (\inner{a'}b) \cdot a' \ne 0, \;\;
b' = \frac{\tilde{b}}{\abs{\tilde{b}}}, \;\;
c' = \cross{a'}{b'}
\end{equation}
Let $B$ be the matrix formed by those basis vectors as columns; then
$B \in \SO3$ and applying $B^{-1} = B^\top$ onto a vector represents it
in the basis $(a', b', c')$.

Now, the columns of the representation of $R$ in this basis are simply
the images under $R$ of the basis vectors.  But since $R$ is linear and
also orthogonal, \emph{it preserves norms and inner products}.  Thus,
we get $Ra'$, $Rb'$ and $Rc'$ by simply applying the recipe of
\eqref{abcBasis} onto $Ra$ and $Rb$ in the places of $a$ and $b$.
Let $B'$ be the matrix whose columns are the vectors $Ra'$, $Rb'$ and $Rc'$,
which are known this way.  Then the full matrix is simply:
\begin{equation*}
R = B' B^\top
\end{equation*}
Hence, we know all that is necessary for calculating the angle.

\end{document}


\documentclass[a4paper,10pt]{article}
%\usepackage{palatino}
\usepackage[english]{babel}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{multirow}
\usepackage[below]{placeins}

% Spelling correction
\newenvironment{centre}{\begin{center}}{\end{center}}

% Formatting
\frenchspacing
\dateenglish
\pagestyle{empty}
\setlength{\parindent}{1em}

\setlength{\topmargin}{-10mm}
\setlength{\oddsidemargin}{0mm}
\setlength{\textwidth}{159mm}
\setlength{\textheight}{246mm}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}
\newtheorem{lemma}{Lemma}
\newcommand{\QED}{\hfill$\Box$}
\newenvironment{proof}{\textbf{Proof:}}{\QED}

\newcommand{\Bq}{\mathbb{B}_q}
\newcommand{\Bqx}{\mathbb{B}_q^{(x)}}
\newcommand{\Bqxx}{\mathbb{B}_q^{(x+1)}}

\begin{document}

\section{Introduction}

Consider a collection of containers of positive integral size, $c_1
\le c_2 \le \ldots \le c_n)$, with $c_i \ge 0$, and a process which at
each step of the process fills $s$ distinct containers with an item of
size 1. We denote such a problem with the tuple $(\{c_i\}, n, s)$. Let
$t$ be the number of steps taken before there are fewer than $s$
containers with non-zero capacity. We wish to develop an upper bound,
$t_{\max}$ for $t$, and show that the algorithm in section
\ref{algorithm} achieves that bound.

We denote the remaining capacity of containers at the ``current'' step
as $c_i$, and at the following step as $c'_i$. Values at an arbitrary
step $x$ are denoted $c_i^{(x)}$.

\section{Upper bounds on the number of steps}

A trivial upper bound, $t_0$, on $t$ can be found by observing that
each step of the process reduces the total capacity by $s$, and so
\begin{equation}
  \label{t0-bound}
  t_0 = \left\lfloor \frac{\sum c_i}{s} \right\rfloor
\end{equation}
is an upper bound on $t$.

Now, consider a very unbalanced set of containers, where a few of them
($q$, where $q < s$) are much larger than the rest. If we place an
item into each of the the $q$ large containers on every step, then the
remaining $s-q$ items must be distributed within the $n-q$ small
containers. If the large containers are large enough, we will fill all
of the small containers first. We therefore have a reduced problem of
placing $s-q$ items at a time into $n-q$ containers, with a
corresponding trivial bound:
\begin{equation}
  \label{tq-bound}
  t_q = \left\lfloor \frac{\sum_{q<i \le n} c_i}{s-q} \right\rfloor
\end{equation}

This bound holds only if the ``small'' containers in the partition are
sufficiently small. What counts as sufficiently small? We are placing
items in all of the ``large'' containers at each step, so each of
those containers must be larger than $t_q$. Since the $c_i$ are
ordered, this means that for $t_q$ to be a bound on the number of
steps, the smallest ``large'' container is $c_q$, and therefore $t_q$
is a bound if
\begin{equation}
  \label{tq-condition}
  c_q \ge t_q.
\end{equation}

An upper bound on the number of steps possible is therefore the
minimum of the trivial bound and the valid $q$-bounds:
\begin{equation}
  \label{upper-bound-eqn}
  t_{\max} = \min\left(t_0, \min_{q: q < s, c_q \ge t_q} t_q\right).
\end{equation}

Any process which achieves this bound will have placed $s t_{\max}$
items into the containers, and if the bounding constraint is $t_q$
(for $0 \le q < s$), then every container $c_i$ with $1 \le i \le q$
will have $t_{\max}$ items in it. The other containers, with $q+1 \le
i \le n$, will be filled with $(s-q)t_q$ items between them, leaving
$(\sum_{q<i\le n} c_i)-(s-q)t_q$ capacity unfilled.

\section{Optimal algorithm}
\label{algorithm}

Take the $s$ containers with the largest free capacity. Add one item
to each of those. Repeat until there are fewer than $s$ containers
with available capacity. Believed to be optimal, but not proven.

\end{document}


% Created 2015-09-14 Mon 00:57
\documentclass{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{fixltx2e}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{soul}
\usepackage{textcomp}
\usepackage{marvosym}
\usepackage{wasysym}
\usepackage{latexsym}
\usepackage{amssymb}
\usepackage{hyperref}
\tolerance=1000
\usepackage[margin=18mm]{geometry}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{parskip}
\usepackage{standalone}
\usepackage{tikz,pgf,pgfplots}
\usetikzlibrary{decorations.pathmorphing,patterns}
\usetikzlibrary{arrows,snakes,backgrounds,patterns,matrix,shapes,fit,calc,shadows,plotmarks,decorations.markings,datavisualization,datavisualization.formats.functions,intersections,external}
\usetikzlibrary{decorations.pathmorphing,patterns}
\pgfplotsset{compat=1.9}
\newcommand*{\mexp}[1]{\ensuremath{\mathrm{e}^{#1}}}
\newcommand*{\laplace}[1]{\ensuremath{\mathcal{L} \{#1\}}}
\newcommand*{\laplaceinv}[1]{\ensuremath{\mathcal{L}^{-1} \{#1\}}}
\newcommand*{\realpart}[1]{\ensuremath{\operatorname{Re}(#1)}}
\newcommand*{\impart}[1]{\ensuremath{\operatorname{Im}(#1)}}
\newcommand*{\vsp}[1]{\rule{0pt}{#1}}
\newcommand*{\tderiv}[1]{\ensuremath{\frac{d^{#1}}{dt^{n}}}}
\newcommand*{\bbm}{\begin{bmatrix}}
\newcommand*{\ebm}{\end{bmatrix}}
\newcommand*{\obsmatrix}{\mathcal{O}}
\newcommand*{\contrmatrix}{\mathcal{C}}
\newcommand*{\cwh}{\ensuremath{\cos \omega h}}
\newcommand*{\swh}{\ensuremath{\sin \omega h}}
\newcommand*{\zethree}{\big(z - \mexp{-3h}\big)}
\providecommand{\alert}[1]{\textbf{#1}}

\title{Computerized control - partial exam 1 (dummy)}
\author{Kjartan Halvorsen}
\date{Due 2015-09-18}
\hypersetup{
  pdfkeywords={},
  pdfsubject={},
  pdfcreator={Emacs Org-mode version 7.9.3f}}

\begin{document}

\maketitle

\section*{Problem 1}
\label{sec-1}

Consider the continuous-time system with the following transfer function
\[ G(s) = \frac{s+1}{s(s+3)}. \]
The system is sampled with sampling interval $h$ using zero-order hold. \textbf{Show that the pulse-transfer function for the sampled system is}
\[ H(z) = \frac{(2z-2+3h)\zethree{} - 2(z-1)^2}{9(z-1)\zethree{}}. \]
\section*{Problem 2}
\label{sec-2}

  The sampled system in Problem 1 is controlled using proportional control with gain equal to 1.
  \begin{center}
  \includestandalone[width=0.5\linewidth]{feedback}
  \end{center}

\begin{enumerate}
\item Calculate the closed-loop pulse-transfer function
\item Let $h=\frac{\ln 2}{3} \approx 0.23$. Is the closed-loop system stable?
\end{enumerate}
\section*{Solutions}
\label{sec-3}
\subsection*{Problem 1}
\label{sec-3-1}

   First calculate the step-response of the continous-time system
   \[G(s)\frac{1}{s} = \frac{s+1}{s^2(s+3)} = \frac{2}{9s} + \frac{1}{3s^2} - \frac{2}{9(s+3)}.\]
   The inverse Laplace-transform gives
   \[ y(t) = \frac{2}{9} + \frac{1}{3}t - \frac{2}{9}\mexp{-3t}. \]
   Sampling this function gives
   \[ y(kh) = \frac{2}{9} + \frac{1}{3}kh - \frac{2}{9}\big(\mexp{-3h}\big)^k, \]
   which has the Z-transform
   \[Y(z) = \frac{2z}{9(z-1)} + \frac{hz}{3(z-1)^2} - \frac{2z}{9\zethree{}}. \]
   Dividing the z-transform of the system response to that of the input (the step) gives
   \begin{align*}
   H(z) &= \frac{Y(z)}{U(z)} = \frac{z-1}{z}Y(z) = \frac{2}{9} + \frac{h}{3(z-1)} - \frac{2(z-1)}{9\zethree{}}\\
        &= \frac{2(z-1)\zethree{} + 3h\zethree{} - 2(z-1)^2}{9(z-1)\zethree{}}\\
        &= \frac{(2z-2+3h)\zethree - 2(z-1)^2}{9(z-1)\zethree{}}.
   \end{align*}
\subsection*{Problem 2}
\label{sec-3-2}

\begin{enumerate}
\item The closed loop system becomes
      \begin{align*}
          H_c(z) &= \frac{H(z)}{1+H(z)}\\
                 &= \frac{(2z-2+3h)\zethree{} - 2(z-1)^2}{(2z-2+3h)\zethree{} - 2(z-1)^2+9(z-1)\zethree{}}\\
                 &= \frac{(2z-2+3h)\zethree{} - 2(z-1)^2}{(2z-2+3h+9z-9)\zethree{} - 2(z-1)^2}\\
                 &= \frac{(2z-2+3h)\zethree{} - 2(z-1)^2}{(11z+3h-11)\zethree{} - 2(z-1)^2}
      \end{align*}
\item Stability of the closed-loop system. Substituting $h=\frac{\ln2}{3}$ gives the characteristic equation
      \[ (11z + \ln2 - 11)\big(z-\frac{1}{2}\big) - 2z^2 +4z - 2
           = 9z^2 - (12.5-\ln2)z + 3.5 - 0.5\ln 2 = 0. \]
      Apply Jury's  criterion:

\begin{center}
\begin{tabular}{llll}
 $9$                                   &  $-11.81$                                      &  $3.15$  &                                                \\
 $3.15$                                &  $-11.81$                                      &  $9$     &  $\alpha_2 = \frac{3.15}{9}$                   \\
\hline
 $9-\frac{3.15^2}{9}\approx 7.9$       &  $-11.81 + \frac{3.15}{9}11.81 \approx -7.68$  &          &                                                \\
 $-7.68$                               &  $7.9$                                         &          &  $\alpha_1 = -\frac{7.68}{7.9} \approx -0.97$  \\
\hline
 $7.9 - 0.96 \cdot 7.68 \approx 0.44$  &                                                &          &                                                \\
\end{tabular}
\end{center}

      The first element in each odd row in the table are positive $\Rightarrow$ \textbf{The closed loop system is stable.}

      Step response (not asked for):
      \begin{center}
      \includegraphics[width=0.7\linewidth]{stepresponse}
      \end{center}
\end{enumerate}

\end{document}


[30 min late]
\section{Planck's Law}
Derived Planck's law.

\section{Identical particles}
Suppose you have two particles moving along a line.
As you know, the system only has one wave function,
which is $\psi(x_1,x_2)$,
not a wave function for each but one wave function for all.
Suppose it's one dimensional and there is the kinetic energy operator
\begin{align}
    -\frac{\hbar^2}{2m}\frac{\partial}{\partial x_1}
    -\frac{\hbar^2}{2m}\frac{\partial}{\partial x_2}
\end{align}
but the new thing in QM is that you can have identical particles.
There is nothing you can do to distinguish them.
That has consequences.
Suppose someone exchanged the two particles,
and I don't measure anything else.
Should I obtain any results?
I shouldn't.

So the magnitude squared of the wave function before and after exchange should
be the same.
\begin{align}
    |\psi(x_1, x_2)|^2 &= |\psi(x_2, x_1)|^2
\end{align}
Putting those two things together,
I can have either
\begin{align}
    \psi(x_1, x_2)^2 &= \pm\psi(x_2, x_1)^2
\end{align}
There could be a phase,
but if I exchange the particle twice, I should get the same thing back.

The not obvious thing is that if I have two electrons,
any two electrons at any time,
this sign is a property of the kind of particle.

All particles in the universe are two types.
Either they don't flip the sign,
called bosons,
or they do,
called fermions.

Another interesting fact,
is particles have spin, as you know,
which could be either integer and they're always bosons,
or they could be fermions, spin in half-integers.

Give me examples of bosons?
Photons.
How do we know?

Two photons of the same wavelength,
one particle mode is excited,
and there are states in the magnetic field with an arbitrary number of photons
with the same momentum,
and they are bosons.
Same with phonons.
It could be very complicated.

In fact,
every time you have a state with two fermions,
the composite system is going to be a boson.
Every time you have an even number of fermions you have bosons.

That's why we don't know if you are a fermion or a boson,
because you'd have to know exactly how many particles you have.

All this is obvious.

Let me do an example.

Consider particles in a 3D box.
Let's say we have a single particle
\begin{align}
    \braket{xyz}{\psi} = \psi(x, y, z) =
    \left( \frac{2}{L} \right)^{3/2}
    \sin\left( \frac{n_x \pi x}{L} \right)
    \sin\left( \frac{n_y \pi x}{L} \right)
    \sin\left( \frac{n_z \pi x}{L} \right)
\end{align}
where $n_x,n_y,n_z=1,2,\ldots$.

And the energy is
\begin{align}
    E_{n_x,n_y,n_z} &=
    \frac{\hbar^2}{2mL^2}\left(
        n_x^2 + n_y^2 + n_z^2
    \right)
\end{align}
by dimension analysis.
Dimension analysis is hard to teach.
People don't know.

And so the Hamiltonian can be indexed by
\begin{align}
    H\ket{m} =  \frac{\hbar^2}{2ML^2} m^2 \ket{m}
\end{align}

All I have to do is create single particle states,
and I remember the Hamiltonian is the sum of kinetic energies.
$H=K_1 + K_2$.
Then what happens?

I act with $K_1$ on the first one,
and I leave along the second one.
Then I act $K_2$ on the second one,
and leave alone the first one.
That's really what I mean by $K_1 + K_2$,
a shorthand for
\begin{align}
    H = K_1\otimes I + I\otimes K_2
\end{align}
What's going to happen?

This gives
\begin{align}
    H\ket{m_1}\otimes\ket{m_2} &=
    E_{m_1}\ket{m_1}\otimes\ket{m_2} +
    E_{m_2}\ket{m_1}\otimes\ket{m_2}\\
    &=
    \left( E_{m_1} + E_{m_2} \right)
    \ket{m_1}\otimes\ket{m_2}
\end{align}
We'll continue next time.


\chapter{Approach}\label{chap:approach}
To semantically segment urban scenes and extract predictions for traversability classes, we proposed to use a deep neural network based on the DeepLab v3+ architecture with a loss function inspired by "U-Net: Convolutional Networks for Biomedical Image Segmentation" by Olaf Ronneberger et al.~\cite{unet}.
In this chapter, the architecture used and an explanation of our loss function will be discussed.

\section{Architecture Selection} \label{section:approach-architectureselection}
We came to use the DeepLab v3+ architecture after running these experiments as it outperformed other networks that were tested.
A further discussion of the results can be found in section and the experiments done to select the network can be found in Section \ref{section:experiments-networkevaluation}.

\subsection{Network Architecture}\label{section:approach-networkarchitecture}
We propose to apply the DeepLab v3+ architecture to our stated problem of curb and curb cut segmentation.
The architecture itself is given in \figref{fig:approach-network} with further descriptions in Tables \ref{tab:drn}-\ref{tab:decoder}\footnote{The full implementation written using the PyTorch framework can be found at \url{github.com/yvan674/CurbNet}}.

\input{figures/approach/network}
\input{tables/drn}
\input{tables/bottleneck}
\input{tables/aspp}
\input{tables/decoder}

\section{Loss Function}\label{section:approach-lossfunction}
We chose to use a modified weighted cross entropy loss due to the severe class imbalance.
Using the assumption that all curbs and curb cuts must be located along the perimeter of roads, we used a loss function inspired by the paper "U-Net: Convolutional Networks for Biomedical Image Segmentation," which we call Masked Cross Entropy (MCE) and is defined in \eqref{eq:mce}~\cite{unet}.
The weighted cross entropy loss function was modified to penalize according to the given weights when labeling within a certain border around road classes, which we call the mask $M$.
We define road classes $\text{class}_{\text{road}}$ as all classes which can reasonably be expected to be found on roads, including road, road markings, potholes, etc.
This mask was calculated using a binary dilation on a full $b \times b$ matrix $B$ where $b$ is $0.05 \times \text{image}_{width}$ on the road class, then subtracted by the road class itself.
Thus, this can be formalized as follows:
\begin{align}
	B &= \underbrace{
			\begin{bmatrix}
				1  & \cdots & 1\\
				\vdots &  \ddots & \vdots\\
				1 &  \cdots & 1
				\end{bmatrix}}_{b \text{ columns and } b \text{ rows}} \\
	M &= \left(\text{class}_{\text{road}} \oplus B\right) - \text{class}_{\text{road}}
\end{align}
A visualization of this mask applied to a color street-level image from the Mapillary dataset results in Figure \ref{fig:approach-mask}.
\input{figures/approach/mask}

The value 0.05 was chosen empirically after looking at samples in the dataset and measuring what area around roads are typically curbs.
The road class was simply taken from the ground truth data.
Any labeling outside of $M$ by the network is then given an increased penalty, incentivizing the network to focus labeling around road edges.
We chose to multiply the penalty for areas outside $M$ by a factor of 3.
This can be seen in the following formalization of the loss function we used:
\begin{align}\label{eq:mce}
	\ell_{MCE}(y, \hat{y}) &=\sum_{m}-y_m\log(\hat{y}_m) \cdot d_m\\
	\text{with } d_m &=
	\begin{cases}
	d_m' & \text{if } x \in M\\
	d_m' \cdot 3 & \text{if } x \notin M
	\end{cases}
\end{align}
where $d_m'$ are the user defined weights.
This loss function operates on the assumption that curbs and curb cuts must be located adjacent to roads.
A visualization of the resulting mask $M$ can be seen in Figure \ref{fig:approach-mask}.


\section{*Off-policy Methods with Approximation}

Function approximation turns out to be more difficult in the case of off-policy learning than it is in the on-policy case. This is because both the update target and the state distribution are different between the target and behaviour policies.

\subsection{Semi-gradient Methods}
Semi-gradient methods alter the update target to correspond to the target policy, but do not address the issue of the update distribution. As such, they may diverge in some cases (but they are often successfully used in practice nonetheless).\\

The tabular off-policy algorithms can be applied in this case, where we simply exchange the estimated value arrays for their equivalents under function approximation and incorporate the importance sampling ratio. For instance, the one-step, state-value algorithm is semi-gradient off-policy TD(0). This has the update
\begin{equation}
    \vec{w}_{t+1} = \vec{w}_t + \alpha \rho_t \delta_t \grad \hat{v}(S_t, \vec{w}_t),
\end{equation}
where $\rho_t \doteq \rho_{t:t}$ is per-step importance sampling ratio. The TD error $\delta_t$ is defined as appropriate, with respect to the episodic and discounted reward or the continuing and undiscounted average reward according to the setting. \\

The corresponding one-step algorithm for state-values is semi-gradient Expected Sarsa. In the tabular case we did not use importance sampling for one-step, action-value methods, but with function approximation (and corresponding generalisation) it is not clear that all actions should be weighted equally.\\

We now give some of the multi-step generalisations.

\subsubsection*{$n$-step Semi-Gradient Expected Sarsa}
The update is
\begin{equation}
    \vec{w}_{t+n} = \vec{w}_t + \alpha \rho_{t+1}\cdots\rho_{t+n-1} [G_{t+n} - \hat{q}(S_t, A_t, \vec{w}_{t+n-1})] \grad \hat{q}(S_t, A_t, \vec{w}_{t+n-1})
\end{equation}
where $\rho_k = 1$ when $k \geq T$ and $G_{t:n} = G_t$ when $t+n\geq T$. The returns targets are defined for the episodic case as
\[
    G_{t:t+n} \doteq \sum_{i=0}^{n-1} \gamma^iR_{t+i+1} + \gamma^n \hat{q}(S_{t+n}, A_{t+n}, \vec{w}_{t+n-1})
\]
and in the continuing case as
\[
    G_{t:t+n} \doteq \sum_{i=0}^{n-1} \left(R_{t+i+1} -  \bar{R}_{t+i}\right) + \hat{q}(S_{t+n}, A_{t+n}, \vec{w}_{t+n-1}).
\]

\subsubsection*{$n$-step Tree Backup Algorithm}
The updates are
\begin{align}
    \vec{w}_{t+n} &= \vec{w}_{t+n-1} + \alpha [G_{t:t+n} - \hat{q}(S_t, A_t, \vec{w}_{t+n-1})]\grad \hat{q}(S_t, A_t, \vec{w}_{t+n-1}) \\
    G_{t:t+n} &\doteq \hat{q}(S_t, A_t, \vec{w}_{t-1}) + \sum_{k=t}^{t+n-1} \delta_k \prod_{i=t+1}^k \gamma \pi(A_i \vert{} S_i),
\end{align}
where $\delta_t$ is defined in the episodic case as
\[
    \delta_t \doteq R_{t+1} + \gamma \sum_a \pi(a \vert{} S_{t+1}) \hat{q}(S_{t+1}, a, \vec{w}_t) -  \hat{q}(S_t, A_t, \vec{w}_t)
\]
and in the continuing case as
\[
    \delta_t \doteq R_{t+1}  - \bar{R}_t + \sum_a \pi(a \vert{} S_{t+1}) \hat{q}(S_{t+1}, a, \vec{w}_t) -  \hat{q}(S_t, A_t, \vec{w}_t)
\]

\setcounter{subsection}{2}
\subsection{The Deadly Triad}
Instability and divergence arise whenever we have all three of \emph{function approximation}, \emph{bootstrapping}, \emph{off-policy training}. Each of these has their benefits, so it is unclear which (if any!) is to be given up. Note that the difficulties are not due to uncertainty in the environment (they arise with DP), nor are they due specifically to control or generalised policy iteration.

\begin{description}
    \item[Function Approximation] allows for generalisation, dimensionality reduction and reduction in complexity. Potentially also
    \item[Bootstrapping] Data and computationally efficient. Memory efficient.
    \item[Off-policy Learning] Essential to some use cases (not yet mentioned in this book). Seems important to be able to learn from hypothetical actions.
\end{description}

\subsection{Linear Value-function Geometry}


\chapter{$k$-safe Labeling of Petersen Graph}\label{safeklabelingpetersen}

In 1898, Petersen produced a trivalent graph with no leaves, now
called the Petersen graph~\cite{holton1993petersen}.  In this chapter
we study $k$-safe labeling for the Petersen graph. We also give upper
bound for the span of the Petersen graph. We provide necessary proof
for the upper bound.

\endinput


\section{The \filli algorithm}
\Label{sec:filli}

The \filli  algorithm in the \cxx
Standard Library \cite[\S 28.6.6]{cxx-17-draft} initializes general
sequences with a particular value.
The signature of our modified variant reads:

\begin{lstlisting}[style=acsl-block]

  void fill(value_type* a, size_type n, value_type v);
\end{lstlisting}

\subsection{Formal specification of \filli}

The following listing shows the formal specification of \specref{filli}.
We can express the postcondition of \filli simply by using the overloaded
predicate \logicref{AllEqual}.

\input{Listings/fill.h.tex}

The \inl{assigns}-clauses formalize that \filli modifies only the
entries of the range \inl{a[0..n-1]}.
In general, when more than one \emph{assigns clause} appears
in a function's specification,
it is permitted to modify any of the referenced memory locations.
However, if no \emph{assigns clause} appears at all,
the function is free to modify any memory location, see
\cite[\S 2.3.2]{ACSLSpec}.
To forbid a function to do any modifications outside its
scope, a clause \inl{assigns \\nothing;}
must be used, as we practised
in the example specifications in Chapter~\ref{cha:non-mutating}.

\subsection{Implementation of \filli}

The implementation of \implref{filli} comes with the loop invariant
\inl{constant} expresses that for each iteration the array is
\emph{filled} with the value of \inl{v} up to the index \inl{i} of the iteration.
Note that we use here again the predicate \logicref{AllEqual}.

\input{Listings/fill.c.tex}

%\clearpage


\section{Image Distortion}
\label{sec:image_distortion}

The images being provided by users for text Captchas, makes it impossible to tell if those images are easy to recognize for bots. Therefore, they might be unsuited for being used as a Captcha token, in there original form. In order to complicate the recognition of the Captcha token, the system uses an image distortion algorithm, which is automatically applied to all uploaded text Captcha tokens.

The image distortion algorithm consists of two steps: the drawing of a horizontal line and a wave transformation.

In the first step it places a horizontal line in the middle of the image, which is colored with the dominant color of the whole image. Afterwards this line will be transformed together with the rest of the image.
The algorithm works via the k-means clustering algorithm implemented in the scipy package. The implementation was taken from \href{http://stackoverflow.com/questions/3241929/python-find-dominant-most-common-color-in-an-image}{\underline{here}}.

In the second step, a standard sinus wave is used which will be modified in frequency and amplitude by according modifiers. The frequency as well as the amplitude of the wave, which will be applied, are dependent on the height of the image. Furthermore, the frequency depends on the width of the image so that one wavelength is at least as wide as the image itself. In addition, the frequency and the amplitude are modified by a random value in order make every transformation unique.

\begin{lstlisting}[style=std]
frequency_modifier = Decimal(random.uniform(0.6, 0.8) *
	(width / height))
frequency = frequency_modifier *
	(Decimal(1) / Decimal((width / 2))) * Decimal(math.pi)
\end{lstlisting}

\begin{lstlisting}[style=std]
amplitude_modifier = random.uniform(5, 7)
amplitude = (height / amplitude_modifier)
\end{lstlisting}

%\lstinputlisting[language=Python, firstline=17, lastline=24]{../server/api/captcha_service/image_distortion.py}

The wave is then used to determine an vertical offset for every pixel in the image. Each point then will be shifted either up or down by the offset so that the new y-position is a point on the wave.

\clearpage

\begin{lstlisting}[style=std]
for x in range(width):
	for y in range(height):
		shift = math.sin(frequency * x) * amplitude
		new_y = y + shift
		if new_y < height and new_y > 0:
			output_data[x, y] = input_data[x, new_y]
\end{lstlisting}

Every pixel that would be shifted out of bounds will be cut off. Additionally the pixels, which were located at the bottom and the top of the the original image will be stretched out vertically to fill the space, which was emptied due to the transformation.

\begin{lstlisting}[style=std]
elif shift < 0:
	output_data[x, y] = input_data[x, 0]
elif shift >= 0:
	output_data[x, y] = input_data[x, (height - 1)]
\end{lstlisting}

\clearpage


%==============================================================
% Introduction
\chapter{Introduction}
%==============================================================
\section{Background}
The use of high energy radiation has many wide spread uses. In
particular, radiation in the X ray regime has applications in medicine,
astronomy and in the study of materials.
One way to better understand how energetic radiation such as X rays
interact with matter is to consider the most fundamental physical
process involved. In this case, it is the quantum mechanical scattering
of X ray photons off a single atom.
The theoretical calculation of the scattering form factor allows the
determination of a number of quantities such as the scattering cross
section and the material's attenuation coefficient.

\section{Aims}
The main aim of this project is to compute the form factor for photons
in the X ray regime scattering of small atoms, in particular Hydrogen
and Helium. This is calculated using the relativistic wave-functions as
obtained using the Dirac equation for Hydrogen and a Dirac-Hartree-Fock
numerical approach for Helium. The two components of the form factor
that will be computed are the normal component which has an angular
dependence, and the anomalous component which depends on the energy of
the incident photon.

\section{Aims}
The main aim of this project is to compute the form factor for photons
in the X ray regime scattering of small atoms, in particular Hydrogen
and Helium. This is calculated using the relativistic wave-functions as
obtained using the Dirac equation for Hydrogen and a Dirac-Hartree-Fock
numerical approach for Helium. The two components of the form factor
that will be computed are the normal component which has an angular
dependence, and the anomalous component which depends on the energy of
the incident photon.

\section{Atomic Form Factor}
    \subsection{The Physical Model}
    The scattering process considered consists of a single atom in it's
    ground state described by a relativistic four component Dirac spinor
    $\psi$. A photon of momentum $\mb{k}$ and a polarisation $\mb{\hat{e_n}}$
    scatters off the atom. The two processes considered will be:
    \begin{itemize}
        \item Photoionisation of one of the atom's electrons.
        \item Elastic scattering where the momentum of the scattered
              photon $\mb{k'}$ is the same as the incident photon.
    \end{itemize}

    \subsection{Normal Component}
    The normal component of the atomic form factor shows the angular
    dependence of the scattering process. It can be considered as the
    Fourier transform of the electronic charge distribution and is
    defined as:
    \begin{equation} \label{eq:nff-general}
        f_0(q) = \int e^{i \mb{q} \cdot \mb{r} } \rho(\mb{r}) d\mb{r}
    \end{equation}
    where the momentum transfer is given by $\mb{q} = |\mb{k} - \mb{k'}|$
    and $\rho(\mb{r})$ is the electron charge density. In non
    relativistic quantum mechanics, this is defined as
    $\rho(\mb{r}) = \psi^{*}(\mb{r}) \psi(\mb{r})$ whereas in
    relativistic quantum mechanics it is defined as
    $\rho(\mb{r}) = \psi^{\dagger}(\mb{r}) \psi(\mb{r})$.
    In the case of a spherically symmetric atom, the normal form factor
    is give by:
    \begin{equation} \label{eq:nff-spherical}
        f_0(q) = 4 \pi \int_{0}^{\infty} \rho(r) \frac{\sin (qr)}{q} r \; dr
    \end{equation}

    \subsection{Anomalous Component}
    The anomalous component of the form factor is defined in terms of
    the scattering amplitude. The cases that are considered are the
    excitement of the ground state electron to a higher energy
    unoccupied state and the excitement of the electron to the
    continuum, that is photoionisation.

    If we consider a relativistic four component spinor $\ket{B}$, which
    is solution to a many particle Dirac equation, the second order
    scattering amplitude (S-matrix) is given by:
    \begin{equation}
        A = - \sum_P
            \left[
                \frac{\bra{M} O_f^* \ket{P} \bra{P} O_i \ket{N} }
                     {E_N - E_P + \hbar \omega_i + i0_+}
                +
                \frac{\bra{M} O_i \ket{P} \bra{P} O_f^* \ket{N} }
                     {E_n - E_p - \hbar \omega_f - i0_+}
            \right]
    \end{equation}
    where $\ket{N}$ is the initial state, $\ket{M}$ is the final state,
    and $\ket{P}$ are intermediate states. We have the following
    definitions:
    \begin{itemize}
        \item $\hbar \omega_i$ is the energy of the incident photon
        \item $\hbar \omega_f$ is the energy of the scattered photon
        \item $E_N, E_M, E_P$ are the energies of the initial, final
              and the intermediate states respectively.
        \item $0_+$ is a small positive number.
    \end{itemize}
    The photon absorption operator $O_{i}$ and the photon emission
    operator $O_{f}^{*}$ are defined as:
    \begin{equation}
        O_i = \sum_j q_j \; \mb{\alpha} \cdot \hat{\epsilon}_i \;
              e^{i \mb{k_i} \cdot \mb{r_j} }
    \end{equation}
    \begin{equation}
        O_{f}^{*} = \sum_j q_j \; \mb{\alpha} \cdot \hat{\epsilon}_j \;
            e^{i \mb{k_f} \cdot \mb{r_j} }
    \end{equation}
    with $\mb{\alpha}$ is the Dirac alpha matrix operator and
    $\hat{\epsilon}_k$ is the polarisation of the photon.