\chapter{Derivations in Arithmetic Theories}

When we showed that all general recursive functions are representable
in~$\Th{Q}$, and in the proofs of the incompleteness theorems, we
claimed that various things are provable in $\Th{Q}$ and~$\Th{PA}$. The
proofs of these claims, however, just gave the arguments informally
without exhibiting actual derivations in natural deduction. We provide
some of these derivations in this capter.

For instance, in \olref[inc][req][bre]{lem:q-proves-add} we proved
that, for all $n$ and $m \in \Nat$, $\Th{Q} \Proves (\num{n} +
\num{m}) = \num{n+m}$. We did this by induction on $m$.

\begin{proof}[Proof of {\olref[inc][req][bre]{lem:q-proves-add}}]
Base case: $m = 0$. Then what has to be proved is that, for all $n$,
$\Th{Q} \Proves \num{n} + \num{0} = \num{n+0}$. Since $\num{0}$ is
just $\Obj 0$ and $\num{n+0}$ is $\num{n}$, this amounts to showing
that $\Th{Q} \Proves (\num{n} + \Obj 0) = \num{n}$. The derivation
\begin{prooftree}
  \AxiomC{$\lforall[x][(x + \Obj 0) = x]$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$(\num{n} + \Obj 0) = \num{n}$}
\end{prooftree}
is a natural deduction derivation of $(\num{n} + \Obj 0) = \num{n}$
with one undischarged assumption, and that undischarged assumption is
an axiom of~$\Th{Q}$.

Inductive step: Suppose that, for any $n$, $\Th{Q} \Proves (\num{n} +
\num{m}) = \num{n+m}$ (say, by a derivation $\delta_{n,m}$). We have
to show that also $\Th{Q} \Proves (\num{n} + \num{m+1}) =
\num{n+m+1}$. Note that $\num{m+1} \ident \num{m}'$, and that
$\num{n+m+1} \ident \num{n+m}'$. So we are looking for a derivation of
$(\num{n} + \num{m}') = \num{n+m}'$ from the axioms of~$\Th{Q}$. Our
derivation may use the derivation $\delta_{n,m}$ which exists by inductive
hypothesis.
\begin{prooftree}
  \AxiomC{}
  \RightLabel{$\delta_{n,m}$}
  \DeduceC{$(\num{n} + \num{m}) = \num{n+m}$}
  \AxiomC{$\lforall[x][\lforall[y][(x+y') = (x+y)']]$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$\lforall[y][(\num{n}+y') = (\num{n}+y)']$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$(\num{n}+\num{m}') = (\num{n}+\num{m})'$}
    \RightLabel{\Elim=}
    \BinaryInfC{$(\num{n}+\num{m}') = \num{n+m}'$}
\end{prooftree}
In the last $\Elim=$ inference, we replace the subterm $\num{n} +
\num{m}$ of the right side $(\num{n} + \num{m})'$ of the right premise
by the term $\num{n+m}$.
\end{proof}

In \olref[inc][req][min]{lem:less-zero}, we showed that $\Th{Q} \Proves
\lforall[x][\lnot x < \Obj 0]$. What does an actual derivation look like?

\begin{proof}[Proof of {\olref[inc][req][min]{lem:less-zero}}]
To prove a universal claim like this, we use $\Intro\lforall$, which
requires a derivation of $\lnot a < \Obj 0$. Looking at axiom $!Q_8$,
this means proving $\lnot \exists z (z' + a) = \Obj 0$. Specifically,
if we had a proof of the latter, $!Q_8$ would allow us to prove the
former (recall that $A \liff B$ is short for $(A \lif B) \land (B \lif
A)$.
\begin{prooftree}\footnotesize
  \AxiomC{$\lnot\lexists[z][(z' + a) = \Obj 0]$}
  \AxiomC{$\lforall[x][\lforall[y][(x < y \liff \lexists[z][(z' + x) = y])]]$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$\lforall[y][(a < y \liff \lexists[z][(z' + a) = y])]$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$a < \Obj 0 \liff \lexists[z][(z' + a) = \Obj 0]$}
  \RightLabel{\Elim\land}
  \UnaryInfC{$a < \Obj 0 \lif \lexists[z][(z' + a) = \Obj 0]$}
  \AxiomC{$\Discharge{a < \Obj 0}{1}$}
  \RightLabel{\Elim\lif}
  \BinaryInfC{$\lexists[z][(z' + a) = \Obj 0]$}
  \RightLabel{\Elim\lnot}
  \insertBetweenHyps{\hskip -3em}
  \BinaryInfC{$\lfalse$}
  \DischargeRule{\Intro\lnot}{1}
  \UnaryInfC{$\lnot a<\Obj 0$}
\end{prooftree}
This is a derivation of $\lnot a<\Obj 0$ from $\lnot\lexists[z][(z' +
a) = \Obj 0]$ (and~$!Q_8$); let's call it~$\delta_1$.

Now how do we prove $\lnot\lexists[z][(z' + a) = \Obj 0]$ from the
axioms of~$\Th{Q}$? To prove a negated claim like this, we'd need a
derivation of the form
\begin{prooftree}
  \AxiomC{$\Discharge{\lexists[z][(z' + a) = \Obj 0]}{2}$}
  \DeduceC{$\lfalse$}
  \DischargeRule{\Intro\lnot}{2}
  \UnaryInfC{$\lnot\lexists[z][(z' + a) = \Obj 0]$}
  \end{prooftree}
To get a contradiction from an existential claim, we introduce a
constant~$b$ for the existentially quantified variable~$z$ and use
\Elim\lexists:
\begin{prooftree}
  \AxiomC{$\Discharge{\lexists[z][(z' + a) = \Obj 0]}{2}$}
  \AxiomC{$\Discharge{(b'+a) = \Obj 0}{3}$}
  \RightLabel{$\delta_2$}
    \DeduceC{$\lfalse$}
    \DischargeRule{\Elim\exists}{3}
    \BinaryInfC{$\lfalse$}
  \DischargeRule{\Intro\lnot}{2}
  \UnaryInfC{$\lnot\lexists[z][(z' + a) = \Obj 0]$}
  \end{prooftree}
Now the task is to fill in $\delta_2$, i.e., prove $\lfalse$ from
$(b'+a) = \Obj 0$ and the axioms of~$\Th{Q}$. $Q_2$ says that $\Obj 0$
can't be the successor of some number, so one way of doing that would
be to show that $(b' + a)$ is equal to the successor of some number.
Since that expression itself is a sum, the axioms for addition must
come into play. If $\eq[a][\Obj 0]$, $Q_5$ would tell us that $\eq[(b'
+ a)][b']$, i.e., $b' + a$ is the successor of some number, namely
of~$b$. On the other hand, if $\eq[a][c']$ for some $c$, then
$\eq[(b'+a)][(b'+c')]$ by \Elim\eq, and $\eq[(b'+c')][(b'+c)']$
by~$Q_6$. So again, $b'+a$ is the successor of a number---in this
case, $b'+c$. So the strategy is to divide the task into these two
cases. We also have to verify that $\Th{Q}$ proves that one of these
cases holds, i.e., $\Th{Q} \Proves a = 0 \lor \lexists[y][(a = y')]$,
but this follows directly from $Q_3$ by \Elim\lforall. Here are the
two cases:

Case 1: Prove $\lfalse$ from $\eq[a][\Obj 0]$ and $\eq[(b'+a)][\Obj
  0]$ (and axioms $Q_2$, $Q_5$):
\begin{prooftree}\footnotesize
  \AxiomC{$\lforall[x][\lnot \Obj 0 = x']$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$\lnot \Obj 0 = b'$}
  \AxiomC{$\lforall[x][(x+\Obj 0) = x]$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$(b' + \Obj 0) = b'$}
  \AxiomC{$a = \Obj 0$}
  \AxiomC{$(b'+a) = \Obj 0$}
  \RightLabel{\Elim=}
  \BinaryInfC{$(b' + \Obj 0) = \Obj 0$}
  \doubleLine
  \UnaryInfC{$\Obj 0 = (b' + \Obj 0)$}
      \insertBetweenHyps{\hskip -.5em}
  \RightLabel{\Elim=}
  \BinaryInfC{$\Obj 0 = b'$}
  \RightLabel{\Elim\lnot}
  \BinaryInfC{$\lfalse$}
\end{prooftree}
Call this derivation~$\delta_3$. (We've abbreviated the derivation of
$\Obj 0 = (b' + \Obj 0)$ from $(b' + \Obj 0) = \Obj 0$ by a double
inference line.)

Case 2: Prove $\lfalse$ from $\lexists[y][a = y']$ and
$\eq[(b'+a)][\Obj 0]$ (and axioms $Q_2$, $Q_6$). We first show how to
derive $\lfalse$ from $\eq[a][c']$ and $\eq[(b'+a)][\Obj 0]$.
\begin{prooftree}\footnotesize
  \AxiomC{$\lforall[x][\lnot \Obj 0 = x']$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$\lnot \Obj 0 = (b'+c)'$}
  \AxiomC{$a = c'$}
  \AxiomC{$(b'+a) = \Obj 0$}
  \RightLabel{\Elim=}
          \insertBetweenHyps{\hskip -.3em}
  \BinaryInfC{$\Obj (b'+c') = \Obj 0$}
  \AxiomC{$\lforall[x][\lforall[y][(x+y') = (x+y)']]$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$\lforall[y][(b'+y') = (b'+y)']$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$(b' + c') = (b'+c)'$}
  \RightLabel{\Elim=}
        \insertBetweenHyps{\hskip .5em}
  \BinaryInfC{$\Obj 0 = (b' + c)'$}
  \RightLabel{\Elim\lnot}
          \insertBetweenHyps{\hskip -.5em}
  \BinaryInfC{$\lfalse$}
\end{prooftree}
Call this $\delta_4$. We get the required derivation $\delta_5$ by
applying $\Elim\lexists$ and discharging the assumption $\eq[a][c']$:
\begin{prooftree}
  \AxiomC{$\lexists[y][a=y']$}
  \AxiomC{$\Discharge{a = c'}{6} \quad \eq[(b'+a)][\Obj 0]$}
  \RightLabel{$\delta_4$}
  \DeduceC{$\lfalse$}
  \DischargeRule{\Elim\exists}{6}
  \BinaryInfC{$\lfalse$}
\end{prooftree}

Putting everything together, the full proof looks like this:
\begin{prooftree}\footnotesize
  \AxiomC{$\Discharge{\lexists[z][(z' + a) = \Obj 0]}{2}$}
  \AxiomC{$\begin{gathered}
  \lforall[x][(x = 0 \lor {}]\\
  \lexists[y][(a = y')])
  \end{gathered}$}
  \RightLabel{\Elim\lforall}
  \UnaryInfC{$\begin{gathered}a = 0 \lor {}\\
  \lexists[y][(a = y')]
  \end{gathered}$}
  \AxiomC{$\begin{gathered}[b]
      \Discharge{a = \Obj 0}{7} \\
      \Discharge{(b'+a) = \Obj 0}{3}
      \end{gathered}$}
    \RightLabel{$\delta_3$}
    \DeduceC{$\lfalse$}
    \AxiomC{$\begin{gathered}[b]
        \Discharge{\lexists[y][a=y']}{7} \\
        \Discharge{(b'+a) = \Obj 0}{3}
        \end{gathered}$\quad}
    \RightLabel{$\delta_5$}
    \DeduceC{$\lfalse$}
    \DischargeRule{\Elim\lor}{7}
    \insertBetweenHyps{\hskip -.5em}
    \TrinaryInfC{$\lfalse$}
    \RightSubproofLabel{$\delta_2$}
    \DischargeRule{\Elim\exists}{3}
    \BinaryInfC{$\lfalse$}
  \DischargeRule{\Intro\lnot}{2}
  \UnaryInfC{$\lnot\lexists[z][(z' + a) = \Obj 0]$}
  \RightLabel{$\delta_1$}
  \DeduceC{$\lnot a<\Obj 0$}
  \RightLabel{\Intro\lforall}
  \UnaryInfC{$\lforall[x][\lnot x < \Obj 0]$}
\end{prooftree}
\end{proof}

In the proof of \olref[inc][inp][ros]{thm:rosser}, we defined
$\ORProv(y)$ as \[\lexists[x][(\OPrf(x, y) \land \lforall[z][(z < x
    \lif \lnot \ORefut(z, y))])].\] $\OPrf(x,y)$ is the formula
representing the proof relation of~$\Th{T}$ (a consistent,
axiomatizable extension of~$\Th{Q}$) in $\Th{Q}$, and $\ORefut(z, y)$
is the formula representing the refutation relation. That means that
if $n$ is the G\"odel number of a proof of~$!A$, then $\Th{Q} \Proves
\OPrf(\num{n}, \gn{!A})$, and otherwise $\Th{Q} \Proves
\lnot\OPrf(\num{n}, \gn{!A})$. Similarly, if $n$ is the G\"odel number
of a proof of $\lnot !A$, then $\Th{Q} \Proves \ORefut(\num{n},
\gn{!A})$, and otherwise $\Th{Q} \Proves \lnot\ORefut(\num{n},
\gn{!A})$. We use the Diagonal Lemma to find a sentence $!R$ such that
$\Th{Q} \Proves !R \liff \lnot \ORProv(\gn{!R})$. Rosser's Theorem
states that $\Th{T} \Proves/ !R$ and $\Th{T} \Proves/ \lnot !R$. Both
claims were proved indirectly: we show that if $\Th{T} \Proves !R$,
$\Th{T}$ is inconsistent, i.e., $\Th{T} \Proves \lfalse$, and the same
if $\Th{T} \Proves \lnot !R$.

\begin{proof}[Proof of {\olref[inc][inp][ros]{thm:rosser}}]
First we prove something things about~$<$. By
\olref[inc][req][min]{lem:less-nsucc}, we know that $\Th{Q} \Proves
\lforall[x][(x < \num {n+1} \lif (\eq[x][\Obj 0] \lor \dots \lor
  \eq[x][\num n]))]$ for every~$n$. So of course also (if $n>1$),
$\Th{Q} \Proves \lforall[x][(x < \num {n} \lif (\eq[x][\Obj 0] \lor
  \dots \lor \eq[x][\num{n-1}]))]$. We can use this to derive
$\eq[a][\Obj 0] \lor \dots \lor \eq[a][\num{n-1}]$ from $a < \num{n}$:
\begin{prooftree}
  \AxiomC{$a < \num{n}$}
  \AxiomC{}
  \DeduceC{$\lforall[x][(x < \num{n} \lif (x = \num{0} \lor
      \dots \lor x = \num{n-1}))]$}
  \RightLabel{\Elim\forall}
  \UnaryInfC{$a < \num{n} \lif (a = \num{0} \lor
      \dots \lor a = \num{n-1})$}
  \RightLabel{\Elim\lif}
  \BinaryInfC{$a = \num{0} \lor \dots \lor a = \num{n-1}$}
\end{prooftree}
Let's call this derivation $\lambda_1$.

Now, to show that $\Th{T} \Proves/ !R$, we assume that $\Th{T}
\Proves !R$ (with a derivation~$\delta$) and show that $\Th{T}$ then
would be inconsistent. Let $n$ be the G\"odel number
of~$\delta$. Since $\OPrf$ represents the proof relation in~$\Th{Q}$,
there is a derivation~$\delta_1$ of $\OPrf(\num{n},
\gn{!R})$. Furthermore, no $k < n$ is the G\"odel number of a
refutation of~$!R$ since $\Th{T}$ is assumed to be consistent, so for
each $k < n$, $\Th{Q} \Proves \lnot \ORefut(\num{k}, \gn{!R})$; let
$\rho_k$ be the corresponding derivation. We get a derivation of
$\ORProv(\gn{!R})$:
\begin{prooftree}\footnotesize
  \AxiomC{}
  \RightLabel{$\delta_1$}
  \DeduceC{$\OPrf(\num{n}, \gn{!R})$}

  \AxiomC{$\Discharge{a < \num{n}}{1}$}
  \RightLabel{$\lambda_1$}
  \DeduceC{$\begin{gathered}[b]
      a= \num{0} \lor \dots {} \\
      {} \lor a = \num{n-1}
      \end{gathered}$}
  \AxiomC{$\dots$}

  \AxiomC{$\Discharge{a=\num{k}}{2}$}
  \AxiomC{}
  \RightLabel{$\rho_k$}
  \DeduceC{$\lnot \ORefut(\num{k}, \gn{!R})$}
  \RightLabel{\Elim=}
  \BinaryInfC{$\lnot \ORefut(a, \gn{!R})$}
  \AxiomC{$\dots$}
  \DischargeRule{$\Elim\lor^*$}{2}
  \doubleLine
  \insertBetweenHyps{\hskip -1pt}
  \QuaternaryInfC{$\lnot \ORefut(a, \gn{!R})$}
  \DischargeRule{\Intro\lif}{1}
  \UnaryInfC{$a < \num{n} \lif \lnot \ORefut(a, \gn{!R})$}
  \RightLabel{\Intro\lforall}
  \UnaryInfC{$\lforall[z][(z < \num{n} \lif \lnot \ORefut(z, \gn{!R}))]$}

    \insertBetweenHyps{\hskip -5pt}
  \RightLabel{\Intro\land}
  \BinaryInfC{$\OPrf(\num{n}, \gn{!R}) \land \lforall[z][(z < \num{n} \lif \lnot \ORefut(\num{z}, \gn{!R}))]$}
  \RightLabel{\Intro\lexists}
  \UnaryInfC{$\lexists[x][(\OPrf(x, \gn{!R}) \land \lforall[z][(z < x \lif \lnot \ORefut(z, \gn{!R}))])]$}
\end{prooftree}
(We abbreviate multiple applications of $\Elim\lor$ by $\Elim\lor^*$
above.)  We've shown that if $\Th{T} \Proves !R$ there would be a
derivation of~$\ORProv(\gn{!R})$.  Then, since $\Th{T} \Proves R \liff
\lnot \ORProv(\gn{!R})$, also $\Th{T} \Proves \ORProv(\gn{!R}) \lif
\lnot R$, we'd have $\Th{T} \Proves \lnot !R$ and $\Th{T}$ would be
inconsistent.

Now let's show that $\Th{T} \Proves/ \lnot !R$. Again, suppose it
did. Then there is a derivation $\rho$ of $\lnot !R$ with G\"odel
number $m$---a refutation of~$!R$---and so $\Th{Q} \Proves
\ORefut(\num{m}, \gn{!R})$ by a derivation~$\rho_1$. Since we assume
$\Th{T}$ is consistent, $\Th{T} \Proves/ !R$. So for all $k$, $k$ is
not a G\"odel number of a derivation of~$!R$, and hence $\Th{Q} \Proves \lnot
\OPrf(\num{k}, \gn{!R})$ by a derivation~$\pi_k$. So we have:

\begin{prooftree}\footnotesize
  \AxiomC{}
  \RightLabel{$\lambda_2$}
  \DeduceC{$\begin{gathered}[b] a = \num{0} \lor \dots \lor\\
      a = \num{m} \lor \num{m} < a\end{gathered}$}
  \AxiomC{$\dots$}
  \AxiomC{$\begin{gathered}\Discharge{\OPrf(a, \gn{!R})}{1}\\
    \Discharge{a = \num{k}}{2}\end{gathered}$}
  \RightLabel{$\pi_k'$}
  \DeduceC{$\lfalse$}
  \RightLabel{$\lfalse_I$}
  \UnaryInfC{$\num{m}<a$}
  \AxiomC{$\dots$}
  \AxiomC{$\Discharge{\num{m} < a}{2}$}
  \DischargeRule{$\Elim\lor^*$}{2}
  \insertBetweenHyps{\hspace{-.1em}}
  \doubleLine
  \QuinaryInfC{$\num{m} < a$}
  \AxiomC{}
  \RightLabel{$\rho_1$}
  \DeduceC{$\ORefut(\num{m}, \gn{!R})$}
  \RightLabel{\Intro\land}
    \insertBetweenHyps{\hspace{-.1em}}
  \BinaryInfC{$\num{m}
    < a \land \ORefut(\num{m}, \gn{!R})$}
  \RightLabel{\Intro\exists}
  \UnaryInfC{$\exists z(z
    < a \land \ORefut(z, \gn{!R}))$}
  \DischargeRule{\Intro\lif}{1}
  \UnaryInfC{$\OPrf(a, \gn{!R}) \lif \exists z(z
    < a \land \ORefut(z, \gn{!R}))$}
  \RightLabel{\Intro\forall}
  \UnaryInfC{$\forall x(\OPrf(x, \gn{!R}) \lif \exists z(z
    < x \land \ORefut(z, \gn{!R})))$}
  \DeduceC{$\lnot\exists x(\OPrf(x, \gn{!R}) \land \forall z(z
    < x \lif \lnot \ORefut(z, \gn{!R})))$}
\end{prooftree}
where $\pi_k'$ is the derivation
\begin{prooftree}
  \AxiomC{}
  \RightLabel{$\pi_k$}
  \DeduceC{$\lnot \OPrf(\num{k}, \gn{!R})$}
  \AxiomC{$a = \num{k}$}
  \AxiomC{$\OPrf(a, \gn{!R})$}
  \RightLabel{\Elim=}
  \BinaryInfC{$\OPrf(\num{k}, \gn{!R})$}
  \RightLabel{\Elim\lnot}
  \BinaryInfC{$\lfalse$}
\end{prooftree}
and $\lambda_2$ is
\begin{prooftree}\footnotesize
  \AxiomC{}
  \RightLabel{$\lambda_3$}
  \DeduceC{$\begin{gathered}[b](a < \num{m} \lor {}\\a = \num{m}) \lor {}\\ \num{m} < a\end{gathered}$}

  \AxiomC{$\Discharge{a < \num{m}}{3}$}
  \RightLabel{$\lambda_1$}
  \DeduceC{$\begin{gathered}[b]a = \num{0} \lor \dots \lor {}\\
  a = \num{m-1}\end{gathered}$}
  %\RightLabel{\Intro\lor}
  \doubleLine
  \UnaryInfC{$\begin{gathered}[b]
      a = \num{0} \lor \dots \lor \\
      a = \num{m} \lor \num{m} < a
    \end{gathered}$}

  \AxiomC{$\Discharge{a = \num{m}}{3}$}
  %\RightLabel{\Intro\lor}
  \doubleLine
  \UnaryInfC{$\begin{gathered}[b]
      a = \num{0} \lor \dots \lor \\
      a = \num{m} \lor \num{m} < a
    \end{gathered}$}

  \AxiomC{$\Discharge{\num{m} < a}{3}$}
  \doubleLine
  \RightLabel{$\Intro\lor^*$}
  \UnaryInfC{$\begin{gathered}[b]
      a = \num{0} \lor \dots \lor \\
      a = \num{m} \lor \num{m} < a
    \end{gathered}$}

  %\insertBetweenHyps{\hskip 5pt}
  \doubleLine
  \DischargeRule{$\Elim\lor^2$}{3}
  \QuaternaryInfC{$a = \num{0} \lor \dots \lor a = \num{m} \lor \num{m} < a$}
\end{prooftree}
(The derivation $\lambda_3$ exists by
\olref[inc][req][min]{lem:trichotomy}. We abbreviate repeated use of
$\Intro\lor$ by $\Intro\lor^*$ and the double use of $\Elim\lor$ to
derive $a = \num{0} \lor \dots \lor a = \num{m} \lor \num{m} < a$ from
$(a < \num{m} \lor a = \num{m}) \lor \num{m} < a$ as $\Elim\lor^2$.)
\end{proof}

\OLEndChapterHook


\subsection{Memory}
We start by defining a memory typeclass, which will define the basic environment for our computations. We model our memory after a stack for simplicity. The memory predicate is defined as follows:

\begin{lstlisting}
class Memory m where
  data Malloc :: * -> * -> *
  malloc :: m -> a -> Malloc a m
  free   :: Malloc a m -> m
  read   :: Malloc a m -> a
  write  :: a -> Malloc a m -> Malloc a m
\end{lstlisting}

Our memory is characterized by three operators:
\begin{itemize}
\item a type function that takes our memory and another type as input and returns the new memory obtained by pushing the second parameter
on top of the input stack
\item a $malloc$ function which adds a value to an existing memory
\item a $free$ function which removes the last allocated value from our memory
\end{itemize}

The $null$ value is a memory where all the values are null ($undefined$). This value can be used to represent a starting memory with a certain size.

\subsection{References and Statements}
We will never work directly with values, since what we are trying to accomplish requires that values are packed inside "smart" containers that are capable of doing more complex operations such as mutating a shared state, sending messages to other processes or tracking dependencies from other smart values to implement reactive updates. For this reason we will represent values in two different ways:
\begin{itemize}
\item as references whenever we wish to represent a pointer to some value inside the current memory
\item as statements whenever we wish to represent the result of arbitrary computations
\end{itemize}

References and statements are defined respectively with the $State$ predicate:

\begin{lstlisting}
class (Memory m,Monad (st m m)) => State st m where
  data Ref st :: * -> * -> *
  eval :: Ref st m a -> st m m a
  (.=) :: Ref st m a -> a -> st m m ()
  delete :: st (Malloc a m) m ()
  new :: a -> st m (Malloc a m) (Ref st (Malloc a m) a)
  (>>+) :: (Memory m', Memory m'') => st m m' a -> (a -> st m' m'' b) -> st m m'' b
\end{lstlisting}

We represent state in a more complete fashion than is usually done. Usually stateful computations are represented with the type of the denotational semantics of a stateful statement where the state is the same at the beginning and end of the evaluation of the statement. In our case a stateful computation has different input and output types for the state, since memory manipulation changes the type of the state.

The $st$ functor applied to the memory $m$ twice is required by this definition to be a monad; thanks to this we can use our operators on references taking advantage of the syntactic sugar that Haskell offers, obtaining code that is much more intuitive to a programmer used to traditional object oriented languages. Since references are strictly linked to their kind of state, we define a type function $Ref$ which returns the type of references associated to state $st$.

The $new$ and $delete$ operators respectively add and remove a single value from our memory.

To concatenate regular statements and state transition statements we define the $(>>+)$ operator, which is a generalized binding operator: $(>>=)$ is defined in terms of $(>>+)$ for the state monad, since $(>>=)$ simply imposes the constraint that both parameters of $st$ are the same.

In our examples, we will use syntactic sugar that is not available in Haskell to make using the $(>>+)$ operator transparent; we call this the $do+$ notation.

(*=) :: State st s => Ref st s a -> (a->a) -> st s s ()
ref *= f = do v <- eval ref
              ref .= (f v)

Before Giuseppe's PhD/Monads/ObjectiveMonad/MonadicObjects/trunk/tex v2/2.stack_ref_and_state.tex


\subsection{Introduction}

\subsection{Input layer}

The input is the feature vector.

\subsection{Pattern layer}

The first layer has a node for each observation in the training set.

In each node, the value is the distance from the input to the comparator.

This can be calculated using Gaussian distribution, or another method.

\subsection{Summation layer}

One neuron for each category.

We map from the pattern layer to the summation layer according to the actual label of each training item.

Ie, if a sample is red, it will be fed only to the red neuron.

The values are summed.

Largest value is selected.


\chapter{Optimal Strategies}

After examining a variety of heuristic strategies that perform variably under different configuration of rules, a natural question arise: for a given set of rules, does there exist an \emph{optimal} strategy? The answer is ``yes''. In this chapter, we will explore several techniques that enable us to find such a strategy.

\section{Overview}

An \emph{optimal strategy} is a strategy that achieves a certain objective in an optimal manner. In this chapter, we will focus on the following three objectives, in order of  strength of optimality:\footnote{In each objective presented, we could replace ``reveal'' with ``determine'', which are subtly different from an information perspective (see 2.4). The overall methodology remains the same. Therefore in this chapter we will focus on strategies that aim to reveal the secret.}

\emph{Min-Steps} objective. To minimize the average number of guesses needed to reveal a secret, assuming each codeword is equally likely to be the secret. This is equivalent to minimizing the total number of guesses needed to reveal all secrets.

\emph{Min-Depth} objective. Subject to being optimal in \emph{Min-Steps} sense, also minimize the maximum number of guesses required to reveal any single secret.

\emph{Min-Worst} objective. Subject to being optimal in \emph{Min-Depth} sense, also minimize the number of secrets that are revealed using the maximum number of guesses.

Note that there may be other objectives for a strategy, such as minimizing the number of guesses evaluated. See, for example, Temporel and Kovacs (2003). However, those objectives are not studied in this chapter.

Optionally, additional constraints could be enforced for an optimal strategy. Commonly used constraints include:

\emph{Max-Depth} constraint. This constraint limits the maximum number of guesses allowed to reveal any single secret. For certain configuration of rules, this constraint does have a meaningful impact on the result. See Koyama(93) for an example.

\emph{Possibility-Only} constraint. This constraint requires all guesses to be made from the remaining possibilities. This significantly reduces the complexity of searching at the cost of yielding sub-optimal solutions, and was employed in 1980's when computation power was limited. For example, see \cite{neuwirth81}.

The theory to find an optimal strategy is simple. Since the number of possible secrets as well as sequence of (non-redundant) guesses are finite, the code breaker can employ an exhaustive search to find out the optimal guessing strategy. However, the scale of the problem is very big and many optimization techniques are required to achieve an efficient (or even feasible) implementation. These technique are discussed in this chapter.

[could copy Neuwirth's illustration of finding an optimal strategy]

[show the search scale of the problem]

[show that an optimal strategy is not unique]

\section{Obvious Guesses}

Some definitions. Partitions, feedback count, etc.

While finding an optimal strategy for the general game is complex, in certain cases it's easy. For example, when there's only one possibility left, we should guess it. When there are only two possibilities left, we should guess (either) one of them. [these appear in Neuwirth 82]. When there are more than two possibilities, it's still possible.

An obviously-optimal guess is an optimal guess that doesn't require too much effort to identify. Depending on the techniques used to identify such, the "obvious"-ty could vary. Here we use the technique introduced by \cite{koyama93}.

[Definition.] An obviously-optimal guess is a guess that partitions the remaining possibilities into discrete cells, i.e.\ where every cell contains exactly one element.

If such a guess exists and comes from the possibility set, then it is optimal because it reveals one potential secret (itself) in the immediate step and reveals all the other potential secrets in two steps. It is easy to see that no other strategy could do better. If no such guess exists in the possibility set but one exists outside the possibility set, then that one is optimal because it reveals all secrets in two steps.

Note that an obviously-optimal guess is fairly generic about the goal -- it is optimal both in terms of the worst-case number of steps and the expected number of steps to determine or reveal the secret.

A necessary condition for an obviously-optimal guess to exist is that the number of remaining possibilities does not exceed the number of distinct feedbacks. For a game with $p$ pegs, the number of distinct feedbacks is $p(p+3)/2$. For example, in a four-peg game, there can be at most 14 secrets left for an obviously-optimal guess to exist. This is a useful check in practice to reduce unnecessary efforts to search for an obviously optimal guess.

Note also that in practice we may only want to check in the remaining possibilities (so that the effort is minimized). In turns out that if we check outside the remaining possibilities, it is equivalent to a full-run of a heuristic function, as we show below.

It turns out (not so surprisingly) that the heuristics introduced in the previous chapter will yield an obviously-optimal guess when one exists. We only need to show that the partition of an obviously-optimal guess (which we will call an \emph{obviously optimal partition} and denote by $Q$ below) achieves the lowest possible heuristic value.

Let $P$ denote any given partition. Let $k$ denote the number of (non-empty) cells in $P$. Let $n_i$ denote the number of elements in the $i$-th cell. Let $n = \sum_{i=1}^k$ denote the total number of elements, which is invariant across different $P$. Finally, let $Q$ denote the partition of an obviously-optimal guess, i.e. one with all singleton cells.

\paragraph{Min-Max}
The \minmax{} heuristic value of an obviously optimal partition is one. This is the minimum value of the heuristic function.
\[
h(P) = \max_{1 \le i \le n} n_i \ge 1 = h(Q).
\]

\paragraph{Min-Avg}
The \minavg{} heuristic value of an obviously optimal partition is one. This is the minimum value of the function.
\[
h(P) = \sum_{i=1}^k \frac{n_i}{n} n_i \ge \min_{1 \le i \le k} n_i \ge 1 = h(Q).
\]

\paragraph{Max-Entropy}
The \maxent{} heuristic value of an obviously optimal partition is ?. This is the maximum value of the function.
\[
h(P) = - \sum_{i=1}^k \frac{n_i}{n} \log \frac{n_i}{n} = ?
\]

\paragraph{Max-Parts}
The \maxpar{} heuristic value of an obviously optimal partition is $n$. This is the maximum value of the function.
\[
h(P) = k \le n = h(Q).
\]

Since all four heuristics introduced yield an obviously optimal guess when one exists, we can insert the step to find an optimal guess into the strategy as a shortcut to save computation time, knowing that this will not alter the output of the heuristic strategy.

\section{Less Obvious Guesses}

\section{Search space pruning}

Two techniques are important in reducing the search space: visit candidate guesses in order of their lower bound, and visit partitions in order of their size.

\section{Other techniques}

(e.g. two-phase optimization, hash collision group)

\section{Using a pre-built strategy tree}

\section{Extended/Adaptive strategy tree}

i.e. the tree not only contains guesses along the chosen strategy path, but also includes guesses if the user made a non-optimal guess halfway. The tree size in this case is much larger, and we must use isomorphism to detect the symmetry.


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 10, Section 2.1; 6, 19–23, 27, 28
 }
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{6.}
$\begin{bmatrix} -5&22 \\ 12&-22 \\ 3&-2   \end{bmatrix}$\\
\subsection*{19.}
$ Ab_3 = ab_1 + Ab_2 $ since $A = A.$Therefore $b_3 = b_1 + b_2$
\subsection*{20.}
$if Ab_1 = Ab_2, $ then if you divide both sides by A then $b_1 = b_2$
\subsection*{21.}
Let bp be the last column of $B$. By hypothesis, the last column of $AB$ is zero. Thus, $Ab_p = 0$. However, $b_p$ is not the zero vector, because $B$ has no column of zeros. Thus, the equation $Ab_p = 0$ is a linear dependence relation among the columns of A, and so the columns of A are linearly dependent.
\subsection*{22.}
If the columns of $B$ are linearly dependent, then there exists a nonzero vector x such that $B_x = 0$. From this, $A(Bx) = A0$ and $(AB)x = 0$ (by associativity). Since x is nonzero, the columns of $AB$ must be linearly dependent.
\subsection*{23.}
If $x$ satisfies $Ax=0$, then $CAx=C0=0$ and so $I_nx=0$ and $x=0$.This means that the equation $Ax=0$ has no free variables so every variable is a basic variable and every column of A is a pivot column meaning each pivot is in a different row, A must have at least as many rows as columns.
\subsection*{27.}
$uv^T = \begin{bmatrix} -2a&-2b&-2c \\ 3a&3b&3c \\ -4a&-4b&-4c   \end{bmatrix}$\\
$vu^T = \begin{bmatrix} -2a&3a&-4a \\ -2b&3b&-4b \\ -2c&3c&-4c   \end{bmatrix}$\\
\subsection*{28.}
By Theorem 3, $(uv^T)^T = (v^T)^Tu^T = vu^T.$

\end{document}


\subsection{Example}\label{sec:set-example}

TK - Introduce, 2D identity map, known observed (square in center with area 1/100, i.e. density value of 100).

\begin{python}
"""
Set up and solve problem with identity map
"""
# import libraries
import bet.sample as sample
import bet.sampling.basicSampling as bsam
import bet.calculateP.simpleFunP as simpleFunP
import bet.calculateP.calculateP as calculateP
import numpy as np
import scipy.stats as sstats
# define input space parameters and model to instantiate sampler object
dimension = 2
numSamples = 100
I = np.eye(dimension)
def model(input_samples):
        return (I@input_samples.T).T
sampler = bsam.sampler(model)
# instantiate objects that hold input/output samples
# default random sample set is uniform over unit domain (normalized space)
input_set = input_samples = bsam.random_sample_set('r',input_obj=dimension, num_samples=numSamples)
param_ref = np.array([0.5, 0.5])
input_set.set_reference_value(param_ref)
# Estimate volumes of Voronoi cells associated with the parameter samples
if MC_assumption is False:
    input_samples.estimate_volume(n_mc_points=5E4)
else:
    input_samples.estimate_volume_mc()

# input_set = bsam.regular_sample_set(input_obj=dimension, num_samples_per_dim=49)
disc = sampler.compute_QoI_and_create_discretization(input_sample_set=input_set)
Qref = disc.get_output().get_reference_value()
print('Reference Value:', param_ref, 'maps to', Qref)
# define inverse problem
disc_1 = disc.copy()
simpleFunP.regular_partition_uniform_distribution_rectangle_size(
        data_set=disc_1, Q_ref=Qref, rect_size=0.2,
        cells_per_dimension = 1)
calculateP.prob(disc_1)
# compare with higher-fidelity discretization of output space
disc_2 = disc.copy()
simpleFunP.regular_partition_uniform_distribution_rectangle_size(
        data_set=disc_2, Q_ref=Qref, rect_size=0.2,
        cells_per_dimension = 2)
calculateP.prob(disc_2)
\end{python}

Note that there is no need to explictly call {\tt disc.compute\_pushforward()}, (or \pythoninline{disc.compute_predicted()}) since it is computed automatically if none have been previously constructed.
When \pythoninline{disc.updated_pdf()} is called, densities are evaluated at the initial set of $\nsamps$ random samples, and stored in \pythoninline{disc._input_sample_set._densities}.
However, the function \pythoninline{disc.predicted_pdf()} is capable of evaluating the solution at any new set of samples (provided a model is available/equipped to the discretization), something we leverage for plotting on a regular grid.

Once our four discretization objects \pythoninline{disc}, \pythoninline{disc_a}, \pythoninline{disc_b} and \pythoninline{disc_c} have been generated, we can use some utility plotting functions to compare the densities:

\begin{python}
"""
Plotting code to generate figures.
"""
# define plotting parameters
nbins = 50
xmn, xmx = 0.25, 0.75
ymn, ymx = 0.25, 0.75
xi, yi = np.mgrid[xmn:xmx:nbins*1j, ymn:ymx:nbins*1j]
# plotting functions computes nearest-neighbors to
# the regular grid of samples.
plot_2d_comparison(xi, yi, disc_1, disc_2,
                   '$M=1, N=%d$'%(numSamples),
                   '$M=4, N=%d$'%(numSamples))
\end{python}

\begin{figure}[ht]
\begin{minipage}{.975\textwidth}
  \includegraphics[width=\linewidth]{./examples/identity/set/M1-N100_N100-vs-M4-N100_N100.png}
\end{minipage}
\caption{
$\nsamps=100$ were used to discretize $\pspace$ and $\ndiscs=1, 4$ (left/right) were used to discretize $\dspace$.
The latter was chosen to visualize the geometry of describing $\pspace$ with $\pborel$.
}
\label{fig:ex:identity_set_1E2}
\end{figure}

We have chosen a uniform density to describe the uncertainty in our output space.
Any value that is within $0.1$ to the left and right of the reference value \pythoninline{Qref = [0.5, 0.5]} in each dimension is treated as equally likely.
This was done so that using $\ndiscs = 1$ samples to discretize $\dspace$ would fully characterize the characteristic function density representing this uncertainty.
The inverse image of this set is a characteristic function defined on $\pborel$, so errors will exist in particular at the boundaries of the region.

The fundamental challenge with the set-based approach is linked to the geometry of the induced Voronoi-cell tesselation on $\pspace$.
With only $\nsamps = 100$ samples, we see in \ref{fig:ex:identity_set_1E2} that the region (which is supposed to be a square) hardly represents one.
There is ample variation in the shape parameters of the induced sets $\VV_\iparam$ when so few samples are used.
It is possible that some small sample sets will result in a Voronoi tesselation that aligns with the boundaries of the true target density $\Chi_{[0.4, 0.6]^2}$, but the figure is representative of the difficulty of using $\nsamps = 100$ random samples to describe a geometry.
More often than not, it is hard to approximate a square with $100$ Voronoi cells that come from Mont-Carlo sampling of $\pspace$.
With $\nsamps = 1000$, there are usually still significant differences in the symmetric difference of approximated and true supports of the densities.

Below, we demonstrate the use of more samples to resolve the geometry of the desired set.
\begin{figure}[ht]
\begin{minipage}{.975\textwidth}
  \includegraphics[width=\linewidth]{./examples/identity/set/M1-N1000_N1000-vs-M4-N1000_N1000.png}
  \includegraphics[width=\linewidth]{./examples/identity/set/M1-N10000_N10000-vs-M4-N10000_N10000.png}
\end{minipage}
\caption{
(Top):$\nsamps=1,000$ were used to discretize $\pspace$ and $\ndiscs=1, 4$ (left/right) were used to discretize $\dspace$.
(Bottom): The same, except with $\nsamps=10,000$, where we finally begin to see something resembling the correct correct geometry.
As $\nsamps$ increases, the negative impact on error of choosing higher $\ndiscs$ is resolved.}
\label{fig:ex:identity_set_1E3_1E4}
\end{figure}

We remark on the fact that in this particular situation, $\ndiscs = 1$ is a ``correct'' choice for the probability measure chosen, and $\ndiscs = 4$ actually introduces errors.
The reason that the latter solutions have different values inside of the support of $\PPspace$ is because there are four sets in output space that the $\nsamps$ parameter samples may be mapped to, so the probability of all four receiving equal proportions of samples depends on increasing $\nsamps$.
When $\ndiscs = 1$, all samples land in the same set and probability is assigned in equal proportion to each of them.
Had we chosen a non-uniform density, this would fail to hold, and we would need to increase $\ndiscs$ and accordingly, $\nsamps$.


% !TEX root = ./main.tex
\section{Empirical Bacterial Growth Laws}
\label{sec:emp_laws}

In recent years, there has been a tremendous advancement in our understanding
bacterial physiology from a theoretical perspective. The group of
Terrence Hwa has proposed and tested a series of empirical ``growth laws'' based on experimental observations of cellular behavior \cite{Scott2010,Deris2013}. A major observation is the law that describes a linear relationship between the fraction of the proteome dedicated to ribosomes and growth rate of the cell,
\begin{equation}
    \lambda = (r_u - r_\mathrm{min})\kappa_t\label{equ:growthlaw1},
\end{equation}
where $\lambda$ is the growth rate, $r_u$ is the proteomic fraction of active ribosomes, $r_\mathrm{min}$ is the minimal fraction of ribosomes needed for growth and $\kappa_t$ is an empirically determined proportionality factor called translational capacity. Another result of their study was that the observed cells responded to translation inhibiting drugs by increasing the ribosome fraction of the proteome. Surprisingly, this relationship was also found to be linear,
\begin{equation}
    \lambda = (r_\mathrm{max} - r_\RM{tot}) b\label{equ:growthlaw2}
\end{equation}
In the absence of drug, these two equations are equal to each other, and $r_\RM{tot} = r_u$. We can use this to determine the proportionality factor $b$,
\begin{eqnarray}
    b&=&\frac{1}{\lambda_0}(r_\RM{max}-r_u) \nonumber\\
    &=&\frac{1}{\lambda_0}\left(r_\RM{max}-\frac{\lambda_0}{\kappa_t}-r_\RM{min}\right)\nonumber \\
    &=&\frac{\Delta r}{\lambda_0} - \frac{1}{\kappa_t},
\end{eqnarray}
where we inserted equation \ref{equ:growthlaw1} for $r_u$. Now we can solve for the total ribosomes $r_\RM{tot}$,
\begin{eqnarray}
    r_\RM{tot} &=& r_\RM{max} - b\lambda \nonumber\\
    &=&r_\RM{max} - \lambda \left(\frac{\Delta r}{\lambda_0} - \frac{1}{\kappa_t}\right) \nonumber\\
    &=& r_\RM{max} - \Delta r\lambda \left( \frac{1}{\lambda_0} - \frac{1}{\kappa_t\Delta r} \right)\label{equ:rtot}.
\end{eqnarray}

\section{Derivation of Drug Inhibition}
\label{sec:drug_inhib}
Tetracycline binds the small subunit of the ribosome and stalls translation by inhibiting tRNA from binding \textcolor{purple}{(citation)}. It can be assumed that the drug enters the cell via passive diffusion \textcolor{purple}{(citation)} with flux $J_\RM{in}$,
\begin{equation}
    J_\RM{in} = P_\RM{in} a_\RM{ex} - P_\RM{out} a,
\end{equation}
where $a_\RM{ex}$ is the extracellular concentration of drug, $a$ is the intracellular drug concentration and $j$ is a proportionality constant. In our model, cells gain resistance by expressing drug specific secondary efflux pumps (antiporter), which draw energy from a chemical gradient. In the case of the tetracycline antiporter, the drug is exported against a protein gradient. We assume that the proton gradient is constant for the duration of the experiment (\purple{something we could think about more in detail later}). In this case, the drug kinetics can be described by a Michaelis-Menten equation describing a flux $J_\RM{ex}$
\begin{equation}
J_\RM{ex} = V_\RM{max} {a \over a+K_M},
\end{equation}
where $K_M$ is the Michaelis constant of the efflux pump and $V_\RM{max}$ is the maximum efflux rate which depends on the expression of the resistance gene. In this model we include growth mediated feedback to gene expression \cite{Deris2013}. That is, if growth rate is reduced, so is the expression of genes in the cell (\purple{is this obvious or do we need to prove, e.g. which Heinemann data}),
\begin{equation}
V_\RM{max} = {\lambda\over\lambda_0} V_0,
\end{equation}
where $V_0$ and $\lambda_0$ are the catalytic rate and growth rate in the absence of drug. A detailed dissection of $V_0$ and how it depends on regulation of the resistance gene is given in section \ref{sec:gene_expression}.\\
Tetracycline binds reversibly to the ribosome, therefore we can describe the binding kinetics with
\begin{equation}
    f(r_u, r_b, a) = -k_\RM{on}a\,(r_u-r_\RM{min}) + k_\RM{off} r_b,
\end{equation}
where $k_\RM{on}$ is the binding rate of drug to free ribosome, $k_\RM{off}$ is the dissociation rate of the drug-ribosome complex, and $r_b$ is the concentration of bound ribosomes. In the binding term, we consider the difference of free ribosome to minimum ribosome concentration needed for growth, the model is based on the empirical growth laws, which are not defined for $r_u < r_\RM{min}$.
Using these terms, we can write down the ODE's describing the concentration of drug and ribosomes,
\begin{eqnarray}
    {d a\over d t} &=& -\lambda a + f(r_u, r_b, a) + J(a_\RM{ex}, a) - V_0 {\lambda\over\lambda_0}{a\over a + K_M},\label{equ:dta}\\
    {dr_u\over dt} &=& -\lambda r_u + f(r_u, r_b, a) + s(\lambda),\label{equ:dtru}\\
    {dr_b\over dt} &=& -\lambda r_b - f(r_u, r_b, a)\label{equ:dtrb},
\end{eqnarray}
where $s(\lambda)$ is the rate at which new ribosomes are synthesized. Here we are looking for steady state solutions to find a relationship between growth rate and extracellular drug concentration.
To start, we solve the first empirical growth law \ref{equ:growthlaw1}  for the concentration of free ribosomes, $r_u = \lambda/\kappa_t + r_\RM{min}$ and use it to find the concentration of bound ribosomes by solving equation \ref{equ:dtrb}
\begin{eqnarray}
    0 &=& -\lambda r_b + k_\RM{on}a\,(r_u-r_\RM{min}) - k_\RM{off} r_b \nonumber\\
    &=& -\lambda r_b + k_\RM{on}a\frac{\lambda}{\kappa_t} - k_\RM{off} r_b \nonumber\\
    \Rightarrow r_b &=& k_\RM{on}a{\lambda/\kappa_t \over \lambda + k_\RM{off} } \approx {k_\RM{on} a\lambda \over k_\RM{off} \kappa_t}={a\lambda \over K_d \kappa_t},
\end{eqnarray}
where we simplified $k_\RM{off} + \lambda \approx k_\RM{off}$ ($k_\RM{off}\sim 10^{-2} s^{-1}$ (\purple{cite Barrenechea et al., 2021}) and $\lambda < 5\times 10^{-4} s{-1}$ (20 min doubling time)) and introduced the dissociation constant of drug-ribosome binding, $K_d = k_\RM{off}/k_\RM{on}$.
Next, we use these two results to solve for the intracellular drug concentration in \ref{equ:dta},
\begin{eqnarray}
    0&=&-\lambda a -\frac{k_\RM{on} a \lambda}{\kappa_t}+ k_\RM{off} {a\lambda \over K_d \kappa_t}+ P_\RM{in} a_\RM{ex} - P_\RM{out} a - V_0 {\lambda \over \lambda_0} {a \over a + K_M} \nonumber\\
    &=& -\lambda a (a+K_M) + P_\RM{in} a_\RM{ex} (a+K_M) - P_\RM{out} a(a+K_M) - V_0 \frac{\lambda}{\lambda_0}\nonumber \\
    &=& a^2 \left( -\lambda - P_\RM{out} \right) + a \left(-\lambda K_M - V_0 {\lambda \over \lambda_0} - P_\RM{out} K_M + P_\RM{in} a_\RM{ex}\right) + P_\RM{in} a_\RM{ex} K_M,\label{equ:lambda_poly_1}
\end{eqnarray}
 Above we found expressions for both unbound ribosomes $r_u$ and bound ribosomes $r_b$, which we can compare to equation \ref{equ:rtot} by noting that $r_\RM{tot} = r_u + r_b$,
\begin{eqnarray}
    r_\RM{max} - \Delta r\lambda \left( \frac{1}{\lambda_0} - \frac{1}{\kappa_t \Delta r} \right) &=& \lambda/\kappa_t + r_\RM{min} + {a\lambda\ \over \kappa_t K_d} \nonumber\\
    r_\RM{max} - \frac{\Delta r\lambda}{\lambda_0} + \frac{\lambda}{\kappa_t }  &=& \lambda/\kappa_t + r_\RM{min} + {a\lambda\ \over \kappa_t K_d} \nonumber
\end{eqnarray}
which we solve for the intracellular drug concentration
\begin{equation}
    a = \frac{\Delta r \kappa_t K_d}{\lambda}\left( 1-\frac{\lambda}{\lambda_0} \right)\label{equ:a_of_lambda}
\end{equation}
By inserting equation \ref{equ:a_of_lambda} into equation \ref{equ:lambda_poly_1} we obtain a polynomial of degree 3. The roots of this polynomial need to be found numerically. We obtain at least one real root for all non-negative $a_\RM{ex}$. At increasing resistance, there is a non-continuity in the response (Figure \ref{fig:model_res}).

\begin{figure}
    \centering
    \includegraphics[width=.7\linewidth]{res_model_edit.pdf}
    \caption{Predicted relative growth rate $\lambda/\lambda_0$ at various levels of resistance $V_0$ for increasing extracellular drug concentration $a_\RM{ex}$.}
    \label{fig:model_res}
\end{figure}
\section{Statistical Mechanics of the Simple Repression Motif}
\section{Growth Rate Measurements}
Liquid cultures were inoculated from -80\degree C freezer stock into LB media (\purple{find exact product}) and grown overnight for 12h at 37\degree C and shaken at 250rpm. Saturated cultures were diluted into
\label{sec:gene_expression}


         %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         % Information sheet for the Matlab lab - Maths 6111 %
         %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[10pt]{article}
\input ma_no_html_header

\usepackage{color}
\usepackage{hyperref}
\hypersetup{breaklinks=true,colorlinks=true}
%\input ma_header
\setlength{\parindent}{0pt}
\pagestyle{myheadings}
% \markright{
% \protect {\protect \epsfxsize=0.2 true cm \protect \epsffile {dolph.line.eps}}
% \it Maths 3018/6111 - Numerical methods \hfill}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\thispagestyle{empty}
\begin{center}
\textbf{\Large Maths 3018/6111 - Numerical Methods \\*[8mm]
Worksheet 6}\\*[.8cm]
\end{center}

\section*{Theory}

\begin{enumerate}
\item Explain the shooting method for BVPs.
\item Give a \emph{complete} algorithm for solving the BVP
  \begin{equation*}
    y'' - 3 y' + 2 y = 0, \quad y(0) = 0, \quad y(1) = 1
  \end{equation*}
  using the finite difference method. Include the description of the
  grid, the grid spacing, the treatment of the boundary conditions,
  the finite difference operators and a description of the linear
  system to be solved. You do not need to say which method would be
  used to solve the linear system, but should mention any special
  properties of the system that might make it easier to solve.
\item Explain how your algorithm would have to be modified to solve
  the BVP where the boundary condition at $x=1$ becomes the Neumann
  condition
  \begin{equation*}
    y'(1) = 1 + \frac{e}{e-1}.
  \end{equation*}
\end{enumerate}

\section*{Coding}

\begin{enumerate}
\item Write a simple shooting method to solve the BVP
  \begin{equation*}
    y'' - 3 y' + 2 y = 0, \quad y(0) = 0, \quad y(1) = 1.
  \end{equation*}
  Use {\tt ode45} to solve the ODE, rewritten in first order form, and
  either a simple bisection method or the in-built Matlab {\tt fsolve}
  routine to find the root. Compare your estimate against the answer
  \begin{equation*}
    y(x) = \frac{e^{2 x - 1} - e^{x - 1}}{e - 1}.
  \end{equation*}
\item Implement your finite difference algorithm algorithm above to
  solve this BVP, using Matlab's in-built linear system solver. Show
  that your result converges to the correct answer.
\item Modify your algorithm for the Neumann boundary condition
  above. Check that it converges to the same answer as for the
  Dirichlet case.
\end{enumerate}

\end{document}


% LaTeX source for ``Python for Informatics: Exploring Information''
% Copyright (c)  2010-  Charles R. Severance, All Rights Reserved

\chapter{Iteration}
\index{iteration}

\section{Updating variables}
\label{update}

\index{update}
\index{variable!updating}

A common pattern in assignment statements is an assignment statement
that updates a variable --
where the new value of the variable depends on the old.

\beforeverb
\begin{verbatim}
x = x+1
\end{verbatim}
\afterverb
%
This means ``get the current value of {\tt x}, add 1, and then
update {\tt x} with the new value.''

If you try to update a variable that doesn't exist, you get an
error, because Python evaluates the right side before it assigns
a value to {\tt x}:

\beforeverb
\begin{verbatim}
>>> x = x+1
NameError: name 'x' is not defined
\end{verbatim}
\afterverb
%
Before you can update a variable, you have to {\bf initialize}
it, usually with a simple assignment:

\index{initialization (before update)}

\beforeverb
\begin{verbatim}
>>> x = 0
>>> x = x+1
\end{verbatim}
\afterverb
%
Updating a variable by adding 1 is called an {\bf increment};
subtracting 1 is called a {\bf decrement}.

\index{increment}
\index{decrement}

\section{The {\tt while} statement}

\index{statement!while}
\index{while loop}
\index{loop!while}
\index{iteration}

Computers are often used to automate repetitive tasks.  Repeating
identical or similar tasks without making errors is something that
computers do well and people do poorly.
Because iteration is so common, Python provides several
language features to make it easier.

One form of iteration in Python is the {\tt while} statement.  Here is
a simple program that counts down from five and then says ``Blastoff!''.

\beforeverb
\begin{verbatim}
n = 5
while n > 0:
    print n
    n = n-1
print 'Blastoff!'
\end{verbatim}
\afterverb
%
You can almost read the {\tt while} statement as if it were English.
It means, ``While {\tt n} is greater than 0,
display the value of {\tt n} and then reduce the value of
{\tt n} by 1.  When you get to 0, exit the {\tt while} statement and
display the word {\tt Blastoff!}''

\index{flow of execution}

More formally, here is the flow of execution for a {\tt while} statement:

\begin{enumerate}

\item Evaluate the condition, yielding {\tt True} or {\tt False}.

\item If the condition is false, exit the {\tt while} statement
and continue execution at the next statement.

\item If the condition is true, execute the
body and then go back to step 1.

\end{enumerate}

This type of flow is called a {\bf loop} because the third step
loops back around to the top.  We call each time we execute the body of
the loop an {\bf iteration}.  For the above loop, we
would say, ``It had five iterations'', which means that the body of
the loop was executed five times.

\index{condition}
\index{loop}
\index{body}

The body of the loop should change the value of one or more variables
so that eventually the condition becomes false and the loop
terminates.
We call the variable that changes each time the loop
executes and controls when the loop finishes the
{\bf iteration variable}.
If there is no iteration variable, the loop will repeat forever,
resulting in an {\bf infinite loop}.

\section{Infinite loops}

An endless source of amusement for
programmers is the observation that the directions on shampoo,
``Lather, rinse, repeat,'' are an infinite loop because
there is no {\bf iteration variable} telling you how many times
to execute the loop.

\index{infinite loop}
\index{loop!infinite}

In the case of {\tt countdown}, we can prove that the loop
terminates because we know that the value of {\tt n} is finite, and we
can see that the value of {\tt n} gets smaller each time through the
loop, so eventually we have to get to 0.  Other times a loop is obviously
infinite because it has no iteration variable at all.

\section{``Infinite loops'' and {\tt break}}
\index{break statement}
\index{statement!break}

Sometimes you don't know it's time to end a loop until you get half
way through the body.  In that case you can write an infinite loop on purpose
and then use the {\tt break} statement to jump out of the loop.

This loop is obviously an {\bf infinite loop} because the logical
expression on the
{\tt while} statement is simply the logical constant {\tt True}:

\beforeverb
\begin{verbatim}
n = 10
while True:
    print n,
    n = n - 1
print 'Done!'
\end{verbatim}
\afterverb
%
If you make the mistake and run this code, you will learn quickly how
to stop a runaway Python process on your system or find where the power-off
button is on your computer.
This program will
run forever or until your battery runs out
because the logical expression at the top of the loop
is always true by virtue of the fact that the expression is the
constant value {\tt True}.

While this is a dysfunctional infinite loop, we can still use this pattern
to build useful loops as long as we carefully add code to the
body of the loop to explicitly exit the loop using {\tt break}
when we have reached
the exit condition.

For example, suppose you want to take input from the user until they
type {\tt done}.  You could write:

\beforeverb
\begin{verbatim}
while True:
    line = raw_input('> ')
    if line == 'done':
        break
    print line
print 'Done!'
\end{verbatim}
\afterverb
%
The loop condition is {\tt True}, which is always true, so the
loop runs repeatedly until it hits the break statement.

Each time through, it prompts the user with an angle bracket.
If the user types {\tt done}, the {\tt break} statement exits
the loop.  Otherwise the program echoes whatever the user types
and goes back to the top of the loop.  Here's a sample run:

\beforeverb
\begin{verbatim}
> hello there
hello there
> finished
finished
> done
Done!
\end{verbatim}
\afterverb
%
This way of writing {\tt while} loops is common because you
can check the condition anywhere in the loop (not just at the
top) and you can express the stop condition affirmatively
(``stop when this happens'') rather than negatively (``keep going
until that happens.'').

\section{Finishing iterations with {\tt continue}}
\index{continue statement}
\index{statement!continue}

Sometimes you are in an iteration of a loop and want to finish the
current iteration and immediately jump to the next iteration.
In that case you can use the {\tt continue}
statement to skip to the next iteration without finishing the
body of the loop for the current iteration.

Here is an example of a loop that copies its input until the user
types ``done'', but treats lines that start with the hash character
as lines not to be printed (kind of like Python comments).

\beforeverb
\begin{verbatim}
while True:
    line = raw_input('> ')
    if line[0] == '#' :
        continue
    if line == 'done':
        break
    print line
print 'Done!'
\end{verbatim}
\afterverb
%
Here is a sample run of this new program with {\tt continue} added.

\beforeverb
\begin{verbatim}
> hello there
hello there
> # don't print this
> print this!
print this!
> done
Done!
\end{verbatim}
\afterverb
%
All the lines are printed except the one that starts with the hash
sign because when the {\tt continue} is executed, it ends
the current iteration and jumps
back to the {\tt while} statement to start the next iteration, thus
skipping the {\tt print} statement.

\section{Definite loops using {\tt for} }
\index{for statement}
\index{statement!for}

Sometimes we want to loop through a {\bf set} of things such
as a list of words, the lines in a file, or a list of numbers.
When we have a list of things to loop through, we can
construct a \emph{definite} loop using a {\tt for} statement.
We call the {\tt while} statement an \emph{indefinite} loop
because it simply loops until some condition becomes {\tt False},
whereas the {\tt for} loop is looping through a known
set of items so it runs through as many iterations as there
are items in the set.

The syntax of a {\tt for} loop is similar to the {\tt while} loop
in that there is a {\tt for} statement and a loop body:

\beforeverb
\begin{verbatim}
friends = ['Joseph', 'Glenn', 'Sally']
for friend in friends:
    print 'Happy New Year:', friend
print 'Done!'
\end{verbatim}
\afterverb
%
In Python terms,
the variable {\tt friends} is a list\footnote{We will
examine lists in more detail in a later chapter.}
of three strings and the {\tt for}
loop goes through the list and executes the body once
for each of the three strings in the list resulting in this output:

\beforeverb
\begin{verbatim}
Happy New Year: Joseph
Happy New Year: Glenn
Happy New Year: Sally
Done!
\end{verbatim}
\afterverb
%

Translating this {\tt for} loop to English is not as direct as the
{\tt while}, but if you think of friends as a {\bf set},
it goes like this: ``Run the statements in the body of the
for loop once for each friend \emph{in} the set named friends.''

Looking at the {\tt for} loop, {\bf for} and {\bf in} are reserved
Python keywords, and {\tt friend} and {\tt friends} are variables.

{\tt {\bf for} friend {\bf in} friends{\bf :}\\
\verb"    "{\bf print} 'Happy New Year', friend }

In particular, {\tt friend} is the {\bf iteration variable} for
the for loop.  The variable {\tt friend} changes for each iteration of
the loop and controls when the {\tt for} loop completes.  The
{\bf iteration variable} steps successively through the
three strings stored in the {\tt friends} variable.

\section{Loop patterns}

Often we use a {\tt for} or {\tt while} loop to go through a list of items
or the contents of a file and we are looking for something such as
the largest or smallest value of the data we scan through.

These loops are generally constructed by:

\begin{itemize}

\item Initializing one or more variables before the loop starts

\item Performing some computation on each item in the loop body,
possibly changing the variables in the body of the loop

\item Looking at the resulting variables when the loop completes

\end{itemize}

We will use a list of numbers to demonstrate the concepts and construction
of these loop patterns.

\subsection{Counting and summing loops}

For example, to count the number of items
in a list, we would write the following {\tt for} loop:

\beforeverb
\begin{verbatim}
count = 0
for itervar in [3, 41, 12, 9, 74, 15]:
    count = count + 1
print 'Count: ', count
\end{verbatim}
\afterverb
%
We set the variable {\tt count} to zero before the loop starts,
then we write a {\tt for} loop to run through the list of numbers.
Our {\bf iteration} variable is named {\tt itervar} and while we do
not use {\tt itervar} in the loop, it does control the loop and cause
the loop body to be executed once for each of the values in the list.

In the body of the loop, we add 1 to the current value of {\tt count}
for each of the values in the list.  While the loop is executing, the
value of {\tt count} is the number of values we have seen ``so far''.

Once the loop completes, the value of {\tt count} is the total number
of items.   The total number ``falls in our lap'' at the end of the
loop.  We construct the loop so that we have what we want when the loop
finishes.

Another similar loop that computes the total of a set of numbers
is as follows:

\beforeverb
\begin{verbatim}
total = 0
for itervar in [3, 41, 12, 9, 74, 15]:
    total = total + itervar
print 'Total: ', total
\end{verbatim}
\afterverb
%
In this loop we \emph{do} use the {\bf iteration variable}.
Instead of simply adding one to the {\tt count} as in the previous loop,
we add the actual number (3, 41, 12, etc.) to the running
total during each loop iteration.
If you think about the variable {\tt total}, it contains the
``running total of the values so far''.  So before the loop
starts {\tt total} is zero because we have not yet seen any values,
during the loop {\tt total} is the running total, and at the end of
the loop {\tt total} is the overall total of all the values
in the list.

As the loop executes, {\tt total} accumulates the sum of the
elements; a variable used this way is sometimes called an
{\bf accumulator}.
\index{accumulator!sum}

Neither the counting loop nor the summing loop are particularly
useful in practice because there are built-in functions
{\tt len()} and {\tt sum()} that compute the number of
items in a list and the total of the items in the list
respectively.

\subsection{Maximum and minimum loops}

\index{loop!maximum}
\index{loop!minimum}
\index{None special value}
\index{special value!None}
\label{maximumloop}
To find the largest value in a list or sequence, we construct the
following loop:

\beforeverb
\begin{verbatim}
largest = None
print 'Before:', largest
for itervar in [3, 41, 12, 9, 74, 15]:
    if largest is None or itervar > largest :
        largest = itervar
    print 'Loop:', itervar, largest
print 'Largest:', largest
\end{verbatim}
\afterverb
%
When the program executes, the output is as follows:

\beforeverb
\begin{verbatim}
Before: None
Loop: 3 3
Loop: 41 41
Loop: 12 41
Loop: 9 41
Loop: 74 74
Loop: 15 74
Largest: 74
\end{verbatim}
\afterverb
%
The variable {\tt largest} is best thought of as
the ``largest value we have seen so far''.
Before the loop, we set {\tt largest} to the constant {\tt None}.
{\tt None} is a special constant value which we can
store in a variable to mark
the variable as ``empty''.

Before the loop starts, the largest value we have seen so far
is {\tt None} since we have not yet seen any values.  While the
loop is executing, if {\tt largest} is {\tt None} then we take
the first value we see as the largest so far.   You can see in
the first iteration when the value of {\tt itervar} is 3,
since {\tt largest} is {\tt None}, we immediately set
{\tt largest} to be 3.

After the first iteration, {\tt largest} is no longer {\tt None},
so the second part of the compound logical expression that checks
{\tt itervar > largest} triggers only when we see a value that is
larger than the ``largest so far''.  When we see a new ``even larger''
value we take that new value for {\tt largest}.  You can see in the
program output that {\tt largest} progresses from 3 to 41 to 74.

At the end of the loop, we have scanned all of the values and
the variable {\tt largest} now does contain the largest value
in the list.

To compute the smallest number, the code is very similar with one
small change:

\beforeverb
\begin{verbatim}
smallest = None
print 'Before:', smallest
for itervar in [3, 41, 12, 9, 74, 15]:
    if smallest is None or itervar < smallest:
        smallest = itervar
    print 'Loop:', itervar, smallest
print 'Smallest:', smallest
\end{verbatim}
\afterverb
%
Again, {\tt smallest} is the ``smallest so far'' before, during, and after the
loop executes.  When the loop has completed, {\tt smallest} contains the
minimum value in the list.

Again as in counting and summing, the built-in functions
{\tt max()} and {\tt min()} make writing these exact loops
unnecessary.

The following is a simple version of the Python built-in
{\tt min()} function:

\beforeverb
\begin{verbatim}
def min(values):
    smallest = None
    for value in values:
        if smallest is None or value < smallest:
            smallest = value
    return smallest
\end{verbatim}
\afterverb
%
In the function version of the smallest code, we removed all of the
{\tt print} statements so as to be equivalent to the {\tt min}
function which is already built in to Python.

\section{Debugging}

As you start writing bigger programs, you might find yourself
spending more time debugging.  More code means more chances to
make an error and more places for bugs to hide.

\index{debugging!by bisection}
\index{bisection, debugging by}

One way to cut your debugging time is ``debugging by bisection.''
For example, if there are 100 lines in your program and you
check them one at a time, it would take 100 steps.

Instead, try to break the problem in half.  Look at the middle
of the program, or near it, for an intermediate value you
can check.  Add a {\tt print} statement (or something else
that has a verifiable effect) and run the program.

If the mid-point check is incorrect, the problem must be in the
first half of the program.  If it is correct, the problem is
in the second half.

Every time you perform a check like this, you halve the number
of lines you have to search.  After six steps (which is much
less than 100), you would be down to one or two lines of code,
at least in theory.

In practice it is not always clear what
the ``middle of the program'' is and not always possible to
check it.  It doesn't make sense to count lines and find the
exact midpoint.  Instead, think about places
in the program where there might be errors and places where it
is easy to put a check.  Then choose a spot where you
think the chances are about the same that the bug is before
or after the check.

\section{Glossary}

\begin{description}

\item[accumulator:] A variable used in a loop to add up or
accumulate a result.
\index{accumulator}

\item[counter:] A variable used in a loop to count the number
of times something happened.  We initialize a counter to
zero and then increment the counter each time we want to
``count'' something.
\index{counter}

\item[decrement:] An update that decreases the value of a variable.
\index{decrement}

\item[initialize:] An assignment that gives an initial value to
a variable that will be updated.

\item[increment:] An update that increases the value of a variable
(often by one).
\index{increment}

\item[infinite loop:] A loop in which the terminating condition is
never satisfied or for which there is no terminating condition.
\index{infinite loop}

\item[iteration:] Repeated execution of a set of statements using
either a function that calls itself or a loop.
\index{iteration}

\end{description}

\section{Exercises}

\begin{ex}
Write a program which repeatedly reads numbers until the user
enters ``done''.
Once ``done'' is entered, print out the total, count, and average
of the numbers.  If the user enters anything other than a number,
detect their mistake using {\tt try} and {\tt except} and
print an error message and skip to the next number.

\begin{verbatim}
Enter a number: 4
Enter a number: 5
Enter a number: bad data
Invalid input
Enter a number: 7
Enter a number: done
16 3 5.33333333333
\end{verbatim}
\end{ex}

\begin{ex}
Write another program that prompts for a list of numbers as above
and at the end prints out both the maximum and minimum of the numbers instead of the average.
\end{ex}


\hypertarget{loops-and-invariants}{%
\section{Loops and Invariants}\label{loops-and-invariants}}

\hypertarget{invariants}{%
\subsection{Invariants}\label{invariants}}

An assertion I is called an invariant, if a command C doesn't change
anything in regards to the assertion I. -\textgreater{} $\models$ \{ I \} C \{ I
\}

E.g. if you drive through a tunnel by car, you pass an emergency exit
every 250 meters. If you are between two exits and you are driving
forward, you are maximizing the distance to the earlier exit and
minimizing the distance to the next exit, but you don't change the
distance between them. The distance between them is invariant.

\hypertarget{invariants-for-loops}{%
\subsubsection{Invariants for loops}\label{invariants-for-loops}}

\begin{itemize}
\tightlist
\item
  here we need invariants for loops
\item
  but at the beginning of the loop body we can assume more than the
  invariant only: we can also assume the loop condition
\end{itemize}

\begin{lstlisting}
while i > 0 do
    { i >= 0 $\land$ i > 0 } --invariant $\land$ condition
    { (i - 1) >= 0 }
    i := i - 1
    { i >= 0 } --invariant
endwhile
\end{lstlisting}

\begin{itemize}
\tightlist
\item
  an invariant I of the body C relative to the condition B of a loop
  indeed is an invariant of the complete loop
\end{itemize}

\begin{lstlisting}
{ I }
while B do
    { I $\land$ B }
    C
    { I }
endwhile
{ I }
\end{lstlisting}

\begin{figure}[H]
\centering
\includegraphics[width=0.7\textwidth]{figures/invariantLoop.png}
\caption{Invariants of a Loop}
\end{figure}

\hypertarget{finding-invariants}{%
\subsubsection{Finding Invariants}\label{finding-invariants}}

\begin{itemize}
\tightlist
\item
  to find an appropriate invariant is a creative process, just like
  programming
\item
  it cannot be automated
\item
  however, after having found an invariant, it can be added as
  annotation to the loop
\item
  and then, the generation of verification conditions can again be
  performed fully automatically
\end{itemize}

\clearpage


\documentclass[11pt,a4paper]{report}
\usepackage{amsmath,amsfonts,amssymb,amsthm,epsfig,epstopdf,titling,url,array}
\usepackage{enumitem}
\usepackage{changepage}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{listings}
\usepackage{color}
\theoremstyle{plain}
\newtheorem{thm}{Theorem}[section]
\newtheorem{lem}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}
\newtheorem*{cor}{Corollary}
\theoremstyle{definition}
\newtheorem{defn}{Definition}[section]
\newtheorem{conj}{Conjecture}[section]
\newtheorem{exmp}{Example}[section]
\newtheorem{exercise}{Exercise}[section]
\theoremstyle{remark}
\newtheorem*{rem}{Remark}
\newtheorem*{note}{Note}
\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]}
\let\endchangemargin=\endlist

\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}

\lstdefinestyle{mystyle}{
	backgroundcolor=\color{backcolour},
	commentstyle=\color{codegreen},
	keywordstyle=\color{magenta},
	numberstyle=\tiny\color{codegray},
	stringstyle=\color{codepurple},
	basicstyle=\footnotesize,
	breakatwhitespace=false,
	breaklines=true,
	captionpos=b,
	keepspaces=true,
	numbers=left,
	numbersep=5pt,
	showspaces=false,
	showstringspaces=false,
	showtabs=false,
	tabsize=2
}

\lstset{style=mystyle}
\begin{document}

\section*{Problem}
5,915,587,277 is the difference between two squares.  What are those numbers?  In other words, find $a$ and $b$ such that
$$a^2 - b^2 = 5,915,587,277$$

\textbf{Hint}: 5,915,587,277 is a prime number.

\section*{Solution}
$a^2 - b^2 = (a + b)(a - b).$  Given that the result is prime, it follows that either $a+b$ or $a-b$ must be 1 and the other quantity must equal the product (otherwise their product is not prime).  The only possibility that works is $a - b = 1$ and $a + b = 5,915,587,277$  so $a = 2,957,793,639$ and $b =  2,957,793,638.$

\end{document}


\chapter{Web-based QPJ-BR questionnaire}
\label{annex:QPJ-BR-questionnaire}
\includegraphics[page=1,width=1\textwidth]{images/annex/QPJ-BR-questionnaire.pdf}

%% ========== %%
\chapter{Data Gathering Instruments for Measuring Participants' Skill and Knowledge}
\label{annex:data-gathering-instruments-skill-knowledge}

\section{Programming Problem: Calculate the Proper Divisors of a Number (P1')}
\label{annex:pilot-study-p1}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-p1.pdf}

\newpage
\section{Programming Problem: Calculate the Sum of Prime Divisors of Number (P2')}
\label{annex:pilot-study-p2}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-p2.pdf}

\newpage
\section{Programming Problem: Calculate Distance of Rebounds for an Elastic Ball (P3')}
\label{annex:pilot-study-p3}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-p3.pdf}

\newpage
\section{Programming Problem: Calculate the Maximum Length of a Hailstone Sequence (P4')}
\label{annex:pilot-study-p4}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-p4.pdf}

\newpage
\section{Programming Problem: Calculate Inverse Fibonacci Sequence on Base $n$ and $m$ (PA')}
\label{annex:pilot-study-pA}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-pA.pdf}

\newpage
\section{Programming Problem: Calculate the Absolute Difference Between Odd and Even Numbers in an Inverse Fibonacci Sequence (PB')}
\label{annex:pilot-study-pB}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-pB.pdf}

\newpage
\section{Programming Problem: Calculate the i-th Prize of a Machine Slot (PC')}
\label{annex:pilot-study-pC}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-pC.pdf}

\newpage
\section{Programming Problem: Calculate the Highest Prize of a Machine Slot (PD')}
\label{annex:pilot-study-pD}
\includegraphics[page=1,width=1\textwidth]{images/annex/pilot-study-pD.pdf}

\newpage
\section{Formative Evaluation: Multiple Choice Knowledge Questionnaires of Cond. Structures (provinha1a)}
\label{annex:first-study-pre}
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pre-1.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/first-study-pre-1.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/first-study-pre-1.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/first-study-pre-1.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pre-2.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pre-list.pdf}

\newpage
\section{Programming Problem: Develop a Simple Virtual Temperature Monitor (P1)}
\label{annex:first-study-p1}
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-p1.pdf}

\newpage
\section{Formative Evaluation: Multiple Choice Knowledge Questionnaires of Cond. Structures (provinha1b)}
\label{annex:first-study-pos}
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pos-1.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/first-study-pos-1.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/first-study-pos-1.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/first-study-pos-1.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pos-2.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pos-list.pdf}

\newpage
\section{Programming Problem: Develop a Basal Metabolic Rate (PA)}
\label{annex:first-study-pA}
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pA.pdf}

\newpage
\section{Programming Problem: Develop a Diet Calculator (PB)}
\label{annex:first-study-pB}
\includegraphics[page=1,width=1\textwidth]{images/annex/first-study-pB.pdf}

%%
\newpage
\section{Formative Evaluation: Multiple Choice Knowledge Questionnaires of Loop Structures (provinha2a)}
\label{annex:second-study-pre}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pre.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/second-study-pre.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/second-study-pre.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pre-list.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/second-study-pre-list.pdf}

\newpage
\section{Programming Problem: Calculate the Proper Divisors of a Number (P2)}
\label{annex:second-study-p2}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-p2.pdf}

\newpage
\section{Programming Problem: Calculate the Maximum Length of a Hailstone Sequence (P3)}
\label{annex:second-study-p3}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-p3.pdf}

%%
\newpage
\section{Formative Evaluation: Multiple Choice Knowledge Questionnaires of Loop Structures (provinha2b)}
\label{annex:second-study-pos}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pos.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/second-study-pos.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/second-study-pos.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/second-study-pos.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pos-list.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/second-study-pos-list.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/second-study-pos-list.pdf}

\newpage
\section{Programming Problem: Calculate a Geometric Sequence (PC)}
\label{annex:second-study-pC}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pC.pdf}

\newpage
\section{Programming Problem: Calculate Global Minimum Coin Changes (PD)}
\label{annex:second-study-pD}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pD.pdf}

\newpage
\section{Programming Problem: Count Number of Semi-primes for RSA (PE)}
\label{annex:second-study-pE}
\includegraphics[page=1,width=1\textwidth]{images/annex/second-study-pE.pdf}

%%
\newpage
\section{Formative Evaluation: Multiple Choice Knowledge Questionnaires of Recursion (provinha3a)}
\label{annex:third-study-pre}
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pre.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/third-study-pre.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/third-study-pre.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/third-study-pre.pdf}
\newpage
\includegraphics[page=5,width=1\textwidth]{images/annex/third-study-pre.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pre-list.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/third-study-pre-list.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/third-study-pre-list.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/third-study-pre-list.pdf}

\newpage
\section{Programming Problem: Calculate Fibonacci Polynomials (P4)}
\label{annex:third-study-p4}
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-p4.pdf}

%%
\newpage
\section{Formative Evaluation: Multiple Choice Knowledge Questionnaires of Recursion (provinha3c)}
\label{annex:third-study-pos}
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pos.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/third-study-pos.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/third-study-pos.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/third-study-pos.pdf}
\newpage
\includegraphics[page=5,width=1\textwidth]{images/annex/third-study-pos.pdf}
\newpage
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pos-list.pdf}
\newpage
\includegraphics[page=2,width=1\textwidth]{images/annex/third-study-pos-list.pdf}
\newpage
\includegraphics[page=3,width=1\textwidth]{images/annex/third-study-pos-list.pdf}
\newpage
\includegraphics[page=4,width=1\textwidth]{images/annex/third-study-pos-list.pdf}

\newpage
\section{Programming Problem: Generation of Planning Poker Sequence (PF)}
\label{annex:third-study-pF}
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pF.pdf}

\newpage
\section{Programming Problem: Counting Palindromes (PG)}
\label{annex:third-study-pG}
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pG.pdf}

\newpage
\section{Programming Problem: Maze Solving Algorithm (PH)}
\label{annex:third-study-pH}
\includegraphics[page=1,width=1\textwidth]{images/annex/third-study-pH.pdf}

%% ========== %%
\chapter{Data Gathering Instruments for Measuring Participants' Motivation}
\label{annex:data-gathering-instruments-motivation}

%\newpage
\section[Web-based Questionnaire for the Adapted Portuguese IMI]{Web-based Questionnaire for the Adapted Portuguese IMI}
\label{annex:IMI-pilot-study}
\includegraphics[width=1\textwidth]{images/annex/IMI-pilot-study.pdf}

\newpage
\section[Paper-based Questionnaire for the Adapted Portuguese IMI]{Paper-based Questionnaire for the Adapted Portuguese IMI}
\label{annex:IMI-first-study}
\includegraphics[width=1\textwidth]{images/annex/IMI-first-study.pdf}

\newpage
\section[Paper-based Questionnaire for the Adapted Portuguese IMMS]{Paper-based Questionnaire for the Adapted Portuguese IMMS}
\label{annex:IMMS-second-study}
\includegraphics[width=1\textwidth]{images/annex/IMMS-second-study.pdf}

\newpage
\section[Web-based Questionnaire for the Adapted Portuguese IMI and IMMS]{Web-based Questionnaire for the Adapted Portuguese IMI and IMMS}
\label{annex:IMI-IMMS-third-study}
\includegraphics[width=1\textwidth]{images/annex/IMI-IMMS-third-study-01.pdf}
\newpage
\includegraphics[width=1\textwidth]{images/annex/IMI-IMMS-third-study-02.pdf}

%% ========== %%


\documentclass{beamer}

\mode<presentation> {
\usetheme{default}
\usecolortheme{dolphin}
}

\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{amsthm}
\usepackage{subfigure}
\usepackage{bbm}
\usepackage{tikz}
\usetikzlibrary{shapes,automata}

\newcommand{\iter}[2]{#1^{(#2)}}
\newcommand{\transpose}[1]{#1^{\mathsf{T}}}
\newcommand{\iter}[2]{#1^{(#2)}}
\newcommand{\parens}[1]{ \left( #1 \right) }
\newcommand{\prob}{\mathbb{P}}

\title[Short title]{Exploring PageRank}
\author{Nathanael Gentry}
\institute[LU]{MATH 321}
\date{24 April 2019}

\begin{document}
\begin{frame}
\titlepage
\end{frame}

\section{First Steps}
\begin{frame}[label=web]{A Miniscule Web}
    	\begin{figure}[h]
		\centering \scalebox{0.75}{\begin{tikzpicture}[
			scale=0.6,
            > = stealth,
			shorten > = 1pt,
			auto,
			node distance = 3cm,
			semithick]
			\tikzstyle{every state}=[
			draw = black,
			thick,
			fill = white,
			minimum size = 4mm]

			\node[state] (a) {$p_1$};
			\node[state] (b) [above right of=a] {$p_2$};
			\node[state] (c) [right of=a] {$p_3$};
			\node[state] (d) [below right of=a] {$p_4$};
			\node[state] (e) [right of=c] {$p_5$};

			\path[->] (a) edge node {} (b);
			\path[->] (a) edge node {} (d);
			\path[->] (a) edge node {} (c);
			\path[->] (c) edge node {} (b);
			\path[->] (d) edge node {} (c);
			\path[->] (e) edge node {} (b);
			\path[->] (b) edge [bend left=45] node {} (e);
			\path[->] (d) edge node {} (b);
			\path[->] (e) edge node {} (c);
			\path[->] (d) edge node {} (e);
		\end{tikzpicture}}
		\caption{A Miniscule Web}
		\label{fig:web}
	\end{figure}
\end{frame}

\begin{frame}{Intuition}
    We view a hyperlink to a page as an endorsement of importance from the linking page.\\
    \vspace{1mm}
    If a page is not very selective of its endorsements, its endorsements should be less prestigious than those given by a more discriminating page.
\end{frame}

\begin{frame}{Intuition}
    Let $N^-_i$ represent the set of $p_i$'s predecessors, and let $N^+_i$ represent the set of $p_i$'s successors, and note that $|N^+_i|$ gives the out-degree of $p_i$. We thus define the importance score $\pi_i$ of a page $p_i$:
 	\begin{equation*}
 		\pi_i = \sum_{p_j\in N^-_i}{\frac{\pi_j}{|N^+_i|}}.
 	\end{equation*}
\end{frame}

\begin{frame}{Iteration}
     Take some initial score vector $\iter{\pi}{0}$, say a uniform distribution $\pi = [1/|\mathcal{W}|]$. Then, we define an iterative calculation for an iterator $t\geq 1$ to refine the initial distribution:
 	 \begin{equation*}
	 	\iter{\pi}{t+1}_i = \sum_{p_j\in N^-_i}{\frac{\iter{\pi}{t}_j}{|N^+_i|}}.
 	\end{equation*}
\end{frame}

\begin{frame}{Hyperlink Matrix}
     We encode the system given by the previous equation over $\mathcal{W}$ in a hyperlink matrix:
 	\begin{equation*}
 		H_{ij}=\begin{cases}
 			1/|N^+_i|, & \text{if } p_i \in N^-_j; \\
 			0, & \text{otherwise}.
 		\end{cases}
 	\end{equation*}
\end{frame}

\againframe{web}

\begin{frame}{A Simple Example}
    \begin{figure}[t!]
        \centering
        \begin{subfigure}
            \centering
            $\begin{bmatrix}
                0 & 1/3 & 1/3 & 1/3 & 0 \\
                0 & 0 & 0 & 0 & 1 \\
                0 & 1 & 0 & 0 & 0 \\
                0 & 1/3 & 1/3 & 0 & 1/3 \\
                0 & 1/2 & 1/2 & 0 & 0
            \end{bmatrix}$
            \caption{Hyperlink Matrix for Figure \ref{fig:web}}
        \end{subfigure}
        \vspace{1em}
        \begin{subfigure}
        \centering
        \begin{tabular}{|c||c|c|c|c|c|}
             \hline
             $k$ & $\iter{\pi_1}{k}$ & $\iter{\pi_2}{k}$ & $\iter{\pi_3}{k}$ & $\iter{\pi_4}{k}$ & $\iter{\pi_5}{k}$ \\
             \hline\hline
             0 & 0.2 & 0.2 & 0.2 & 0.2 & 0.2 \\
             1 & 0 & 0.433 & 0.233 & 0.067 & 0.267 \\
             2 & 0 & 0.389 & 0.156 & 0 & 0.456 \\
             3 & 0 & 0.383 & 0.228 & 0 & 0.390 \\
             4 & 0 & 0.422 & 0.194 & 0 & 0.383 \\
             \hline
        \end{tabular}
        \caption{Score Iteration for Figure \ref{fig:web}}
        \end{subfigure}

        \label{fig:iterative_example}
    \end{figure}
\end{frame}

\section{Markov Chains}
\subsection{Stochasticity}
\begin{frame}{Stochasticity}
    Consider a vector $d$, where $d_i=1$ if page $p_i$ dangles and $d_i=0$ otherwise, and let $v$ be a distribution over all pages in $\mathcal{W}$, given as a row vector. We stochasticize $H$ thusly:
	\begin{equation*}
	    \label{eqn:S}
		S=H + d \transpose{v}.
	\end{equation*}
\end{frame}

\begin{frame}{Stochasticity}
	\begin{theorem}
	    All stochastic matrices $S$ have spectral radius $\rho(S)=1$, with an eigenvalue $\lambda = \rho(S)$ on the spectral circle.
	\end{theorem}
	\begin{proof}
	    Let $\mathbbm{1}$ represent a vector of ones. The row-stochasticity of $S$ implies $\Vert S \Vert_{\infty} = 1$, i.e., $S \mathbbm{1} = \mathbbm{1}$. Thus, $(1, \mathbbm{1})$ provides a right eigenpair for every stochastic matrix. Now, consider an eigenpair $(v, \lambda)$ of a stochastic matrix $S$ and suppose $|\lambda| > 1$. Then, the vector given by $|\lambda|^n v = S^n v$ grows exponentially, so at least one $(S^n)_{ij} > 1$ for some $n$. This, however, violates the stochasticity of $S$ and its powers. Thus, we conclude that $|\lambda| \leq 1$, and since we have already 1 as an eigenvalue, we conclude the spectral radius $\rho(S)=1$.
	\end{proof}

\end{frame}

\begin{frame}{Markov Property}
    A discrete-time stochastic process $\{X_t\}_{t \in T}$ for a countable set $T$ over a finite state space $\Omega = \{s_i\}_{i=0}^{n}$ is a Markov chain if the Markov property holds for each $t$:
	\begin{align*}
		\prob(X_{t+1} = s_j \mid &X_t = s_{i_t}, X_{t-1} = s_{i_{t-1}},\cdots, X_0 = s_{i_0}) \\ &= \prob(X_{t+1} = s_j \mid X_t=s_{i_{t}}).
	\end{align*}
	\vspace{2mm}
	\pause \centering \textit{Markov chains are memoryless.}
\end{frame}

\begin{frame}{Transition Probabilities}
    We encode the $k$-step transition probabilities for a homogeneous Markov chain in a stochastic matrix of dimension $|\Omega| \times |\Omega|$:
	\begin{equation*}
		\iter{P}{k} = [ \iter{p}{k}_{ij} ] = [ \prob(X_{t+k} = s_j \mid X_t = s_i) ] = [ \prob(X_{t} = s_j \mid X_0 = s_i) ].
	\end{equation*}
\end{frame}

\begin{frame}{Markovian Iteration}
    Consider a distribution $\iter{v}{\ell}$ over $\Omega$, defined by $\iter{v}{\ell}_i = \prob(X_\ell = i)$. Then, by the total law of probability, we calculate the following:
    \begin{align*}
        \iter{v}{\ell}_i &= \sum_{s_k\in\Omega}{\prob(X_0 = s_k) \prob(x_\ell = s_i \mid X_0 = s_k)} \\
        &= \sum_{k\leq |\Omega|}{\parens{\iter{v}{0}_k \iter{p}{\ell}_{ki}}}.
    \end{align*}\\
    \vspace{2mm}
    We conclude that $\iter{v}{\ell} = \iter{v}{0} \iter{P}{\ell} = \iter{v}{0} P^\ell$, for some initial distribution $\iter{v}{0}$.
\end{frame}

\subsection{Stationarity}
\begin{frame}{Stationary Distribution}
     We can take the stochastic process $\{X_t\}_{t \in T}$ underlying a Markov chain as a categorical distribution over $\mathcal{S}$, the state space. \\
     \vspace{2mm}
     By Brouwer's fixed point theorem, within this process must be a $P$-invariant distribution, i.e., an eigenvector of $P$. \\
     \vspace{2mm}
     We denote this fixed distribution $\pi$, given as a row vector, such that $\pi P = \pi$. That is, $\pi_i = \prob(X_t = s_i)$ for all $t$ over all states.
\end{frame}

\begin{frame}{Limiting Distribution}
    Consider a stationary distribution $\pi$ that is also a limiting distribution:
	\begin{equation*}
		\pi = \lim_{k\to\infty}{{\iter{\pi}{k}}} = \lim_{k\to\infty}{\parens{{\iter{\pi}{0}} P^k}}.
	\end{equation*}\\
	\vspace{2mm}
	\textit{Note}: In a Markov chain where the transition matrix converges, an almost-arbitrary initial distribution $\iter{\pi}{0}$ will eventually converge upon $\pi$.
\end{frame}T22tshow

\section{Perron-Frobenius Theory}
\begin{frame}{Irreducibility of Chains}
    In a Markov chain, states $s_i$ and $s_j$ communicate if $\iter{P}{k}_{ij} > 0$, for some iterator (nonnegative integer) $k$. That is, there is a random walk of nonzero probability between states $s_i$ and $s_j$. Communicativity is easily shown to be an equivalence relation.\\
    \vspace{2mm}
    A Markov chain is irreducible if it contains a single communicating class.
\end{frame}

\begin{frame}{Irreducibility of Graphs}
    Equivalently, a graph is irreducible if it has no strongly connected components smaller than itself. If a graph $\mathcal{G}$ is \textit{reducible}, we can permute its vertices to obtain a canonical form for the acjacency matrix $A$:
    \begin{equation*}
	    R = \transpose{P} A P =
	    \begin{bmatrix}
		    X & Y \\
		    0 & Z
	    \end{bmatrix}.
	\end{equation*} \\
	\vspace{2mm}
	If $\iter{A}{k}_{ij} > 0$ for all distinct vertex pairs $(i,j)$ and $k = k_{(i,j)}$, then $\mathcal{G}$ is strongly connected and $A$ is therefore irreducible.
\end{frame}

\begin{frame}{Significance of Irreducibility}
    If the chain is reducible so not all pages communicate, then there is a nonzero probability that the random surfer ventures off and visits some pages only finitely many times. Thus, as time passes, the probability of visiting these pages drops to zero.
\end{frame}

\begin{frame}{Periodicity}
    We define the period of a state by $d(s_i)=\text{gcd}\{n\in \mathbb{Z}_{\geq 0} : \iter{p}{n}_{ii} > 0 \}$. In other words, a random walk from state $s_i$ can only return to $s_i$ in multiples of $d(s_i)$ steps. \\
    \begin{theorem}
        If Markov states $s_i$ and $s_j$ communicate, then $d(s_i) = d(s_j)$.
    \end{theorem}\\
    \vspace{2mm}
     When $P$ is irreducible, $d(P)$ is well-defined; every state has the same period. When $d(P) = 1$, the chain is aperiodic.
\end{frame}

\begin{frame}{Primitivity}
    \begin{lemma}
        For an irreducible aperiodic Markov chain with transition matrix $P$, there is some bound $t_i$ beyond which $\iter{p}{t}_{ii} > 0$ for all $t \geq t_i$ over all states.
    \end{lemma}

    \begin{theorem}
        For the chain considered above, there is a $t$ that gives $\iter{p}{t}_{ij} > 0$ over all states, and so $P^t > 0$.
    \end{theorem}\\
    \vspace{2mm}
    We take this result as the definition of matrix primitivity.
\end{frame}

\begin{frame}{Perron-Frobenius Theorem}
	If $A$ is a primitive matrix, then
	\begin{enumerate}
		\item The spectral radius $\rho(A)$ equals some real eigenvalue $\lambda_1$;
		\item This eigenvalue (the Perron root) is simple, and it is the only eigenvalue on the spectral circle;
		\item The Perron root has unique, strictly positive left and right eigenvectors (Perron vectors);
		\item These left and right Perron vectors are, up to normalization, the only nonnegative eigenvectors of $A$.
	\end{enumerate}
\end{frame}

\section{Limiting Distributions}
\begin{frame}{Matrix Limits}
    \begin{theorem}
       For a square real matrix $A$, we have $A^n \to 0$ as $n\to\infty$ if and only if $\rho(A) < 1$, and the limit does not exist when $\rho(A) > 1$.
    \end{theorem}

    \begin{theorem}
        For $A$ given ahove, when $\rho(A) = 1$ is a simple eigenvalue with respective left and right eigenvectors $x$ and $y$, the limit converges:
        \begin{equation*}
            \lim_{k\to\infty}{A^k} = \frac{x y^*}{x^* y},
        \end{equation*}
        where $\frac{x y^*}{x^* y}$ projects onto the eigenspace $\mathcal{E}_1$.
    \end{theorem}
\end{frame}

\begin{frame}{Limiting Distributions}
    All stochastic matrices have right eigenvector $\mathbbm{1}$, and a primitive matrix $P$ has left eigenvector $\pi$ (the stationary distribution). Thus, we have:
    \begin{align*}
        \lim_{k\to\infty}{{\iter{\pi}{k}}} &= \lim_{k\to\infty}{\parens{{\iter{\pi}{0}} P^k}} \\ &= \iter{\pi}{0} \parens{\frac{\mathbbm{1} \pi}{\pi \mathbbm{1}}} \\ &= \parens{\iter{\pi}{0} \mathbbm{1}} \pi \\ &= \pi.
    \end{align*}
\end{frame}

\begin{frame}{Power Iteration}
    Suppose a square matrix $A$ has $n$ eigenpairs $\{ (\lambda_i,x_i)_{i\leq n} \}$ with $\lambda_1$ as a dominant eigenvalue.\\
    \vspace{2mm}
    Consider a starting vector $x^{(0)}=\sum_i{\gamma_i x_i}$ for scalars $\gamma_i$. Then, we again consider an iterative calculation:
	\begin{equation*}
		\iter{x}{k} = A^k \iter{x}{0} = \sum_{i=1}^{k}{\gamma_i \lambda_i^k x_i} = \lambda_1^k\parens{\gamma_1 x_1 + \sum_{i=2}^{k}{\gamma_i \frac{\lambda_i}{\lambda_1} x_i}} \to \lambda_1^k \gamma_1 x_1.
	\end{equation*}
	\pause Since $\lambda_i<\lambda_1$ for $i>1$, the sum approaches 0 as $k\to\infty$.
\end{frame}

\begin{frame}{The Web Graph [1]}
        \begin{figure}
        \centering
        \includegraphics[scale=0.6]{../paper/images/bow-tie}
        \label{fig:bow-tie}
    \end{figure}
\end{frame}

\begin{frame}{Taming the Web}
    We interpreted the stochastic Web matrix $S$ as the matrix of probabilities that a random web surfer follows links from a page or chooses another randomly when it hits a dead end. However, \textit{human} Web surfers often get bored. They might suddenly elect to teleport -- shift course altogether and visit a random page.\\
    \vspace{2mm}
    \pause Consider a distribution $u$ over all Web pages. We model this teleportation behavior with a rank-one matrix $T = \mathbbm{1} \cdot \transpose{u}$.
\end{frame}

\begin{frame}{The Google Matrix}
    Now, consider a matrix $G$ formed from a convex combination of $S$ and $T$:
	\begin{equation*}
	    \label{google_matrix}
		G = \alpha S + (1-\alpha)T.
	\end{equation*}

	\pause Every page has some connection of nonzero probability to every other page, and our process is still stochastic. The teleportation adjustment thus provides primitivity and guarantees a limiting distribution, from which we can rank pages.
\end{frame}

\begin{frame}{Convergence Speed}
    The speed of convergence depends upon how quickly $\parens{\frac{\lambda_i}{\lambda_1}}^k \to 0$. Unfortunately, due to its \textit{scale-free} nature, the raw Web graph as $\lambda_2$ (a subdominant eigenvalue) close to 1. \\
    \vspace{2mm}
    \pause Even with the primitivity adjustment, the Web graph is only weakly irreducible. \\
    \vspace{2mm}
    \pause Choosing $\alpha \approx 0.85$ has been empirically shown to balance fidelity with efficiency.
\end{frame}

\begin{frame}{Further Investigations}
    \begin{enumerate}
        \item What about link spammers -- those who try to trick PageRank by manipulating the link structure of the Web?
        \item How sensitive is the calculation to the always-changing Internet?
        \item How can we coax the iterative calculation to converge more quickly?
    \end{enumerate}
\end{frame}

\begin{frame}[allowframebreaks]
        \frametitle{References}
        \bibliographystyle{amsalpha}
        \bibliography{../references.bib}
\end{frame}
\end{document}