\subsection{Stochastic problem and how to solve them}%
\label{sub:stochastic_problem_and_how_to_solve_them}

A stochastic problem, modelized as a \gls{mdp} (see section~\ref{sub:mdp}) is a \gls{mdp} in which \(P_{a}(s, s') \leq 1\).
Meaning that the transition between two states \(s\) and \(s'\) by action \(a\) is not guaranteed to succeed.
Therefore, applying any action to any state may, each time, results in different state.

To be clearer, let's consider a small problem.
The rules are simple:

\begin{itemize}
    \item The game is played in a grid of dimension \(W \times H\).
    \item The player starts on a random cell of the grid.
    \item Each turn the player can go \textit{up} , \textit{right} , \textit{down} or \textit{left}.
    \item When choosing a direction, the player may walk one, two or three cells in that direction with a unknown probability.
    \item The player can not exit the grid and is stuck at the edge if he tries.
    \item Each turn, the player freeze a bit and gain \(-1\) point.
    \item An exit is placed on a random cell of the grid.
    \item The goal is for the player to start a turn on the exit.
\end{itemize}

Take the time to understand these rules, they are not very complex.
A state of the game can be represented by the coordinates of the player.
It is a stochastic or random game because, when applying any action to any states, we do not know which state we can obtain.
Exception due to states where the player is on the edges of the grid and try to go beyond.
Now imagine a state where the player is on the next next to the exit.
If he chooses to go toward the exit he might go beyond it.

The difficulty of a stochastic problem is that a sequence \(S\) of \(n\) actions may not contain enough action to solve the game.
Furthermore, many stochastic problems, once modelized as a \gls{mdp} tends to posses a very large set of states.
For some of them, if we were to see the entire tree of the game, the ramification may be too large.
That is not say that for some non stochastic problem, the number of states is not immensely large.
In the game of Go, which is not stochastic, the number of state is \(3^{361}\) which is enormous.
Some problem even have an infinity amount of states.

So the real problem is really the fact that sequence, when applied to a state may not always lead to terminal state.
Also, in some problems, actions can be illegal so a sequence \(S={a1, a2}\) when applied from state \(s\) might lead to a final state \(s^{*}\) but also to another state for which th\(a2\) is illegal.
This poses a new problem as to how define a sequence in a stochastic context.
For some problem, sequences that always lead from root state to a terminal one exists but, in general, it is unlikely.
Furthermore, we are more interested in solving more difficult problem.


\achapter{Canonical Structures}
\aauthor{Assia Mahboubi and Enrico Tassi}

\label{CS-full}
\index{Canonical Structures!presentation}

This chapter explains the basics of Canonical Structure and how thy can be used
to overload notations and build a hierarchy of algebraic structures.
The examples are taken from~\cite{CSwcu}.  We invite the interested reader
to refer to this paper for all the details that are omitted here for brevity.
The interested reader shall also find in~\cite{CSlessadhoc} a detailed
description of another, complementary, use of Canonical Structures:
advanced proof search.  This latter papers also presents many techniques one
can employ to tune the inference of Canonical Structures.

\section{Notation overloading}

We build an infix notation $==$ for a comparison predicate.  Such notation
will be overloaded, and its meaning will depend on the types of the terms
that are compared.

\begin{coq_eval}
Require Import Arith.
\end{coq_eval}

\begin{coq_example}
Module EQ.
  Record class (T : Type) := Class { cmp : T -> T -> Prop }.
  Structure type := Pack { obj : Type; class_of : class obj }.
  Definition op (e : type) : obj e -> obj e -> Prop :=
    let 'Pack _ (Class _ the_cmp) := e in the_cmp.
  Check op.
  Arguments op {e} x y : simpl never.
  Arguments Class {T} cmp.
  Module theory.
    Notation "x == y" := (op x y) (at level 70).
  End theory.
End EQ.
\end{coq_example}

We use Coq modules as name spaces.  This allows us to follow the same pattern
and naming convention for the rest of the chapter.  The base name space
contains the definitions of the algebraic structure.  To keep the example
small, the algebraic structure \texttt{EQ.type} we are defining is very simplistic,
and characterizes terms on which a binary relation is defined, without
requiring such relation to validate any property.
The inner \texttt{theory} module contains the overloaded notation \texttt{==} and
will eventually contain lemmas holding on all the instances of the
algebraic structure (in this case there are no lemmas).

Note that in practice the user may want to declare \texttt{EQ.obj} as a coercion,
but we will not do that here.

The following line tests that, when we assume a type \texttt{e} that is in the
\texttt{EQ} class, then we can relates two of its objects with \texttt{==}.

\begin{coq_example}
Import EQ.theory.
Check forall (e : EQ.type) (a b : EQ.obj e), a == b.
\end{coq_example}

Still, no concrete type is in the \texttt{EQ} class.  We amend that by equipping \texttt{nat}
with a comparison relation.

\begin{coq_example}
Fail Check 3 == 3.
Definition nat_eq (x y : nat) := nat_compare x y = Eq.
Definition nat_EQcl : EQ.class nat := EQ.Class nat_eq.
Canonical Structure nat_EQty : EQ.type := EQ.Pack nat nat_EQcl.
Check 3 == 3.
Eval compute in 3 == 4.
\end{coq_example}

This last test shows that Coq is now not only able to typecheck \texttt{3==3}, but
also that the infix relation was bound to the \texttt{nat\_eq} relation.  This
relation is selected whenever \texttt{==} is used on terms of type \texttt{nat}.  This
can be read in the line declaring the canonical structure \texttt{nat\_EQty},
where the first argument to \texttt{Pack} is the key and its second argument
a group of canonical values associated to the key.  In this case we associate
to \texttt{nat} only one canonical value (since its class, \texttt{nat\_EQcl} has just one
member).  The use of the projection \texttt{op} requires its argument to be in
the class \texttt{EQ}, and uses such a member (function) to actually compare
its arguments.

Similarly, we could equip any other type with a comparison relation, and
use the \texttt{==} notation on terms of this type.

\subsection{Derived Canonical Structures}

We know how to use \texttt{==} on base types, like \texttt{nat}, \texttt{bool}, \texttt{Z}.
Here we show how to deal with type constructors, i.e. how to make the
following example work:

\begin{coq_example}
Fail Check forall (e : EQ.type) (a b : EQ.obj e), (a,b) == (a,b).
\end{coq_example}

The error message is telling that Coq has no idea on how to compare
pairs of objects.  The following construction is telling Coq exactly how to do
that.

\begin{coq_example}
Definition pair_eq (e1 e2 : EQ.type) (x y : EQ.obj e1 * EQ.obj e2) :=
  fst x == fst y /\ snd x == snd y.
Definition pair_EQcl e1 e2 := EQ.Class (pair_eq e1 e2).
Canonical Structure pair_EQty (e1 e2 : EQ.type) : EQ.type :=
  EQ.Pack (EQ.obj e1 * EQ.obj e2) (pair_EQcl e1 e2).
Check forall (e : EQ.type) (a b : EQ.obj e), (a,b) == (a,b).
Check forall n m : nat, (3,4) == (n,m).
\end{coq_example}

Thanks to the \texttt{pair\_EQty} declaration, Coq is able to build a comparison
relation for pairs whenever it is able to build a comparison relation
for each component of the pair.  The declaration associates to the key
\texttt{*} (the type constructor of pairs) the canonical comparison relation
\texttt{pair\_eq} whenever the type constructor \texttt{*} is applied to two types
being themselves in the \texttt{EQ} class.

\section{Hierarchy of structures}

To get to an interesting example we need another base class to be available.
We choose the class of types that are equipped with an order relation,
to which we associate the infix \texttt{<=} notation.

\begin{coq_example}
Module LE.
  Record class T := Class { cmp : T -> T -> Prop }.
  Structure type := Pack { obj : Type; class_of : class obj }.
  Definition op (e : type) : obj e -> obj e -> Prop :=
    let 'Pack _ (Class _ f) := e in f.
  Arguments op {_} x y : simpl never.
  Arguments Class {T} cmp.
  Module theory.
  Notation "x <= y" := (op x y) (at level 70).
  End theory.
End LE.
\end{coq_example}

As before we register a canonical \texttt{LE} class for \texttt{nat}.

\begin{coq_example}
Import LE.theory.
Definition nat_le x y := nat_compare x y <> Gt.
Definition nat_LEcl : LE.class nat := LE.Class nat_le.
Canonical Structure nat_LEty : LE.type := LE.Pack nat nat_LEcl.
\end{coq_example}

And we enable Coq to relate pair of terms with \texttt{<=}.

\begin{coq_example}
Definition pair_le e1 e2 (x y : LE.obj e1 * LE.obj e2) :=
  fst x <= fst y /\ snd x <= snd y.
Definition pair_LEcl e1 e2 := LE.Class (pair_le e1 e2).
Canonical Structure pair_LEty (e1 e2 : LE.type) : LE.type :=
  LE.Pack (LE.obj e1 * LE.obj e2) (pair_LEcl e1 e2).
Check (3,4,5) <= (3,4,5).
\end{coq_example}

At the current stage we can use \texttt{==} and \texttt{<=} on concrete types,
like tuples of natural numbers, but we can't develop an algebraic
theory over the types that are equipped with both relations.

\begin{coq_example}
Check 2 <= 3 /\ 2 == 2.
Fail Check forall (e : EQ.type) (x y : EQ.obj e), x <= y -> y <= x -> x == y.
Fail Check forall (e : LE.type) (x y : LE.obj e), x <= y -> y <= x -> x == y.
\end{coq_example}

We need to define a new class that inherits from both \texttt{EQ} and \texttt{LE}.

\begin{coq_example}
Module LEQ.
  Record mixin (e : EQ.type) (le : EQ.obj e -> EQ.obj e -> Prop) :=
    Mixin { compat : forall x y : EQ.obj e, le x y /\ le y x <-> x == y }.
  Record class T := Class {
    EQ_class    : EQ.class T;
    LE_class    : LE.class T;
    extra : mixin (EQ.Pack T EQ_class) (LE.cmp T LE_class) }.
  Structure type := _Pack { obj : Type; class_of : class obj }.
  Arguments Mixin {e le} _.
  Arguments Class {T} _ _ _.
\end{coq_example}

The \texttt{mixin} component of the \texttt{LEQ} class contains all the extra content
we are adding to \texttt{EQ} and \texttt{LE}.  In particular it contains the requirement
that the two relations we are combining are compatible.

Unfortunately there is still an obstacle to developing the algebraic theory
of this new class.

\begin{coq_example}
  Module theory.
  Fail Check forall (le : type) (n m : obj le), n <= m -> n <= m -> n == m.
\end{coq_example}

The problem is that the two classes \texttt{LE} and \texttt{LEQ} are not yet related by
a subclass relation.  In other words Coq does not see that an object
of the \texttt{LEQ} class is also an object of the \texttt{LE} class.

The following two constructions tell Coq how to canonically build
the \texttt{LE.type} and \texttt{EQ.type} structure given an \texttt{LEQ.type} structure
on the same type.

\begin{coq_example}
  Definition to_EQ (e : type) : EQ.type :=
    EQ.Pack (obj e) (EQ_class _ (class_of e)).
  Canonical Structure to_EQ.
  Definition to_LE (e : type) : LE.type :=
    LE.Pack (obj e) (LE_class _ (class_of e)).
  Canonical Structure to_LE.
\end{coq_example}
We can now formulate out first theorem on the objects of the \texttt{LEQ} structure.
\begin{coq_example}
  Lemma lele_eq (e : type) (x y : obj e) : x <= y -> y <= x -> x == y.
   now intros; apply (compat _ _ (extra _ (class_of e)) x y); split. Qed.
  Arguments lele_eq {e} x y _ _.
  End theory.
End LEQ.
Import LEQ.theory.
Check lele_eq.
\end{coq_example}

Of course one would like to apply results proved in the algebraic
setting to any concrete instate of the algebraic structure.

\begin{coq_example}
Example test_algebraic (n m : nat) :  n <= m -> m <= n -> n == m.
 Fail apply (lele_eq n m). Abort.
Example test_algebraic2 (l1 l2 : LEQ.type) (n m : LEQ.obj l1 * LEQ.obj l2) :
  n <= m -> m <= n -> n == m.
 Fail apply (lele_eq n m). Abort.
\end{coq_example}

Again one has to tell Coq that the type \texttt{nat} is in the \texttt{LEQ} class, and how
the type constructor \texttt{*} interacts with the \texttt{LEQ} class.  In the following
proofs are omitted for brevity.

\begin{coq_example}
Lemma nat_LEQ_compat (n m : nat) : n <= m /\ m <= n <-> n == m.
\end{coq_example}
\begin{coq_eval}

split.
  unfold EQ.op; unfold LE.op; simpl; unfold nat_le; unfold nat_eq.
  case (nat_compare_spec n m); [ reflexivity | | now intros _ [H _]; case H ].
  now intro H; apply nat_compare_gt in H; rewrite -> H; intros [_ K]; case K.
unfold EQ.op; unfold LE.op; simpl; unfold nat_le; unfold nat_eq.
case (nat_compare_spec n m); [ | intros H1 H2; discriminate H2 .. ].
intro H; rewrite H; intros _; split; [ intro H1; discriminate H1 | ].
case (nat_compare_eq_iff m m); intros _ H1.
now rewrite H1; auto; intro H2; discriminate H2.
Qed.
\end{coq_eval}
\begin{coq_example}
Definition nat_LEQmx := LEQ.Mixin nat_LEQ_compat.
Lemma pair_LEQ_compat (l1 l2 : LEQ.type) (n m : LEQ.obj l1 * LEQ.obj l2) :
n <= m /\ m <= n <-> n == m.
\end{coq_example}
\begin{coq_eval}

case n; case m; unfold EQ.op; unfold LE.op; simpl.
intros n1 n2 m1 m2; split; [ intros [[Le1 Le2] [Ge1 Ge2]] | intros [H1 H2] ].
  now split; apply lele_eq.
case (LEQ.compat _ _ (LEQ.extra _ (LEQ.class_of l1)) m1 n1).
case (LEQ.compat _ _ (LEQ.extra _ (LEQ.class_of l2)) m2 n2).
intros _ H3 _ H4; apply H3 in H2; apply H4 in H1; clear H3 H4.
now case H1; case H2; split; split.
Qed.
\end{coq_eval}
\begin{coq_example}
Definition pair_LEQmx l1 l2 := LEQ.Mixin (pair_LEQ_compat l1 l2).
\end{coq_example}

The following script registers an \texttt{LEQ} class for \texttt{nat} and for the
type constructor \texttt{*}.  It also tests that they work as expected.

Unfortunately, these declarations are very verbose.  In the following
subsection we show how to make these declaration more compact.

\begin{coq_example}
Module Add_instance_attempt.
  Canonical Structure nat_LEQty : LEQ.type :=
    LEQ._Pack nat (LEQ.Class nat_EQcl nat_LEcl nat_LEQmx).
  Canonical Structure pair_LEQty (l1 l2 : LEQ.type) : LEQ.type :=
    LEQ._Pack (LEQ.obj l1 * LEQ.obj l2)
      (LEQ.Class
        (EQ.class_of (pair_EQty (to_EQ l1) (to_EQ l2)))
        (LE.class_of (pair_LEty (to_LE l1) (to_LE l2)))
        (pair_LEQmx l1 l2)).
  Example test_algebraic (n m : nat) : n <= m -> m <= n -> n == m.
   now apply (lele_eq n m). Qed.
  Example test_algebraic2 (n m : nat * nat) : n <= m -> m <= n -> n == m.
   now apply (lele_eq n m). Qed.
End Add_instance_attempt.
\end{coq_example}

Note that no direct proof of \texttt{n <= m -> m <= n -> n == m} is provided by the
user for \texttt{n} and \texttt{m} of type \texttt{nat * nat}.  What the user provides is a proof of
this statement for \texttt{n} and \texttt{m} of type \texttt{nat} and a proof that the pair
constructor preserves this property.  The combination of these two facts is a
simple form of proof search that Coq performs automatically while inferring
canonical structures.

\subsection{Compact declaration of Canonical Structures}

We need some infrastructure for that.

\begin{coq_example*}
Require Import Strings.String.
\end{coq_example*}
\begin{coq_example}
Module infrastructure.
  Inductive phantom {T : Type} (t : T) : Type := Phantom.
  Definition unify {T1 T2} (t1 : T1) (t2 : T2) (s : option string) :=
    phantom t1 -> phantom t2.
  Definition id {T} {t : T} (x : phantom t) := x.
  Notation "[find v | t1 ~ t2 ] p" := (fun v (_ : unify t1 t2 None) => p)
    (at level 50, v ident, only parsing).
  Notation "[find v | t1 ~ t2 | s ] p" := (fun v (_ : unify t1 t2 (Some s)) => p)
    (at level 50, v ident, only parsing).
  Notation "'Error : t : s" := (unify _ t (Some s))
    (at level 50, format "''Error'  :  t  :  s").
  Open Scope string_scope.
End infrastructure.
\end{coq_example}

To explain the notation \texttt{[find v | t1 \textasciitilde t2]} let us pick one
of its instances: \texttt{[find e | EQ.obj e \textasciitilde T | "is not an EQ.type" ]}.
It should be read as: ``find a class e such that its objects have type T
or fail with message "T is not an EQ.type"''.

The other utilities are used to ask Coq to solve a specific unification
problem, that will in turn require the inference of some canonical
structures.  They are explained in mode details in~\cite{CSwcu}.

We now have all we need to create a compact ``packager'' to declare
instances of the \texttt{LEQ} class.

\begin{coq_example}
Import infrastructure.
Definition packager T e0 le0 (m0 : LEQ.mixin e0 le0) :=
    [find e  | EQ.obj e ~ T       | "is not an EQ.type" ]
    [find o  | LE.obj o ~ T       | "is not an LE.type" ]
    [find ce | EQ.class_of e ~ ce ]
    [find co | LE.class_of o ~ co ]
    [find m  | m ~ m0             | "is not the right mixin" ]
  LEQ._Pack T (LEQ.Class ce co m).
Notation Pack T m := (packager T _ _ m _ id _ id _ id _ id _ id).
\end{coq_example}

The object \texttt{Pack} takes a type \texttt{T} (the key) and a mixin \texttt{m}.  It infers all
the other pieces of the class \texttt{LEQ} and declares them as canonical values
associated to the \texttt{T} key.  All in all, the only new piece of information
we add in the \texttt{LEQ} class is the mixin, all the rest is already canonical
for \texttt{T} and hence can be inferred by Coq.

\texttt{Pack} is a notation, hence it is not type checked at the time of its
declaration.  It will be type checked when it is used, an in that case
\texttt{T} is going to be a concrete type.  The odd arguments \texttt{\_} and \texttt{id} we
pass to the
packager represent respectively the classes to be inferred (like \texttt{e}, \texttt{o}, etc) and a token (\texttt{id}) to force their inference.  Again, for all the details the
reader can refer to~\cite{CSwcu}.

The declaration of canonical instances can now be way more compact:

\begin{coq_example}
Canonical Structure nat_LEQty := Eval hnf in Pack nat nat_LEQmx.
Canonical Structure pair_LEQty (l1 l2 : LEQ.type) :=
  Eval hnf in Pack (LEQ.obj l1 * LEQ.obj l2) (pair_LEQmx l1 l2).
\end{coq_example}

Error messages are also quite intelligible (if one skips to the end of
the message).

\begin{coq_example}
Fail Canonical Structure err := Eval hnf in Pack bool nat_LEQmx.
\end{coq_example}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "Reference-Manual"
%%% End:


\documentclass[a4paper]{article}
\usepackage{etoolbox}
\usepackage[colorlinks,bookmarksopen,bookmarksnumbered,citecolor=red,urlcolor=red]{hyperref}
\usepackage[affil-it]{authblk}
\usepackage[cm]{fullpage}
\usepackage{amsmath, amssymb, amsthm, amscd, mathtools, stmaryrd}
\usepackage{fullpage}
\usepackage{algorithm,algcompatible}
\usepackage{listings}
\usepackage{color}
\usepackage{graphicx}
\usepackage{multirow}
\usepackage{subfig}
\usepackage{mathrsfs}
\usepackage{booktabs,tabularx}
\usepackage{caption}

% For diagrams
\usepackage{tikz}
\usetikzlibrary{matrix}

%%%%%%%%% Custom math commands %%%%%%%%%
\DeclarePairedDelimiter\norm{\lVert}{\rVert}
\DeclarePairedDelimiter\jump{\llbracket}{\rrbracket}
\DeclarePairedDelimiter\avg{\{\!\!\{}{\}\!\!\}}
\DeclarePairedDelimiter{\both}{\lbrace}{\rbrace}
\renewcommand{\vec}[1]{\boldsymbol{#1}}
\newcommand{\zhat}{\hat{\vec{z}}}
\newcommand{\rhat}{\hat{\vec{r}}}
\newcommand{\ddt}[1]{\frac{\partial #1}{\partial t}}
\newcommand{\Uspace}{\mathbb{U}}
\newcommand{\Vspace}{\mathbb{V}}
\newcommand{\Wspace}{\mathbb{W}}
\newcommand{\Hdiv}{\texttt{HDiv}}
\newcommand{\Hcurl}{\texttt{HCurl}}

\newcommand{\thgsays}[1]{{\bfseries Thomas says:} {\color{blue} #1}}
\newcommand{\cjcsays}[1]{{\bfseries Colin says:} {\color{red} #1}}

\title{Code verification for the manuscript:
``Slate: extending Firedrake's domain-specific abstraction to hybridized solvers for geoscience and beyond''}

\author[$\dagger$,1]{Thomas H. Gibson}
\affil[1]{Department of Mathematics, Imperial College, South Kensington Campus, London SW7 2AZ}
\affil[$\dagger$]{Email: \texttt{t.gibson15@imperial.ac.uk}}

\date{\today}
\begin{document}

\maketitle

\section{Code verification}\label{subsec:hybridpoisson}
To verify our computed results, we now perform a simple convergence study for a model Dirichlet
problem. We seek a solution to the Poisson equation as a first-order system:
\begin{align}\label{eq:modelpoisson}
	\boldsymbol{u} + \nabla p &= 0 \text{ in } \Omega = \lbrack 0, 1\rbrack^2, \\
	\nabla\cdot\boldsymbol{u} &= f \text{ in } \Omega, \\
	p &= p_0 \text{ on } \partial\Omega_D,
\end{align}
where $f$ and $p_0$ are chosen so that the analytic solution is the sinusoid
$p(x, y) = \sin(\pi x)\sin(\pi y)$ and its negative gradient. We solve this problem
by hybridizing the mixed formulation of \eqref{eq:modelpoisson},
and employ our static condensation preconditioner described in Section
4.1.1 of the manuscript. All results were obtained in serial, with MUMPS providing the
LU factorization algorithms for the condensed trace system.

Each mesh in our convergence study is obtained by generating a quadrilateral mesh with $2^r$ cells
in each spatial direction, and dividing each quadrilateral cell into two equal simplicial elements.
Once the solutions are obtained, we compute a post-processed scalar solution using the method
described in Section 3.3.1 of the manuscript
via Slate-generated kernels. Figure \ref{fig:h-rt-conv}
displays the results for the hybridized Raviart-Thomas (RT) method. Our computations are in full agreement
with the theory.
\begin{figure}[!htbp]
	\centering
	\includegraphics[width=\textwidth]{HMM/HRT-convergence}
	\caption{Error convergence rates for our implementation of the hybridized RT method of
		orders 0, 1, 2, and 3. We observe the expected rates for the scalar
		and flux solutions of the standard RT method:
		$k+1$ in the $L^2$-error for both the scalar and flux approximations.
		Additionally, we see the effects of post-processing the scalar solution, yielding
		superconvergent $k+2$ rates.}
	\label{fig:h-rt-conv}
\end{figure}

We repeat this experiment for the LDG-H method with varying choices of $\tau$
in order to verify how $\tau$ influences the convergence rates, comparing with the
expected rates for the LDG-H method given a particular order of $\tau$
(see Table \ref{table:ldg-h-rates} for a summary).
In all our experiments, we use the post-processing methods described in Sections
3.3.1 and 3.3.2 to produce approximations $p_h^\star$ and $\boldsymbol{u}_h^\star$.
Error convergence plots from our tests are shown in Figure \ref{fig:ldgh-conv} that confirm the expected
rates. This rather sensitive test verifies that our software framework is generating correct code.
\begin{table}[!htbp]
	\centering
	\caption{The expected convergence rates of the LDG-H method with a
		stability parameter $\tau$ of a particular order.}
	\begin{tabular}{ccccc}
		\hline
		parameter &
		\multicolumn{4}{c}{expected rates of convergence ($k \geq 1$)} \\
		\cline{2-5}
		$\tau$ & $\norm{p - p_h}_{L^2(\Omega)}$ &
		$\norm{\boldsymbol{u} - \boldsymbol{u}_h}_{\boldsymbol{L}^2(\Omega)}$ &
		$\norm{p - p^\star_h}_{L^2(\Omega)}$ &
		$\norm{\boldsymbol{u} - \boldsymbol{u}^\star_h}_{\boldsymbol{L}^2(\Omega)}$ \\
		\hline
		$\mathcal{O}(1)$ & $k+1$ & $k+1$ & $k+2$ & $k+1$ \\
		\hline
		$\mathcal{O}(h)$ & $k$ & $k+1$ & $k+2$ & $k+1$ \\
		\hline
		$\mathcal{O}\left(h^{-1}\right)$ & $k+1$ & $k$ & $k+1$ & $k$ \\
		\hline
	\end{tabular}
	\label{table:ldg-h-rates}
\end{table}
\begin{figure}[!htbp]
	\centering
	\includegraphics[width=0.8\textwidth]{LDGH/LDGH-convergence}
	\caption{Error convergence rates for our implementation of the LDG-H method with
		$\tau = h$ and $\tau = \frac{1}{h}$.
		The expected sensitivity of this discretization subject to appropriate choices of
		stabilization parameter $\tau$ is verified. We see no change in the convergence
		rates between the scalar and post-processed scalar solutions when $\tau = \frac{1}{h}$.
		Superconvergence is achieved when taking $\tau = h$. The post-processed flux rates in both
		cases match the rates of the unprocessed flux.}
	\label{fig:ldgh-conv}
\end{figure}

\end{document}


\section{Data Structures}
\label{sec:datastruct}
%\ifthenelse{\boolean{mtc}}{\minitoc}{}

Access to the facilities provided by \verb|psblas-ext| is mainly
through the data types that are provided within.
The data classes are derived from the base  classes in PSBLAS, through
the Fortran~2003 mechanism of \emph{type extension}~\cite{MRC:11}.

The data classes are divided between the general purpose CPU
extensions, the GPU interfaces and the RSB interfaces.

In the description we will make use of the notation introduced in
Table~\ref{tab:notation}.

\begin{table}[ht]
\caption{Notation for parameters describing a sparse matrix}
\begin{center}
{\footnotesize
\begin{tabular}{ll}
\hline
Name & Description \\
\hline
M		& Number of rows in matrix		 \\
N		& Number of columns in matrix		 \\
NZ              & Number of nonzeros in matrix   \\
AVGNZR          & Average number of nonzeros per row  \\
MAXNZR          & Maximum number of nonzeros per row  \\
NDIAG           & Numero of nonzero diagonals\\
AS	        & Coefficients 	array		 \\
IA	        & Row indices array			 \\
JA	        & Column  indices array			 \\
IRP	        & Row start pointers array			 \\
JCP	        & Column start pointers array			 \\
NZR 	        & Number of nonzeros per row array \\
OFFSET          & Offset for diagonals			 \\
\hline
\end{tabular}
}
\end{center}
\label{tab:notation}
\end{table}

\begin{figure}[ht]
	\centering
%		\includegraphics[width=5.2cm]{images/mat.eps}
		\includegraphics[width=5.2cm]{images/mat.pdf}
	\caption{Example of sparse matrix}
	\label{fig:dense}
\end{figure}

\subsection{CPU-class extensions}

\subsubsection*{ELLPACK}

The ELLPACK/ITPACK format (shown in Figure~\ref{fig:ell})
comprises  two 2-dimensional arrays \verb|AS| and
\verb|JA|  with \verb|M| rows and \verb|MAXNZR| columns, where
\verb|MAXNZR| is the maximum
number of nonzeros in any row~\cite{ELLPACK}.
Each row of the arrays \verb|AS| and \verb|JA| contains the
coefficients and column indices; rows shorter than
\verb|MAXNZR| are padded with zero coefficients and appropriate column
indices, e.g. the last valid one found in the same row.

\begin{figure}[ht]
	\centering
%		\includegraphics[width=8.2cm]{images/ell.eps}
		\includegraphics[width=8.2cm]{images/ell.pdf}
	\caption{ELLPACK compression of matrix in Figure~\ref{fig:dense}}
	\label{fig:ell}
\end{figure}

\begin{algorithm}
\lstset{language=Fortran}
\small
  \begin{lstlisting}
    do i=1,n
      t=0
      do j=1,maxnzr
        t = t +  as(i,j)*x(ja(i,j))
      end do
      y(i) = t
    end do
  \end{lstlisting}
  \caption{\label{alg:ell} Matrix-Vector product in ELL format}
\end{algorithm}
The matrix-vector product $y=Ax$ can be computed with the code shown in
Alg.~\ref{alg:ell}; it costs  one  memory write per outer iteration,
plus three memory reads  and two floating-point operations per inner
iteration.

Unless all rows have exactly the same number of nonzeros, some of the
coefficients in the \verb|AS| array will be zeros; therefore this
data structure will have  an overhead both in terms of memory space
and redundant operations (multiplications by zero).  The overhead can
be acceptable if:
\begin{enumerate}
\item The maximum number of nonzeros per row is not much larger than
  the    average;
\item The regularity of the data structure allows for faster  code,
  e.g. by allowing vectorization, thereby offsetting the additional
  storage requirements.
\end{enumerate}
In the extreme case where the input matrix has one full row, the
ELLPACK structure would require more memory than the normal 2D array
storage. The ELLPACK storage format was very popular in the vector
computing days; in modern CPUs it is not quite as popular, but it
is  the basis for many GPU formats.

The relevant data type is \verb|psb_T_ell_sparse_mat|:
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
  type, extends(psb_d_base_sparse_mat) :: psb_d_ell_sparse_mat
    !
    ! ITPACK/ELL format, extended.
    !

    integer(psb_ipk_), allocatable :: irn(:), ja(:,:), idiag(:)
    real(psb_dpk_), allocatable :: val(:,:)

  contains
    ....
  end type psb_d_ell_sparse_mat
\end{minted}

\subsubsection*{Hacked ELLPACK}

The \textit{hacked ELLPACK} (\textbf{HLL}) format
alleviates the main problem of the ELLPACK format, that is,
the  amount of  memory required by  padding for  sparse matrices in
which the maximum row length is  larger than the average.

The number of  elements  allocated to padding is $[(m*maxNR) -
(m*avgNR) = m*(maxNR-avgNR)]$
for both \verb|AS|  and \verb|JA| arrays,
where $m$ is equal to the number of rows of the matrix, $maxNR$ is the
maximum number of nonzero elements
in every row and $avgNR$ is the average number of nonzeros.
Therefore a single densely populated row can seriously affect the
total size of the allocation.

To limit this effect, in the HLL format  we break the original matrix
into equally sized groups of rows (called \textit{hacks}), and then store
these groups as independent matrices in ELLPACK format.
The groups can be arranged selecting rows in an arbitrarily manner;
indeed, if the rows are sorted by decreasing number of nonzeros we
obtain essentially the JAgged Diagonals format.
If the rows are not in the original order, then an   additional vector
\textit{rIdx} is required, storing the actual row index  for each row
in the data structure.

The multiple ELLPACK-like buffers are stacked together inside a
single, one dimensional array;
an additional  vector \textit{hackOffsets} is provided to keep track
of the individual submatrices.
All hacks have the same number of rows  \textit{hackSize}; hence,
the \textit{hackOffsets} vector is  an array of
$(m/hackSize)+1$ elements, each one pointing  to the first index of a
submatrix inside the stacked \textit{cM}/\textit{rP} buffers, plus an
additional element pointing past the end of the last block, where the
next one would begin.
We thus have the property that
the elements of the $k$-th \textit{hack} are stored between \verb|hackOffsets[k]| and
\verb|hackOffsets[k+1]|, similarly to what happens in the CSR format.

\begin{figure}[ht]
	\centering
%		\includegraphics[width=8.2cm]{../images/hll.eps}
	 \includegraphics[width=.72\textwidth]{../images/hll.pdf}
	\caption{Hacked ELLPACK compression of matrix in Figure~\ref{fig:dense}}
	\label{fig:hll}
\end{figure}

With this data structure a very long row only affects one hack, and
therefore the additional memory is limited to the hack in which the
row appears.

The relevant data type is \verb|psb_T_hll_sparse_mat|:
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
  type, extends(psb_d_base_sparse_mat) :: psb_d_hll_sparse_mat
    !
    ! HLL format. (Hacked ELL)
    !
    integer(psb_ipk_) :: hksz
    integer(psb_ipk_), allocatable :: irn(:), ja(:), idiag(:), hkoffs(:)
    real(psb_dpk_), allocatable :: val(:)

  contains
   ....
  end type
\end{minted}

\subsubsection*{Diagonal storage}

The DIAgonal (DIA) format (shown in Figure~\ref{fig:dia})
has   a 2-dimensional array \verb|AS| containing in each column the
coefficients along a  diagonal of the matrix, and an integer array
\verb|OFFSET|  that determines  where each diagonal starts. The
diagonals in \verb|AS| are padded with zeros as necessary.

The code to compute the matrix-vector product $y=Ax$ is shown in Alg.~\ref{alg:dia};
it costs one  memory read per outer iteration,
plus three memory reads, one memory write  and two floating-point
operations per inner iteration. The accesses to  \verb|AS| and
\verb|x| are in strict sequential order,  therefore no indirect
addressing is required.

\begin{figure}[ht]
	\centering
%		\includegraphics[width=8.2cm]{images/dia.eps}
		\includegraphics[width=.72\textwidth]{images/dia.pdf}
	\caption{DIA compression of matrix in Figure~\ref{fig:dense}}
	\label{fig:dia}
\end{figure}

\begin{algorithm}
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
    do j=1,ndiag
      if (offset(j) > 0) then
        ir1 = 1; ir2 = m - offset(j);
      else
        ir1 = 1 - offset(j); ir2 = m;
      end if
      do i=ir1,ir2
        y(i) = y(i) + alpha*as(i,j)*x(i+offset(j))
      end do
    end do
\end{minted}
  \caption{\label{alg:dia} Matrix-Vector product in DIA format}
\end{algorithm}

The relevant data type is \verb|psb_T_dia_sparse_mat|:
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
  type, extends(psb_d_base_sparse_mat) :: psb_d_dia_sparse_mat
    !
    ! DIA format, extended.
    !

    integer(psb_ipk_), allocatable :: offset(:)
    integer(psb_ipk_) :: nzeros
    real(psb_dpk_), allocatable :: data(:,:)

  end type
\end{minted}

\subsubsection*{Hacked DIA}

Storage by DIAgonals is an attractive option for matrices whose
coefficients are located on a small set of diagonals, since they do
away with storing explicitly the indices and therefore reduce
significantly memory traffic. However, having a few coefficients
outside of the main set of diagonals may  significantly increase the
amount of needed padding; moreover, while the DIA code is easily
vectorized, it does not necessarily make optimal use of the memory
hierarchy. While processing each diagonal we are updating entries in
the output vector \verb|y|, which is then accessed multiple times; if
the vector \verb|y| is too large to remain in the cache memory, the
associated cache miss penalty is paid multiple times.

The \textit{hacked DIA} (\textbf{HDIA}) format was designed to contain
the amount of padding, by  breaking  the original matrix
into equally sized groups of rows (\textit{hacks}), and then storing
these groups as independent matrices in DIA format. This approach is
similar to that of HLL, and requires using an offset vector for each
submatrix. Again, similarly to HLL, the various submatrices are
stacked inside a linear array to improve memory management. The fact
that the matrix is accessed in slices helps in reducing cache misses,
especially regarding accesses to the %output
vector \verb|y|.

An additional vector \textit{hackOffsets} is provided to complete
the matrix format; given  that \textit{hackSize} is the number of rows of each hack,
the \textit{hackOffsets} vector is made by an array of
$(m/hackSize)+1$ elements,  pointing to the first diagonal offset of a
submatrix inside the stacked \textit{offsets} buffers, plus an
additional element equal to the number of nonzero diagonals in the whole matrix.
We thus have the property that
the number of diagonals of the $k$-th \textit{hack} is given by
\textit{hackOffsets[k+1] - hackOffsets[k]}.

\begin{figure}[ht]
	\centering
%		\includegraphics[width=8.2cm]{../images/hdia.eps}
		\includegraphics[width=.72\textwidth]{../images/hdia.pdf}
	\caption{Hacked DIA compression of matrix in Figure~\ref{fig:dense}}
	\label{fig:hdia}
\end{figure}

The relevant data type is \verb|psb_T_hdia_sparse_mat|:
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
  type pm
     real(psb_dpk_), allocatable  :: data(:,:)
  end type pm

  type po
     integer(psb_ipk_), allocatable  :: off(:)
  end type po

  type, extends(psb_d_base_sparse_mat) :: psb_d_hdia_sparse_mat
    !
    ! HDIA format, extended.
    !

    type(pm), allocatable :: hdia(:)
    type(po), allocatable :: offset(:)
    integer(psb_ipk_) :: nblocks, nzeros
    integer(psb_ipk_) :: hack = 64
    integer(psb_long_int_k_) :: dim=0

  contains
   ....
  end type
\end{minted}

\subsection{GPU-class extensions}

For computing on the GPU we define a dual memorization strategy in
which each variable on the CPU (``host'') side has a GPU (``device'')
side. When a GPU-type variable is initialized, the data contained is
(usually) the same on both sides. Each operator invoked on the
variable may change the data so that only the host side or the device
side are up-to-date.

Keeping track of the updates to data in the variables  is essential: we want
to perform most  computations on the GPU, but we cannot afford the time
needed to move data between the host  memory and the device memory
because the bandwidth of the interconnection bus would become the main
bottleneck of the computation. Thus, each and every computational
routine in the library is built according to the following principles:
\begin{itemize}
\item If the data type being handled is {GPU}-enabled, make sure that
  its device copy is up to date, perform any arithmetic operation on
  the {GPU}, and if the data has been altered as a result, mark
  the main-memory copy as outdated.
\item The main-memory copy is never updated unless this is requested
  by the user either
\begin{description}
\item[explicitly] by invoking a synchronization method;
\item[implicitly] by invoking a method that involves other data items
  that are not {GPU}-enabled, e.g., by assignment ov a vector to a
  normal array.
\end{description}
\end{itemize}
In this way, data items are put on the {GPU} memory ``on demand'' and
remain there as long as ``normal'' computations are carried out.
As an example, the following call to a matrix-vector product
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
    call psb_spmm(alpha,a,x,beta,y,desc_a,info)
\end{minted}
will transparently and automatically be performed on the {GPU} whenever
all three data inputs \fortinline|a|, \fortinline|x|  and
\fortinline|y| are {GPU}-enabled. If a program makes many such calls
sequentially, then
\begin{itemize}
\item The first kernel invocation will find the data in main memory,
  and will copy it to the {GPU} memory, thus incurring a significant
  overhead; the result is however \emph{not} copied back, and
  therefore:
\item Subsequent kernel invocations involving the same vector will
  find the data on the {GPU} side so that they will run at full
  speed.
\end{itemize}
For all invocations after the first the only data that will have to be
transferred to/from the main memory will be the scalars \fortinline|alpha|
and \fortinline|beta|, and the return code \fortinline|info|.

\begin{description}
\item[Vectors:] The data type \fortinline|psb_T_vect_gpu| provides a
  GPU-enabled extension of the inner type \fortinline|psb_T_base_vect_type|,
  and must be used together with the other inner matrix type to make
  full use of the GPU computational capabilities;
\item[CSR:] The data type \fortinline|psb_T_csrg_sparse_mat| provides an
  interface to the GPU version of CSR available in the NVIDIA CuSPARSE
  library;
\item[HYB:] The data type \fortinline|psb_T_hybg_sparse_mat| provides an
  interface to the HYB GPU storage  available in the NVIDIA CuSPARSE
  library. The internal structure is opaque, hence the host side is
  just CSR; the HYB data format is only available up to CUDA version
  10.
\item[ELL:] The data type \fortinline|psb_T_elg_sparse_mat| provides an
  interface to the  ELLPACK implementation from SPGPU;

\item[HLL:] The data type \fortinline|psb_T_hlg_sparse_mat| provides an
  interface to the  Hacked ELLPACK implementation from SPGPU;
\item[HDIA:] The data type \fortinline|psb_T_hdiag_sparse_mat| provides an
  interface to the  Hacked DIAgonals implementation from SPGPU;
\end{description}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "userguide"
%%% End:


%!TEX root = ../notes.tex
\section{March 15, 2022}
\subsection{Number Fields \emph{continued}}
\recall from last class...
\begin{definition*}
    A complex number $\alpha$ is called \ul{algebraic} if it is algebraic over $\QQ$.
\end{definition*}
\begin{proposition*}
    The set $\overline{\QQ}$ of algebraic numbers is a subfield of $\CC$.
\end{proposition*}
\begin{definition*}
    A number field is a subfield $K$ of $\CC$ such that $[K: \QQ]<\infty$.
\end{definition*}
\begin{theorem*}[Primitive Element Theorem]
    If $K$ is a number field, then $K = \QQ(\theta)$ for some $\theta\in \overline{\QQ}$.
\end{theorem*}
\begin{proof}[Crux of proof]
    Suppose $K = K_1(\alpha, \beta)$. Thus $K = K_1(\theta)$ for some $\theta$ that is easily found as a function of $\alpha$ and $\beta$.

    Write the minimum polynomials of $\alpha, \beta$ over $K_1$(factored over $\CC$)
    \begin{align*}
        (t-\alpha_1)(t-\alpha_2)\cdots (t-\alpha_n)\quad \alpha_i\in\overline{\QQ}, \text{ and }\alpha =: \alpha_1 \\
        (t-\beta_1)(t-\beta_2)\cdots (t-\beta_m)\quad \beta_i\in\overline{\QQ}, \text{ and }\beta =: \beta_1
    \end{align*}
    These are separable. Hence for each $i$ and each $k\neq 1$, there exists at most one $x\in K$ such that
    \[\alpha_i + x\beta_k = \alpha_1 + x\beta_1\]
    Hence, since there are only finitely many of these equations, we can choose $0\pm c\in K$, such that
    \[\alpha_i + c\beta_k \neq \alpha_1 + c\beta_1\]
    for any $1\leq i\leq n$ and $2\leq k\leq m$. We define $\theta = \alpha + c\beta$. We claim $K = K_1(\theta)$.
\end{proof}
The proof up to this claim is actually useful in finding a primitive element.
\begin{example}[p.39 \cite{stewart2015algebraic}]
    Let $K = \QQ(\sqrt{2}, \sqrt[3]{5})$.
    \[\alpha_1 = \sqrt{2}, \beta_1 = \sqrt[3]{5}\]
    We then have $\alpha_2 = -\sqrt{2}$ and we can let $\beta_2 = \zeta_3\sqrt[3]{5}, \beta_2 = \zeta_3^2\sqrt[3]{5}$ where $\zeta_3$ is the primitive 3rd root of unity.

    $c = 1$ has the property that
    \[\alpha_i + c\beta_k \neq \alpha_1 + c\beta_1\]
    for $1\leq i\leq 2$ and $2\leq k\leq 3$.

    Therefore, we can conclude that $\sqrt{2} + \sqrt[3]{5}$ is a primitive element for $\QQ(\sqrt{2}, \sqrt[3]{5})/\QQ$.
\end{example}

\subsection{Conjugates of Algebraic Numbers}
\begin{theorem}[p.40 \cite{stewart2015algebraic}]
    Let $K = \QQ(\theta)$ be a number field of degree $n$ over $\QQ$. Then there exists exactly $n$ distinct field embedding of $K$ into $\CC$. (We label these $\sigma_i : K \hookrightarrow \CC, 1\leq i \leq n$.)

    The $\sigma_i(\theta)=:\theta_i$ are the zeros in $\CC$ of the minimal polynomial of $\theta$ over $\QQ$.
\end{theorem}
\begin{proof}
    Suppose $\sigma : K\hookrightarrow \CC$ is an embedding. We have that $\sigma$ is the identity on $\QQ$ (since $\sigma(1) = 1$), so
    \[0=\sigma(f(\theta)) = f(\sigma(\theta))\qquad \text{where $f:= \minpoly_\QQ(\theta)$}.\]
    Hence $\sigma(\theta)$ is a root of $f$.

    Conversely, for each root $\theta_i$ of $f$, there is a field isomorphism\footnote{We find isomorphisms $\QQ(\theta)\overset{\sim}{\longrightarrow} \QQ[x]/f$ and similarly $\QQ(\theta_i)\overset{\sim}{\longrightarrow} \QQ[x]/f$ where $f :=\minpoly_\QQ(\theta) = \minpoly_\QQ(\theta_i)$.} taking
    \[\QQ(\theta)\overset{\sigma_i}{\longrightarrow}\QQ(\theta_i)\]
    such that $\sigma_i (\theta) = \theta_i$. Therefore we've shown a bijection between the roots of $f$ and the embeddings of $K$ into $\CC$.
\end{proof}

\subsection{Discriminants of Bases, Vandermonde Determinant}
Let $K = \QQ(\theta)$ be a number field of degree $n$, and let $\{\alpha_1, \alpha_2, \dots, \alpha_n\}$ be a basis of $K$ as a vector space over $\QQ$. Let $\sigma_i : K\hookrightarrow \CC$, $1\leq i\leq n$ be the embeddings of $K$ into $\CC$.

\begin{definition}
    The discriminant of $\{\alpha_1, \alpha_2, \dots, \alpha_n\}$ is
    \begin{align*}
        \Delta[\alpha_1, \alpha_2, \dots, \alpha_n] & = \det\left(\sigma_i(\alpha_j)\right)^2 \\
                                                    & = \det
        \begin{pmatrix}
            \sigma_1(\alpha_1) & \sigma_1(\alpha_2) & \cdots & \sigma_1(\alpha_n) \\
            \sigma_2(\alpha_1) & \sigma_2(\alpha_2) & \cdots & \sigma_2(\alpha_n) \\
            \vdots             & \vdots             & \ddots & \vdots             \\
            \sigma_n(\alpha_1) & \sigma_n(\alpha_2) & \cdots & \sigma_n(\alpha_n) \\
        \end{pmatrix}^2
    \end{align*}
\end{definition}
If $\{\beta_1, \dots, \beta_n\}$ is another basis, then $\forall 1\leq k\leq n$,
\[\beta_k = \sigma_{i = 1}^n C_{ik}\alpha_i, \quad C_{ik}\in \QQ,\]
where $\det(C_{ik})\neq 0$. Then it is a fact that
\[\Delta[\beta_1, \dots, \beta_n] = \left(\det(C_{ik})\right)^2\cdot \Delta[\alpha_1, \dots, \alpha_n]\]
\begin{theorem}[p.42 \cite{stewart2015algebraic}]\label{thm:2.7}
    The discriminant of any basis for $K = \QQ(\theta)$ is rational and nonzero.
\end{theorem}
\begin{proof}
    It suffices to show that this holds for $\{1, \theta, \theta^2, \dots, \theta^{n-1}\}$ by the above fact.

    Write $\theta = \theta_1$, $\theta_1, \theta_2, \dots, \theta_n$ for the conjugates of $\theta_1$. Then
    \[\Delta[1, \theta, \theta^2, \dots, \theta^{n-1}] = \left(\det(\theta_i^j)\right)^2\]
    We use a general observation
    \begin{definition}[Vandermonde Matrix]
        A (square) \ul{Vandermonde matrix} is a matrix of the form
        \[V = \begin{pmatrix}
                1      & t_1    & t_1^2  & \dots  & t_1^{n-1} \\
                1      & t_2    & t_2^2  & \dots  & t_2^{n-1} \\
                \vdots & \vdots & \vdots & \ddots & \vdots    \\
                1      & t_n    & t_n^2  & \dots  & t_n^{n-1} \\
            \end{pmatrix}\]
    \end{definition}
    \begin{claim}
        The determinant of $V$ is
        \[\prod_{1\leq i < j\leq n}(t_i - t_j).\]
    \end{claim}
    Going back to the proof of our claim about discriminants, we can take $t_i = \theta_i := \sigma_i(\theta)$ to get
    \begin{align*}
        \Delta[1, \theta, \dots, \theta^{n-1}] & = \prod_{i<j}(\theta_i - \theta_j)^2  \\
                                               & = \mathsf{disc}(\minpoly_\QQ(\theta))
    \end{align*}
\end{proof}


\section*{Part B, Problem 1: Decision Tree (30 pts) (Hao and Yiting)}

\subsection*{1.1 Build Your Own Decision Tree (14 pts)}

The following is a small synthetic data set where we try to predict the usage of individual mobile phones based on their income, age, education, and marital status. In this section, you can assume that the decision tree is built using the ID3 algorithm, where each attribute is used only as an internal node.

\begin{center}
  \begin{tabular}{ l l l l | l }
    \hline
    Income & Age & Education & Martial Status & Usage \\ \hline \hline
    Low & Old & University & Married & Low \\ \hline
    Medium & Young & College & Single & Medium  \\ \hline
    Low & Old & University & Married & Low  \\ \hline
    High & Young & University & Single & High  \\ \hline
    Low & Old & University & Married & Low  \\ \hline
    High & Young & College & Single & Medium  \\ \hline
    Medium & Young & College & Married & Medium  \\ \hline
    Medium & Old & High School & Single & Low  \\ \hline
    High & Old & University & Single & High  \\ \hline
    Low & Old & High School & Married & Low  \\ \hline
    Medium & Young & College & Married & Medium  \\ \hline
    Medium & Old & High School & Single & Low  \\ \hline
    High & Old & University & Single & High  \\ \hline
    Low & Old & High School & Married & Low  \\ \hline
    Medium & Young & College & Married & Medium  \\ \hline
  \end{tabular}
\end{center}

\begin{enumerate}

\item\textbf{[2 points]} What is the initial entropy of Usage?
\\\textbf{Answer:}\\
Usage:
\begin{itemize}
	\item Low: 7
	\item Medium: 5
	\item High: 3
\end{itemize}
Therefore,
$$H(Y)=-\frac{7}{15}\log_2\frac{7}{15}-\frac{5}{15}\log_2\frac{5}{15}-\frac{3}{15}\log_2\frac{3}{15}=1.506$$

\item\textbf{[5 points]} Which attribute should be chosen at the root of the tree? Show your calculation for the information gains (IG) and explain your choice in a sentence.
\\\textbf{Answer:}\\
\begin{itemize}
	\item Income:
	\begin{itemize}
		\item Low: 5 (L: 5, M: 0, H: 0)
		$$P(Low)H(Y|Low)=-\frac{5}{15}(\frac{5}{5}\log_2\frac{5}{5}+\frac{0}{5}\log_2\frac{0}{5}+\frac{0}{5}\log_2\frac{0}{5})=0.000$$
		\item Medium: 6 (L: 2, M: 4, H: 0)
		$$P(Medium)H(Y|Low)=-\frac{6}{15}(\frac{2}{6}\log_2\frac{2}{6}+\frac{4}{6}\log_2\frac{4}{6}+\frac{0}{6}\log_2\frac{0}{5})=0.367$$
		\item High: 4 (L: 0, M: 1, H: 3)
		$$P(Medium)H(Y|High)=-\frac{4}{15}(\frac{0}{4}\log_2\frac{0}{4}+\frac{1}{4}\log_2\frac{1}{4}+\frac{3}{4}\log_2\frac{3}{4})=0.216$$
	\end{itemize}
	$$IG(Y;Income)=H(Y)-H(Y|Income)=1.506-0.000-0.367-0.216=0.923$$
	\item Age:
	\begin{itemize}
		\item Young: 6 (L: 0, M: 5, H: 1)
		$$P(Young)H(Y|Young)=-\frac{6}{15}(\frac{0}{6}\log_2\frac{0}{6}+\frac{5}{6}\log_2\frac{5}{6}+\frac{1}{6}\log_2\frac{1}{6})=0.260$$
		\item Old: 9 (L: 7, M: 0, H: 2)
		$$P(Old)H(Y|Old)=-\frac{9}{15}(\frac{7}{9}\log_2\frac{7}{9}+\frac{0}{9}\log_2\frac{0}{9}+\frac{2}{9}\log_2\frac{2}{9})=0.459$$
	\end{itemize}
	$$IG(Y;Age)=H(Y)-H(Y|Age)=1.506-0.260-0.459=0.787$$
	\item Education:
	\begin{itemize}
		\item High School: 4 (L: 4, M: 0, H: 0)
		$$P(High School)H(Y|High School)=-\frac{4}{15}(\frac{4}{4}\log_2\frac{4}{4}+\frac{0}{4}\log_2\frac{0}{4}+\frac{0}{4}\log_2\frac{0}{4})=0.000$$
		\item College: 5 (L: 0, M: 5, H: 0)
		$$P(College)H(Y|College)=-\frac{5}{15}(\frac{0}{5}\log_2\frac{0}{5}+\frac{5}{5}\log_2\frac{5}{5}+\frac{0}{5}\log_2\frac{0}{5})=0.000$$
		\item University: 6 (L: 3, M: 0, H: 3)
		$$P(University)H(Y|University)=-\frac{6}{15}(\frac{3}{6}\log_2\frac{3}{6}+\frac{0}{6}\log_2\frac{0}{6}+\frac{3}{6}\log_2\frac{3}{6})=0.400$$
	\end{itemize}
	$$IG(Y;Education)=H(Y)-H(Y|Education)=1.506-0.000-0.000-0.400=1.106$$
	\item Martial Status:
	\begin{itemize}
		\item Single: 7 (L: 2, M: 2, H: 3)
		$$P(Single)H(Y|Single)=-\frac{7}{15}(\frac{2}{7}\log_2\frac{2}{7}+\frac{2}{7}\log_2\frac{2}{7}+\frac{3}{7}\log_2\frac{3}{7})=0.726$$
		\item Married: 8 (L: 5, M: 3, H: 0)
		$$P(Married)H(Y|Married)=-\frac{8}{15}(\frac{5}{8}\log_2\frac{5}{8}+\frac{3}{8}\log_2\frac{3}{8}+\frac{0}{8}\log_2\frac{0}{8})=0.509$$
	\end{itemize}
	$$IG(Y;Martialtion)=H(Y)-H(Y|Martial)=1.506-0.726-0.509=0.271$$
\end{itemize}
Because $IG(Y;Education)$ is the largest information gain, the ``Education" attribute should be chosen at the root of the tree.

\item\textbf{[7 points]} Draw the full decision tree for the data.
\\\textbf{Answer:}\\
\begin{itemize}
	\item Education = High School $\Rightarrow$ Usage = Low
	\item Education = College $\Rightarrow$ Usage = Medium
	\item Education = University
	\begin{itemize}
		\item Martial Status = Married $\Rightarrow$ Usage = Low
		\item Martial Status = Single $\Rightarrow$ Usage = High
	\end{itemize}
\end{itemize}

\end{enumerate}

\subsection*{1.2 Decision Tress Analysis (16 pts)}

\begin{enumerate}

\item\textbf{[8 points]} Suppose $X$ and $Y$ are discrete variables. Let $IG$ be the information gain and $H$ be the entropy. Show that $IG(X;Y) = H(X,Y) - H(X|Y) - H(Y|X).$
\\\textbf{Answer:}\\
\begin{equation}
\nonumber
\begin{array}{rcl}
IG(X;Y) & = & H(Y) - H(Y|X) \\
		& = & -\sum_y P(y)\log_2P(y) + \sum_x P(x) \sum_y P(y|x)\log_2P(y|x) \\
		& = & -\sum_{x,y} P(x,y)\log_2P(y) -\sum_{x,y} P(x,y)\log_2P(x|y) \\
		&   & +\sum_{x,y} P(x,y)\log_2P(x|y) +\sum_{x,y} P(x,y)\log_2P(y|x) \\
		& = & -\sum_{x,y} P(x,y)\log_2P(x,y) \\
		&   & + \sum_y P(y) \sum_x P(x|y)\log_2P(x|y) + \sum_x P(x) \sum_y P(y|x)\log_2P(y|x) \\
		& = & H(X,Y) - H(X|Y) - H(Y|X)
\end{array}
\end{equation}

\item\textbf{[4 points]} Occam's Razor can be interpreted as simpler hypotheses are generally better than the complex ones. Does ID3 follow Occam's Razor? How about C4.5? Explain briefly (no more than 3 sentences).
\\\textbf{Answer:}\\
The ID3 does not follow Occam's Razor, and C4.5 follows Occam's Razor, because the C4.5 has an additional step to prune back tree to reduce over-fitting and get simpler hypotheses.

\item\textbf{[4 points]} Consider a decision tree $T$ learned on a training set of $n$ instances. Assume that there are two identical instances $X$ and $X\prime$ (i.e. they have exactly the same attributes and labels) in the training set. Can removing $X\prime$ out of the training set produce a different $T$? Explain Briefly (no more than 4 sentences).
\\\textbf{Answer:}\\
Yes, the tree may change, because decision tree classifier requires the availability of all training data to build the tree. E.g. the mid-points of features in C4.5 may change even we just remove a duplicated point.

\end{enumerate}

\newpage


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% fphw Assignment
% LaTeX Template
% Version 1.0 (27/04/2019)
%
% This template originates from:
% https://www.LaTeXTemplates.com
%
% Authors:
% Class by Felipe Portales-Oliva (f.portales.oliva@gmail.com) with template
% content and modifications by Vel (vel@LaTeXTemplates.com)
%
% Template (this file) License:
% CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------

\documentclass[
	12pt, % Default font size, values between 10pt-12pt are allowed
	%letterpaper, % Uncomment for US letter paper size
	%spanish, % Uncomment for Spanish
]{fphw}

% Template-specific packages
\usepackage[utf8]{inputenc} % Required for inputting international characters
\usepackage[T1]{fontenc} % Output font encoding for international characters
\usepackage{fontspec,unicode-math} % Required for using utf8 characters in math mode
\usepackage{parskip}  % To add extra space between paragraphs
\usepackage{graphicx} % Required for including images
\usepackage{booktabs} % Required for better horizontal rules in tables
% \usepackage{listings} % Required for insertion of code
\usepackage{enumerate}% To modify the enumerate environment
\usepackage{import}   % This 4 packages and the command allow importing pdf
\usepackage{xifthen}  % figures generated with inkscape
\usepackage{pdfpages} % Source: https://castel.dev/post/lecture-notes-2/
\usepackage{transparent}
\newcommand{\incfig}[1]{%
    \def\svgwidth{0.45\columnwidth}
    \small
        \import{./images/}{#1.pdf_tex}
}
\setlength{\parindent}{15pt}
\setlength{\headheight}{22.66pt}

%----------------------------------------------------------------------------------------
%	ASSIGNMENT INFORMATION
%----------------------------------------------------------------------------------------

\title{Task 5 \\ Cylinder Variations} % Assignment title

\author{Emilio Domínguez Sánchez} % Student name

\date{May 9th, 2021} % Due date

\institute{University of Murcia \\ Faculty of Mathematics} % Institute or school name

\class{Geometría Global de Superficies} % Course or class name

\professor{Dr. Luis J. Alías Linares} % Professor or teacher in charge of the assignment

%----------------------------------------------------------------------------------------
%	Definitions
%----------------------------------------------------------------------------------------

\setmathfont{latinmodern-math.otf}
\setmathfont[range=\setminus]{asana-math.otf}
\usepackage{physics}
\usepackage{cleveref}
\usepackage{amsthm}
\newtheorem{lemma}{Lemma}
\newcommand{\R}{\mathbb{R}}
\newcommand{\clsr}[1]{\overline{#1}}
\newcommand{\inner}[2]{\left\langle #1, \; #2 \right\rangle}
\DeclareMathOperator{\len}{len}

\begin{document}

\maketitle % Output the assignment title, created automatically using the information in the custom commands above

%----------------------------------------------------------------------------------------
%	ASSIGNMENT CONTENT
%----------------------------------------------------------------------------------------

\section*{Problem}

\begin{problem}
    Consider a regular curve $α:[a,b] \to \R^3$ parametrized by arc
    of the form $α(t) = \mqty(α_1(t) & α_2(t) & 0)$.
    Let $S \subset \R^3$ be the cylinder built over $α$,
    given by its natural parametrization $X : [a,b] \times \R \to S$
    \begin{equation*}
        X(s,v) = α(s) + \mqty(0 & 0 & v) = \mqty(α_1(s) & α_2(s) & v).
    \end{equation*}

    \begin{enumerate}
        \item Let $f : [a,b] \times (-ε,ε) \to \R$ be a differentiable function.
        Prove that the application $φ : [a,b] \times (-ε,ε) \to S$
        \begin{equation*}
            φ(s, t) = α(s) + \mqty(0 & 0 & f(s, t)) = \mqty(α_1(s) & α_2(s) & f(s, t))
        \end{equation*}
        defines a proper normal variation of $α$ if and only if
        the function $f$ satisfies $f(s,0) = f(a,t) = f(b,t) = 0$
        for all $s \in [a,b]$ and $t \in (-ε,ε)$.

        \item In the conditions of the previous item,
        when $φ$ is a proper normal variation,
        let $L(t)$ be the length of the curve $α_t(s) = φ(s, t)$,
        \begin{equation*}
            L(t) = \int_a^b \norm{α_t'(s)} \dd{s}.
        \end{equation*}
        Find $L'(0)$ and $L''(0)$.
    \end{enumerate}
\end{problem}

%----------------------------------------------------------------------------------------

We start by writing the properties of a normal proper variation.
\begin{itemize}
    \item $φ$ is differentiable because it is the sum of two differentiable functions.
    Therefore, it is continous.
    \item $α_t$ is a regular curve because $φ$ is differentiable and
    \vspace{-4mm}
    \begin{equation*}
        \norm{α_t'(s)}^2 = \norm{α'(s)}^2 + \norm{\mqty(0 & 0 & f'(s, t))}^2
            \ge \norm{α'(s)}^2.
        \vspace{-5mm}
    \end{equation*}
    \item $α_0(s) = α(s) + f(s,0)e_3 = α(s)$ if and only if $f(s,0) = 0$.
    \item $α_t(a) = α(a) + f(a,t)e_3 = α(a)$ if and only if $f(a,t) = 0$.
    \item $α_t(b) = α(b) + f(b,t)e_3 = α(b)$ if and only if $f(b,t) = 0$.
    \item $φ_t(s,0) = f_t(s,0)e_3$ is always normal to $α'(s)$.
\end{itemize}
This concludes the first part of the exercise.

%----------------------------------------------------------------------------------------

From now on, we suppose the conditions hold and
we start the study of the length function.
Applying the Leibniz integral rule to differentiate the integral,
\begin{equation*}
    L'(t) = \int_a^b \pdv{t} \norm{a_t'(s)} \dd{s} =
    \int_a^b \pdv{t} \norm{φ_s(s,t)} \dd{s} =
    \int_a^b \frac{\inner{φ_{st}(s,t)}{φ_s(s,t)}}{\norm{φ_s(s,t)}} \dd{s}.
\end{equation*}
In this case, $φ_{st} = f_{st}e_3$, $φ_s(s,t) = α'(s) + f_s(s,t)e_3$,
and the inner product is
\begin{equation*}
    \inner{φ_{st}(s,t)}{φ_s(s,t)} = f_{st}(s,t)f_s(s,t).
\end{equation*}
Hence, the length derivative can be computed as
\begin{equation}\label{eq:dv}
    L'(t) = \int_a^b \frac{f_{st}(s,t)f_s(s,t)}{\norm{φ_s(s,t)}} \dd{s},
\end{equation}
but $\eval{f_s(s, t)}_{t = 0} = \pdv{s} f(s, 0) = 0$,
and so $L'(0) = 0$.

For the second derivative, we use, once more, the Leibniz integral rule
applied to the equation \eqref{eq:dv}.
\begin{align*}
    L''(t) & = \int_a^b \pdv{t} \frac{f_{st}(s,t)f_s(s,t)}{\norm{φ_s(s,t)}} \dd{s} \\
    & = \int_a^b \qty\Bigg[
        \frac{f_{stt}(s,t)f_s(s,t) + f_{st}(s,t)^2}{\norm{φ_s(s,t)}}
        - \frac{\qty[f_{st}(s,t)f_s(s,t)]^2}{\norm{φ_s(s,t)}^2}
    ] \dd{s}.
\end{align*}
Again $\eval{f_s(s,t)}_{t = 0} = 0$, and the second derivative at $t = 0$ is
\begin{align*}
    L''(0) = \int_a^b \frac{f_{st}(s,0)^2}{\norm{φ_s(s,0)}} \dd{s}.
\end{align*}
Finally, we can clean the formula a little bit more using that
$φ_s(s, 0) = \dv{s} φ(s, 0) = \dv{s} α(s) = α'(s)$
and that $α$ is parametrized by arc.
\begin{equation*}
    L''(0) = \int_a^b f_{st}(s,0)^2 \dd{s}.
\end{equation*}

% ALTENATE FORM
% But we are interested in $L'(0)$.
% In particular, $\eval{φ_s(s,t)}_{t = 0} = \pdv{s} α(s) = α'(s)$.
% Using that $α$ is parametrized by arc and the change
% $\inner{φ_{st}}{φ_s} = \pdv{s} \inner{φ_t}{φ_s} - \inner{φ_t}{φ_{ss}}$,
% we can further simplify the expression as
% \begin{align*}
%     L'(0) & = \int_a^b \qty\Bigg[
%         \dv{s} \inner{φ_t(s,0)}{φ_s(s,0)} - \inner{φ_t(s,0)}{φ_{ss}(s,0)}
%     ] \dd{s} \\
%     & = \eval(\inner{φ_t(s,0)}{φ_s(s,0)}|_a^b
%         - \int_a^b \inner{φ_t(s,0)}{φ_{ss}(s,0)} \dd{s} \\
% \end{align*}
% The first part is $0$ because $φ$ is a proper variation:
% $φ_t(a,t) = \dv{t} α(a) = 0 = φ_t(b,t) = \dv{t} α(b) = 0$.
% The second part is also $0$ because
% $φ_t(s,0) = f_t(s,0)e_3$ and $φ_{ss}(s,0) = \dv[2]{s}α(s) \perp e_3$.
% Therefore $L'(0) = 0$.

%----------------------------------------------------------------------------------------

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{MAT257 Notes}
\author{Jad Elkhaleq Ghalayini}
\date{October 26 2018}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{mathtools}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{cancel}

\usepackage[margin=1in]{geometry}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{definition}{Definition}
\newtheorem*{corollary}{Corollary}
\newtheorem{exercise}{Exercise}

\DeclareMathOperator{\Int}{Int}
\DeclareMathOperator{\grad}{grad}

\newcommand{\reals}[0]{\mathbb{R}}
\newcommand{\nats}[0]{\mathbb{N}}
\newcommand{\ints}[0]{\mathbb{Z}}
\newcommand{\rationals}[0]{\mathbb{Q}}
\newcommand{\brac}[1]{\left(#1\right)}
\newcommand{\sbrac}[1]{\left[#1\right]}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\eval}[3]{\left.#3\right|_{#1}^{#2}}
\newcommand{\ip}[2]{\left\langle#1,#2\right\rangle}
\newcommand{\prt}[2]{\frac{\partial #1}{\partial #2}}
\newcommand{\mb}[1]{\mathbf{#1}}

\begin{document}

\maketitle

\section*{Extremum Problems}
We're going to be considering functions, for open sets \(U \subseteq \reals^n\),
\[f: U \to \reals\]
If \(f\) has a local maximum or minimum at a point \(a\) where
\[\prt{f}{x_i}(a)\]
exists, then it must be zero. This is true because the partial derivative is just the derivative along a one-dimensional subspace: a line. That is,
\[\prt{f}{x_i}(a) = g_i'(0) = 0, g_i(t) = f(a + t\mb{e}_i)\]
If \(f\) is differentiable at \(a\), what is the direction of maximum increase (we could also ask for decrease) of \(f\) at \(a\)? It's the direction of the gradient, which I guess is what's called the gradient. Why is this? If we want to look at the increase of \(f\) in a particular direction, we have to look at the value of the directional derivative of \(f\) in that particular direction. We we want to maximize
\[D_{\mb{e}}f(a) = Df(a)(e) = \ip{\grad{f(a)}}{\mb{e}}\]
for some unit vector \(\mb{e}\). In general, of course, what is this inner product? It's
\[|\mb{e}||\grad f(a)|\cos\theta = |\grad f(a)|\cos\theta\]
This, of course, is maximized when \(\theta = 0\), i.e. \(\mb{e}\) points in the direction of \(\grad f(a)\).

What about the converse: does a point where the derivative is zero imply an extremum. Here, just like in first year, this is false, however, in this case, the converse is false \textit{even} assuming \(g_i''(0) \neq 0\). For example, consider
\[f(x, y) = x^2 - y^2\]
This graph is ``saddle shaped''. The second derivative with respect to both \(x\) and \(y\) are nonzero at the origin, the first partial derivatives are zero, and yet we have neither a local maximum or a minimum: in fact, we have a maximum in the \(y\) direction and a minimum in the \(x\) direction!

Let's do an example: consider an acute angled triangle with vetices \(P_i = (x_i, y_i)\). Find a fourth point \(P = (x, y)\) such that the sum of distances to each \(P_i\) is as small as possible. We have
\[f(x, y) = r_1 + r_2 + r_3\]
where
\[r_i = \sqrt{(x - x_i)^2 + (y - y_i)^2}\]
One thing that's we can see is that there \textit{is} a minimum, and it occurs on a closed disk whose boundary contains \(P_i\). \(f\) is differentiable except at \(P_i\), but the minimum can't occur at \(P_i\), since the triangle is acute, implying we can take the intersection of an edge and the perpendicular from a vertex and get a smaller value for \(f\). This isn't the only place where we'll use the fact that the triangle is acute angled, however.

So we know the maximum must occur at a critical point. At a critical point:
\[\prt{f}{x} = \frac{x - x_1}{r_1} + \frac{x - x_2}{r_2} + \frac{x - x_3}{r_3} = 0\]
\[\prt{f}{y} = \frac{y - y_1}{r_1} + \frac{y - y_2}{r_2} + \frac{y - y_3}{r_3} = 0\]
i.e. if
\[u_i = \left(\frac{x - x_i}{r_i}, \frac{y - y_i}{r_i}\right), u_1 + u_2 + u_3 = 0\]

\end{document}


\section*{Method}

Our method consists of imposing additional structure on top of existing machine learning
approaches to reconstruction. For some function $f(p, t)\to\mathbb{R}^n, t\in[0,1]$ which is modelled through
some learned approach, we decompose it into a function $f(p)\to\mathbb{R}^{n\times O}, B_O(f(p), t)$, where
$O$ is the order of the Bezier spline, $f$ is the learned control points, and $B_O$ is the evaluation
of the $O$'th order Bezier spline with control points defined by $f(p)$.

\subsection*{Architecture}

Specifically for the case of dynamic NeRF, we formulate $f(p)$ as
$\text{MLP}(x,y,z)\to\mathbb{R}^{3\times O}$. We raymarch from a camera with known position and
view direction through the scene, and at every point we compute the set of control points for
the amount the ray is being bent. We then are able to evaluate it at any time $t=[0,1]$ in order
to reconstruct movement, $\Delta_0(x)$. The number of spline points is variable, but we are able
to reconstruct movement with as low as 4 spline points, but experiment with at least 16 spline
points. In order to evaluate the Bezier curve in a numerically stable way, we use De Casteljau's
algorithm, and further plan on extending it to handle rational bezier splines, which would allow
for even greater control of movement.

De Casteljau's algorithm evaluated at time $t$ is characterized by the recurrence relation:
\[ \beta_i^{(0)} = \beta_i \]
\[ \beta_i^{j} = (1-t)\beta_i^{(j-1)} + t\beta_{i-1}^{(j-1)} \]
Which can be conceptually thought of as linearly interpolating between adjacent control points until there is only a single point remaining. This takes $O(n^2)$ operations to evaluate, where $n$ is the number of control points.

In addition, we are also interested in constructing a reasonable canonical NeRF, and arbitrarily select $t = 0$ to be the canonical NeRF. Thus, we are interested in Bezier Curves where $B_O(0) = \overrightarrow{0}$. This can be achieved in two different ways, either by assigning $p_0 = \overrightarrow{0}$, and only compute $f(p) = p_{1\cdots O-1}$. Then, we can use the Bezier spline with the control points as $[\overrightarrow{0}, p_{1\cdots O-1}]$. Alternatively, we can compute $p_{0\cdots O-1}$ and use the Bezier spline with control points $[p_{0\cdots O-1}]-p_0$, and the final change in position is $B_O(f(p)-f(p)_0,t)+p_0$. While both formulations are in theory equivalent, and explicitly concatenating 0 leads to fewer degrees of freedom, we find the second one to lead to better convergence near 0, so we use that in our models.

Following NR-NeRF, we also apply a rigidity to every ray, which is also computed as a function of position:
\[ \text{Rigidity}\in[\epsilon, 1] =\sigma_{\epsilon^\uparrow}(\text{MLP}(x,y,z)) \]
\[ \sigma_{\epsilon^\uparrow}(v) = \sigma(v)(1-\epsilon) + \epsilon \]
Which essentially rescales the
difficulty of learning movement, making it more easy to represent and classify static scene
objects. The final change in position is thus defined as $\Delta(x) = \text{Rigidity}\times
\Delta_0(x)$.

In order to reconstruct RGB values, we also diverge from traditional NeRF, and only allow for
positional dependent colors:
\[ \text{RGB} = \sigma_{\epsilon^\Leftrightarrow}(\text{MLP}(x,y,z)) \]
\[ \sigma_{\epsilon^\Leftrightarrow}(v) = \sigma(v)(1+2\epsilon) - \epsilon \]
Using the
activation function as described in MIP-NeRF~\cite{barron2021mipnerf} in order to better reconstruct colors. Because of the low number of
samples for a moving object at a given view, it is much more difficult to learn specular
reflection, thus it becomes necessary to only model the Lambertian component of the color. This is in line with NR-NeRF and D-NeRF, which models the diffuse component.

\subsection*{Loss}
While training, we also introduce an additional loss term, as the $\ell_2$ loss may have
difficulty when there is a large pixel-wise gap in movement between the learned and predicted
component. We formulate the loss term as $\text{MSE}(\text{FFT}(I_\text{GT}),
\text{FFT}(I_\text{predicted}))$, where the FFT is the 2D fast fourier transform of an image.
The MSE here is defined over the complex domain as the FFT has both a real and imaginary
component, and is defined as \[ L_\text{FFT} =\text{MSE}(a\in\mathbb{C}^k,b\in\mathbb{C}^k) \]
\[ = \frac{1}{k}\Sigma_{i=0}^k|a_i-b_i| \]
\[ |z\in\mathbb{C}| = (\Re(z)+\Im(z))(\Re(z)-\Im(z)) \]
We introduce this term as a cheap replacement to structural similarity, since the FFT captures
information about the structure of the whole image, which is useful in a dynamic setting due to
the disjointedness of where a predicted object may be as opposed to its final position\footnote{In a published iteration, we would likely perform an ablation of this loss function.}.

Our final loss formulation is thus:
\[ \ell_2(I_\text{predicted}, I_\text{GT}) + L_\text{FFT}(I_\text{predicted}, I_\text{GT}) \]
without any additional regularization terms.

Our formulation is simpler compared to NR-NeRF since there is no need to regularize over
temporal consistency, and that makes the optimization process simpler. Our training process also does not require adding in additional frames over time for relatively short sequences, so we randomly sample from the set of frames. This is contrast to D-NeRF, which requires adding frames in during training since it has to enforce that at $t=0$ the predicted movement is 0.

\subsection*{Training}

For training, we sample random crops of given frames, computing L2 and FFT loss and back-propagating through the whole network. We use autograd to optimize the splines, but note that there are also classical approaches to solving them which may lead to faster optimization in the future. We use the Adam optimizer~\cite{Kingma2015AdamAM} with cosine simulated annealing~\cite{loshchilov2017sgdr} to go from $\num{2e-4}$ to $\num{5e-5}$ over the course of a day.

In order to ensure that our model is robust to small changes in camera angle, we introduce subpixel jitter in the camera, identical to NeRF. We dot not introduce any jitter in the time-domain, since we are guaranteed smoothness from our formulation.

\subsection*{Long Duration $C^0$ Continuity}

While the above structure is sufficient to model short-term $C^0$ continuity, it runs into
computational instability and higher cost the longer the sequence is, due to the $O(n^2)$
evaluation cost of De Casteljau's algorithm. We also design an additional architecture which
composes the previous architecture for many small $C^0$ curves. Fundamentally, this idea permits
for reconstruction of infinitely long sequences with guaranteed smoothness at little extra cost.

We define the architecture akin to poly-splines, or a composition of many small Bezier splines.
For a known-length sequence, we divide it into $K$ different segments, and without loss of
generality assume that $K$ evenly divides the total number of frames. This implies that $t$ is
now in the range $[0, K]$, and can be decomposed into a segment number and fractional component:
\[ k\in\mathbb{Z}_K=\lfloor t\rfloor, t'\in[0,1)=t-k \]
Then we define an embedding $\text{Emb} = \mathbb{R}^{Z\times(K+1)}$, where $Z$ is some latent
dimension size. We then create an "anchor" network:
\[
    \textit{anchor}(x\in\mathbb{R}^3,z\in\mathbb{R}^Z) = \text{MLP}(x,z)\to(\mathbb{R}^3,\mathbb{R}^{Z'})
\]
So named because it computes "anchors"
or endpoints of the curves, as well as a representative latent vector. Between these two
anchors, we are interested in computing some number of intermediate spline points. We define the
control point estimation network as
\[
\textit{control}(x\in\mathbb{R}^3, z_1, z_2\in\mathbb{R}^{Z'}) = \text{MLP}(x,z_1,z_2)\to\mathbb{R}^{O\times\mathbb{R}^3}
\]
Where $z_1,z_2$ are the
representative latent vectors from the anchor network, and it outputs $O$ spline points in
$R^3$. We optionally also can include a per-segment rigidity value, similar to above, but lose
some guarantees of continuity by doing so.

In order to evaluate the spline at a given time $t$ for a point $x$, we first compute $k, t'$. Then, we compute
the anchor points at
\[ p_0,z_1=\textit{anchor}(x,\text{Emb}[k]) \]
\[ p_\text{end},z_2=\textit{anchor}(x,\text{Emb}[k+1]) \]
And the middle control points as \[ p_{1\cdots O-2} = \text{control}(x,z_1,z_2) \]
The final set of control points we use
is the concatenation of the anchors with the intermediate points:
$[p_0, p_{1\cdots O-2}, p_\text{end}]$. Using this set of control points, evaluate the Bezier
curve spline at $t'$ using De Casteljau's algorithm:
\[ \text{De Casteljau}([p_0, p_{1\cdots O-2}, p_\text{end}], t) = \Delta(x) \]
Because we are using adjacent points from
the embedding for each anchor, we are guaranteed that the endpoints between each spline segment
are continuous. Information is also carried over between segments through the anchor's
representative latent vector, allowing the network to maintain velocity between curves if necessary.

This architecture enforces a guarantee of consistency, regardless of sequence length. It also
naturally allows for disentangling motion between distant time-steps, as while the endpoints of
the spline are fixed, movement in the middle is allowed to be fully independent of other
segments. This architecture is independent from dynamic NeRF, and we propose it for the broader task of generating $C^0$-continuous signals in any domain.

One may question as to why a more complex architecture is necessary as compared to just predicting all the spline points and using only a subset of them. In the case of extremely long sequences, we may expect that the number of spline points increases linearly with the sequence, and for dynamic scene reconstruction we would then have to sample $O(H\times W\times D\times t)$ at each point in space, even though we are only interested in a small subset. With this formulation, we are able to sample sparsely, maintaining a constant memory usage with respect to time.

We do not demonstrate the plausibility of this architecture, due to lack of data known to the
author of long videos with known camera positions, but hope to test it in future work\footnote{If this were to be published, I would demonstrate this architecture, but cannot due to time constraints.}.


\chapter{Background and Literature review} \label{chap-3}

In this section, we will establish some common notation and present several core algorithms for computation on skew-polynomials, as well as other operations that may be used as a "toolbox" for any other algorithms discussed.

\section{Background}
The complexity analysis of algorithms over a finite field is traditionally done using an algebraic model counting elementary field operations $+, -, \times, \cdot^{-1}$ at unit cost. Modular composition notably does not admit a simple analysis in terms of algebraic complexity, and is typically only used to compute the action of powers of the Frobenius map on field elements. For algorithms where this is necessary, two runtimes may be stated: an algebraic one where applications of Frobenius maps are treated with unit cost and a bit complexity model where the cost of using modular composition is counted.

Throughout this work, we will let $\F_{q}$ denote a field of order $q$ a prime power. We will let $\L$ denote a finite extension of $\F_q$ with $[\L:\F_q] = n$. Let $\sigma$ denote an endomorphism of $\L$. The ring of \textit{skew-polynomials} $\L[\tau;\sigma]$ are polynomials with coefficients in $\L$ in the variable $\tau$ subject to the relation $ \tau c = \sigma(c) \tau $ for $c \in L$.  Note that this relation induces a canonical $\F_q$-algebra homomorphism

    \begin{align*}
    \L[\tau; \sigma] & \mapsto \textnormal{End}_{\F_q}[\L] \\
    \tau & \mapsto \sigma
    \end{align*}

Typically, $\sigma$ will be the $q$-order Frobenius $\sigma(c) = c^q$, and since we will often implicitly associate a skew polynomial with its corresponding endomorphism we will simply write $\L\{\tau\}$ for skew-polynomial rings. This also suggests a natural isomorphism between $\L[\tau]$ and the $\F_q$-algebra $\L[x]^q$ of polynomials in $x$ whose terms all have exponents $q^i$ for $i \geq 0$ and whose product is ordinary polynomial composition rather than multiplication.

Commutative polynomials of degree at most $n$ over a $\F_q$ can be multiplied in $O(n\log n)$ field operations, while arithmetic over $\F_q$ itself will be assumed to run in $O(\log q \log \log q \log \log \log q)$. We will let $\omega$ denote a real number such that two $s \times s$ matrices over any ring $R$ can be multiplied in $O(s^{\omega})$ ring operations. A highly celebrated result due to Kedlaya and Umans in \cite{Kedlaya:2011:FPF:2340436.2340448} established a quasi-optimal algorithm for modular composition of polynomials $f(g) \bmod h$ with $f,g,h \in \F_q[x]$ of degree at most $n$ with bit complexity $(n^{1 + \epsilon} \log q)^{1+o(1)}$

\section{Elementary Algorithms on Linearized Polynomials}

\subsection{Skew-Polynomial Multiplication}

Perhaps the most elementary non-trivial algorithm one would consider on skew-polynomials is that of multiplication. This appears to be a somewhat less straightforward question than in the commutative case, with at least two algorithms co-existing depending on the chosen parameters. A common feature of the time complexity analysis is the assumption that a preprocessing step can be done in negligible time to determine the action of a map on a finite field. In the next section we analyze multiplication algorithms due to Puchinger and Wachter-Zeh \cite{PUCHINGER2017b} as well as Caruso and Le Borgne \cite{CaLe17}, and let $\SM(d)$ denote the complexity of of multiplying two degree $d$ skew polynomials. In particular we focus on using a bit complexity model, which allows us to leverage the modular composition algorithm of Kedlaya and Umans \cite{Kedlaya:2011:FPF:2340436.2340448} to explicitly compute Frobenius powers. The analysis done here is a repeat of what was done in my Master's thesis \cite{Musleh}.

\begin{problem}[Skew-polynomial Multiplication]
Given two skew-polynomials $a, b \in \F_{q^n}[\tau]_s$, compute $a \cdot b$.
\end{problem}

\subsubsection{The Puchinger \& Wachter-Zeh Algorithm}

When $s \ll n$, the algorithm described by \cite[Th.7]{PUCHINGER2017b} has the better bit complexity. Set the factors $a = \sum_{i=0}^s a_i \tau^i$, $b = \sum_{i=0}^s b_i \tau^i $, and set $s^* = \lceil \sqrt{s + 1} \rceil $. We can split $a$ into a sum of $s^*$ terms of the form $a^{(i)} = \sum_{j=0}^{s^* - 1} a_{is^* + j}\tau^{id^* + j}$ such that $a = \sum_{i=0}^{s^* - 1} a^{(i)}$. This in turn splits the product $c$ into $s^*$ terms $c = \sum_{i=0}^{s^* - 1} c^{(i)}$ with $c^{(i)} = a^{(i)}b$. We can multiply through the expression for $c^{(i)}$:

\begin{equation*}
    c^{(i)} = \sum_{j=0}^{s^* - 1} a_{is^* + j}\tau^{is^* + j} \bigg( \sum_{k = 0}^{s} b_k \tau^k \bigg) = \sum_{k=0}^{s + s^* - 1} \bigg( \sum_{j=0}^{k} a_{is^* +j}\sigma^{is^* + j}(b_{k -j})\tau^{is^* + k} \bigg)
\end{equation*}

Construct matrices $A, B, C$ such that:

\begin{align*}
    A_{i,j} & = \sigma^{-is^*}(a_{is^* + j}) \\
    B_{i,k} & = \sigma^{i}(b_{k-i}) \\
    C_{i,k} & = \sigma^{-is^*}(c_k^{(i)})
\end{align*}

Where $0 \leq i,j \leq s^* -1$, $0 \leq k \leq s + s^* - 1 $. Thus, we may compute $c$ by computing the matrix product $C = A \cdot B$ to determine the entries $\sigma^{-is^*}(c_k^{(i)})$, through which we may compute $c = \sum_{i = 0}^{s^* - 1}\sum_{k = 0}^{s + s^* - 1} c_k^{(i)}$. The computationally intensive steps are computing the action of $O(s^{1/2})$ distinct automorphisms $\sigma^{\pm is^*}, \sigma^i$ for up to $O(s^{3/2})$ elements. When the cost of multiplication is considered, the total cumulative bit complexity is $(s^{(\omega + 1)/2} n^{1 + \epsilon} \log q)^{1 + o(1)}$.

\subsubsection{The Caruso \& Le Borgne Algorithm}

Here we will provide an overview of the algorithm given in \cite[Sec.2]{CaLe17}. The critical insight of the algorithm is their proposition 1.6, which relates the action of skew-polynomials to commutative multiplication modulo $T^n - 1$. We restate the proposition here:

\begin{proposition*}{\cite[Prop. 1.6]{CaLe17}}
Let $T$ be a commutative variable, and fix a normal basis $b_0, \ldots, b_{n-1}$ for $\L/\F_q$ and let $b = \sum_{i=0}^{n-1} b_i T^i$. Further let $a = \sum_{i=0}^{n-1} a_i \tau^i\in \L[\tau]$ and set $a(T) = \sum_{i=0}^{n-1} a_i T^i$, $c_i = a(b_i)$ and $c(T) = \sum c_iT^i$. Then

\begin{equation*}
    c(T) = a(T)b(T) \mod T^n -1
\end{equation*}

\end{proposition*}

Namely, the above proposition reduces computing the action of $a$ on a normal basis to commutative polynomial multiplication when both are considered modulo $T^n - 1$. Given any two $a, a' \in \L[\tau]/(X^n - 1)$, the product $aa'$ can be computed as follows:

\begin{enumerate}
    \item Compute $c(T) = a(T) b(T)$, $c'(T) = a'(T)b(T)$.
    \item Extract from $c,c'$ the matrices $M,M'$ respectively for the action of $a, a'$ on $\L/\F_q$ where the domain uses the normal basis and the co-domain uses the standard basis.
    \item Compute $M B M'$ where $B$ is the change of basis matrix from the standard basis to the normal basis. This gives the action of the product $aa'$ on $\L/\F_q$.
    \item Given the product $MBM'$ compute the corresponding $\hat{c}(T)$ such that $\hat{c}b^{-1}(T) = a(T)a'(T)$ and extract the coefficients of $aa'$
\end{enumerate}

The main computational bottlenecks are the computation of the matrix product $MBM'$, which costs $O(n^{\omega})$ $\F_q$ operations, as well as the construction of a normal basis for $\L$ over $\F_q$.

\subsection{Division and Factorization}

The ring $\sring$ is a right and left Euclidean domain and therefore admits a non-commutative analog of the Euclidean algorithm \cite{ore}.

\begin{problem}[Right Euclidean Division]
Given skew-polynomials $a, b$, compute skew-polynomials $\omega, \rho$ such that $\skewdeg(\rho) < \skewdeg(b)$ and $a = \omega b + \rho$. In this case we write $\RED(a, b) = (\omega, \rho)$.
\end{problem}

Elementary division problems concerning skew polynomials were first studied in $\cite{ore}$, with the first thorough algorithmic treatment, including the first non-trivial factorization scheme for skew polynomials, given in \cite{GIESBRECHT1998463}. In \cite{CaLe17} an algorithm for right Euclidean division of a degree $d_1$ skew polynomial $a$ by a degree $d_2$ skew polynomial $b$ with complexity in $\tilde{O}(\SM(d_1))$. The procedure is based on a classical algorithm for Euclidean division on commutative polynomials seen in \cite{Gathen:2003:MCA:945759}.

Define the pseudo-inversion operator of order $\delta$ on $\sring_{\delta}$:

\begin{equation}
    T_{\delta}\bigg(\sum_{i=0}^{\delta} a_i \tau^i \bigg) = \sum_{i=0}^{\delta} a_{\delta - i} \tau^{-i}
\end{equation}

For a skew polynomial $\rho = \sum_{j=0}^{d}\rho_j\tau^j$, let $\rho^{(i)} = \sum_{j=0}^{d}\sigma^i(\rho_j)\tau^j$. It is fairly straightforward to check that $T_{\delta}$ is linear and satisfies the relation $T_{d_1 + d_2}(ab) = T_{d_1}(a)T_{d_2}(b^{(\delta_1)})$. Let $\overline{\omega} = \sum_{i=0}^{\infty} \overline{\omega}_i\tau^i$ be such that $\overline{\omega}T_{d_2}(b^{d_1 - d_2}) = 1$. From the preceding product formula, we have that $T_{d_1}(a) = T_{d_1 - d_2}(\omega)T_{d_2}(b) + T_{d_1}(\rho)$. Let $\overline{\omega}^{[d_1 - d_2]} = \sum_{i=0}^{d_1 - d_2 - 1} \overline{\omega}_i\tau^i$; and observe that $T_{d_1}(a)\overline{\omega}^{[d_1 - d_2]} b - T_{d_1}(a)$ must have degree at least $d_1 - d_2$, implying that $T_{d_1 - d_2}(\rho) = T_{d_1}(a)\overline{\omega}^{[d_1 - d_2]} b - T_{d_1}(a)$. We can compute $\overline{\omega}^{[d_1 - d_2]}$ using Newton iteration in $O(\log d_1)$ steps, each of which has cost at most $O(M(d_1))$.

\subsection{Minimal Subspace Polynomial \& Multipoint Evaluation}

Let $V \subset \L$ be a linear subspace of dimension $m \leq n$ over $\F_q$. The \textit{minimal subspace polynomial} $\MSP(V)$ is the monic skew-polynomial of lowest skew-degree whose kernel contains $V$. The MSP of a linear subspace is unique and has skew-degree $d$ if $V$ has dimension $d$. Consequently, a monic degree $d$ skew-polynomial is equal to $\MSP(V)$ for some subspace $V$ if and only if it has a set of $d$ linearly independent roots.

\begin{problem}[Minimal Subspace Polynomial]
Given a linear subspace $V \subset \L$ over $\F_q$, determine its minimal subspace polynomial, $\MSP(V)$.
\end{problem}

\begin{problem}[Multipoint Evaluation]
Given a set of $m$ points $v_1, \ldots, v_s$ and $\rho \in \sring$, compute the set $\MPE(\rho; v_1, \ldots, v_s) = \{ \rho(v_1), \ldots, \rho(v_s) \}$.
\end{problem}

In the case of Multipoint evaluation, it suffices to compute the evaluation on a maximal linearly independent subset of $v_1, \ldots, v_s$ with respect to cardinality. Two mutually recursive algorithms were proposed in $\cite{PUCHINGER2017b}$ to solve both problems:

\begin{algorithm}
\caption{Minimal Subspace Polynomial \cite{PUCHINGER2017b}}
\label{msp1}
\hspace*{\algorithmicindent} \textbf{Input} A basis $\{ v_1, \ldots, v_d \}$ for $V \subset \F_{q^n}$\\
 \hspace*{\algorithmicindent} \textbf{Output} A monic skew-polynomial $\rho \in \F_{q^n}\{\tau\}_d$ of degree $d$ such that $\rho(v_i) = 0$ for each $1 \leq i \leq d$.  \\
 \begin{algorithmic}[1]
 \If{$d = 1$}{}
            \State $\rho \gets 1 \textnormal{ if } v_1 = 0$

            \State  $ \rho \gets \tau - v_i^{q-1} \textnormal{ otherwise}$

\Else {
      \State $ a \gets \MSP(v_1, \ldots v_{d/2})$

    \State $\{v_{d/2 + 1}', \ldots, v_d' \} \gets \MPE(a; v_{d/2 + 1},\ldots, v_d)$

    \State $ b \gets \MSP(v_{d/2 + 1}', \ldots, v_d')$

    \State $\rho \gets ba$}
    \EndIf
\end{algorithmic}
\end{algorithm}

\begin{algorithm}
\caption{Multipoint Evaluation \cite{PUCHINGER2017b}}
\label{mpe1}
\hspace*{\algorithmicindent} \textbf{Input} A skew polynomial $\rho \in \F_{q^n}\{\tau \}$ of degree $d$, and a set of linearly independent evaluation points $\{ v_1, \ldots, v_s \} \subset \F_{q^n}$. \\
 \hspace*{\algorithmicindent} \textbf{Output} The set $\{ \rho(v_1), \ldots, \rho(v_s) \} \subset  \F_{q^n}$ \\
 \begin{algorithmic}[1]
 \If{$\skewdeg(\rho) = 1$}
            \State $\textnormal{\textbf{Return}} \{\rho(v_1), \ldots, \rho(v_s) \}$

\Else {
      \State $ a \gets \MSP(v_1, \ldots v_{s/2})$
      \State $ b \gets \MSP(v_{s/2 + 1}, \ldots v_{s})$
      \State $ (\omega_a, \rho_a) \gets \RED(\rho, a) $
      \State$ (\omega_b, \rho_b) \gets \RED(\rho, b) $
      \State $\{ \rho(v_1), \ldots, \rho(v_{s/2}) \} \gets \MPE(\rho_a; v_1, \ldots v_{s/2}))$
      \State $\{ \rho(v_{s/2 + 1}), \ldots, \rho(v_s) \} \gets \MPE(\rho_b; v_{s/2 + 1}, \ldots v_{s}))$}
      \EndIf
 \end{algorithmic}
\end{algorithm}

A detailed complexity analysis for these mutually recursive algorithms was given in $\cite{PUCHINGER2017b}$. The runtimes $\MSPr(d)$ and $\MPEr(d, s)$ of both algorithms can be expressed recursively.
\begin{align*}
    \MSPr(d) & = 2\MSPr(d/2) + \MPEr(d/2, d/2) + \Mr(d/2) \\
    \MPEr(d, s) & = 2\MSPr(s/2) + 2\MPEr(d/2, d/2) + \REDr(s)
\end{align*}

By memoizing the output of $\MSP$ at line 5, the call to $\MSP$ at line 4 can be avoided, and the recursive expression for $\MPEr$ becomes

\begin{equation*}
    \MPEr(d, s) = \MSPr(s/2) + 2\MPEr(d/2, d/2) + \REDr(s).
\end{equation*}

Both expressions are simultaneously bounded by a complexity class $C(d)$ satisfying the recurrence

\begin{equation*}
    C(d) = 3C(d/2) + \REDr(d)
\end{equation*}

Using the master theorem, this yields a complexity of $C(d) = O(d^{\max(\log_2(3), (\omega + 1)/2)}\log d)$ operations in $\L$.

\section{Drinfeld Modules}

We begin with the ring of regular maps $\F_q[x]$ and a map $\gamma : \F_q[x] \to \L$. Moreover let $\frakp$ denote the monic irreducible polynomial of degree $m$ generating the kernel of $\gamma$, and let $\F_{\frakp} = \F_q[x]/(\frakp)$. Then a \textit{Drinfeld Module} over $(\L, \gamma)$ is a ring homomorphism $\phi: \F_q[x] \to \L\{\tau \}$ such that

\begin{equation*}
    \phi(a) = \sum_{i=1}^{d}c_i\tau^i + \gamma(a)
\end{equation*}

Keeping with the literature, we will often use the notation $\phi_a = \phi(a)$. A Drinfeld module may be specified by its action on $x$, and thus we may refer to a Drinfeld module by specifying the $r$ coefficients $(\Delta_r, \ldots, \Delta_1)$ such that $\Delta_r \neq 0$ and $\phi_x = \sum_{i=1}^{r}\Delta_i \tau^i + \gamma(x)$. The integer $r$ is referred to as the \textit{rank} of the Drinfeld module. A morphism from a Drinfeld module $\phi$ to $\psi$ is an element $u \in \L\{ \tau\}$  such that $u \phi_a = \psi_a u$ for all $u \in \F_q[x]$. The absolute Frobenius map is the element $\tau^n \in \L\{ \tau^n \}$. The minimal polynomial of the Frobenius element is the monic polynomial $\Gamma_{\phi}(X) = X^d - \sum_{i=0}^{d-1} a_iX^i \in \F_q[x][X]$ of least degree such that

\begin{equation*}
    \tau^{dn} - \sum_{i=0}^{d-1}\phi_{a_i} \tau^{ni} = 0
\end{equation*}

and such that $\deg a_i \leq \frac{(r - i)n}{r}$. An explicit formula to compute $a_0$ is given in \cite{GaPa18}:

\begin{equation}\label{norm}
    a_0 = (-1)^{r + n(r+1)}N_{\F_{\mathfrak{p}}/\F_q}(\Delta_r)^{-1} \mathfrak{p}^{ \frac{n d}{ r m}}
\end{equation}

The coefficient $a_0$ is referred to as the \textit{Frobenius norm}, and the preceding formula allows for its calculation in $(n\log q)^{1+o(1)}$ bit operations.

\subsection{Finding the Minimal Polynomial when $r = 2$}
When $r = 2$, the Frobenius element is known to satisfy the relation

\begin{equation}\label{charrank2}
    \tau^{2n} - \phi_{a}\tau^n - \phi_b = 0
\end{equation}

Such that either $\Gamma_{\phi}(X) = X^2 - aX - b$ or $\Gamma_{\phi}(X)^2 = X^2 - aX - b$. We may determine $b$ using equation $(\ref{norm})$, and therefore it suffices to compute $a$ and determine whether the resulting polynomial is a square. The coefficient $a$ is referred to as the $\textit{Frobenius trace}$.

The classical algorithm for computing the minimal polynomials of Drinfeld modules is due to Gekeler in \cite{frobdist}. Write $\phi_{x^i} = \sum_{j=0}^{2i}f_{i,j}\tau^j$, and these $\phi_{x^i}$ satisfy

\begin{equation*}
    \phi_{x^{i+1}} = \phi_{x}\phi_{x^i} = (\Delta_2 \tau^2 + \Delta_1 \tau + \gamma(x))\phi_{x^i}
\end{equation*}

the coefficients $f_{i,j}$ therefore satisfy the recurrence:

\begin{equation}
 f_{i+1, j} =  \gamma(x)f_{i,j} + \Delta_1f_{i,j-1}^{q} + \Delta_2 f_{i, j-2}^{q^2}
\end{equation}

This recurrence allows the computation of $\phi_{x^2}, \ldots, \phi_{x^k}$ in $(k^2n\log^2q)^{1+o(1)}$ bit operations. Computing up to $k = n/2$ and substituting $\phi_a = \sum_{i=0}^{n/2} a_i \phi_{x^i} = \sum_{i=0}^{n/2}\sum_{j = 0}^{2i} a_i f_{i,j}\tau^{j}$ into equation \ref{charrank2}, along with the computed value for $\phi_b$, generates a linear system from which the coefficients $a_i$ can be extracted.

\subsubsection{A Schoof-Like Algorithm}

The classical algorithm due to Schoof \cite{schoof85} looks at the action of the Frobenius on the torsion points of an Elliptic curve via division polynomials, whose roots are the $x$ co-ordinates of $\ell$-torsion points. Computing the Frobenius trace on this restricted subgroup gives the trace reduced modulo $\ell$, from which the Chinese Remainder Theorem can be used to reconstruct the trace once computed for sufficiently many $\ell$. In the Drinfeld setting, the analog of the $\ell$-torsion subgroup appears to have orders that are too large for efficient computation.

Instead, we compute $a \bmod E_i$ for irreducible polynomials $E_1, \ldots, E_k \in \F_q[x]$ such that $\deg(E_1 \ldots E_k) > n/2$ and $\deg(E_i) \in O(\log n)$ for all $i \leq k$. If $q > n/2$, then it suffices to let $k = \frac{n}{2} + 1$ and $E_i = x - e_i$ for some $e_i \in \F_q$. Reducing equation \ref{charrank2} right modulo the image $\phi_{E_i}$ of an irreducible $E_i$, we obtain

\begin{equation}\label{modchar}
    \tau^{2n} \bmod \phi_{E_i} - \phi_{a \bmod E_i}\tau^n \bmod_{E_i} - \phi_{b \bmod E_i} = 0
\end{equation}

Let $d_i = \deg E_i$. Given $b \mod E_i$ and $\tau^n \bmod \phi_{E_i}$, we now only need to compute $\phi_{x^2}, \ldots, \phi_{x^{d_i -1}}$ to solve for $a$.  We use a companion matrix method to compute $\tau^n \bmod \phi_{E_i}$; for $\rho \in \sring_{2d_i}$, let $T(\rho) = \tau \rho \mod E_i$ denote left multiplication by $\tau$ right modulo $E_i$. Set $\mathbf{T}$ to be its $2d_i \times 2d_i$ companion matrix acting on coefficient vectors $\hat{\rho} \in \F_q^{2d_i} $ for $\rho \bmod \phi_{E_i}$, and let $\mathbf{T}^{i}$ denote entry-wise exponentiation by $q^i$. Then

\begin{equation}\label{companion}
    \hat{\tau}^n = \mathbf{T}\mathbf{T}^{1} \mathbf{T}^{2} \ldots \mathbf{T}^{n-1}\hat{1}
\end{equation}

The cost of computing $\phi_{x^2}, \ldots, \phi_{x^{d_i -1}}$ and $\phi_{b \bmod E_i}$ is $(n\log^2q)^{1+o(1)}$ bit operations. Computing $\hat{\tau}^n$ via equation \ref{companion} takes $O(n\log n)$ modular compositions, resulting in a bit complexity of $(n^{2+\epsilon}\log q)^{1+o(1)}$. The overall runtime is then $(n^{2+\epsilon}\log q + n \log^2q)^{1+o(1)}$

\subsubsection{Using Hankel Systems}

From the definition of the minimal polynomial, $\phi_{a} = \tau^{2n} - \phi_{b}$, set $\phi_{a} := \sum_{i=0}^{\lfloor n/2 \rfloor} c_i\phi_{x}^i$ and $r := \tau^{2n} - \phi_{b}$. For a uniformly random choice of $\F_q$-linear projection $\ell : \L \to \F_q$, $\alpha \in \L$, and for any $j > 0$ evaluate both sides of the equation at $\alpha$ to obtain:

%\begin{equation*}
%    \sum_{i=0}^{\lfloor n/2 \rfloor} c_i\phi_{x}^i(\alpha) = r(\alpha)
%\end{equation*}

%For any $j > 0$, apply $\phi_x^j$ to the preceding expression

\begin{equation*}
    \sum_{i=0}^{\lfloor n/2 \rfloor} c_i\phi_{x}^{i +j}(\alpha) = \phi_x^jr(\alpha)
\end{equation*}

This yields the following $\nu \times \nu$ Hankel system over $\F_q$:

\begin{equation*}
    \begin{bmatrix}
    \ell(\alpha) & \ldots & \ell(\phi_x^{\nu - 1}(\alpha)) \\
    \ell(\phi_x(\alpha)) & \ldots & \ell(\phi_x^{\nu}(\alpha)) \\
    \vdots & & \vdots \\
    \ell(\phi_x^{\nu - 1}(\alpha) & \ldots & \ell(\phi_x^{2\nu - 2}(\alpha)
    \end{bmatrix}
    \begin{bmatrix}c_0 \\ \vdots \\ c_{\nu - 1} \end{bmatrix} =
    \begin{bmatrix} \ell (r(\alpha)) \\ \vdots \\ \ell(\phi_x^{\nu - 1}r(\alpha)) \end{bmatrix}
\end{equation*}

Let $\Gamma_{\ell, \alpha}$ denote the minimal polynomial of the sequence $\{ \ell (\phi_x^i(\alpha))\}$, and $\Gamma$ the minimal polynomial of $\phi_x$. The solution to the preceding system is unique and yields the coefficients of the minimal polynomial if and only if $\Gamma_{\ell, \alpha} = \Gamma$. Under a uniform choice of $\ell, \alpha$, a classical argument due to Wiedemann in \cite{Wiedemann:1986:SSL:13738.13744} can be used to lower bound $\textnormal{Pr}[\Gamma_{\ell, \alpha} = \Gamma]$, and use of the Schwartz-Zippel lemma allows for a lower bound of $1-2n/q$. If $n$ is even, we may not be able to determine $c_{\lfloor n/2 \rfloor}$, in which case we let $\nu = \lfloor n/2 \rfloor$ and compute $c_{\lfloor n/2 \rfloor}$ via the formula
\begin{equation*}
    c_{\lfloor n/2 \rfloor} = \textnormal{Tr}_{\F_{q^2}/\F_{q}}(\textnormal{N}_{\L/\F_{q^2}}(\Delta)^{-1})% \hspace{4mm} \cite{gekeler},
\end{equation*}
here $\F_{q^2}$ is a quadratic extension of $\F_q$ contained in $\L$, and $\textnormal{Tr}$ and $\textnormal{N}$ are the field trace and norm respectively. One can compute up to $2n$ elements of the sequence $\{ \ell (\phi_x^i(\alpha))\}$ using $(n^2 \log^2q)^{1 + o(1)}$ bit operations. $r(\alpha)$ and the resulting sequence $\ell(\phi_{x^i}(r(\alpha)))$ for $i \leq \nu -1$ can be computed using at most $(n^2 \log^2q)^{1 + o(1)}$ bit operations. Finally, the resulting Hankel system can be solved in $(n^2 \log q)^{1 + o(1)}$ bit operations, allowing for an overall runtime of $(n^2 \log^2q)^{1 + o(1)}$.

\subsection{Algorithms for Higher Ranks}

The following approach is due to \cite{GaraiPapikian} and can be used when $\L = \F_{\frak{p}}$. Rewrite the expression for the minimal polynomial as $f_i' + f_i = 0$ where

\[ f_i = \phi_{a_{r-i}}\tau^{n (r-i)} + \phi_{a_{r - i -1 }}\tau^{n (r-i-1)} + \ldots + \phi_{a_0} \]

\[ f_i' = -\tau^{nr} + \phi_{a_{r -1 }}\tau^{n (r-1)} + \ldots + \phi_{a_{r-i + 1}}\tau^{n(r - i + 1)}. \]

The term of skew degree $n(r - i + 1)$ in $f_i'$ is  $\gamma(a_{r-i+1}) \tau^{n(r-i + 1)}$; consequently each $\gamma(a_{r-i+1})$ can be computed by determining the $n(r - i + 1)$ skew-degree term of $f_i$.

The bit-complexity of computing all of $\phi_x, \phi_{x^2}, \ldots, \phi_{x^n}$ is \newline
$\Theta(r\log(r)n^3\log(q))$. Computing each $f_i$ is linear in the degree of the term being added, which is $O(rn)$, which is done for $O(r)$ iterations, so the overall cost is $O((r^2n + r\log(r)n^3)\log(q) )$.