\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage{physics}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{hyperref}

\usetikzlibrary{automata,positioning}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ : \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\bea}{\begin{flalign*}}
\newcommand{\eea}{\end{flalign*}}

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Homework\ \#1}
\newcommand{\hmwkDueDate}{Due on 14th January, 2019}
\newcommand{\hmwkClass}{Dynamical Systems}
\newcommand{\hmwkClassTime}{}
\newcommand{\hmwkClassInstructor}{}
\newcommand{\hmwkAuthorName}{\textbf{Aditya Vijaykumar}}

%
% Title Page
%

\title{
    %\vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{\hmwkDueDate\ }\\
%    \vspace{3in}
}

\author{\hmwkAuthorName}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}

\maketitle
(\textbf{Acknowledgements} - I would like to thank Divya Jagannathan for discussions.)
\\

\begin{homeworkProblem}
	\begin{equation}
	\dot{x} = x^{1/3} \implies \dfrac{dx}{x^{1/3}} = dt \implies \dfrac{2}{3} \qty(x^{2/3} - x_0^{2/3}) = t \implies x(t) = \qty(x_0^{2/3} + \dfrac{3}{2}t)^{3/2}
	\label{eq1}
	\end{equation}
	For $ x_0 = 0 $, we have $ x(t) = \qty(\dfrac{3t}{2})^{3/2} $. As the function $ f(y) = y^{3/2} $ is only defined for $ y \ge 0 $, this solution has maximum interval of existence $ t \in [0, \infty)$. For $ x_0 = 0 $, we also have the trivial solution $ x(t) = 0 $ which has maximum interval of existence $ t \in (-\infty, \infty) $. So there are at least two distinct solutions for $ x_0=0 $. One can also imagine patching up the above solutions at origin and forming other possible solutions.
	\\

	For the case where $ x_0 \ne 0 $, we are only left with [\ref{eq1}]. As $ x_0^{2/3} > 0, \forall x_0 \ne 0 $, the maximal interval of existence for $ x(t) $ is $ \left[ -\dfrac{2}{3} x_0^{2/3},\infty\right) $.

	If we try to solve the problem numerically, we only get the trivial solution $ x(t) = 0  $ for $ x_0 = 0 $.
\end{homeworkProblem}

\begin{homeworkProblem}
	\textbf{Part (a)}\\
	\begin{equation*}
	\dot{x} = x ( x^2 - 1)
	\end{equation*}
	For $ x_0 = 0,1,-1 $, $ x(t) = 0, 1, -1 $ respectively is a solution for all times.
	\begin{align*}
	\therefore \dfrac{dx}{x ( x-1 )(x+1)} &= dt\\
	\therefore - \dfrac{dx}{x} +\dfrac{1}{2}\qty[\dfrac{dx}{x+1} + \dfrac{dx}{x-1}] &= dt\\
	\dfrac{1}{2} \log \abs{\dfrac{x_0^2(x^2 -1)}{x^2 (x_0^2 - 1)}} &= t \\
	\dfrac{(x^2 -1)}{x^2 } &= \pm \dfrac{x_0^2 - 1}{x_0^2}e^{2t}\\
	x(t) &= \sqrt{\dfrac{1}{1  \pm {\dfrac{x_0^2 - 1}{x_0^2}} e^{2t}}}
	\end{align*}
	We take only the $ - $ sign in the above expression as the $ + $ sign does not satisfy the initial condition.
	\begin{equation*}
	x(t) =\dfrac{x_0}{\abs{x_0}}  \sqrt{\dfrac{1}{1  - {\dfrac{x_0^2 - 1}{x_0^2}} e^{2t}}}
	\end{equation*}
	where the $ \dfrac{x_0}{\abs{x_0}} $ has been multiplied to take care of sign.
	For the problem to have the above solution, one must have,
	\begin{align*}
	1 - {\dfrac{x_0^2 - 1}{x_0^2}} e^{2t} > 0 &\implies t < -\dfrac{1}{2} \log {\dfrac{x_0^2 - 1}{x_0^2}} \qq{if}  x_0^2 > 1 \\
	&\qq{and} t \in (-\infty, \infty) \qq{if} x_0^2 < 1
	\end{align*}

	\begin{figure}[!h]
	\caption{Q2 (a)}
	\centering
	\includegraphics[scale=0.6]{2a_1.png}
	\includegraphics[scale=0.45]{2a_3.png}
	\includegraphics[scale=0.6]{2a_2.png}
	\end{figure}

	\textbf{Part (b)}\\
	\begin{equation*}
	\dot{x} = -x^3
	\end{equation*}
	The above equation has the trivial solution $ x(t) = 0; \forall t $ for $ x_0 = 0 $. We proceed to find solutions for other values of $ x_0 $.
	\begin{align*}
	\dot{x} = - x^3 \implies -\dfrac{dx}{x^3} = dt &\implies \eval{\dfrac{1}{2x^2}}_{x_0}^{x} = t \implies \dfrac{1}{x^2} = \dfrac{1}{x_0^2} + 2t\\
	\implies &x = + \dfrac{ x_0}{\sqrt{1 + 2 x_0^2 t}}
	\end{align*}
	The above solution will exist for $ t > -\dfrac{1}{2x_0^2} ; x_0 \ne 0$. The region of existence is also plotted below.

	\begin{figure}[h]
		\caption{Q2 (b)}
		\centering
		\includegraphics[scale=0.6]{2b_3.png}
		\includegraphics[scale=0.6]{2b_1.png}
		\includegraphics[scale=0.6]{2b_2.png}

	\end{figure}

	\textbf{Part (c)}\\
	Given that
	\begin{align*}
	E &= \dfrac{x_2^2}{2} + \dfrac{x_1^2}{2} - \dfrac{x_1^4}{4} \\
	\dot{E} &= x_2 \dot{x}_2 + x_1 \dot{x}_1 - {\dot{x}_1 x_1^3}\\
	&= -x_1 x_2 + x_1^3 x_2 + x_1 x_2 - x_2 x_1^3 = 0\\
	E&= constant
	\end{align*}
	Hence the system as defined in the question is a Hamiltonian system. Now we can proceed to analytically solve this equation,
	\begin{align*}
	\dfrac{\dot{x}_1^2}{2} + \dfrac{x_1^2}{2} - \dfrac{x_1^4}{4} = E \\
	\implies \dot{x}_1 = \sqrt{2 E + \dfrac{x_1^4 }{2}-  x_1^2}\\
	\implies x = \int dx_1 \sqrt{2 E + \dfrac{x_1^4 }{2}-  x_1^2}
	\end{align*}

	\begin{figure}[!h]
	\caption{Q2 (c)}
	\centering
	\includegraphics[scale=0.6]{2c_1.png}
	\end{figure}
\end{homeworkProblem}

\begin{homeworkProblem}
	\textit{Solved completely in the Jupyter Notebook.}
\end{homeworkProblem}

\begin{homeworkProblem}
	One definition of religion is \footnote{Dictionary of Philosophy of Religion, Taliaferro \& Marty 2010} :-

	\textit{A religion involves a communal, transmittable body of teachings and prescribed practices about an ultimate, sacred reality or state of being \textbf{that calls for reverence or awe}, a body which guides its practitioners into what it describes as a saving, illuminating or emancipatory relationship to this reality through a personally transformative life of prayer, ritualized meditation, and/or moral practices like repentance and personal regeneration.}
	\\

	One can argue whether this definition is really general, but I think to first order religions like Hinduism, Islam and Christianity agree with this definition.
	\\

	If one is pedantic about it, science, unlike religion, is only a recently coined term and has been in vogue only since the 19th Century\footnote{https://plato.stanford.edu/entries/religion-science/\#WhatScieWhatReli}. The most accepted definition of a scientific hypothesis was given by Karl Popper in 1959, w\textit{hich, in short, states that \textit{scientific hypotheses should in-principle be falsifiable}. There is no reference made to an absolute authority of science}, which then is the most striking difference between science and religion.
	\\

	Note that this is not to say there aren't any similarities between science and religion - science communities also have cults like religion.

\end{homeworkProblem}

\end{document}


\documentclass{memoir}
\usepackage{linalg}

% \begin{figure}[ht]
%     \centering
%     \incfig{riemmans-theorem}
%     \caption{Riemmans theorem}
%     \label{fig:riemmans-theorem}
% \end{figure}

\begin{document}
\section{Singular Value Decomposition}
\label{cha:singular_value_decomposition}
Let \(V, \langle \cdot, \cdot \rangle \) be a finite-dimensional inner product space.

\begin{defn}[Singular Values]
	The \textbf{singular values} \(s_1,\ldots,s_n\) of \(T\) are the eigenvalues of \(\sqrt{T^{*}T} \), each one repeated \( \textrm{dim}E(s_i,\sqrt{T^{*}T} )\) many times.
\end{defn}
We can order the singular values \(s_1,\ldots,s_n\) of \(T\) such that
\begin{align*}
	s_1\geq s_2\geq \ldots\geq s_n\geq 0.
\end{align*}
\begin{thm}[Singular Value Decomposition]
	Let \((V, \langle \cdot, \cdot \rangle )\) be an inner product space over \(\R\) of dimension \(n\). Let \(T \in \mathcal{L}(V)\) be a linear operator with singular values \(s_1,\ldots,s_n\). There exists distinct orthonormal bases \(e_1,\ldots,e_n\) and \(f_1,\ldots,f_n\) of \(V\) such that
	\begin{align*}
		\mathcal{M}(T,(e_1,\ldots,e_n),(f_1,\ldots,f_n)) = \begin{bmatrix} s_1 & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & s_n \end{bmatrix}
	\end{align*}
Furthermore,
\begin{align*}
	T(v) = s_1 \langle v, e_1 \rangle f_1 + \ldots + s_n \langle v, e_n \rangle f_n \quad \forall v \in V
\end{align*}
\end{thm}
This is a decomposition. If \(h_1,\ldots,h_n\) is the standard basis of \(\R^{n}\), then using the standard inner product, let \(A = \mathcal{M}(T,(h_1,\ldots,h_n)\). The SVD theorem is equivalent to saying there are matrices \(U,\Sigma,V\) such that
\begin{align*}
	A = U\Sigma V^T
\end{align*}
This is basically a series of changes of bases.\\

The matrix \(V\) represents the matrix of the operator \(T_V: \R^{n}\to \R^{n}\) that turns the standard basis vectors \(h_1,\ldots,h_n\) to the basis \(e_1,\ldots,e_n\). Therefore the columns of the matrix consist of the coefficients of the \(e_i\) when expressed in terms of the standard basis \(h_1,\ldots,h_n\). This matrix is orthogonal, and thus \(V^{-1}=V^{T}\)\\

The matrix of \(U\) has as columns the coefficients of the \(f_i\) when they are expressed in terms of the standard basis \(h_1,\ldots,h_n\). This matrix is also orthogonal.

\subsection{Computing an SVD}
\label{sec:computing_an_svd}

\textbf{Computing V}
\begin{enumerate}
	\item Compute \(A^{T}A\)
	\item Find an orthonormal basis of eigenvectors for \(A^{T}A\).
	\item Form \(V\) by using these eigenvectors as its columns.
\end{enumerate}
This also gives us \(s_1^2,\ldots,s_n^2\).\\

\textbf{Computing \(\Sigma\)}
\begin{enumerate}
	\item Compute \(V\). When finding the eigenvectors, keep the eigenvalues (compute eigenvectors of \(A^{T}A\)).
	\item The eigenvalues are \(s_1^2,\ldots,s_n^2\), so take the square root of these (in order) to get the singular values
	\item Place these in order along the diagonal
\end{enumerate}

\textbf{Computing U}
\begin{enumerate}
	\item Calculate \(Av_i\) for each \(v_i \in V\)
	\item Divide each \(Av_i \) by \(s_i\)
	\item This yields \(u_i\), which are the columns of \(U\).
\end{enumerate}

\end{document}


\documentclass[main.tex]{subfiles}
\begin{document}

\subsection{Differential geometry}

\marginpar{Monday\\ 2021-11-15}

A point in \(\mathbb{R}^{n}\) is described by the \(n\)-tuple \((x_1, \dots, x_n)\).
These are a dense set (and also a complete one).

Open sets in \(\mathbb{R}^{n}\) are defined as follows: \(S \subseteq \mathbb{R}^{n}\) is open if for all \(x \in S\) we can make a ball \(B_r(x)\) with \(r > 0\) such that \(B_x \subseteq S\), where a ball is a set
%
\begin{align}
B_r (x) = \qty{y \in \mathbb{R}^{n} | \abs{x - y} < r}
\,.
\end{align}

A topological space is a collection of open sets which is closed under arbitrary intersections and finite unions.

A map \(\rho \) between two sets \(M\) and \(N\) takes a point \(x \in M\) to a point \(y = f(x) \in N\).

If we consider a set \(S \subseteq M\) we can look at the set of the images of \(S\) under \(f\), denoted as \(T = \rho (S) \subseteq N\).
We can also define the inverse image, \(S = \rho^{-1} (T)\).

\textbf{Composition} of maps between different sets can also be defined: suppose we have three spaces \(M\), \(N\), \(Z\), with some maps \(f \colon M \to N\) and \(g \colon N \to Z\); their composition will be \(g \circ f \colon M \to Z\).

A map of \(M\) is \textbf{into} \(N\) if all points of \(M\) are mapped to \(N\); a map of \(M\) is \textbf{onto} \(N\) if all points of \(N\) have inverse maps to \(M\).

A \textbf{continuous} map \(f\) between topological spaces \(M\) and \(N\) is one for which, given any point \(x \in M\) such that \(y = f(x) \in N\), there is an open set of \(N\) containing \(f(x)\), which is the image of an open set of \(M\).

\todo[inline]{This is less stringent than the \(\epsilon \)-\(\delta \) definition of continuity because there is no size requirement\dots ??}

A differentiable map is defined as follows: a function \(f (x) \), where \(x \in S\) and \(S \subseteq \mathbb{R}^{n}\) is open, is of order \(\mathcal{C}^{k}\) if all partial derivatives of order up to \(k\) exist and are continuous.

A \textbf{manifold} \(M\) is a collection of points such that each of them has an open neighborhood which has a continuous one-to-one map with an open set of \(\mathbb{R}^{n}\).
The number \(n\) is called the \textbf{dimension} of the manifold.

\todo[inline]{He talks about the locality of this definition as being about the fact that \(M\) is a subset of a larger space\dots}

A coordinate system or \emph{chart} is a pair \((M, \rho )\), where \(M\) is an open set while \(\rho \) is a map.

Suppose we have two coordinate systems \((U, f)\) and \((V, g)\), where \(U\) and \(V\) overlap at least at a point (so, in a region as well).

Consider the region \(f(U \cap V)\): in order to move from it to \(g(U \cap V)\) we need to apply the map \(g \circ f^{-1}\).
This is, therefore, a \textbf{change of coordinates}, which we will be able to write as \(\vec{y} = \vec{y} (\vec{x})\).

A \(\mathcal{C}^{k}\) differentiable \textbf{manifold} is one for which
\begin{enumerate}
    \item each point of \(M\) belongs at least to an open set (and a corresponding chart);
    \item each chart is \(\mathcal{C}^{k}\) related to other charts it overlaps with.
\end{enumerate}

These changes of coordinates can be written as \(y^{i} = F^{i}(x^{j})\), for which we can define a Jacobian determinant:
%
\begin{align}
J = \abs{\pdv{F^{i}}{x^{j}}}
\,.
\end{align}

If \(J \neq 0\) at a point \(P\), there is an onto, one-to-one map in some neighborhood of \(P\).

A \textbf{curve} is a path in \(M\) such that we associate a number, called the parameter, to each point in it, and this provides a map between it and a path in \(\mathbb{R}^{n}\), called the image of the curve.

This is typically denoted as \(\gamma \colon s \in [a, b] \to [x^{1}(s), \dots, x^{\mu }(s)]\).

The quantities
%
\begin{align}
\dv{x^{i}}{s} = \left[\begin{array}{ccc}
\displaystyle \dv{x^{1}}{s}
& \dots &
\displaystyle \dv{x^{n}}{s}
\end{array}\right]
\,
\end{align}
%
are the \emph{components of the tangent vector}.

Suppose we want to make a coordinate change, \(x'(x)\): how does our tangent vector change?
By the chain rule, it will read
%
\begin{align}
\dv{x^{\prime i}}{s} = \pdv{x^{\prime i}}{x^{j}} \dv{x^{j}}{s}
\,.
\end{align}

This kind of vector is called a \textbf{contravariant vector}, since it transforms with the Jacobian matrix.

We want to show that directional derivatives form a vector space at a point \(P\).
Consider a curve with its parameter \(\lambda \) and a differentiable function \(\phi (x_1, \dots, x_m)\).
The derivative
%
\begin{align}
\dv{\phi }{\lambda } = \pdv{\phi }{x^i} \dv{x^{i}}{\lambda }
\,.
\end{align}

With this in mind, we can define the \textbf{directional derivative operator}
%
\begin{align}
\dv{}{\lambda } = \dv{x^{i}}{\lambda } \pdv{}{x^{i}}
\,.
\end{align}

However, we already know that the \(\dv*{x^{i}}{\lambda }\) are the \emph{components of the tangent vector}.

If we have two curves, \(x^{i}(\lambda )\) and \(x^{i} (s)\), we can define a directional derivative for each of them.
We can then sum these, by defining the sum as
%
\begin{align}
\dv{}{\lambda } + \dv{}{s} = \qty(\dv{x^{i}}{\lambda } + \dv{x^{i}}{s}) \pdv{}{x^{i}}
\,.
\end{align}

This vector will still be tangent to the curve, and we will be able to find a third parameter \(\mu \) such that
%
\begin{align}
\dv{}{\mu } = \dv{x^{i}}{\mu } \pdv{}{x^{i}}
\,.
\end{align}

We can also scale the directional derivative operator:
%
\begin{align}
a \dv{}{\lambda } = \underbrace{\qty(a \dv{x^{i}}{\lambda })}_{\dv*{x^{i}}{\sigma }} \pdv{}{x^{i}}
\,.
\end{align}

This proves (?) that the space of directional derivatives is a vector space.
This space is denoted as \(T_P\), where \(\dv*{}{\lambda }\) is a vector.

The crucial idea here is that vectors at different points belong to different vector spaces, and cannot be directly compared.

Coordinate lines are the ones for which all but one of the coordinates we are using remain constant.
What this means is that directional derivatives along these read
%
\begin{align}
\dv{}{x^{i}} = \dv{x^{j}}{x^{i}} \pdv{}{x^{j}} = \delta^{j}_{i} \pdv{}{x^{j}} = \pdv{}{x^{i}}
\,.
\end{align}

This also means that any directional derivatives can be expressed (uniquely) as linear combinations of these \(\partial_{i}\), which are therefore a basis for the tangent space at each point.

We have a one-to-one connection between the tangents of curves at a point \(P\) and the derivatives along a curve at \(P\).
Because of this, we associate the directional derivative \(\dv*{}{\lambda }\) to a tangent vector to a curve \(x^{i}(\lambda )\).

We denote the \(i\)-th basis vector as \(\vec{e}_{(i)} = \pdv{}{x^{i}}\).
The number between round brackets enumerates the vectors in the basis, it is not a spatial index.

We can then express any vector in a more conventional notation as \(\vec{A} = A^{i}\vec{e}_{(i)}\).
If we change coordinates, the basis will shift like
%
\begin{align}
\vec{e}_{k'} = \widetilde{\Lambda}^{j}_{k'} \vec{e}_j
\,,
\end{align}
%
where \(\widetilde{\Lambda}\) is the inverse of the Jacobian.

\paragraph{One-forms}

A one-form \(\widetilde{q}\) is a real-valued, linear function of vectors:
%
\begin{align}
\widetilde{q}(a \vec{V} + \vec{W}) = a \widetilde{q}(\vec{V}) + \widetilde{q}(\vec{W})
\,.
\end{align}

We can sum one-forms by summing their action, and multiply them by scalars by multiplying the result of their application by a scalar.
These are then a vector space, which is called the \emph{dual}: \(T_P^{*}\).

The reason for the name is the dual symmetry: taking \(\widetilde{q}(\vec{V}) \simeq \vec{V} (\widetilde{q})\).

The basis for the dual is called the conjugate basis \(\widetilde{\omega}\), and it is convenient to select it so that \(\widetilde{\omega}^{(i)} (\vec{V}) = V^{i}\).
Alternatively, we can write this as \(\widetilde{\omega}^{(i)} (\vec{e}_{(j)}) = \delta^{i}_{j}\).

Any 1-form can also be written in terms of the basis covectors, \(\widetilde{q} = q_{i} \widetilde{\omega}^{(i)}\).

We can make the manipulation
%
\begin{align}
\widetilde{q} (\vec{V}) =
\widetilde{q} (V^{i} \vec{e}_{(i)}) =
V^{i} \widetilde{q} (\vec{e}_{(i)}) =
\widetilde{\omega}^{(i)}(\vec{V}) \widetilde{q} (\vec{e}_{(i)}) = q^{i} V_i
\,.
\end{align}
%

\end{document}


% ===> this file was generated automatically by noweave --- better not edit it
\section{Introduction}

An \emph{affine} transformation is a transformation of the form
\[
  T(x) = Ax + b
\]
The {\Tt{}affine\nwendquote} module provides routines to create, apply, and
manipulate affine transformations on Euclidean 3-space.

\section{Interface}

\nwfilename{affine.nw}\nwbegincode{1}\sublabel{NW3kVjcs-3IKtGs-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-3IKtGs-1}}}\moddef{affine.h~{\nwtagstyle{}\subpageref{NW3kVjcs-3IKtGs-1}}}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
#ifndef AFFINE_H
#define AFFINE_H

\LA{}exported functions~{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-1}}\RA{}

#endif /* AFFINE_H */
\nwnotused{affine.h}\nwendcode{}\nwbegindocs{2}\nwdocspar

We represent affine transformations by a 3-by-4 double
precision matrix in column-major format.  The leading
3-by-3 submatrix represents the linear part ($A$),
and the trailing column vector represents the translation
($b$).

The {\Tt{}identity\nwendquote} function initializes the matrix to the
representation of the identity transform; $A = I$, $b = 0$.

\nwenddocs{}\nwbegincode{3}\sublabel{NW3kVjcs-2gXoUc-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-1}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-3IKtGs-1}}\nwprevnextdefs{\relax}{NW3kVjcs-2gXoUc-2}\nwenddeflinemarkup
void affine_identity(double* T);

\nwalsodefined{\\{NW3kVjcs-2gXoUc-2}\\{NW3kVjcs-2gXoUc-3}\\{NW3kVjcs-2gXoUc-4}}\nwused{\\{NW3kVjcs-3IKtGs-1}}\nwendcode{}\nwbegindocs{4}\nwdocspar

There are several functions which modify an existing
transformation $T$ by composing it with a simple transformation
$S$ to get a new transformation $ST$.
In the {\Tt{}translate\nwendquote} and {\Tt{}translate{\_}xyz\nwendquote} functions, $S$ is
a simple translation ($A = 0$, $b$ is specified by the arguments).
The {\Tt{}rotate{\_}ox\nwendquote}, {\Tt{}rotate{\_}oy\nwendquote}, and {\Tt{}rotate{\_}oz\nwendquote} functions
apply rotations by some fixed number of radians about the $x$,
$y$, and $z$ axes, respectively.  The rotations are oriented
according to the right hand rule.  The {\Tt{}scale\nwendquote}, {\Tt{}scale{\_}x\nwendquote},
{\Tt{}scale{\_}y\nwendquote}, and {\Tt{}scale{\_}z\nwendquote} apply a scaling, either of all
variables or just of the individual components.

\nwenddocs{}\nwbegincode{5}\sublabel{NW3kVjcs-2gXoUc-2}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-2}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-3IKtGs-1}}\nwprevnextdefs{NW3kVjcs-2gXoUc-1}{NW3kVjcs-2gXoUc-3}\nwenddeflinemarkup
void affine_translate    (double* T, double* c);
void affine_translate_xyz(double* T, double x, double y, double z);
void affine_rotate_ox    (double* T, double ox);
void affine_rotate_oy    (double* T, double oy);
void affine_rotate_oz    (double* T, double oz);
void affine_scale        (double* T, double s);
void affine_scale_x      (double* T, double s);
void affine_scale_y      (double* T, double s);
void affine_scale_z      (double* T, double s);
\nwused{\\{NW3kVjcs-3IKtGs-1}}\nwendcode{}\nwbegindocs{6}\nwdocspar

The {\Tt{}compose\nwendquote} function composes two transformations
to get $S := TS$.

\nwenddocs{}\nwbegincode{7}\sublabel{NW3kVjcs-2gXoUc-3}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-3}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-3IKtGs-1}}\nwprevnextdefs{NW3kVjcs-2gXoUc-2}{NW3kVjcs-2gXoUc-4}\nwenddeflinemarkup
void affine_compose(double* T, double* S);

\nwused{\\{NW3kVjcs-3IKtGs-1}}\nwendcode{}\nwbegindocs{8}\nwdocspar

There are several versions of the {\Tt{}apply\nwendquote} transformation.
The basic {\Tt{}apply\nwendquote} transformation computes $x := T(x)$,
but there is also {\Tt{}apply{\_}A\nwendquote} to apply just the linear
part of the transformation ($x := Ax$), and {\Tt{}apply{\_}AT\nwendquote}
to apply the transpose of the linear part ($x := A^T x$).
The basic {\Tt{}apply\nwendquote} functions assume that the argument vector
is laid out contiguously in memory, but there are also
versions of the functions which assume the elements are
separated by a constant stride.

\nwenddocs{}\nwbegincode{9}\sublabel{NW3kVjcs-2gXoUc-4}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-4}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NW3kVjcs-2gXoUc-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-3IKtGs-1}}\nwprevnextdefs{NW3kVjcs-2gXoUc-3}{\relax}\nwenddeflinemarkup
void affine_apply          (double* T, double* x);
void affine_apply_stride   (double* T, double* x, int stride);
void affine_apply_A        (double* T, double* x);
void affine_apply_A_stride (double* T, double* x, int stride);
void affine_apply_AT       (double* T, double* x);
void affine_apply_AT_stride(double* T, double* x, int stride);

\nwused{\\{NW3kVjcs-3IKtGs-1}}\nwendcode{}\nwbegindocs{10}\nwdocspar

\section{Implementation}

\nwenddocs{}\nwbegincode{11}\sublabel{NW3kVjcs-dYkGQ-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-dYkGQ-1}}}\moddef{affine.c~{\nwtagstyle{}\subpageref{NW3kVjcs-dYkGQ-1}}}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
#include <sugar.h>

#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <affine.h>

\LA{}functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}\RA{}
\nwnotused{affine.c}\nwendcode{}\nwbegindocs{12}\nwdocspar

\nwenddocs{}\nwbegincode{13}\sublabel{NW3kVjcs-nRuDO-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{\relax}{NW3kVjcs-nRuDO-2}\nwenddeflinemarkup
void affine_identity(double* T)
\{
    memset(T, 0, 12*sizeof(double));
    T[0] = 1;
    T[4] = 1;
    T[8] = 1;
\}

\nwalsodefined{\\{NW3kVjcs-nRuDO-2}\\{NW3kVjcs-nRuDO-3}\\{NW3kVjcs-nRuDO-4}\\{NW3kVjcs-nRuDO-5}\\{NW3kVjcs-nRuDO-6}\\{NW3kVjcs-nRuDO-7}\\{NW3kVjcs-nRuDO-8}\\{NW3kVjcs-nRuDO-9}\\{NW3kVjcs-nRuDO-A}\\{NW3kVjcs-nRuDO-B}\\{NW3kVjcs-nRuDO-C}}\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{14}\nwdocspar

Note that the {\Tt{}translate\nwendquote} function overwrites $T$ with
the composition of translation with $T$, not with the
composition of $T$ with the translation.  The same holds
for the rotations.

\nwenddocs{}\nwbegincode{15}\sublabel{NW3kVjcs-nRuDO-2}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-2}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-1}{NW3kVjcs-nRuDO-3}\nwenddeflinemarkup
void affine_translate(double* T, double* c)
\{
    T[9 ] += c[0];
    T[10] += c[1];
    T[11] += c[2];
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{16}\nwdocspar

\nwenddocs{}\nwbegincode{17}\sublabel{NW3kVjcs-nRuDO-3}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-3}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-2}{NW3kVjcs-nRuDO-4}\nwenddeflinemarkup
void affine_translate_xyz(double* T, double x, double y, double z)
\{
    T[9 ] += x;
    T[10] += y;
    T[11] += z;
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{18}\nwdocspar

The rotation functions obey the right-hand rule, which usually
confuses me.  That is why there is a sign reversal on the angle
for the rotation about the $y$ axis.  Since these are purely
linear transformations, we just need to transform each column
of $T$ in turn.

\nwenddocs{}\nwbegincode{19}\sublabel{NW3kVjcs-nRuDO-4}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-4}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-3}{NW3kVjcs-nRuDO-5}\nwenddeflinemarkup
void affine_rotate_ox(double* T, double ox)
\{
    double c = cos(ox);
    double s = sin(ox);
    int i;

    for (i = 0; i < 4; ++i) \{
        double y =  c*T[1] + s*T[2];
        double z = -s*T[1] + c*T[2];
        T[1] = y;
        T[2] = z;
        T += 3;
    \}
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{20}\nwdocspar

\nwenddocs{}\nwbegincode{21}\sublabel{NW3kVjcs-nRuDO-5}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-5}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-4}{NW3kVjcs-nRuDO-6}\nwenddeflinemarkup
void affine_rotate_oy(double* T, double oy)
\{
    double c =  cos(oy);
    double s = -sin(oy);
    int i;

    for (i = 0; i < 4; ++i) \{
        double x =  c*T[0] + s*T[2];
        double z = -s*T[0] + c*T[2];
        T[0] = x;
        T[2] = z;
        T += 3;
    \}
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{22}\nwdocspar

\nwenddocs{}\nwbegincode{23}\sublabel{NW3kVjcs-nRuDO-6}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-6}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-5}{NW3kVjcs-nRuDO-7}\nwenddeflinemarkup
void affine_rotate_oz(double* T, double oz)
\{
    double c = cos(oz);
    double s = sin(oz);
    int i;

    for (i = 0; i < 4; ++i) \{
        double x =  c*T[0] + s*T[1];
        double y = -s*T[0] + c*T[1];
        T[0] = x;
        T[1] = y;
        T += 3;
    \}
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{24}\nwdocspar

The {\Tt{}scale\nwendquote} function scales everything in sight by the given constant.

\nwenddocs{}\nwbegincode{25}\sublabel{NW3kVjcs-nRuDO-7}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-7}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-6}{NW3kVjcs-nRuDO-8}\nwenddeflinemarkup
void affine_scale(double* T, double s)
\{
    int i;
    for (i = 0; i < 12; ++i)
        *T++ *= s;
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{26}\nwdocspar

The {\Tt{}scale{\_}x\nwendquote}, {\Tt{}scale{\_}y\nwendquote}, and {\Tt{}scale{\_}z\nwendquote} functions only scale
the appropriate rows.

\nwenddocs{}\nwbegincode{27}\sublabel{NW3kVjcs-nRuDO-8}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-8}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-7}{NW3kVjcs-nRuDO-9}\nwenddeflinemarkup
void affine_scale_x(double* T, double s)
\{
    int row = 0;
    \LA{}scale row~{\nwtagstyle{}\subpageref{NW3kVjcs-yJ7l-1}}\RA{}
\}

void affine_scale_y(double* T, double s)
\{
    int row = 1;
    \LA{}scale row~{\nwtagstyle{}\subpageref{NW3kVjcs-yJ7l-1}}\RA{}
\}

void affine_scale_z(double* T, double s)
\{
    int row = 2;
    \LA{}scale row~{\nwtagstyle{}\subpageref{NW3kVjcs-yJ7l-1}}\RA{}
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{28}\nwdocspar

\nwenddocs{}\nwbegincode{29}\sublabel{NW3kVjcs-yJ7l-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-yJ7l-1}}}\moddef{scale row~{\nwtagstyle{}\subpageref{NW3kVjcs-yJ7l-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-nRuDO-8}}\nwenddeflinemarkup
int i;
for (i = 0; i < 4; ++i, T += 3)
    T[row] *= s;
\nwused{\\{NW3kVjcs-nRuDO-8}}\nwendcode{}\nwbegindocs{30}\nwdocspar

The {\Tt{}compose\nwendquote} function does \emph{not} necessarily do a purely
linear transformation.  The composition $TS$ will have the form
\begin{eqnarray*}
  TS(x) & = & A_T (A_S x + b_S) + b_T \\
        & = & A_T A_S x + T(b_S)
\end{eqnarray*}
So we apply $A_T$ to the first three columns of $S$, and then
apply $T$ to the last column of $S$.

\nwenddocs{}\nwbegincode{31}\sublabel{NW3kVjcs-nRuDO-9}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-9}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-8}{NW3kVjcs-nRuDO-A}\nwenddeflinemarkup
void affine_compose(double* T, double* S)
\{
    int i;
    for (i = 0; i < 3; ++i) \{
        double y[3];

        y[0] = T[0]*S[0] + T[3]*S[1] + T[6]*S[2];
        y[1] = T[1]*S[0] + T[4]*S[1] + T[7]*S[2];
        y[2] = T[2]*S[0] + T[5]*S[1] + T[8]*S[2];

        S[0] = y[0];
        S[1] = y[1];
        S[2] = y[2];

        S += 3;
    \}

    affine_apply(T, S);
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{32}\nwdocspar

The {\Tt{}apply\nwendquote}, {\Tt{}apply{\_}A\nwendquote}, and {\Tt{}apply{\_}AT\nwendquote} functions are
implemented as simple wrappers around their strided
counterparts.

\nwenddocs{}\nwbegincode{33}\sublabel{NW3kVjcs-nRuDO-A}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-A}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-9}{NW3kVjcs-nRuDO-B}\nwenddeflinemarkup
void affine_apply(double* T, double* x)
\{
    affine_apply_stride(T, x, 1);
\}

void affine_apply_stride(double* T, double* x, int stride)
\{
    double x0 = x[0*stride];
    double x1 = x[1*stride];
    double x2 = x[2*stride];

    x[0*stride] = T[0]*x0 + T[3]*x1 + T[6]*x2 + T[9 ];
    x[1*stride] = T[1]*x0 + T[4]*x1 + T[7]*x2 + T[10];
    x[2*stride] = T[2]*x0 + T[5]*x1 + T[8]*x2 + T[11];
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{34}\nwdocspar

\nwenddocs{}\nwbegincode{35}\sublabel{NW3kVjcs-nRuDO-B}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-B}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-A}{NW3kVjcs-nRuDO-C}\nwenddeflinemarkup
void affine_apply_A(double* T, double* x)
\{
    affine_apply_A_stride(T, x, 1);
\}

void affine_apply_A_stride(double* T, double* x, int stride)
\{
    double x0 = x[0*stride];
    double x1 = x[1*stride];
    double x2 = x[2*stride];

    x[0*stride] = T[0]*x0 + T[3]*x1 + T[6]*x2;
    x[1*stride] = T[1]*x0 + T[4]*x1 + T[7]*x2;
    x[2*stride] = T[2]*x0 + T[5]*x1 + T[8]*x2;
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}\nwbegindocs{36}\nwdocspar

\nwenddocs{}\nwbegincode{37}\sublabel{NW3kVjcs-nRuDO-C}\nwmargintag{{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-C}}}\moddef{functions~{\nwtagstyle{}\subpageref{NW3kVjcs-nRuDO-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3kVjcs-dYkGQ-1}}\nwprevnextdefs{NW3kVjcs-nRuDO-B}{\relax}\nwenddeflinemarkup
void affine_apply_AT(double* T, double* x)
\{
    affine_apply_AT_stride(T, x, 1);
\}

void affine_apply_AT_stride(double* T, double* x, int stride)
\{
    double x0 = x[0*stride];
    double x1 = x[1*stride];
    double x2 = x[2*stride];

    x[0] = T[0]*x0 + T[1]*x1 + T[2]*x2;
    x[1] = T[3]*x0 + T[4]*x1 + T[5]*x2;
    x[2] = T[6]*x0 + T[7]*x1 + T[8]*x2;
\}

\nwused{\\{NW3kVjcs-dYkGQ-1}}\nwendcode{}

\nwixlogsorted{c}{{affine.c}{NW3kVjcs-dYkGQ-1}{\nwixd{NW3kVjcs-dYkGQ-1}}}%
\nwixlogsorted{c}{{affine.h}{NW3kVjcs-3IKtGs-1}{\nwixd{NW3kVjcs-3IKtGs-1}}}%
\nwixlogsorted{c}{{exported functions}{NW3kVjcs-2gXoUc-1}{\nwixu{NW3kVjcs-3IKtGs-1}\nwixd{NW3kVjcs-2gXoUc-1}\nwixd{NW3kVjcs-2gXoUc-2}\nwixd{NW3kVjcs-2gXoUc-3}\nwixd{NW3kVjcs-2gXoUc-4}}}%
\nwixlogsorted{c}{{functions}{NW3kVjcs-nRuDO-1}{\nwixu{NW3kVjcs-dYkGQ-1}\nwixd{NW3kVjcs-nRuDO-1}\nwixd{NW3kVjcs-nRuDO-2}\nwixd{NW3kVjcs-nRuDO-3}\nwixd{NW3kVjcs-nRuDO-4}\nwixd{NW3kVjcs-nRuDO-5}\nwixd{NW3kVjcs-nRuDO-6}\nwixd{NW3kVjcs-nRuDO-7}\nwixd{NW3kVjcs-nRuDO-8}\nwixd{NW3kVjcs-nRuDO-9}\nwixd{NW3kVjcs-nRuDO-A}\nwixd{NW3kVjcs-nRuDO-B}\nwixd{NW3kVjcs-nRuDO-C}}}%
\nwixlogsorted{c}{{scale row}{NW3kVjcs-yJ7l-1}{\nwixu{NW3kVjcs-nRuDO-8}\nwixd{NW3kVjcs-yJ7l-1}}}%
\nwbegindocs{38}\nwdocspar

\nwenddocs{}


\section{Numerical experiments}\label{sec:numerics}
In this section, we present some numerical results to illustrate the
efficiency and potential of MgNet as described in Algorithm
\ref{alg:mgnet}.

\subsection{Data sets and model structure }
We choose CIFAR-10 and CIFAR-100
\cite{krizhevsky2009learning}
as two data sets for numerical tests.
Here, the CIFAR-10 dataset consists of 60000 32x32 colour
images in 10 classes, with 6000 images per class.
The CIFAR-100 dataset is just like the CIFAR-10,
except it has 100 classes containing 600 images each.
We split these two data sets with 50000 training images
and 10000 test images.

We will mainly carry out
 a comparison with study between MgNet and  ResNet \cite{he2016deep}
on these two data sets, so we choose some
similar process techniques such as there will
be a average pooling before linear regression
layers:
\begin{equation}\label{eq:ave-pooling}
R_{ave}: \mathbb{R}^{m_{J-1} \times n_{J-1} \times c_{J-1}} \mapsto \mathbb{R}^{c_{J-1}}.
\end{equation}
We use the similar ideas in MgNet to take $\nu_{J} = 0$, thus to choose
$$
u^{J} = \Pi_{J-1}^J u^{J-1, m_{J-1}} \in \mathbb{R}^{c_{J-1}},
$$
with
$$
\Pi_{J-1}^J  = R_{ave}.
$$
This can be true also thanks to our structure that
\begin{equation}\label{eq:c_u}
c_{u,\ell} = c_{u}, \quad 1 \le \ell \le J.
\end{equation}
Given an image $f$, similar to ResNet, we apply our MgNet as follows:
\begin{equation}\label{final-mg}
y = S \circ \theta \circ u^{J}(f),
\end{equation}
where $u^J(f)$ is the output from our MgNet as described in Algorithm
\ref{alg:mgnet},  $S$ is the soft-max mapping  and
\begin{equation}\label{final-theta}
\theta: \mathbb{R}^{c_u} \mapsto \mathbb{R}^\kappa,
\end{equation}
represents a fully linear layer with $\kappa = 10$ for CIFAR-10 and
$\kappa = 100$ for CIFAR-100.

We will make the following choice of hyper parameters
for the MgNet:
\begin{itemize}
	\item $J$: the number of grids. As all images in CIFAR-10 or CIFAR-100
	are $32\times 32 \times 3$,  we choose $J = 4$ to be consistent with ResNet.
	\item $\nu_\ell$:  the number of smoothings in each grids. To be consistent with
	ResNet-18 or ResNet-34 we choose $\nu_\ell = 2$ or $\nu_\ell = 4$.
	\item $c_u$ and $c_f$: the number of feature and data channels.
	\item $A^\ell$: the data-feature mapping. We choose the linear case in \eqref{linearA}.
	\item $B^{\ell,i}$: the feature extractor. We choose the variable extractors as in \eqref{extractor-ell}.
	\item $R_{\ell}^{\ell+1}$: the restriction operator in \eqref{restrict-f}.
	Here we choose it as a convolution with stride $2$ which need to be trained.
	\item $\Pi_\ell^{\ell+1}$: the interpolation operator in
	\eqref{interpolation}.  Here we compare these next three
	different choices:
	\begin{enumerate}
		\item {$\Pi_0$: } $\Pi_\ell^{\ell+1} = 0$;
		\item {$\Pi_1$: }convolution with stride $2$ which need to be
		trained;
		\item {$\Pi_2$: }channel-wise interpolation as in
		\eqref{Pi}, with $\bar P_{\ell}^{\ell+1}$ as a convolution
		with one channel and stride $2$ which also need to be trained.
	\end{enumerate}
\end{itemize}

\subsection{Top 1 and Top 5 accuracy}
Once you trained your model, you have the CNN model with soft-max layer.
This can be defined by
\begin{equation}
y = H(f) = S \circ \theta \circ u^{J}(f),
\end{equation}
as in \eqref{final-mg}.

Now, suppose that we have test data like:
\begin{equation}
\{(f_i, y_i)\}_{i=1}^n.
\end{equation}

Then the two accuracy are defined by:
\begin{description}
\item[Top 1]
\begin{equation}
\frac{1}{n}\left( \# \left\{i~:~  y_i =  \max_{ j = 1:\kappa}\{[H(f_i)]_j \} \right \} \right).
\end{equation}
\item[Top 5]
\begin{equation}
\frac{1}{n}\left( \# \left\{i~:~  y_i \in \max_5 \{ [H(f_i)]_1, \cdots, [H(f_i)]_\kappa \} \right \} \right)
\end{equation}
where $\max_5$ means the first 5 largest elements:
\begin{equation}
{\rm max}_5\{x_1, \cdots, x_\kappa \} = \{x_{j_1}, \cdots, x_{j_5}\},
\end{equation}
with
\begin{equation}
x_{j_1} \ge \cdots \ge x_{j_5} \ge x_{j}, \quad \forall j \neq j_1 \cdots, j_5.
\end{equation}

\end{description}

\subsection{Training algorithm}\label{subsec:traning}
While there are many different choices of training algorithms \cite{bottou2018optimization},
in our test, we adopt the popular
stochastic gradient descent (SGD) with mith-batch and momentum for
cross-entropy loss function.
\begin{breakablealgorithm}
	\caption{SGD with mini-batch and momentum}
	\label{alg:sgd}
	\begin{algorithmic}
\State {\bf Input}: learning rate $\eta_t$, batch size $m$, parameter Initialization $ w_0$, number of epochs $K$.
\For{Epoch $k = 1:K$} \\
\State Shuffle data and get mini-batch $B_1, \cdots, B_{\frac{N}{m}}$, choose mini-batch as: $B_{i_t}$ with
$$
i_t \equiv t \mod(\frac{N}{m}),
$$
\State Compute the gradient on $B_{i_t}$:
$$
g_t = \nabla_{w} \frac{1}{m} \sum_{i \in B_{i_t}} h_i(w_{t}).
$$
\State Compute the momentum:
\begin{equation}
v_t = \alpha v_{t-1} - \eta_t g_t \quad (v_0 = 0).
\end{equation}
\State Update $w$:
\begin{equation}
w_{t+1} = w_t + v_t.
\end{equation}
\EndFor
\end{algorithmic}
\end{breakablealgorithm}

Here we have $h_i(w_t) = l(H(f_i;w_t),y_i)$ as defined in \eqref{eq:3}, where $w_t$ notes all free parameters in MgNet and $\theta$ in \eqref{final-theta}.
We use the SGD with momentum of 0.9.
The mini-batch size is chosen as
$m=128$. The learning rate starts from 0.1 and is divided by $10$ for
every $30$ epochs, and the models are trained for up to $K=120$ epochs.
We adopt batch normalization (BN) after each convolution and before
activation, following \cite{ioffe2015batch}.  Initialization strategy
is the same with ResNet as in \cite{he2015delving}.  We
do not use weight decay and dropout.  The final Top-1 test accuracy is
shown in Table~\ref{comparison}.
\begin{table}[!htbp]
	\caption{ResNet and MgNet on CIFAR-10 and CIFAR-100.
	Our methods are named with $\nu_\ell$, ($c_u$, $c_f$), $\Pi_\ell^{\ell+1}$ by definition above.}
	\label{comparison}
	\vskip 0.15in
	\begin{center}
				%		\resizebox{.5\textwidth}{!}{
				\begin{tabular}{cccc}
					\hline
					Models & CIFAR-10 & CIFAR-100 & Params \\
					\hline
					ResNet-18 & 92.24 & 71.96 & 11.2M   \\
					ResNet-34 & 92.80 & 71.93 & 21.3M   \\
					\hline
					%				{$\nu_\ell$,  $(c_u, c_f)$, $\Pi_\ell^{\ell+1}$} & Accuracy & Accuracy & Numbers  \\
					$2, (256,256)$, $\Pi_0$ & 92.02 & 68.29 & 7.1M  \\
					$2, (256,256)$, $\Pi_1$ & 93.04 & 72.32 & 8.9M  \\
					$2, (256,512)$, $\Pi_1$ & 93.20 & 72.42 & 19.5M  \\
					$4, (256,512)$, $\Pi_2$ & 93.53& 74.26 & 17.7M  \\
					\hline
				\end{tabular}
		%		}
	\end{center}
	\vskip -0.1in
\end{table}

From the above numerical results, we find that the modified CNN models
based on MgNet structure have competitive and sometimes better
performance in comparison with standard ResNet models when applied to
both CIFAR-10 and CIFAR-100 data sets. Generally speaking, the more
channels the better performance you can achieve (see WideResNet
\cite{zagoruyko2016wide} for similar observation). Furthermore,
$\Pi_1$ and $\Pi_2$ work better than $\Pi_0$, and $\Pi_2$ can even
work better than $\Pi_1$ with fewer parameters for big enough
channel numbers.

\section{Concluding remarks}\label{sec:conclusion}
By carefully studying the connections between the traditional
multigrid method and the convolutional neural network (especially the
ResNet type) models, the MgNet established in this paper provides a
unified framework that connects both multigrid and CNN in a technical
level.  Comparing with other existing works that discuss the
connection between multigrid and CNN, MgNet goes beyond formal or
qualitative comparisons and identifies key model components that play
the same corresponding roles, from an abstract viewpoint, for these two different
methodologies.  As a result, how and why CNN models work can be
mathematically understood in a similar fashion as for multigrid method
which has a much more mature and better developed theory.  Motivated
from various known techniques from multigrid method, many variants and
improvements of CNN can then be naturally obtained.  For example, as
demonstrated from our preliminary numerical experiments, the resulting
modified CNN models quipped with fewer weights and hyper parameters
actually exhibit competitive and sometimes better performance than
standard ResNet models.

The MgNet framework opens a new door to the
mathematical understanding, analysis and improvements of deep learning
models.  The very preliminary results presented in
this paper have demonstrated the great potential of MgNet from both
theoretical and practical viewpoints.  Obviously many aspects of MgNet
should be further explored and expect to be much improved.  In fact, only very
few techniques from multigrid method have been tried in this paper and
many more in-depth techniques from multigrid require further study for
deep neural networks, especially CNN.
In particular, we believe that the MgNet framework will
lead to improved CNN that only has a small fraction of the number
of weights that are required by the current CNN. On the other hand,
the techniques in CNN can also be used to develop new generation of multigrid
and especially  algebraic multigrid methods \cite{xu2017algebraic} for solving
partial differential equations. Our ongoing works have
demonstrated great potentials for research in these directions and  many
more results will be reported in future papers.