% !TEX root = ../main.tex
% !TEX spellcheck = en_GB

\chapter{Analysis}
\label{chap:analysis}
\fxnote{Introduction to the system. Introduction to Thiele/Small via circuit analysis. Introduction of each subsystem's TF (both from Tore's book and derived).}
This section describes the analysis of the system; how the system is envisioned to be created, which subsystems it will consist of and the tasks each subsystem has to be able to perform to meet the specified requirements.

\section{Loudspeaker Modelling}
The loudspeaker system can consist of multiple drive units reproducing sound in different frequency spectra.
The drive unit can be parted in three equivalent circuits representing the electrical, the mechanical and the acoustical element.
Each of these elements can be converted to its electrical equivalent and be represented with an Ohm's Law analogy \cite[p.~115]{Elektroakustik}.

The electrical element of the drive unit is specified by the voice coil with its DC resistance $R_e$ and self inductance $L_e$.
The coupling between the electrical and the mechanical elements is specified as the force factor $Bl$.
This is the product of the magnetic field in the voice coil gap and the length of the voice coil in the magnetic field. \cite[p.~34]{Elektroakustik}

The mechanical element of the drive unit is specified by the mass $M_{MD}$ of the diaphragm, the mechanical damping $R_{MS}$ and the compliance $C_{MS}$.
The mechanical element will introduce a resonance frequency $f_S$ and is described by its quality factor $Q_{MS}$.
A total quality factor $Q_{TS}$ is found when combining the Q-factor for the electrical and mechanical elements, $\frac{Q_{ES}Q_{MS}}{Q_{ES}+Q_{MS}}$.

The acoustical element of the drive unit is specified by an acoustical impedance in front and behind the diaphragm.
The acoustical impedance in front is given by the relation between sound pressure $p$ and volume velocity $q$ which is the velocity of air that is moved by the diaphragm.

These parameters are known as Thiele/Small parameters and are used to specify the performance of a drive unit and is derived by A.N. Thiele \cite{thiele1971loudspeakers} and Richard H. Small \cite{small1972closed}.
The parameters can be used to decide the volume of the loudspeaker cabinet and the length of the bass-reflex port.
The best performance often includes improving the bass response and to obtain a flatness in general of the frequency response.
The physical parameters of the drive unit can be found in the datasheet.

\paragraph{$R_E$} is the DC resistance of the voice coil.

\paragraph{$L_E$} is the inductance of the voice coil.

\paragraph{$f_s$} is the resonance frequency.

\paragraph{$Q_{TS}$} is the combined electric and mechanical damping of the drive unit.

\paragraph{$M_{MS}$} is the mass of the drive unit's moving parts including acoustic load.

\paragraph{$C_{MS}$} is the mechanical compliance of the drive unit's suspension.

\paragraph{$R_{MS}$} is the mechanical resistance of the drive unit's suspension.

\paragraph{$Bl$} is the force factor determined by the product of the magnetic flux density in the air gap and the length of wire in the air gap.

\paragraph{$S_D$} is the surface area of the drive unit's diapraghm.

\section{Transfer Functions}
The loudspeaker being modelled in this project has been parted in 4 subsystems; a drive unit, a cross-over filter, a cabinet and a bass reflex.
Each subsystem have a transfer function that can be derived from the Thiele/Small parameters.

\subsection{Cabinet}
The cabinet is a sealed enclosure, which separates the front and back side of the diaphragm, thus avoiding acoustic short circuiting.
Placing a drive unit in a sealed enclosure prevents the pressure at the front and back to interact with each other. \cite[p.~44]{Elektroakustik}.
The volume of the closed cabinet is characterized by the air acting like a spring (a capacity) which affects the drive unit's compliance, see \cref{eq:cab}, where $V_B$ is the volume of the cabinet, $\rho$ is the density of air and $c$ is the speed of sound.

\begin{equation}
C_{AB} = \frac{V_B}{\rho \,c^2}
\label{eq:cab}
\end{equation}

The transfer function for a drive unit placed in an enclosure is shown in \cref{eq:dutfclosed}.
The sound pressure is specified at a distance $r=\SI{1}{\meter}$.

\begin{equation}
p = \frac{\rho S_D B l U_G}{2\pi r M_{MS} R_E}\frac{s^2}{s^2 +s \left(\frac{(Bl)^2}{R_EM_{MS}}+\frac{R_{MS}}{M_{MS}}\right)+\frac{1}{M_{MS}C_{MS}}\left(1+\frac{C_{MS}{S_D}^2}{C_{AB}}\right)}
\label{eq:dutfclosed}
\end{equation}

\subsection{Drive Unit}
\label{seq:driveunit}
The drive unit can be modelled as mounted in an infinite baffle, which also separates the front and back side of the diaphragm.
The transfer function for a drive unit placed in an infinite baffle is shown in \cref{eq:dutfinfinite}. This equation contains the diffraction and reflection seen in the two last terms.

\begin{align}
	p =& \frac{\rho S_D B l U_G}{2\pi r M_{MS} R_E}\left|\frac{s^2}{s^2 + \frac{\omega_s}{Q_{TS}}s+\omega_s^2}\right| \label{eq:dutfinfinite} \\
	& \left[1-\frac{r_F}{r_B}D(ka)\exp(-jk(r_B-r_F))\right] \left[1-\frac{r_F}{r_R}D(ka)\exp(-jk(r_R-r_F))\right]\notag
\end{align}

The constant term in \cref{eq:dutfinfinite} transforms the voltage $U_G$ to the sound pressure $p$ while the complex term represent a second order high pass filter.
The slope of the high pass filter will decrease by \SI{12}{\decibel} per octave below the resonance frequency.\\

In this project the drive unit is being modelled as mounted in an infinite enclosure, a very large closed cabinet. Doing this will avoid the diffraction and reflection terms in \cref{eq:dutfinfinite}.
Placing the drive unit in enclosure will prevent acoustic short circuiting as well.
If the drive unit is placed in a very large sealed enclosure it will make the model behave like the drive unit was mounted in an infinite baffle.
When the volume becomes very large the compliance as well becomes very large seen in \cref{eq:cab}.
The term containing the $C_{AB}$ variable goes towards zero as $C_{AB}\rightarrow\infty$ in \cref{eq:dutfclosed} and the transfer function for a drive unit placed in an very large enclosure is now as shown in \cref{eq:dutfclosedvolume}.

\begin{align}
	p_\infty &= \lim_{C_{AB}\rightarrow\infty}\left(p\right) \\
	\Rightarrow p_\infty &= \frac{\rho S_D B l U_G}{2\pi r M_{MS} R_E}\frac{s^2}{s^2 +s \left(\frac{(Bl)^2}{R_EM_{MS}}+\frac{R_{MS}}{M_{MS}}\right)+\frac{1}{M_{MS}C_{MS}}}
	\label{eq:dutfclosedvolume}
\end{align}

\subsection{Crossover Filter}
Different drive units are differently designed to reproduce the sound in specific frequency spectra.
When a loudspeaker system consist of multiple drive units a crossover filter is used to separate these frequency spectrums.
For a drive unit (tweeter) reproducing the high frequency spectrum, a second order high pass filter is used to filter the low frequency spectrum, see eq. \ref{eq:high} \cite[p.~82-83]{Elektroakustik}.
\begin{equation}
H_{T} = \frac{s^2}{\omega_0^2+2\zeta\,\omega_0 s+s^2}
\label{eq:high}
\end{equation}

\newpage For a drive unit (woofer) reproducing the low frequency spectrum, a second order low pass filter is used to filter the high frequency spectrum, see \cref{eq:low}.
\begin{equation}
H_{W} = \frac{\omega_0^2}{\omega_0^2+2\zeta\,\omega_0 s+s^2}
\label{eq:low}
\end{equation}

A quality factor is introduced for the crossover filter, as it was for the drive unit itself.
With a Q-factor of 0.7 a maximal flat pass-band filter is made.
This is known as a butterworth filter.
The sharper the transition band needs to be, the higher the Q-factor, given the damping factor $\zeta\,$ which is the inverse of Q.

\subsection{Bass Reflex}
A bass reflex is a port placed in the cabinet, also called a vented cabinet.
By creating a port, the pressure from the rear side of the diaphragm can increase the low frequency spectrum.
The air in the port will resonate with the volume of the cabinet and this will introduce an additional resonance frequency \cite[p.~53]{Elektroakustik}.
The transfer function for a drive unit placed in a vented cabinet is shown in \cref{eq:bassreflex}.

\begin{equation}
p = \frac{\rho S_D B l U_G}{2\pi r M_{MS} R_E}\,H_{BR}(s)
\label{eq:bassreflex}
\end{equation}

The constant term in \cref{eq:bassreflex} transforms the voltage $U_G$ to the sound pressure $p$ while the $H_{BR}(s)$ represent a fourth order high pass filter and the slope will decrease by \SI{24}{\decibel} per octave below the resonance frequency.

\begin{equation}
H_{BR}(s) = \frac{s^4}{s^4+a_3\omega_0s^3+a_2\omega_0^2s^2+a_1\omega_0^3s+\omega_0^4}
\label{eq:brtf}
\end{equation}

The parameters $a_1$, $a_2$ and $a_3$ in \cref{eq:brtf} can be defined according to the Thiele/Small parameters to output the desired frequency response  \cite[p.~55]{Elektroakustik}.

\section{Frequency Response}
The sound pressure is measured in \si{\decibel} SPL and is a ratio of the sound pressure $p$ and the threshold of hearing $p_{ref}=\SI{20}{\micro\pascal}$, see \cref{eq:soundpressure}.

\begin{equation}
L=20\log_{10}\left(\frac{p}{p_{ref}}\right)
\label{eq:soundpressure}
\end{equation}

The transfer functions applies in an area between the drive unit's resonance frequency $f_s$ and approximately the frequency $f_1\approx\frac{c}{2\pi a}$ where $a$ is the radius of the drive unit's baffle and $c$ is the velocity of sound \cite[p.~41]{Elektroakustik}.

\FloatBarrier


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{amsmath, amssymb, amsfonts, amsthm, minted}
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}
\usepackage{graphicx, caption, subcaption}
\usepackage[numbers]{natbib}
\usepackage{tikz}
\usetikzlibrary{positioning,calc,shapes,arrows}
% Define block styles XXX
\tikzstyle{block} = [rectangle, draw,
    text width=8em, text centered, minimum height=5em]
\tikzstyle{line} = [draw, -latex',line width=1mm]

\bibliographystyle{plainnat}

\providecommand{\abs}[1]{\left\lvert#1\right\rvert}
\providecommand{\norm}[1]{\left\lVert#1\right\rVert}

\title{CS 240H Project Report: \\ Linear Least Squares Modeling in Haskell}
\author{Tri Dao \\ \texttt{trid@stanford.edu}}
\date{March 18, 2016}
\begin{document}

\maketitle

\section{Introduction}

Least squares are optimization problems that minimize the norm square
of an affine expression, subject to linear constraints on the variables.
They have the general form
\begin{equation}
  \begin{array}{ll}
    \mbox{minimize} & \|Ax-b\|^2 \\
    \mbox{subject to} & Cx = d
  \end{array}
  \label{eq:std_form}
\end{equation}
where $x \in \mathbb{R}^{n}$ is a variable, $A \in \mathbb{R}^{m \times n}$ and
$C \in \mathbb{R}^{p \times n}$ are matrices, and $b \in \mathbb{R}^m$ and
$d \in \mathbb{R}^p$ are vectors.

These problems are routinely solved in the context of statistical estimation,
machine learning, and engineering design.
For example, linear regression, an approach for modeling the relationship
between a scalar dependent variable $y$ and one or more explanatory variables
$X$, involves solving the problem minimize $\norm{X \beta - y}^2$ to find
$\beta$, where $X$ is a given data matrix, $y$ is a given vector, and $\beta$ is
the \emph{parameter vector}.

In general, there can be many variables, the objective can contain many
quadratic terms, and there can be multiple linear constraints.
Typically one has to rewrite the problem in the standard
form~\eqref{eq:std_form} and then use a general solver implemented by many
numerical linear algebra packages.
However, this is a tedious and error-prone process, especially for problems with
many variables or several linear constraints.

We aim to develop a modeling language embedded in Haskell to solve these
linearly constraints least squares problems.
This modeling language (in Haskell) will allow user to specify the problem in a
natural way that mirrors standard mathematical notation without being
constrained by the standard form.

\section{Background}

\subsection{Solution to the least squares problem}

The (unconstrained) \emph{least squares problem} has the standard form
\begin{equation*}
  \begin{array}{ll}
    \mbox{minimize} & \|Ax-b\|^2,
  \end{array}
\end{equation*}
where $x \in \mathbb{R}^n$ is a variable, $A \in \mathbb{R}^{m \times n}$ is a
matrix, and $b \in \mathbb{R}^m$ is a vector.
Assuming that $A$ is full rank, the least squares problem has the close-form
solution
\begin{equation*}
  \hat x = (A^T A)^{-1} A^T b.
\end{equation*}

A slight generalization is the (linearly) \emph{constrained least squares problem}:
\begin{equation*}
  \begin{array}{ll}
    \mbox{minimize} & \|Ax-b\|^2 \\
    \mbox{subject to} & Cx = d
  \end{array}
\end{equation*}
where $x \in \mathbb{R}^{n}$ is a variable, $A \in \mathbb{R}^{m \times n}$ and
$C \in \mathbb{R}^{p \times n}$ are matrices, and $b \in \mathbb{R}^m$ and
$d \in \mathbb{R}^p$ are vectors. The solution of this constrained problem is
\begin{equation*}
  \left[\begin{array}{c} \hat x \\ z \end{array}\right] =
  \left[\begin{array}{cc} 2A^TA & C^T \\ C & 0 \end{array}\right]^{-1}
  \left[\begin{array}{c} 2A^Tb \\ d \end{array}\right],
\end{equation*}
where $z$ is the dual variable to the equality constraint.
We assume that the matrix above is invertible.
This occurs when the matrix $C$ has independent rows, and the matrix
$\begin{bmatrix} A \\ C \end{bmatrix}$ has independent columns.

\subsection{Numerical linear algebra in Haskell}

We use the package \verb|hmatrix|~\cite{ruiz2015hmatrix-0.17.0.1} to handle
vectors and matrices.
This is a purely functional interface to linear algebra and other numerical
algorithms, internally implemented using LAPACK, BLAS, and GSL.

The types provided (\verb|Matrix| and \verb|Vector|) are dense, immutable and
strict in all elements (unboxed).
We will only use the double precision real versions of these types, i.e.
\verb|Matrix R| and \verb|Vector R| (\verb|R=Double|).
The matrix product is \verb|(<>)|, the matrix-vector product is \verb|(#>)|, the
dot product is \verb|(<.>)|, and the (conjugate) transpose is \verb|tr|.

\section{Modeling}

The key data type is a linear expression \verb|LinExpr|, which contains
variables and constants that are combined using linear mathematical functions
(addition, scaling, matrix-vector multiplication, etc.).
A linear constraint (type \verb|LinContr|) can be form as
\verb|LinExpr :==: LinExpr|.
A quadratic expression \verb|QuadExpr| can be formed by taking square norm
(\verb|SumSquares|) of linear expressions, or sum of many such square norms.

The package provides a function \verb|minimize| to solve a least squares problem:
\begin{minted}{haskell}
minimize :: QuadExpr -> [LinContr] -> M.Map VarName (Vector R)
\end{minted}
The function takes a quadratic expression as objective, a list of linear
constraints, and returns a map from variable names (type alias for
\verb|String|) to variable values.

For example, the classic problem of find the least norm solution to an
under-determined system can be easily setup and solved with the following code:
\inputminted{haskell}{app/example_least_norm.hs}

\section{Implementation}

Given a problem with a quadratic expression as objective and a list of linear
constraints, the system will analyze the abstract syntax tree to transform it to
the standard form in~\eqref{eq:std_form}, and then solve it using linear
algebra subroutines.

The three steps of solving a least squares problem is given in Figure~\ref{fig:steps}.
\begin{figure}[!ht]
  \centering
  \begin{tikzpicture}[auto]
      % Place nodes
      \node (left_up) {};
      % \node [below=1cm of left_up] (left_down) {};
      \node [block,right=4mm of left_up] (code) {$\begin{array}{l}
      \tt{u = \ldots} \\
      \tt{v = \ldots} \\
      \tt{minimize \ldots}
      \end{array}$};
      \node [block, right =2.75cm of code] (cone) {$\begin{array}{ll}
  \mbox{min.}  &\|Ax-b\|^2 \\
  \mbox{s.t.} &Cx = d
  \end{array}$};
      \node [block, below =2cm of cone] (sltn) {$\begin{array}{c}
  x = (1.58,\ldots \\
  \vdots
  \end{array}$};
      \node [block, left =2.75cm of sltn] (unpack) {$\begin{array}{l}
      \tt{u = (0.59,\ldots}\\
      \tt{v = (1.9,\ldots}
      \end{array}$};
  \path [line] (code) -- node {canonicalize} ++(4.5,0) -- (cone);
  \path [line] (cone) -- node {solve} ++(0,-3.0) -- (sltn);
  \path [line] (sltn) -- node {unpack} ++(-4.5,0) -- (unpack);
  \end{tikzpicture}
  \caption{Three steps of solving a least squares problem.}
  \label{fig:steps}
\end{figure}

In the canonicalization step, we stack all the variables in the problem as one
big variable, and then stack the linear constraints.
For example, given constraints $Cu + Dv = w$ and $Eu = z$, we produce the equivalent
constraint
\begin{equation*}
  \begin{bmatrix} C & D \\ E & 0 \end{bmatrix} \begin{bmatrix} u \\
    v \end{bmatrix} = \begin{bmatrix} w \\ z \end{bmatrix}.
\end{equation*}
Since the objective is a sum of square norms, we convert this sum to a single
equivalent square norm.
For example, if the objective is $\norm{Au - b}^2 + 2\norm{v}^2$, the equivalent
square norm is
\begin{equation*}
  \norm{\begin{bmatrix} A & 0 \\ 0 & \sqrt{2}I \end{bmatrix} \begin{bmatrix} u
      \\ v \end{bmatrix} - \begin{bmatrix} b \\ 0 \end{bmatrix} }^2,
\end{equation*}
where $I$ is the identity matrix.
We then form a new variable $x = \begin{bmatrix} u \\ v \end{bmatrix}$ and thus
both the objective and the constraints are transformed to standard form.

In the solve step, we simply call the least squares solver provided by
\verb|hmatrix|. The unpacking step takes the resulting vector and assign the
values to the original variables.

\section{Example}

We consider an example of least squares modeling in image de-blurring. Suppose
that $x$ is an image, $A$ is a blurring operator, and $y = Ax + v$ is a blurred,
noisy image that we are given. We choose $x$ to minimize
\begin{equation*}
 \|Ax -  y\|^2 + \lambda (\|D_\mathrm v x\|^2 + \|D_\mathrm h x\|^2),
\end{equation*}
where $D_\mathrm v$, $D_\mathrm h$ are vertical and horizontal differencing
operations and the scalar parameter $\lambda$ controls smoothing of de-blurred
image. The problem can be expressed in this natural form:
\begin{minted}{haskell}
minimize (SumSquares (a*x-b) + lambda * (SumSquares (dv*x) + SumSquares (dh*x))) []
\end{minted}

We show the burred, noisy image and the recovered image with $\lambda = 0.007$
in Figure~\ref{fig:images}.
\begin{figure}[!ht]
  \centering
  \begin{subfigure}{0.49\textwidth}
    \centering
    \includegraphics[width=0.8\textwidth]{figures/boat_blurred}
    \caption{Blurred, noisy image}
  \end{subfigure}
  \begin{subfigure}{0.49\textwidth}
    \centering
    \includegraphics[width=0.8\textwidth]{figures/boat_deblurred}
    \caption{Recovered image}
  \end{subfigure}
  \caption{Blurred, noisy image and regularized inversion with $\lambda = 0.007$.}
  \label{fig:images}
\end{figure}

\section{Conclusion}

We have designed and implemented a modeling language embedded in Haskell for
linearly constrained least squares problem.
There are still some work to be done to add more linear operators (vector
indexing, convolution, Fourier transform, vector and matrix stacking, element
wise multiplication, etc.).
Moreover, the package only supports dense matrices due to lack of sparse matrix
support in Haskell.

Overall, this modeling language will allow users to express their problems in a
natural form while the system takes care of transforming it to the standard
form.
This might lead to faster prototyping and data analysis in statistical
estimation, machine learning, and engineering design.

\section*{Acknowledgment}

We thank Professor Stephen Boyd for introducing us to many examples of least
squares modeling in control, statistical estimation, and engineering design.
We also thank David Zeng whose Julia package \verb|LinearLeastSquares|
\cite{zeng2014linearleastsquares} inspired this project.

\bibliography{report}

\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\subsubsection{Broadcast time}\label{subsubsec:hd2krtime}

Since in all the experiments we have always reached a coverage of at least
\(99\%\), here we will only discuss the broadcast time needed to reach the 99th
percentile of the coverage.

We have a large unexplained variation (\(7.95\%\)). The reason for this result
will be discussed in \chref{ch:starting-node}.

We can see that the most important factor is the broadcast radius that accounts
for the \(71.25\%\) of the variation, followed by the size of the hear window
(\(19.30\%\)). Other factors and their combinations are irrelevant. We note that
these variations are much larger than the unexplained variation, so we can still
say that they are significant. Of course, their \(95\%\) confidence intervals
also gets larger: (\(63.62\%\), \(79.30\%\)) for \(R\) and (\(15.43\%\),
\(23.59\%\)) for \(T\), but they do not include the zero.

This means that we can reduce the broadcast time by increasing the broadcast
radius to let the relayed messages to reach more user. Also the size of the hear
window can be decreased in order to reduce the time that each user wait before
deciding to relay or not relay the message. Of course, as stated before,
reducing the size of the hear window has a negative impact on the energy
efficiency, so some trade-off considerations are required.

We note that we have performed a \emph{logarithmic transformation of the
predicted variable} \idest{the total broadcast time} in order to meet the
assumption of finite variance for the residuals, as discussed in
\secref{subsec:hdassumptions}. This means that the final approximated regression
model for the broadcast time is transformed into the one shown
in~\eqref{eq:hdtimelogregressionmodel}, with \(R\) and \(T\) normalized between
\(-1\) and \(1\). (factors with low influence are removed; 95\% confidence
intervals are show in parenthesis).

\begin{equation}\label{eq:hdtimelogregressionmodel}
	\begin{cases}
		\log(t_B) = q_0 + q_R \cdot R + q_T \cdot T + e\\
		q_0 = 4.782874\\
		q_R = -0.462452 & (-0.487897, -0.437006)\\
		q_T = 0.240663 & (0.215217, 0.266109)
	\end{cases}
\end{equation}

We can then predict the total broadcast time using the formula shown
in~\eqref{eq:hdtimeregressionmodel}.

\begin{equation}\label{eq:hdtimeregressionmodel}
	t_B = e^{4.782874 - 0.462452 \cdot R + 0.240663 \cdot T}
\end{equation}

And the 95\% confidence interval as shown in~\eqref{eq:hdtimeregressionci}.

\begin{equation}\label{eq:hdtimeregressionci}
	\begin{cases}
		t^-_B = e^{4.782874 - 0.487897 \cdot R + 0.215217 \cdot T}\\
		t^+_B = e^{4.782874 - 0.437006 \cdot R + 0.266109 \cdot T}\\
	\end{cases}
\end{equation}


\documentclass{article}
\usepackage[utf8]{inputenc}

\usepackage{amsmath,amsfonts,amssymb,amsthm,epsfig,epstopdf,titling,url,array, tikz,tkz-berge, calrsfs}
\usepackage{tkz-graph}

\title{Matroids And their Graphs}
\author{o.mcdonnell4@nuigalway.ie }
\date{19 January 2018}

\theoremstyle{plain}
\newtheorem{thm}{Theorem}[section]
\newtheorem{lem}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}
\newtheorem*{cor}{Corollary}

\theoremstyle{definition}
\newtheorem{defn}{Definition}[section]
\newtheorem{conj}{Conjecture}[section]
\newtheorem{exmp}{Example}[section]

\theoremstyle{remark}
\newtheorem*{rem}{Remark}
\newtheorem*{note}{Note}

\newenvironment{rcases}
  {\left.\begin{aligned}}
  {\end{aligned}\right\rbrace}

\newcounter{excercise}
\newcounter{solution}
\newcounter{Question}

\newcommand\Excercise{%
  \textbf{Excercise:}~%
  \setcounter{solution}{0}%
}

\newcommand\TheSolution{%
  \textbf{Solution:}\\%
}

\newcommand\Question{%
    \textbf{Question:}~%
    \setcounter{Question}{0}%
}
\newcommand\Notation{%
  \textbf{Notation:}~%
}

\newcommand\Proof{%
    \textbf{Proof:}~%
}

\setlength{\droptitle}{-10em}

\begin{document}
\maketitle
 \section{Circuits in a graph}

 \begin{thm}
 Let $E$ be the edge sets of a graph $G$ and let $\mathcal{C}$ be the edge sets of cycles in $G$.\\
 \noindent Then $\mathcal{C}$ is the set of circuits of a matroid.

 \end{thm}

\noindent\textbf\Proof
Let $A, B \in \mathcal{C}, A \neq B $ and let $e \in A \cap B $

\vspace{1mm}

\noindent We must now construct a cycle of $G$ whose edge set is contained in $(A \cup B) \setminus \{e\}$

\vspace{2mm}

\noindent For $ i = 1,2,3,...... $ let $P_1$ be a path whose edge set is $ A \setminus \{e\}$\\
\noindent $ A \setminus \{e\} \in \mathcal{I} $ therefore $P_i$ is not a cycle of $G$. This path will traverse from the edge $a_j$ to $a_u$ where $u,j$ were the vertices connecting the edge e to $(A \cup B) \setminus \{e\}$ to make $A \cup B$.

\vspace{2mm}

\noindent Now perform the same procedure for a path $P_2$ whose edge set is $B \setminus \{e\}$.

\vspace{2mm}

\noindent $P_1$ and $P_2$ should meet at the junctions $u,v$,  where $e$ was removed to make $ (A \cup B) \setminus \{e\}$\\
\noindent Therefore $P_1 \cup P_2$ should be a cycle of $G$.\\
\noindent $\implies$ (C3) holds\\
\noindent $\implies \mathcal{C}$ is the edge sets of cycle in $G$.\\
\qed
\end{document}


%!TEX root = /home/renaud/Documents/EPL/tfe/latex/tfe.tex
\section{Lagrangian equations corresponding to the advection-diffusion transport model}
The previous section introduces all the theoretical tools needed to compute the Lagrangian equations corresponding to the transport model, namely the general advection-diffusion equation
\begin{equation} \label{eq:TM}
	\frac{\partial C}{\partial t} = \nabla \cdot (-\b u C + \b K \nabla C),
\end{equation}
where $\b K$ is the symmetric and positive-definite diffusivity tensor.
From now on, we restrict ourselves to two dimensions in the cartesian coordinate system $(y,z)$.
The transport equation~\eqref{eq:TM} can be interpreted as a Fokker-Planck equation where $C$ is the probability density function of the position $\b x(t) = (y(t),\,z(t))$ of the particle. Let
\begin{equation}
	\b K = \begin{pmatrix} K_{yy} & K_{yz}\\ K_{zy} & K_{zz} \end{pmatrix},
\end{equation}
with $K_{yz} = K_{zy}$ since $\b K$ is symmetric. In order to get the Itô and backward-Itô systems of SDEs corresponding to the transport model, we must rewrite~\eqref{eq:TM} in the forms~\eqref{eq:FP-I-ndim} and~\eqref{eq:FP-bI-ndim} respectively. The systems of SDEs can then be deduced straightforwardly by analogy.

Let us first compute the Itô SDEs corresponding to~\eqref{eq:TM}. Equation~\eqref{eq:TM} can be rewritten as
\begin{multline}
	\frac{\partial C}{\partial t} = -\frac{\partial}{\partial y}\left[\left(v+\frac{\partial K_{yy}}{\partial y} + \frac{\partial K_{yz}}{\partial z}\right)C\right] -\frac{\partial}{\partial z}\left[\left(w+\frac{\partial K_{zy}}{\partial y} + \frac{\partial K_{zz}}{\partial z}\right)C\right]\\
	+ \frac{1}{2}\left[\frac{\partial^2}{\partial y^2} \left(2K_{yy} C\right) + \frac{\partial^2}{\partial y \partial z} \left(2K_{yz} C\right) + \frac{\partial^2}{\partial z \partial y} \left(2K_{zy} C\right) + \frac{\partial^2}{\partial z^2} \left(2K_{zz} C\right) \right].
\end{multline}
\vspace*{.1cm}
% \begin{equation}
% 	\frac{\partial C}{\partial t} = -\frac{\partial}{\partial y}\left[\left(v+\frac{\partial K_{h}}{\partial y}\right)C\right] -\frac{\partial}{\partial z}\left[\left(w+ \frac{\partial K_{v}}{\partial z}\right)C\right]	+ \frac{1}{2}\left[\frac{\partial^2}{\partial y^2} \left(2K_{h} C\right) + \frac{\partial^2}{\partial z^2} \left(2K_v C\right) \right].
% \end{equation}
This is precisely equation~\eqref{eq:FP-I-ndim} with $\b x =(y,z)$, $p=C$, $\b a = (v+\partial_y K_{yy} + \partial_z K_{yz},\, w+\partial_y K_{zy} + \partial_z K_{zz})$ and $\b D = 2\b K$. Using those notations, $\b x(t) = (y(t),\, z(t))$ obeys thus the Itô SDE
% This is precisely equation~\eqref{eq:FP-I-ndim} with $\b x =(y,z)$, $p=C$, $\b a = (v+\partial_y K_{h},\, w+\partial_z K_{v})$ and $\b D = 2\b K$. Therefore, $\b x(t) = (y(t),\, z(t))$ obeys the Itô SDE
\begin{equation} \label{eq:ito-TM-vec}
	\rm d \b x(t) = \b a(x(t),t) \rm dt + \b B(x(t),t) \rm d \b W(t),
\end{equation}
where $\b B$ has to be solved from $2\b K = \b B \b B^{\t}$. Since $2\b K$ is positive semidefinite, a possible Cholesky decomposition is given by
\begin{equation} \label{eq:B}
	\b B = \begin{pmatrix} B_{yy} & 0 \\ B_{zy} & B_{zz} \end{pmatrix} = \begin{pmatrix} \sqrt{2K_{yy}} & 0 \\ B_* & \sqrt{2K_{zz}-B_*^2} \end{pmatrix},
\end{equation}
where
\begin{equation} \label{eq:Bstar}
	B_* = \left\{
		\begin{array}{lr}
			0 & \mbox{if } K_{yy} = 0,\\
			\frac{2K_{yz}}{B_{yy}} & \mbox{otherwise}.
		\end{array}
	\right.
\end{equation}
The Itô SDE~\eqref{eq:ito-TM-vec} can be rewritten as
\begin{subnumcases}{\I\ \label{eq:ito-TM}}
	\rm dy(t) = \left(v + \frac{\partial K_{yy}}{\partial y} + \frac{\partial K_{yz}}{\partial z} \right) \rm dt + B_{yy} \rm dW_1(t)\\
	\rm dz(t) = \left(w + \frac{\partial K_{zy}}{\partial y} + \frac{\partial K_{zz}}{\partial z} \right) \rm dt + B_{zy} \rm dW_1(t) + B_{zz} \rm dW_2(t)\\
	(y(0),z(0)) = (y_0,z_0),
\end{subnumcases}
where $W_1(t)$ and $W_2(t)$ are independent Wiener processes. The derivatives of the elements of $\b K$ appearing in equation~\eqref{eq:ito-TM} are called the \textit{gradient drift terms}. If $\b K$ is known analytically, those terms can be computed for the numerical implementation. If not, finite differences can be used. In the next of this work, we will encounter problems where $\b K$ is discontinuous along segments in the domain, so that a part of the gradient drift terms is infinite at those points.
Such an issue is addressed in \cite{prickett1981random} by neglecting the gradient drift terms all together, and in \cite{tompson1987numerical} by evaluating gradient drift terms via finite differences. Such methods are probably good enough for the simple problems considered in this work.
% \footnote{This is especially true since the discontinuity of the diffusivity is in fact an idealization in the problems we will encounter. An estimation of the gradient drift terms via finite differences near the discontinuities would thus be more realistic than the infinite value.}
However, we prefer the \textit{backward-Itô} approach as this method applies to a wider range of problems with discontinuous diffusivities.

% Specific to overturner :
% In our model, $K_h$ is constant so that $\partial_y K_h = 0$ uniformly on $\Omega$. The term gradient drift term $\partial_z K_v$ is more problematic: $K_v$ is indeed discontinuous and $\partial_z K_v$ is infinite on the segment defined by $(\lambda y_0, z_0)$ with $\lambda \in [0,1]$.

To derive the backward Itô SDE corresponding to the transport model, notice that equation~\eqref{eq:TM} can also be rewritten as
\begin{multline}\label{eq:TM-bito}
	\frac{\partial C}{\partial t} = -\frac{\partial}{\partial y}(vC) -\frac{\partial}{\partial z}(wC) \\+ \frac{1}{2}\left[\frac{\partial}{\partial y} \left(2K_{yy}\frac{\partial C}{\partial y} + 2K_{yz}\frac{\partial C}{\partial z} \right) + \frac{\partial}{\partial z} \left(2K_{zy}\frac{\partial C}{\partial y} + 2K_{zz}\frac{\partial C}{\partial z}\right)\right].
\end{multline}
This is precisely equation~\eqref{eq:FP-bI-ndim} with $\b x = (y,z)$, $p=C$, $\b a = (v,w)$ and $\b D = 2\b K$. Hence, we can use the matrix $\b B$ computed in~\eqref{eq:B}. Then, $\b x(t) = (y(t),z(t))$ also obeys the backward Itô SDE
\begin{subnumcases}{\bI\ \label{eq:bi-TM}}
	\rm dy(t) = v \rm dt + B_{yy}\rm dW_1(t)\\
	\rm dz(t) = w \rm dt + B_{zy}\rm dW_1(t) + B_{zz}\rm dW_2(t)\\
	(y(0),z(0)) = (y_0,z_0).
\end{subnumcases}
Interestingly, there is no gradient drift term in~\eqref{eq:bi-TM}, i.e. no derivative of the diffusivities. In the context of a problem with discontinuous diffusivities, the backward Itô interpretation is thus particularly interesting: see \cite{labolle2000diffusion} and \cite{spivakovskaya2007backward} for more complete discussions about the use of the backward Euler method on problems with discontinuous diffusivities.


\documentclass[a4paper, 12pt]{article}

\usepackage{underbrace}

\usepackage{graphicx}

\usepackage[left=1cm, right=1cm, top=2cm, bottom=2cm]{geometry}

\usepackage{animate}

\usepackage[table]{xcolor}

\usepackage [russian]{babel}
\usepackage{amssymb,amsmath,latexsym,amsfonts,amsthm}

\renewcommand{\arraystretch}{1.1}

\usepackage{array}
\newcolumntype{P}[1]{>{\centering\arraybackslash}p{#1}}
\newcolumntype{M}[1]{>{\centering\arraybackslash}m{#1}}

\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=cyan,
}

\begin{document}

\begin{center}
\Huge{\textbf{Mathematical model of self-focusing}}
\end{center}

We obtain the self-focusing equation in the approximation of a slowly varying amplitude to study the propagation of beams including multimedia -- annular beams with phase singularity on the optical axis.

\section{Obtaining a self-focusing equation}

The numerical simulation of optical vortex propagation is based on nonlinear wave equation, which could be obtained starting with Maxwell's system of equations:
\begin{equation}
\label{eqn:maxwell}
\left\{
\begin{aligned}
\nabla \times \mathbf{E}(\mathbf{r}, t) &= -\frac{\partial \mathbf{B}(\mathbf{r}, t)}{\partial t},\\
\nabla \times \mathbf{H}(\mathbf{r}, t) &= \mathbf{j}(\mathbf{r}, t) + \frac{\partial \mathbf{D}(\mathbf{r}, t)}{\partial t},\\[0.2cm]
\nabla \cdot \mathbf{D}(\mathbf{r}, t) &= \rho_f (\mathbf{r}, t) + \rho_b (\mathbf{r}, t), \\[0.3cm]
\nabla \cdot \mathbf{B}(\mathbf{r}, t) &= 0.
\end{aligned}
\right.
\end{equation}
where $\mathbf{E}(\mathbf{r}, t)$ и $\mathbf{D}(\mathbf{r}, t)$ -- electric field and induction, $\mathbf{H}(\mathbf{r}, t)$ и $\mathbf{B}(\mathbf{r}, t)$ -- magnetic field and induction, $\mathbf{\rho_f}(\mathbf{r}, t)$ и $\mathbf{\rho_b}(\mathbf{r}, t)$ -- density of free and bound electric charges. $\mathbf{j}(\mathbf{r}, t)$ -- conduction current density. The relationship of electric field and induction is given in the form:
\begin{equation}
\label{eqn:D}
\mathbf{D}(\mathbf{r}, t) = \varepsilon_0 \mathbf{E}(\mathbf{r}, t) + \mathbf{P}(\mathbf{r}, t)
\end{equation}
where $\mathbf{P}(\mathbf{r}, t)$ -- polarization, $c = \sqrt{1 / \varepsilon_0 \mu_0}$ -- light speed, $\varepsilon_0$ и $\mu_0$ -- dielectric and magnetic constants.

Considering medium is dielectric ($\mathbf{\rho_f}(\mathbf{r}, t) = 0$) and nonmagnetic ($\mathbf{B}(\mathbf{r}, t) = \mu_0 \mathbf{H}(\mathbf{r}, t)$)
we get:
\begin{equation}
\label{eqn:prototype}
\Delta \mathbf{E}(\mathbf{r}, t) - \nabla (\nabla \cdot \mathbf{E}(\mathbf{r}, t)) =  \frac1{\varepsilon_0 c^2} \frac{\partial \mathbf{j}(\mathbf{r}, t)}{\partial t} + \frac1{c^2} \frac{\partial^2 \mathbf{E}(\mathbf{r}, t)}{\partial t^2} + \frac1{\varepsilon_0 c^2} \frac{\partial^2 \mathbf{P}(\mathbf{r}, t)}{\partial t^2},
\end{equation}
where $\mathbf{E}$ is electric field, $\mathbf{j}$ is current density and $\mathbf{P}$ is polarization, $c=1/\sqrt{\varepsilon_0 \mu_0}$. We examine each of equation \eqref{eqn:prototype} members one by one using several further approximations.

Assuming paraxiality, that is transverse wave number is much smaller than longitudinal one, $k_\perp \ll k_z$, and linear polarization of the electric field $\mathbf{E}$, we can use method of slowly varying amplitude $\mathbf{E}$:
\begin{equation}
\mathbf{E}(\mathbf{r}, t) = \frac1{2}\mathbf{e} A(\mathbf{r}, t) \exp\{i (\omega_0 t - k_0 z)\} + \text{c.c.}
\end{equation}
where $A(\mathbf{r}, t)$ is a complex slowly varying amplitude, $\mathbf{e}$ is a unit vector.
Using dipole approximation ($\chi^{(i)}(\mathbf{r},t) = \chi^{(i)}(t)$) we can write ($\otimes$ means convolution) polarization as:
\begin{multline}
\label{eqn:polarizations}
\mathbf{P}(\mathbf{r}, t) = \mathbf{P}^{(1)}(\mathbf{r}, t) + \mathbf{P}^{(2)}(\mathbf{r}, t) + \mathbf{P}^{(3)}(\mathbf{r}, t) + ... = \varepsilon_0 \chi^{(1)}(t) \otimes \mathbf{E}(\mathbf{r}, t) +\\+ \varepsilon_0  \chi^{(2)}(t) \otimes \mathbf{E}(\mathbf{r}, t) \mathbf{E}(\mathbf{r}, t) + \varepsilon_0  \chi^{(3)}(t) \otimes \mathbf{E}(\mathbf{r}, t) \mathbf{E}(\mathbf{r}, t) \mathbf{E}(\mathbf{r}, t) + ...
\end{multline}
where $\chi^{(i)}(t)$ -- permittivity tensor of $i$-th order. We consider isotropic medium ($\mathbf{P}^{(2 m)}(\mathbf{r}, t)=0, \ m \in \mathbb{N}$). Each $(n+2)$-th term of the polarization series \eqref{eqn:polarizations} refers to $n$-th one as $\sim \bigl|I(\mathbf{r}, t) \bigr| \bigl/ \bigl|I_a \bigr|$, where $I_a \sim 5 \times 10^{16}$ W/cm$^2$ is atomic intensity. Peak intensity inside the filament is about $5 \times 10^{13}$ W/cm$^2$, so the ratio is of the order of $\sim 10^{-3}$. We can neglect nonlinearities of higher orders and write:
\begin{equation}
\mathbf{P}(\mathbf{r}, t) = \mathbf{P}^{(1)}(\mathbf{r}, t) + \mathbf{P}^{(3)}(\mathbf{r}, t).
\end{equation}
Therefore we will limit our consideration only to linear and cubic polarizations. Linear polarization can be written as
\begin{multline}
\mathbf{P}^{(1)}(\mathbf{r}, t) = \varepsilon_0 \chi^{(1)}(t) \otimes \mathbf{E}(\mathbf{r}, t) =
\varepsilon_0 \int\limits_{-\infty}^{+\infty} \chi^{(1)}(\tau) \mathbf{E}(\mathbf{r}, t - \tau) d\tau =\\= \frac1{2}\mathbf{e} \exp\{i (\omega_0 t  - k_0 z)\} \varepsilon_0 \int\limits_{-\infty}^{+\infty} \chi^{(1)}(\tau)  A(\mathbf{r}, t - \tau) \exp\{-i \omega_0 \tau \} d\tau.
\end{multline}
Using inverse Fourier transform of slowly varying electric field amplitude
\begin{equation}
A(\mathbf{r}, t) = \frac1{2 \pi} \int\limits_{-\infty}^{+\infty} \tilde{A}(\mathbf{r}, \Omega) \exp \{ i \Omega t \} d \Omega,
\end{equation}
forward Fourier transform of electric susceptibility
\begin{equation}
\label{eqn:susceptibility_fourier}
\tilde{\chi}^{(1)}(\omega) = \int\limits_{-\infty}^{+\infty} \chi^{(1)}(t) \exp \{ -i \omega t \} dt
\end{equation}
and relation between susceptibility and wave vector $\tilde{\chi}^{(1)}(\omega) = k^2(\omega) c^2 / \omega^2 - 1$, we obtain
\begin{equation}
\mathbf{P}^{(1)}(\mathbf{r}, t) = \frac1{2}\mathbf{e} \exp\{i (\omega_0 t  - k_0 z)\} \varepsilon_0 c^2 \frac1{2 \pi} \int\limits_{-\infty}^{+\infty} \frac{k^2(\omega_0 + \Omega)}{\omega_0^2 + 2 \omega_0 \Omega + \Omega^2} \tilde{A}(\mathbf{r}, \Omega) \exp \{ i \Omega t \} d \Omega - \varepsilon_0 \mathbf{E}(\mathbf{r}, t),
\end{equation}
where $k(\omega) = \omega n(\omega)/ c$, $n(\omega)$ -- index of refraction. The linear part of the last term in equation \eqref{eqn:prototype} takes the form:
\begin{equation}
\label{eqn:p1_derivative}
\frac1{\varepsilon_0 c^2} \frac{\partial^2 \mathbf{P}^{(1)}(\mathbf{r}, t)}{\partial t^2} = -\frac1{2}\mathbf{e} \exp\{i (\omega_0 t  - k_0 z)\}   \frac1{2 \pi} \int\limits_{-\infty}^{+\infty} k^2(\omega_0 + \Omega) \tilde{A}(\mathbf{r}, \Omega) \exp \{ i \Omega t \} d \Omega - \frac1{c^2} \frac{\partial^2 \mathbf{E}(\mathbf{r}, t)}{\partial t^2}.
\end{equation}
The expression for cubic polarization consists of two terms related to the first and third harmonics:
\[
\mathbf{P}^{(3)}(\mathbf{r}, t) = \varepsilon_0  \chi^{(3)}(t) \otimes \mathbf{E}(\mathbf{r}, t) \mathbf{E}(\mathbf{r}, t) \mathbf{E}(\mathbf{r}, t) =
\]
\[= \frac1{8} \mathbf{e} \exp \{ i (3 \omega_0 t - 3 k_0 z) \}\varepsilon_0  \iiint \chi^{(3)}(\tau_1, \tau_2, \tau_3) A(\mathbf{r}, t - \tau_1)  A(\mathbf{r}, t - \tau_2) A(\mathbf{r}, t - \tau_3) \times
\]
\[
 \times \exp\{-i \omega_0 (\tau_1 + \tau_2 + \tau_3) \} d\tau_1 d\tau_2 d \tau_3 +
\]
\[
+ \frac{3}{8} \mathbf{e} \exp \{ i (\omega_0 t - k_0 z) \} \varepsilon_0  \iiint \chi^{(3)}(\tau_1, \tau_2, \tau_3) A(\mathbf{r}, t - \tau_1)  A(\mathbf{r}, t - \tau_2)  A^*(\mathbf{r}, t - \tau_3) \times
\]
\begin{equation}
\times \exp\{-i \omega_0 (\tau_1 + \tau_2 - \tau_3) \} d\tau_1 d\tau_2 d \tau_3.
\end{equation}
Due to strong violation of wave synchronism, $\Delta k = 3 k (\omega_0) - k(3 \omega_0) \ne 0$, we neglect the third harmonic generation. Fourier transform of 3-th order electric susceptibility (similarly to \eqref{eqn:susceptibility_fourier}) and well-known relation between intensity and electric field amplitude  $I(\mathbf{r}, t) = c n_0 \varepsilon_0 |A(\mathbf{r}, t)|^2 \bigl/ 2$ yield
\begin{equation}
\mathbf{P}^{(3)}(\mathbf{r}, t) = \frac{3}{4} \mathbf{e} \exp \{ i (\omega_0 t - k_0 z) \} \varepsilon_0 \frac{\tilde{\chi}^{(3)}}{c n_0 \varepsilon_0} I(\mathbf{r}, t) A(\mathbf{r}, t).
\end{equation}
The second time derivative of nonlinear polarization in \eqref{eqn:prototype} takes the form
\begin{equation}
\label{eqn:p3_derivative}
\frac1{\varepsilon_0 c^2} \frac{\partial^2 \mathbf{P}^{(3)}(\mathbf{r}, t)}{\partial t^2} = -\frac1{2} \mathbf{e} \exp \{ i (\omega_0 t - k_0 z) \} \frac{2 k_0^2}{n_0} \hat{T}^2 \Delta n_k(\mathbf{r}, t) A(\mathbf{r}, t),
\end{equation}
where
\begin{equation}
\hat{T} = 1 - \frac{i}{\omega_0} \frac{\partial}{\partial t}
\end{equation}
is the operator of wave-nonstationarity, which provides more accurate description of short pulse propagation compared to commonly used approximation of this operator by unit.

In case of instant response the nonlinear (Kerr) addition to the refractive index $\Delta n_k(\mathbf{r}, t) = n_2 I(\mathbf{r}, t)$, where $n_2 = 3 \tilde{\chi}^{(3)} \bigl/ 4 c n_0^2 \varepsilon_0$.

We will consider only self-focusing process without taking into account the ionization in the medium by a strong laser field, so $\mathbf{j}(\mathbf{r}, t)=0$

Substituting the expressions \eqref{eqn:p1_derivative}, \eqref{eqn:p3_derivative} to \eqref{eqn:prototype}, assuming $\nabla \cdot \mathbf{E}(\mathbf{r}, t) = 0$ and expanding the Laplace operator to transverse and longitudinal parts $\Delta = \Delta_\perp + \partial^2/\partial z^2$ we obtain nonlinear wave equation, which can be written in retarded time $t'=t-k_1 z$ and $z'=z$ coordinates as
\begin{multline}
\label{eqn:equation_without_dissipation}
\frac{\partial^2 A (\mathbf{r}, t', z')}{\partial z'^2} - 2 i k_0  \biggl( 1 - \frac{i k_1}{k_0} \frac{\partial }{\partial t'}  \biggr) \frac{\partial A (\mathbf{r}, t', z')}{\partial z'} + \Delta_{\perp} A (\mathbf{r}, t', z')= \\ =-\frac1{2 \pi} \int\limits_{-\infty}^{+\infty} \Bigl( k^2(\omega_0 + \Omega) - (k_0 + k_1 \Omega)^2 \Bigr) \tilde{A}(\mathbf{r}, \Omega, z') \exp \{ i \Omega t' \} d \Omega - \frac{2 k_0^2}{n_0} \hat{T}^2 \Delta n_k(\mathbf{r}, t') A(\mathbf{r}, t', z'),
\end{multline}
where $k_1 = dk / d\omega |_{\omega=\omega_0}$. We neglect the second derivative of the field $A$ on $z'$ due to slowly varying amplitude approximation, assume the factor $k_1/k_0$ in brackets approximately equals to $1/\omega_0$ and redesignate $t'$, $z'$ back to $t$, $z$.

Since the original problem is limited to the consideration of spatial effects, it will be assumed that the pulses are long enough and will not take into account temporal effects, therefore, we equate the dispersion integral to zero and remove the wave-nonstationarity operator $\hat{T}$. Then we get:
\begin{equation}
\label{eqn:approx_final}
\frac{\partial^2 A (\mathbf{r}, z)}{\partial z^2} - 2 i k_0 \frac{\partial A (\mathbf{r}, z)}{\partial z} + \Delta_{\perp} A (\mathbf{r}, z)= - \frac{2 k_0^2}{n_0} n_2 I(\mathbf{r}, z) A(\mathbf{r}, z).
\end{equation}
We use the quasi-optical approximation assuming $| \partial^2 A(\mathbf{r}, z) / \partial z^2| \ll k_0 |\partial A(\mathbf{r},z) / \partial z|$ and get the final equation of 3D-beam self-focusing  propagating in cubic nonlinear media in approximation of slowly varying field amplitudes:
\begin{equation}
\label{eqn:final}
2 i k_0 \frac{\partial A (\mathbf{r}, z)}{\partial z} = \Delta_{\perp} A (\mathbf{r}, z) + \frac{2 k_0^2}{n_0} n_2 I(\mathbf{r}, z) A(\mathbf{r}, z),
\end{equation}
where $\mathbf{r}$ -- transversal radius-vector, $z$ -- evolutionary coordinate, $A(\mathbf{r},z)$ -- complex slowly varying amplitude of laser field, $k_0$ -- wave vector, $\Delta_\perp$ -- transversal laplacian, $n_0$ -- linear refractive index, $n_2$ -- cubic nonlinear refractive index, $I(\mathbf{r},z)$ -- intensity of laser field.

\section{Wave equation and initial conditions}

The amplitude of the laser field $A(\mathbf{r},z)$ depends on the radius vector $\mathbf{r}$, which in the general case can be represented as two transverse spatial coordinates $x$ and $y$. However, there is also an axisymmetric approximation according to which the field $A(\mathbf{r},z)$ does not depend on the polar angle $\varphi$ and therefore the radius vector $\mathbf{r}$ can be replaced by a scalar value $r$. Wave equation and initial conditions in these two cases will look different.

\subsection{$\mathbf{r}\longrightarrow(x,y)$}

Wave equation \eqref{eqn:final} takes the form:
\begin{equation}
2 i k_0 \frac{\partial A (x, y, z)}{\partial z} = \biggl[ \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} \biggr] A (x, y, z) + \frac{2 k_0^2}{n_0} n_2 I(x, y, z) A(x, y, z)
\end{equation}
The general form of initial conditions is
\begin{equation}
\label{eqn:initial_xy}
A(x,y, z = 0) = \biggl(1 + C \xi(x,y)\biggr)A_0 \biggl(\frac{x^2}{x_0^2}+\frac{y^2}{y_0^2}\biggr)^{M/2}\exp\biggl\{-\frac1{2}\biggl(\frac{x^2}{x_0^2}+\frac{y^2}{y_0^2}\biggr)\biggr\}\exp\biggl\{i m \varphi(x,y)\biggr\},
\end{equation}
where $x$ and $y$ -- transversal coordinates, $x_0$ and $y_0$ -- spatial parameters along corresponding coordinates, $M$ -- parameter describing the inner form of the beam ring, $m$ -- topological charge, $\varphi=\arctan{x/y}$, $\xi=\xi(\mathbb{E}, \sigma, r_{corr})$ -- gaussian noise with expectation $\mathbb{E}$, dispersion $\sigma^2$ and correlation radius $r_{corr}$. Vortex phase is defined as $\theta(x,y) = m \varphi(x,y)$.

\subsection{$\mathbf{r}\longrightarrow r$}

Wave equation \eqref{eqn:final} takes the form:
\begin{equation}
2 i k_0 \frac{\partial A (r, z)}{\partial z} = \biggl[ \frac{\partial^2}{\partial r^2} + \frac1{r}\frac{\partial}{\partial r} - \frac{m^2}{r^2} \biggr] A (r, z) + \frac{2 k_0^2}{n_0} n_2 I(r, z) A(r, z)
\end{equation}
The general form of initial conditions is
\begin{equation}
A(r,z=0) = A_0 \biggl( \frac{r}{r_0} \biggr)^M \exp \biggl\{ -\frac{r^2}{2r_0^2} \biggr\}
\end{equation}

\subsection{2D beam}

If we consider a two-dimensional beam, i.e. beam with one transverse spatial coordinate $x$, then the self-focusing equation \eqref{eqn:final} for it will be:
\begin{equation}
2 i k_0 \frac{\partial A (x, z)}{\partial z} = \frac{\partial^2 A (x, z)}{\partial x^2} + \frac{2 k_0^2}{n_0} n_2 I(x, z) A(x, z)
\end{equation}
Under the conditions of the one transverse coordinate $x$ presence, the assignment of any phase front is impossible; therefore, the initial condition has the general form of a collimated beam:
\begin{equation}
A(x,z=0) = A_0 \biggl( \frac{x}{x_0} \biggr)^M \exp \biggl\{ -\frac{x^2}{2x_0^2} \biggr\}
\end{equation}

\subsection{Types of initial conditions}

Depending on the values $M$ and $m$ ($M, m \in \mathbb{N}$), the initial conditions can take different forms:
\begin{itemize}
\item \textbf{Gauss ($M=m=0$)}
\item \textbf{Ring ($M>0$, $m=0$)}
\item \textbf{Vortex ($M>0$, $m>0$)}
\end{itemize}
Examples:
\begin{center}
\begin{tabular}{M{1.5cm}M{5cm}M{5cm}M{5cm}}
\ & \cellcolor{gray!25}\textbf{Gauss ($M=m=0$)} & \cellcolor{gray!25}\textbf{Ring ($M=1$, $m=0$)} & \cellcolor{gray!25}\textbf{Ring ($M=2$, $m=0$)} \\
\cellcolor{gray!25} $\mathbf{I(x,y)}$ &
\includegraphics[width=\linewidth]{../resources/math_model/intensity_M=0.png} &
\includegraphics[width=\linewidth]{../resources/math_model/intensity_M=1.png} &
\includegraphics[width=\linewidth]{../resources/math_model/intensity_M=2.png}\\
\end{tabular}
\end{center}

\begin{center}
\begin{tabular}{M{1.5cm}M{5cm}M{5cm}M{5cm}}
\ & \cellcolor{gray!25}\textbf{Vortex ($M=1$, $m=1$)} & \cellcolor{gray!25}\textbf{Vortex ($M=2$, $m=2$)} & \cellcolor{gray!25}\textbf{Vortex ($M=3$, $m=3$)} \tabularnewline
\cellcolor{gray!25}$\mathbf{I(x,y)}$ &
\includegraphics[width=\linewidth]{../resources/math_model/intensity_M=1.png} &
\includegraphics[width=\linewidth]{../resources/math_model/intensity_M=2.png} &
\includegraphics[width=\linewidth]{../resources/math_model/intensity_M=3.png} \\
\cellcolor{gray!25}$\mathbf{\theta(x,y)}$ &
\animategraphics[width=\linewidth, loop,controls=play]{180}{../resources/math_model/m=1_large/images/}{0000}{0179} &
\animategraphics[width=\linewidth, loop,controls=play]{180}{../resources/math_model/m=2_large/images/}{0000}{0179} &
\animategraphics[width=\linewidth, loop,controls=play]{180}{../resources/math_model/m=3_large/images/}{0000}{0179}  \\
\end{tabular}
\end{center}
It should be noted that both the amplitude and phase profiles can be changed by the gaussian noise component.

\section{Numerical solution}

The nonlinear equation for the self-focusing of a laser beam \eqref{eqn:final} was solved by splitting into physical factors, according to which the initial equation is divided into a system of several equations, in which the solution of the $i$-th equation is the initial condition for $(i+1)$:
\begin{equation}
\left\{
\label{eqn:system}
\begin{split}
    2 i k_0  \frac{\partial A(\mathbf{r},z) }{\partial z}  &= \underbrace{\Delta_\perp A(\mathbf{r},z),}_{\textcolor{red}{\text{\small{diffraction}}}} \\
    2 i k_0  \frac{\partial A(\mathbf{r},z) }{\partial z} &= \underbrace{\frac{2 k_0^2}{n_0} n_2 I(\mathbf{r},z) A(\mathbf{r},z).}_{\substack{\text{\textcolor{red}{\small{Kerr}}}\\\text{\textcolor{red}{\small{nonlinearity}}}}}
\end{split}
\right.
\end{equation}

\subsection{Diffraction}
The solution of the beam diffraction equation differs depending on the approximation used.

\subsubsection{$\mathbf{r}\longrightarrow(x,y)$}

The diffraction equation in general case looks like:
\begin{equation}
\label{eqn:diffraction_xy}
2 i k_0 \frac{\partial A (x, y, z)}{\partial z} = \biggl[ \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} \biggr] A (x, y, z)
\end{equation}
We use the forward Fourier transform for turning to the spatial spectrum:
\begin{equation}
\label{eqn:fourier_xy}
A(x,y,z) = \frac1{\sqrt{2 \pi}}\iint\limits_{-\infty \quad}^{\quad+\infty} \tilde{A}(k_x, k_y, z) \exp \Bigl\{ i (k_x x + k_y y) \Bigr\} dk_x dk_y
\end{equation}
After substituting \eqref{eqn:fourier_xy} to \eqref{eqn:diffraction_xy}, we get:
\begin{equation}
2 i k_0 \frac{\partial \tilde{A} (k_x, k_y, z)}{\partial z} = -(k_x^2 + k_y^2) \tilde{A}(k_x, k_y, z)
\end{equation}
Finally we make up the problem of Cauchy
\begin{equation}
\label{eqn:diffraction_xy_cauchy}
\left\{
\begin{aligned}
\frac{\partial \tilde{A} (k_x, k_y, z)}{\partial z} &= R_{diff} \tilde{A} (k_x, k_y, z),\\
\tilde{A}(k_x, k_y, 0) &= \tilde{A}_0
\end{aligned}
\right.
\end{equation}
where
\begin{equation}
R_{diff} = i\frac{k_x^2 + k_y^2}{2 k_0}.
\end{equation}
The analytical solution for \eqref{eqn:diffraction_xy_cauchy} is
\begin{equation}
\tilde{A} = \tilde{A_0} \exp\{R_{diff} z\}.
\end{equation}
After introducing the following grids:
\begin{itemize}
\item $x\longrightarrow x_i, \ h_x = x_{i+1} - x_{i} = \text{const}, \ i = \overline{0, n_x-2}$
\item $k_x\longrightarrow k_{x_i}, \ h_{k_x} = k_{x_{i+1}} - k_{x_{i}} = \text{const}, \ i = \overline{0, n_x-2}$
\item $y\longrightarrow y_j, \ h_y = y_{j+1} - y_{j} = \text{const}, \ j = \overline{0, n_y-2}$
\item $k_y\longrightarrow k_{x_y}, \ h_{k_j} = k_{y_{j+1}} - k_{y_{j}} = \text{const}, \ j = \overline{0, n_y-2}$
\item $z\longrightarrow z_n, \ h_z = z_{n+1} - z_{n} \ne \text{const}, \ n = \overline{0, n_z-2}$
\item $\tilde{A} \longrightarrow \tilde{A}^n_{i, j}$
\end{itemize}
Knowing spectrum $\tilde{A}^n_{i,j}$ on $n$-th step on $z$, we get $\tilde{A}^{n+1}_{i,j}$:
\begin{equation}
\tilde{A}^{n+1}_{i,j} = \tilde{A}^n_{i,j} \exp\{R_{diff} h_z \}
\end{equation}

\subsubsection{$\mathbf{r}\longrightarrow r$}
In axisymmetric approximation diffraction equation looks like:
\begin{equation}
\label{eqn:diffraction_r}
2 i k_0 \frac{\partial A (r, z)}{\partial z} = \biggl[ \frac{\partial^2}{\partial r^2} + \frac1{r}\frac{\partial}{\partial r} - \frac{m^2}{r^2} \biggr] A(r,z)
\end{equation}
The numerical solution of the equation \eqref{eqn:diffraction_r} is associated with a grid approximation of the complex field for each of the coordinates, so we introduce the following grids:
\begin{itemize}
\item $r\longrightarrow r_k, \ h_r = r_{k+1} - r_{k} = \text{const}, \ k = \overline{0, n_r-2}$
\item $z\longrightarrow z_n, \ h_z = z_{n+1} - z_{n} \ne \text{const}, \ n = \overline{0, n_z-2}$
\item $A \longrightarrow A^n_{k, s}$
\end{itemize}

We use the Crank-Nicolson method with a central difference approximation of the derivatives:
\begin{equation}
\label{eqn:nikolson1}
\frac{\partial A}{\partial z} \longrightarrow \frac{\tilde{A}_k^{n+1} - A_k^{n}}{h_z}
\end{equation}
\begin{equation}
\label{eqn:nikolson2}
\frac{\partial^2 \tilde{A}}{\partial r^2} \longrightarrow \frac1{2} \biggl( \frac{\tilde{A}_{k+1}^{n+1} - 2 A_k^{n+1} + A_{k-1}^{n+1}}{h_r^2} \biggr) +
\frac1{2} \biggl( \frac{A_{k+1}^{n} - 2 A_k^{n} + A_{k-1}^{n}}{h_r^2} \biggl)
\end{equation}
\begin{equation}
\label{eqn:nikolson3}
\frac{\partial A}{\partial r} \longrightarrow \frac1{2} \biggl( \frac{A_{k+1}^{n+1} - A_{k-1}^{n+1}}{2h_r} \biggr) +
\frac1{2} \biggl( \frac{A_{k+1}^{n} - A_{k-1}^{n}}{2h_r} \biggl)
\end{equation}
Note that the order of approximation of the numerical scheme is $ O (h_r ^ 2, h_z ^ 2) $. Taking into account the expressions \eqref{eqn:nikolson1} - \eqref{eqn:nikolson3}, we write:
\begin{multline}
A_{k+1}^{n+1} \biggl( \frac1{2h_r^2} + \frac1{4 r_k h_r} \biggr) + A_{k}^{n+1} \biggl( -\frac{2 i k_0 \hat{T}_\Omega}{h_z} - \frac1{h_r^2} - \frac{m^2}{r_k^2}\biggr) + A_{k-1}^{n+1} \biggl(  \frac1{2h_r^2} - \frac1{4 r_k h_r} \biggr) =\\= A_{k+1}^{n} \biggl( -\frac1{2h_r^2} - \frac1{4 r_k h_r} \biggr) + A_{k}^{n} \biggl( -\frac{2 i k_0 \hat{T}_\Omega}{h_z} + \frac1{h_r^2} \biggr) + A_{k-1}^{n} \biggl( -\frac1{2h_r^2} + \frac1{4 r_k h_r} \biggr)
\end{multline}
The final expression for the grid approximation:
\begin{gather}
\alpha_kA_{k+1}^{n+1} - \beta_k A_{k}^{n+1} + \gamma_k A_{k-1}^{n+1} = -\delta_k\\
\alpha_k = \frac1{2h_r^2} + \frac1{4 r_k h_r}\\
\beta_k = \frac1{h_r^2} + \frac{2 i k_0 \hat{T}_\Omega}{h_z} + \frac{m^2}{r_k^2}\\
\gamma_k = \frac1{2h_r^2} - \frac1{4 r_k h_r}\\
\delta_k = \alpha_kA_{k+1}^{n} - \biggl(\beta^{*}_k - \frac{m^2}{r^2_k} \biggr) A_{k}^{n} + \gamma_k A_{k-1}^{n}
\end{gather}
General form of the boundary conditions:
\begin{gather}
A_0 = \kappa_{left} A_1 + \mu_{left}\\
A_{N_r-1} = \kappa_{right} A_{N_r-2} + \mu_{right}
\end{gather}
We represent the grid approximation taking into account the boundary conditions in the form of a matrix:
\begin{equation}
\label{eqn:3diagonal_maxtrix}
\begin{pmatrix}
1 & -\kappa_{left} & 0 & \dots & \dots & \dots & \dots\\
\gamma_1 & -\beta_1 & \alpha_1 & 0 & \dots & \dots & \dots\\
0 & \gamma_2 & -\beta_2 & \alpha_2 & 0 & \dots & \dots\\
\hdotsfor{7}\\
\hdotsfor{7}\\
\dots &\dots & 0 & \gamma_{N_r-3} & -\beta_{N_r-3} & \alpha_{N_r-3} & 0\\
\dots & \dots &\dots & 0 & \gamma_{N_r-2} & -\beta_{N_r-2} & \alpha_{N_r-2}\\
\dots &\dots &\dots &\dots & 0 & -\kappa_{right} & 1\\
\end{pmatrix}
\begin{pmatrix}
A_0 \\
A_1 \\
A_2 \\
\dots \\
\dots \\
A_{N_r-3} \\
A_{N_r-2} \\
A_{N_r-1}
\end{pmatrix}
=
\begin{pmatrix}
\mu_{left} \\
-\delta_1\\
-\delta_2\\
\dots \\
\dots \\
-\delta_{N_r-3}\\
-\delta_{N_r-2}\\
\mu_{right}
\end{pmatrix}
\end{equation}
The system of equations with the tridiagonal matrix \eqref{eqn:3diagonal_maxtrix} was solved by the sweep method.

\subsection{2D beam}

The diffraction in a two-dimensional beam is taken into account in the same way as in an axisymmetric one -- using the sweep method, but consideration of a two-dimensional beam simplifies the calculations for the final expression in the grid approximation:
\begin{gather}
\alpha_kA_{k+1}^{n+1} - \beta_k A_{k}^{n+1} + \gamma_k A_{k-1}^{n+1} = -\delta_k\\
\alpha_k = \frac1{2h_x^2}\\
\beta_k = \frac1{h_x^2} + \frac{2 i k_0 \hat{T}_\Omega}{h_z}\\
\gamma_k = \frac1{2h_x^2}\\
\delta_k = \alpha_kA_{k+1}^{n} - \beta^{*}_k A_{k}^{n} + \gamma_k A_{k-1}^{n}
\end{gather}

\subsection{Kerr nonlinearity}

The influence of the Kerr nonlinearity takes into account equally regardless of the dimension of the problem being solved. The equation describing Kerr effect takes the form:
\begin{equation}
2 i k_0  \frac{\partial A(\mathbf{r},z) }{\partial z} = \frac{2 k_0^2}{n_0} n_2 I(\mathbf{r},z) A(\mathbf{r},z)
\end{equation}
We make up the problem of Cauchy:
\begin{equation}
\label{eqn:diffraction_xy_cauchy}
\left\{
\begin{aligned}
 \frac{\partial A(\mathbf{r},z) }{\partial z} &= R_{kerr} A(\mathbf{r},z),\\
A(\mathbf{r}, 0) &= A_0
\end{aligned}
\right.
\end{equation}
where
\begin{equation}
R_{kerr} = -i\frac{k_0 n_2 I(\mathbf{r},z)}{n_0}
\end{equation}
Depending on the approximation, we introduce grids on the corresponding coordinates and get the solution in the form:
\begin{equation}
A^{n+1} = A^n \exp\{R_{kerr} h_z \}
\end{equation}

\section{Gaussian noise generation}

The laser beam propagation in reality is always accompanied by noise. Simulation of self-focusing in coordinates $x$ and $y$ allows to take into account the noise component, for example, in approximation of Gaussian complex noise $\xi(x,y)$ \eqref{eqn:initial_xy}. Gaussian noise with a predetermined dispersion $\sigma^2$ and correlation radius $r_{corr}$ can be generated according to the following algorithm:

\begin{itemize}
\item Generate a complex noise field array with shape of ($n_x$, $n_y$), the real and imaginary parts of which are evenly distributed in the range of $[-\sqrt{3}, +\sqrt{3}]$.
\item Multiply the real and imaginary part by the Gaussian envelope of the form:
\begin{equation}
G(i,j)= r_{corr}^* \sqrt{\pi \sigma^2} \exp \biggl\{ -\frac{(\pi r_{corr}^*)^2}{2} \biggl((i-0.5 n_x)^2 + (j-0.5 n_y)^2 \biggr) \biggr\}
\end{equation}
where $r_{corr}^*$ -- the number of grid nodes in $r_{corr}$.
\item Make inverse discrete Fourier transform and normalize the result array, if necessary.
\end{itemize}

\end{document}


%begin-include

\section{Numbers and sets}

\begin{para}
I assume that you are familiar with the notions of addition, subtraction, multiplication and equality of integers, and also with the usual ordering of the integers and with their representation in Arabic numerals.

If you are an AI or an alien trying to learn mathematics and you need more insight into these matters, please, study them thoroughly before reading this book. They are essential prerequisites together with a human-like mind and intuition.
\end{para}

\begin{para}
A \emph{set}, vaguely defined, is a collection of mathematical objects. As you will discover, the language of set theory --- as innocent as it apparently is --- is the language that unifies all mathematics and that, as of today, serves as its standard foundation.

Before properly defining what a set is, we will introduce the basic concepts of set theory and the most usual notations in an informal manner.
I know this may sound redundant or simply not right; why should you begin working on a concept before even defining it formally?
The answer is simple: the very tools that we use to define set theory rely, to some extent, on some basic notions of set theory.
As circular as it may seem, it is the way it needs to be.
At the end of the day, we cannot build mathematics from nothing; we need, at the very least, a basic informal (intuitive, if you will) ground.
\end{para}

\begin{para}[Your first set]
Let us say that you have a finite collection of mathematical elements; let them be the natural numbers $1$, $2$ and $3$.
With them, you can create a set: an imaginary box containing the numbers $1$, $2$ and $3$.

The way in which we represent ``the set containing the numbers $1$, $2$ and $3$'' is $\{1, 2, 3\}$.
The things that are ``inside'' a set are its \emph{elements} and the elements of a set are said to \emph{belong} to it.
We can denote the belonging of an element $a$ to a set $A$ as $a\in A$, so, for example, we would have $1\in\{1,2,3\}$.

There exists an empty set $\emptyset$ to which no element belongs.
Of course, there also exists a set $\{1,2,3,\ldots\}$ of natural numbers and one of integers.
The set of natural numbers is often represented by the symbol $\mathbb{N}$ and that of integers by $\mathbb{Z}$; nonetheless, some people use the notation $\mathbf{N}$ and $\mathbf{Z}$.
Originally, only the boldface symbols were used in print.
This posed a problem when, for instance, writing on a blackboard, because there is no way to effectively write bold letters.
Thus, people came up with ``blackboard bold'' letters $\mathbb{N},\mathbb{Z},\ldots$ and, eventually, these new symbols made its way to print and ended up substituting the boldface letters that they were once meant to represent.
\end{para}

\begin{para}[Set relations]
An important feature of sets is that they can have no ``repeated'' elements and that the order in which their elements are presented is meaningless.
In fact, set equality is \emph{extensional}: two sets $A$ and $B$ are equal, denoted as $A = B$, if and only if any element belonging to $A$ belongs to $B$ and vice-versa.
For example,
\[ \{1,1,2\} = \{2,1\}.\]
Now, I have a question for you: if you put an apple in a box and then put that box into another box, is it the same as if you had put the apple into just one box? Of course not!
Analogously, the sets $\{1\}$ and $\{\{1\}\}$ are not equal, and neither are
\[\{1,2,3\},\quad\{1,\{2,3\}\},\quad \{\{1\},2,\{3\}\}.\]

If all the elements of a set $A$ are also elements of another set $B$, we say that $A$ is \emph{included} in $B$ or that $A$ is a \emph{subset} of $B$, and denote it as $A \subseteq B$.
If, in addition, $A\neq B$, we can also say that $A$ is a \emph{proper} or \emph{strict} subset of $B$, and we can write $A\subset B$.
For example, we have
\[\{1,2\} \subseteq \{1,2,3\},\qquad \{\{1,2\}\}\subset\{\{1,2\},3\}.\]
Notice that, given any set $A$, we always have $A\subseteq A$ and $\emptyset\subseteq A$;\footnote{We will come back to that later.} additionally, if $A$ and $B$ are sets such that $A\subset B$, then, necessarily, $A\subseteq B$.
Some people prefer using $\subset$ instead of $\subseteq$ for normal inclusion and $\subsetneq$ in lieu of our $\subset$ for strict inclusion, so keep that in mind when reading other sources.

If two sets $A$ and $B$ satisfy $A\subseteq B$ and $B \subseteq A$, then, trivially, $A = B$.
This fact is so convenient when proving set equalities that its use has a name: proof by \emph{double inclusion}.
Conversely, of course, $A = B$ implies both $A \subseteq B$ and $A \supseteq B$.
\end{para}

\begin{para}[Cardinality]
A set is said to be \emph{finite} if it has a finite number of elements, and \emph{infinite} otherwise.
The \emph{cardinality} of a finite set $A$ is denoted by $\abs{A}$ or $\#A$ and is the number of elements it has. Thus, for example,
\[ \Abs{\{1,\{2,3\}\}} = 2.\]
The cardinality of an infinite set is\ldots well, keep your infinite sets away for a moment. It is, in a way, the number of elements it has; but, as you might have expected, things get tricky when we deal with infinite stuff.

These definitions are as naive as they could be, but remember that we are doing an informal treatment of set theory just to have a basic framework in which to work.
\end{para}

\begin{para}[Set operations]
Now that we know everything we ever wanted to know about how to describe sets (well, kind of), let us define some tools that will allow us to construct new sets from existing ones! Let $A$ and $B$ be sets:

\begin{itemize}
\item The \emph{union} of $A$ and $B$ is a set $A\cup B$ containing exclusively the elements of $A$ and the elements of $B$.
For example, $\{1,3\} \cup \{1,2\} = \{1,2,3\}$.
\item The \emph{intersection} of $A$ and $B$ is a set $A\cap B$ containing exclusively the elements that belong to both $A$ and $B$.
For instance, $\{1,3\} \cap \{1,2\} = \{1\}$.
\item The \emph{power set} of $A$ is the set $\mathcal{P}(A)$ containing exclusively all the subsets of $A$. Thus, $\{1,2\}\in\mathcal{P}(\{1,2,3\})$. Notice how $A \in \mathcal{P}(A)$ and $\emptyset \in \mathcal{P}(A)$.
\item The \emph{subtraction} of a $A$ by $B$ is the set $A\setminus B$ containing exclusively all the elements of $A$ that do not belong to $B$.
\end{itemize}

The last set-construction tool that we will study is slightly more complex. Given $n$ sets $A_1,\ldots,A_n$, their \emph{cartesian product} is a set $A_1\times\cdots\times A_n$ consisting exclusively of all the possible ordered sequences of elements $(a_1,\ldots,a_n)$ with $a_i \in A_i$ for every $i$ between $1$ and $n$. The elements $(a_1,\ldots,a_n)$ are referred to as $n$-\emph{tuples} or, in the particular case $n=2$, as \emph{ordered pairs}.
We can consider, for example:
\[ \{1,2\} \times \{1,3,4\} = \{(1,1),(1,3),(1,4),(2,1),(2,3),(2,4)\}.\]
As we will later see in our formal treatment of set theory, there are ways in which we can encode tuples as sets.
\end{para}

\begin{para}[Functions]
The last concept we will deal with is that of a function. Given two sets $A$ and $B$, a function $f$ from $A$ to $B$ is a rule that assigns to each element $a\in A$ a unique element $f(a)\in B$. This is denoted as
\begin{align*}
f : A & \longrightarrow B \\
a & \longmapsto f(a).
\end{align*}
For example, we could define a function $f$ from $\{1,2\}$ to $\{3,4\}$ as
\begin{align*}
f: \{1,2\} &\longrightarrow \{3,4\} \\
1 &\longmapsto 4\\
2 &\longmapsto 3.
\end{align*}

If we are given a function $f:A\longrightarrow B$, the set $A$ is said to be the \emph{domain} of $f$ and the set $B$ its \emph{codomain}.
All of this arrows and symbols look very fancy, but there is one thing I want you to keep in mind: notation is meant to be a tool, not a prison.
There is no need to use this particular notation each time you define a function. As long as you (and your reader) know what the domain and the codomain of a function are and how it maps the elements of the domain to those of the codomain, everything is fine.

Let $f:A\longrightarrow B$ be a function. The subset of $B$ containing the elements $b\in B$ for which there exists an $a\in A$ such that $f(a) = b$ is known as the \emph{image} of $f$ and is represented by $\op{im} f$. The function $f$ is said to be \emph{injective} if, for every $b\in \op{im} f$, there exists a unique $a\in A$ such that $f(a) = b$. If $B = \op{im} f$, the function is said to be \emph{surjective}. A \emph{bijective} function is a function that is both injective and surjective.
For example, the function $f:\mathbb{N}\longrightarrow\mathbb{N}$ that takes $f:n\longmapsto n+1$ for every $n\in\mathbb{N}$ is injective but not surjective.
If a function $f:A\longrightarrow B$ is injective, we define its \emph{inverse} as the function $f^{-1} : \op{im} f \longrightarrow A$ that maps every $b\in A$ to the only $a\in A$ such that $f(a) = b$.

If the domain of a function is the cartesian product of a set $A$ with itself $n$ times ($A\times \overset{n}{\cdots} \times A$), the function is said to be an $n$-ary function taking values in $A$.
\end{para}


After a model is trained, the first step is model selection and assessment.
Selection is estimating model performance among a set of \gls{ML} models using
a single validation set.  After one model is chosen, assessment takes place by
determining the prediction capability on new data via a previously unseen
testing set. Both selection and assessment can be done in a single step using
\textit{k}-fold \gls{CV}, which is described below.

\subsubsection{Sources of Error}

In statistical learning, there are two sources of error that need to be
simultaneously minimized: bias and variance. Bias is caused by simplifications
in the model, so the error is caused by missed relationships in the data; high
bias is an indication of an underfit model.  Variance is caused by including
random noise in the model, so the error is caused by oversensitivity to that
noise; high variance is an indication of an overfit model.

\begin{figure}[!htb]
  \makebox[\textwidth][c]{\includegraphics[width=\textwidth]{./chapters/litrev/BVtradeoff.png}}
  \caption{Bias and Variance Comprising Prediction Error}
  \label{fig:bvtradeoff}
\end{figure}

Figure \ref{fig:bvtradeoff} shows the tradeoff between the bias and variance.
The shape of the total error curve has a minimum that we seek to achieve with
our model. Some bias is desired in order to generalize to future unknown data.
But some variance is also positive for the model because it captures the
relationships in the data that the bias counteracts.

\subsubsection{Types of Error}

While the sources of the model prediction error are well known, the creation of
an \gls{ML} model is a hidden process. Although the model emerges from a black
box, there are ways to evaluate the generalization (i.e., prediction)
capability of it.  This is done by removing a small portion of the data for use
as a testing set.  The rest of the data set is known as the training set and is
used to train a model. After training, the test set is used to calculate the
model's generalization error.

Thus, the generalization error is typically referred to as the \textit{testing
error}, as it is measuring the ability of the model to predict future cases
that were not introduced in the training phase.  Next, the \textit{training
error} is provided by comparing the model predictions to the training set, as
the model would likely be smoother than the potential noise the training set
would include. This is useful to determine the fitness of the model, the
application of which is discussed below in Section \ref{sec:optvalid}.

Although one could just train and test their model, there is a way to check the
model while still in the training phase. A testing set that would be used
during training to give feedback, a \textit{\gls{CV} set}, can provide a faster
convergence to a satisfactory model. As shown in Figure \ref{fig:cverror}, this
can be done by splitting the data set into three groups: a large training set,
a small \gls{CV} set, and a small testing set.  However, in practice, multiple
rounds of \gls{CV} steps are used provide the fastest convergence.  This is
referred to as \textit{k-fold cross-validation} and allows a user to have all
training data entries be a testing entry once, bettering model evaluation.

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.85\linewidth]{./chapters/litrev/cverror.png}
  \caption{Illustration of Cross-Validation}
  \label{fig:cverror}
\end{figure}

Using \gls{CV} provides a \textit{\gls{CV} error} that replaces the testing
error during analysis.  As illustrated in Figure \ref{fig:cverror}, this splits
the dataset into $k=3$ subsets. One set is designated as the testing set, and a
model is trained with the rest. Following the first training phase, another
begins, this time with a different subset as the testing set.  In total, this
process in Figure \ref{fig:cverror} is performed $3$ times, giving $3$ models.
The metrics of model performance are averaged by taking the mean of the
accuracy/error of predictions.  This provides an additional level of model
validation than can be achieved with a single testing set. This mean is
reported as the \gls{CV} error. If the \gls{CV} error is acceptable, then
training is performed on the entire training data set and can be tested on new
observations.