\section{Output}
\subsection{Files}

Explain the created C++ files and executables, and how to use them.

\subsection{Two-scale algorithm}

Explain the structure of the two-scale algorithm classes and how they
work and are plugged together.

The stucture of the two-scale algorithm used in \flexisusy\ is as follows,%

Initial guess:
\begin{enumerate}
\item Guess gauge couplings $g_{1,2,3}$ at $m_Z$
\item Apply user-defined low-scale constraint (\code{LowScaleInput})
\item Guess Yukawa couplings $y^{u,d,e}_{ij}$ at $m_Z$
\item Run model to the high-scale (\code{HighScaleFirstGuess})
\item Apply high-scale constraint (\code{HighScaleInput})
\item Run model to the low-scale (\code{LowScaleFirstGuess})
\item Solve EWSB eqs.\ at the tree-level
\item Calculate \DRbar\ masses
\end{enumerate}
%
Solving the renormalization group equations:
\begin{enumerate}
\item \label{rge-step-one} Run model to the low-scale (\code{LowScale})
  \begin{enumerate}
  \item Calculate \DRbar\ masses
  \item Recalculate low-scale
  \item Calculate \DRbar\ gauge and Yukawa couplings $g_{1,2,3}$,
    $y^{u,d,e}_{ij}$, see
    Section~\ref{sec:matching-to-the-standard-model}.
  \item Apply user-defined low-scale constraint (\code{LowScaleInput})
  \end{enumerate}
\item Run model to the high-scale (\code{HighScale})
  \begin{enumerate}
  \item Recalculate high-scale
  \item Apply high-scale constraint (\code{HighScaleInput})
  \end{enumerate}
\item Run model to the susy-scale (\code{SUSYScale})
  \begin{enumerate}
  \item Calculate \DRbar\ masses
  \item Recalculate low-scale
  \item Apply susy-scale constraint (\code{SUSYScaleInput})
  \item Solve EWSB eqs.\ at the one-loop level
  \end{enumerate}
\item If not converged yet, goto \ref{rge-step-one}
\end{enumerate}
%
Calculating the particle spectrum:
\begin{enumerate}
\item Run model to the susy-scale
\item Calculate the pole masses
\item Run model to the parameter output scale (SLHA block MODSEL 12)
\end{enumerate}

\begin{figure}[tb]
  \centering
  \tikzumlset{fill class=white}
  \begin{tikzpicture}
    \umlclass[x=0, y=0, type=abstract]{Beta\_function}{
      -- scale : double\\
      -- loops : unsigned\\
      -- numPars : unsigned
    }{
      + \umlvirt{get() : const Eigen::ArrayXd} \\
      + \umlvirt{set(v : const Eigen::ArrayXd\&) : void}\\
      + \umlvirt{beta() : Eigen::ArrayXd}\\
      + run\_to(scale : double, eps : double) : void\\
    }
    \umlclass[x=8, y=-12, template={T}]{MSSM}{}{}
    \umlclass[x=8, y=-8, type=abstract]{Two\_scale\_model}{}{
      + \umlvirt{calculate\_spectrum() : void}\\
      + \umlvirt{run\_to(scale : double, eps : double) : void}\\
      + \umlvirt{set\_precision(precision : double) : void}
    }
    \umlclass[x=0, y=-4]{MSSM\_susy\_parameters}{}{
      + get() : const Eigen::ArrayXd \\
      + set(v : const Eigen::ArrayXd\&) : void\\
      + beta() : Eigen::ArrayXd
    }
    \umlclass[x=0, y=-8]{MSSM\_soft\_parameters}{}{
      + get() : const Eigen::ArrayXd \\
      + set(v : const Eigen::ArrayXd\&) : void\\
      + beta() : Eigen::ArrayXd
    }
    \umlclass[x=0, y=-12]{MSSM$<$Two\_scale$>$}{}{
      + calculate\_spectrum() : void\\
      + run\_to(scale : double, eps : double) : void\\
      + set\_precision(precision : double) : void
    }
   \umlinherit{MSSM\_susy\_parameters}{Beta\_function}
   \umlinherit{MSSM\_soft\_parameters}{MSSM\_susy\_parameters}
   \umlinherit{MSSM$<$Two\_scale$>$}{MSSM\_soft\_parameters}
   \umlinherit{MSSM$<$Two\_scale$>$}{Two\_scale\_model}
   \umldep[arg1={$<<$bind$>>$}, mult1={T $\rightarrow$ Two\_scale}, pos1=0.5]{MSSM$<$Two\_scale$>$}{MSSM}
  \end{tikzpicture}
  \caption{Two-scale model class hierarchy}
  \label{fig:two-scale-model-class-hierarchy}
\end{figure}

\begin{figure}[tb]
  \centering
  \tikzumlset{fill class=white}
  \begin{tikzpicture}
    \umlclass[x=0, y=0, type=abstract]{Two\_scale\_model}{}{
      + \umlvirt{calculate\_spectrum() : void}\\
      + \umlvirt{run\_to(scale : double, eps : double) : void}\\
      + \umlvirt{set\_precision(precision : double) : void}
    }
    \umlclass[x=0, y=-4]{RGFlow$<$Two\_scale$>$}{}{
      + solve() : void
    }
    \umlclass[x=8, y=0]{Constraint$<$Two\_scale$>$}{}{
      + \umlvirt{apply() : void}\\
      + \umlvirt{get\_scale() : double}\\
      + \umlvirt{set\_model(model : Two\_scale\_model*) : void}
    }
    \umlclass[x=8, y=4, template={T}]{Constraint}{}{}
    \umlclass[x=0, y=-7]{Initial\_guesser$<$Two\_scale$>$}{}{
      + \umlvirt{guess() : void}
    }
    \umlclass[x=0, y=-11, template={T}]{Initial\_guesser}{}{}
    \umlclass[x=8, y=-7]{Convergence\_tester$<$Two\_scale$>$}{}{
      + \umlvirt{accuracy\_goal\_reached() : bool}\\
      + \umlvirt{max\_iterations() : unsigned}
    }
    \umlclass[x=8, y=-11, template={T}]{Convergence\_tester}{}{}
    \umlclass[x=8, y=-4, template={T}]{RGFlow}{}{}
    \umldep[arg1={$<<$bind$>>$}, mult1={T $\rightarrow$ Two\_scale}, pos1=0.5]{RGFlow$<$Two\_scale$>$}{RGFlow}
    \umldep[mult1=1..*]{RGFlow$<$Two\_scale$>$}{Two\_scale\_model}
    \umldep[mult1=1..*]{RGFlow$<$Two\_scale$>$}{Constraint$<$Two\_scale$>$}
    \umldep{RGFlow$<$Two\_scale$>$}{Initial\_guesser$<$Two\_scale$>$}
    \umldep{RGFlow$<$Two\_scale$>$}{Convergence\_tester$<$Two\_scale$>$}
    \umldep[mult1={$<<$bind$>>$}, arg1={T $\rightarrow$ Two\_scale}, pos1=0.5]{Initial\_guesser$<$Two\_scale$>$}{Initial\_guesser}
    \umldep[mult1={$<<$bind$>>$}, arg1={T $\rightarrow$ Two\_scale}, pos1=0.5]{Convergence\_tester$<$Two\_scale$>$}{Convergence\_tester}
    \umldep[arg1={$<<$bind$>>$}, mult1={T $\rightarrow$ Two\_scale}, pos1=0.5]{Constraint$<$Two\_scale$>$}{Constraint}
  \end{tikzpicture}
  \caption{Two-scale renormalization group solver class hierarchy}
  \label{fig:two-scale-rgflow-class-hierarchy}
\end{figure}

\subsubsection{Matching to the Standard Model}
\label{sec:matching-to-the-standard-model}

\paragraph{Gauge couplings}

At the low-scale $M_Z$ we calculate
%
\begin{align}
  \alpha_{\text{e.m.},\text{susy}}^{\text{\DRbar}}(M_Z) &=
  \frac{\alpha_{\text{e.m.},\text{SM}}^{(5),\text{\MSbar}}(M_Z)}{1 -
    \Delta\alpha_{\text{e.m.},\text{SM}}(M_Z) -
    \Delta\alpha_{\text{e.m.},\text{susy}}(M_Z)} ,\\
  \Delta\alpha_{\text{e.m.},\text{SM}}(\mu) &=
  \frac{\alpha_\text{e.m.}}{2\pi} \left[\frac{1}{3}
    - \frac{16}{9} \log{\frac{m_t}{\mu}} \right],\\
  \Delta\alpha_{\text{e.m.},\text{susy}}(\mu) &=
  \frac{\alpha_\text{e.m.}}{2\pi} \left[ -\sum_{\text{susy particle }
      i}
    C_i \log{\frac{m_i}{\mu}} \right],\\
    e_{\text{susy}}^{\text{\DRbar}}(M_Z) &=
    \sqrt{4\pi\alpha_{\text{e.m.},\text{susy}}^{\text{\DRbar}}(M_Z)}
\end{align}
%
\begin{align}
  \alpha_{\text{s},\text{susy}}^{\text{\DRbar}}(M_Z) &=
  \frac{\alpha_{\text{s},\text{SM}}^{(5),\text{\MSbar}}(M_Z)}{1 -
    \Delta\alpha_{\text{s},\text{SM}}(M_Z)
    - \Delta\alpha_{\text{s},\text{susy}}(M_Z)} ,\\
  \Delta\alpha_{\text{s},\text{SM}}(\mu) &=
  \frac{\alpha_\text{s}}{2\pi} \left[
    -\frac{2}{3} \log{\frac{m_t}{\mu}} \right],\\
  \Delta\alpha_{\text{s},\text{susy}}(\mu) &=
  \frac{\alpha_\text{s}}{2\pi}\left[ \frac{1}{2}-\sum_{\text{susy
        particle } i} C_i \log{\frac{m_i}{\mu}} \right] ,\\
  g_{3,\text{susy}}^{\text{\DRbar}}(M_Z) &=
  \sqrt{4\pi\alpha_{\text{s},\text{susy}}^{\text{\DRbar}}(M_Z)} ,
\end{align}
%
where
%
\begin{align}
  \alpha_{\text{e.m.},\text{SM}}^{(5),\text{\MSbar}}(M_Z) &= 1/127.944,\\
  \alpha_{\text{s},\text{SM}}^{(5),\text{\MSbar}}(M_Z) &= 0.1185,
\end{align}
%
are the \MSbar electromagnetic and strong cougling constants in the
Standard Model including only $5$ quark flavours
\cite{Beringer:1900zz}.  Afterwards, SARAH's tree-level expressions
for the Weinberg angle and the Hypercharge and left gauge couplings
$g_Y$ and $g_2$ in terms of $M_{W,\text{susy}}^{\text{\DRbar}}(M_Z)$,
$M_{Z,\text{susy}}^{\text{\DRbar}}(M_Z)$ and
$e_{\text{susy}}^{\text{\DRbar}}(M_Z)$ are used to calculate
$g_{1,\text{susy}}^{\text{\DRbar}}(M_Z)$ and
$g_{2,\text{susy}}^{\text{\DRbar}}(M_Z)$.  In the MSSM, for example,
we'll have
%
\begin{align}
  \theta_{W,\text{susy}}^{\text{\DRbar}}(M_Z) &=
  \arcsin\sqrt{1 - \left(\frac{M_{W,\text{susy}}^{\text{\DRbar}}(M_Z)}{M_{Z,\text{susy}}^{\text{\DRbar}}(M_Z)}\right)^2} ,\\
  g_{1,\text{susy}}^{\text{\DRbar}}(M_Z) &=
  \sqrt{\frac{5}{3}} \frac{e_{\text{susy}}^{\text{\DRbar}}(M_Z)}{\cos\theta_{W,\text{susy}}^{\text{\DRbar}}(M_Z)} ,\\
  g_{2,\text{susy}}^{\text{\DRbar}}(M_Z) &=
  \frac{e_{\text{susy}}^{\text{\DRbar}}(M_Z)}{\sin\theta_{W,\text{susy}}^{\text{\DRbar}}(M_Z)} ,
\end{align}
%
where
%
\begin{align}
  \left(M_{W,\text{susy}}^{\text{\DRbar}}(M_Z)\right)^2 &=
  \left(M_W^{\text{pole}}\right)^2 + \re \Pi_{WW}^T(p^2 = (M_W^{\text{pole}})^2) ,\\
  \left(M_{Z,\text{susy}}^{\text{\DRbar}}(M_Z)\right)^2 &=
  \left(M_Z^{\text{pole}}\right)^2 + \re \Pi_{ZZ}^T(p^2 = (M_Z^{\text{pole}})^2)
\end{align}
%
and $M_W^{\text{pole}} = 80.404\unit{GeV}$, $M_Z^{\text{pole}} =
91.1876\unit{GeV}$ \cite{Beringer:1900zz}.

\paragraph{Fermion masses}

The expressions for the SM-Yukawa coupling matrices extracted from the
tree-level mass matrices of the SM-fermions.  In the MSSM, for
example, they read
%
\begin{align}
  y_u^{\text{\DRbar}}(M_Z) &= \frac{\sqrt{2} m_{u}^T}{v_u} , &
  y_d^{\text{\DRbar}}(M_Z) &= \frac{\sqrt{2} m_{d}^T}{v_d} , &
  y_e^{\text{\DRbar}}(M_Z) &= \frac{\sqrt{2} m_{e}^T}{v_d}.
\end{align}
%
At the low-scale the fermion mass matrices are composed as
%
\begin{align}
  m_u =
  \begin{pmatrix}
    m_{u}^{\userinput} & 0 & 0 \\
    0 & m_{c}^{\userinput} & 0 \\
    0 & 0 & m_{t,\text{susy}}^{\text{\DRbar}}(M_Z)
  \end{pmatrix} ,\\
  m_d =
  \begin{pmatrix}
    m_{d}^{\userinput} & 0 & 0 \\
    0 & m_{s}^{\userinput} & 0 \\
    0 & 0 & m_{b,\text{susy}}^{\text{\DRbar}}(M_Z)
  \end{pmatrix} ,\\
  m_e =
  \begin{pmatrix}
    m_{e}^{\userinput} & 0 & \\
    0 & m_{\mu}^{\userinput} & 0 \\
    0 & 0 & m_{\tau,\text{susy}}^{\text{\DRbar}}(M_Z)
  \end{pmatrix}.
\end{align}
%
Only the 3rd generation quark masses are calculated in the \DRbar
scheme from the user input quantities $m_t^\text{pole}$,
$m_{b,\text{SM}}^{\text{\MSbar}}(M_Z)$,
$m_{\tau,\text{SM}}^{\text{\MSbar}}(M_Z)$.  In detail, the top quark
\DRbar mass is calculated via
%
\begin{align}
  \begin{split}
    m_{t,\text{susy}}^{\text{\DRbar}}(\mu) &= m_t^\text{pole} +
    \re\Sigma_{tt}^{S,\text{heavy}}(m_t^\text{pole}) \\
    &\phantom{=\;} + m_t^\text{pole}
    \left[ \re\Sigma_{tt}^{L,\text{heavy}}(m_t^\text{pole}) +
      \re\Sigma_{tt}^{R,\text{heavy}}(m_t^\text{pole}) + \Delta
      m_t^{(1),\text{qcd}} + \Delta m_t^{(2),\text{qcd}} \right] ,
  \end{split}
\end{align}
%
where the $\Sigma_{ff}^\text{heavy}$ denotes the self-energy of
fermion $f$ without the Standard Model contributions.  The appearing
QCD corrections $\Delta m_t^{(1),\text{qcd}}$ and $\Delta
m_t^{(2),\text{qcd}}$ are taken from
\cite[Eq. (58),(61)]{Bednyakov:2002sf} and read
%
\begin{align}
  \Delta m_t^{(1),\text{qcd}} &= -\frac{g_3^2 \left(5-3 \log\left(\frac{m_t^2}{\mu^2}\right)\right)}{12 \pi^2},\\
  \begin{split}
    \Delta m_t^{(2),\text{qcd}} &= \left(\Delta
      m_t^{(1),\text{qcd}}\right)^2 \\
    &\phantom{=\;} - \frac{g_3^4 \left[396
        \log^2\left(\frac{m_t^2}{\mu^2}\right)-1476
        \log\left(\frac{m_t^2}{\mu^2}\right)-48 \zeta(3)+2011+16 \pi
        ^2 (1+\log 4)\right]}{4608 \pi^4}.
  \end{split}
\end{align}
%
The \DRbar mass of the bottom quark is calculated via
\cite{Baer:2002ek,Skands:2003cj}
%
\begin{align}
  m_{b,\text{susy}}^{\text{\DRbar}}(\mu) &=
  \frac{m_{b,\text{SM}}^{\text{\DRbar}}(\mu)}{1 -
    \re\Sigma_{bb}^{S,\text{heavy}}(m_{b,\text{SM}}^\text{\MSbar})/m_b
    - \re\Sigma_{bb}^{L,\text{heavy}}(m_{b,\text{SM}}^\text{\MSbar}) -
    \re\Sigma_{bb}^{R,\text{heavy}}(m_{b,\text{SM}}^\text{\MSbar})} ,\\
  m_{b,\text{SM}}^{\text{\DRbar}}(\mu) &=
  m_{b,\text{SM}}^{\text{\MSbar}}(\mu) \left(1 - \frac{\alpha_s}{3
      \pi} - \frac{23}{72} \frac{\alpha_s^2}{\pi^2} + \frac{3
      g_2^2}{128 \pi^2} + \frac{13 g_Y^2}{1152 \pi^2}\right) ,
\end{align}
%
where $m_{b,\text{SM}}^{\text{\MSbar}}(M_Z)$ is user input.  The
\DRbar mass of the $\tau$ is calculated via
%
\begin{align}
  \begin{split}
    m_{\tau,\text{susy}}^{\text{\DRbar}}(\mu) &=
    m_{\tau,\text{SM}}^{\text{\DRbar}}(\mu) +
    \re\Sigma_{\tau\tau}^{S,\text{heavy}}(m_{\tau,\text{SM}}^\text{\MSbar}) \\
    &\phantom{=\;} + m_{\tau,\text{SM}}^{\text{\DRbar}}(\mu) \left[
      \re\Sigma_{\tau\tau}^{L,\text{heavy}}(m_{\tau,\text{SM}}^\text{\MSbar})
      +
      \re\Sigma_{\tau\tau}^{R,\text{heavy}}(m_{\tau,\text{SM}}^\text{\MSbar})
    \right] ,
  \end{split}\\
  m_{\tau,\text{SM}}^{\text{\DRbar}}(\mu) &= m_{\tau,\text{SM}}^{\text{\MSbar}}(\mu)
  \left(1 - 3 \frac{g_Y^2 - g_2^2}{128 \pi^2}\right),
\end{align}
%
where $m_{\tau,\text{SM}}^{\text{\MSbar}}(M_Z)$ is user input.  To
convert the fermion masses from the \MSbar to the \DRbar scheme the
Yukawa coupling conversion from \cite[Eq.~(19)]{Skands:2003cj} was
used and it was assumed that the VEV is already given in the \DRbar
scheme.

\subsection{Lattice algorithm}

Explain the structure of the lattice algorithm classes and how they
work.


\documentclass[12pt]{article}
\setlength{\parindent}{0pt}
		\usepackage{graphicx}
		\usepackage[utf8]{inputenc}
		\usepackage{german}
		\usepackage{ngerman}
		\usepackage{amsmath}
		\usepackage{amssymb}
	    \usepackage[nottoc]{tocbibind}

\begin{document}

\tableofcontents

 \pagebreak

    \section{Introduction}

  \section{Artificial Neural Networks}

   The study of artificial neural networks is motivated by their similarity to successfully working biological systems, which - in comparison to the overall system - consist of very simple but numerous nerve cells that work massively in parallel and (which is probably one of the most significant
   aspects) have the capability to learn.\\
   There is no need to explicitly program a neural network. For instance, it can learn from training samples or by means of encouragement.\footnote{D. Kriesel, A brief introduction to Neural Networks, http://www.dkriesel.com/media/science/neuronalenetze-en-zeta2-2col-dkrieselcom.pdf, p. 4}\\

    \begin{center}
    	\includegraphics[width=0.4\textwidth]{Bild3}
    \end{center}

   They consist of an input layer, multiple hidden layers, and an output layer. Every node in one layer is connected to every other node in the next layer. We make the network deeper by increasing the number of hidden layers.\footnote{https://medium.com/towards-data-science/applied-deep-learning-part-1-artificial-neural-networks-d7834f67a4f6}\\

   Technically a neural network consists of simple processing units, the
   neurons, and directed, weighted connections between those neurons. The strength of a connection (or the connecting weight ) between two neurons
   i and j is referred to as \(w_{i, j}\).\\
   So the weights can be implemented in a square weight matrix W or, optionally, in a weight vector W. Data are transferred between neurons via
   connections with the connecting weight being either excitatory or inhibitory.\\
   For a neuron j the propagation function receives the outputs of the other neurons, which are connected to j, and transforms them in consideration of the connecting weights \(w_{i, j}\) into the network input that can be further processed by the activation function. Here the weighted sum is very popular: The multiplication of the output of each neuron i by \(w_{i, j}\), and the summation of the results.\\
   Based on the model of nature every neuron is, to a certain extent, at all times active. The activation state \(a_j\) from a neuron j indicates the extent of the neuron’s activity and results from the activation function.\footnote{D. Kriesel, A brief introduction to Neural Networks, p. 33 ff.} Typically it can take any value between 0 and 1 or between -1 and 1.\footnote{http://neuralnetworksanddeeplearning.com/chap1.html} \\
   Another important parameter is the threshold value, which marks the position of the maximum gradient value of the activation function.\footnote{D. Kriesel, A brief introduction to Neural Networks, p. 36 ff.}. \\

   Lets take a formal look at the activation function:\\

   \[ a^i_j = \sigma (\sum_k (w_{jk}^i * a^{i-1}_k) + b^i_j) \]

   where\\
   \( \sigma \) is the activation function \\
   \( a^i_j \) represents the activation value of the j-th neuron in the i-th layer\\
   \( w_{jk}^i \)  is the weight from the k-th neuron in the i-1-th layer to the j-th neuron in the i-th layer \\
   \( b^i_j\) is the bias of the jth neuron in the ith layer, where bias = threshold * -1\\

   There is a couple of different activation functions. Each of it has its own strengths and weaknesses. When you know the function you are trying to approximate has certain characteristics, you can choose an activation function which will approximate the function faster leading to faster training process. Otherwise it may be usefull to try out to try out some.\\

   Popular activation functions are:
   \begin{itemize}
   	\item  the sigmoid or logit function\footnote{http://neuralnetworksanddeeplearning.com/chap1.html}:

  \[ \sigma(z) = \frac{1}{1+exp(-z)} \]

  \item the relu function:\footnote{https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0}

  \[ \sigma(z) = max(z,0)\]

  \item the tanh \footnote{https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0}

  \[ \sigma(z) = tanh(z) = \frac{2}{1+exp(-2z)} \]

  \item the cos function

  \[ \sigma(z) = cos(z)\]

  \end{itemize}

The  next important feature is the learning algorithm that is used to change and thereby train the neural network, so that the network produces a desired output for a given input.\footnote{D. Kriesel, A brief introduction to Neural Networks, p. 39}. \\

We will work with back propagation. To implement this, we need a differentiable activation function and an error function Err, which receives all n weights as arguments and assigning them to the output error, i.e.being
n -dimensional. Backpropagation is a gradient descent procedure. On Err a point of small error or even a point of the smallest error is sought by means of the gradient descent. Thus backpropagation trains the weights of the neural network.\footnote{D. Kriesel, A brief introduction to Neural Networks, p. 86}. \\

   \section{Autoencoders}

   An Autoencoder is a specific type of a neural network which is trying to learn an approximation to the identity function, so as to give an output that is similar to the input. The identity function seems a particularly trivial function to be trying to learn; but by placing constraints on the network, such as by limiting the number of hidden units, we can discover interesting structure about the data.\footnote{http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/}

   It consists of three components: encoder, code and decoder. The encoder compresses the input and produces the code, the decoder then reconstructs the input only using this code.\\

   \begin{center}
   	\includegraphics[width=0.4\textwidth]{Bild1}
   \end{center}

   Autoencoders are mainly a dimensionality reduction (or compression) algorithm with a couple of important properties:

    \begin{itemize}

    \item Data-specific: Autoencoders are only able to meaningfully compress data similar to what they have been trained on. Since they learn features specific for the given training data, they are different than a standard data compression algorithm like gzip. For example, we can’t expect an autoencoder trained on handwritten digits to compress landscape photos.

    \item Lossy: The output of the autoencoder will not be exactly the same as the input, it will be a close but degraded representation. If you want lossless compression they are not the way to go.

   \item Unsupervised: Unsupervised learning means that the training set only consists of input patterns, the network tries by itself to detect similarities and to generate pattern classes.\footnote{D. Kriesel, A brief introduction to Neural Networks, p. 52}\\
   Autoencoders are considered an unsupervised learning technique since they don’t need explicit labels to train on. But to be more precise they are self-supervised because they generate their own labels from the training data.

  \end{itemize}\footnote{https://medium.com/towards-data-science/applied-deep-learning-part-3-autoencoders-1c083af4d798}

  To build an autoencoder three things are needed: an encoding method, decoding method, and a loss function to measure the information loss between the compressed representation of your data and the decompressed representation.\footnote{https://medium.com/towards-data-science/applied-deep-learning-part-3-autoencoders-1c083af4d798}\\

  An autoencoder can be visualized as follows:\\

  \begin{center}
  	\includegraphics[width=0.4\textwidth]{Bild2}
  \end{center}

   First the input passes through the encoder, which is an artificial neural network, to produce the code. The decoder, which has the similar artificial neural network structure, then produces the output only using the code. The goal is to get an output identical with the input. The only requirement is that the dimensionality of the input and output needs to be the same. Anything in the middle can be played with.\\

   Four hyperparameters need to be set before training an autoencoder:

    \begin{itemize}

    \item Code size: number of nodes in the middle layer. Smaller size results in more compression.

    \item Number of layers: the autoencoder can be as deep as we like. In the figure above we have 2 layers in both the encoder and decoder, without considering the input and output.

    \item Number of nodes per layer: we work here with stacked autoencoders, so called because the layers are stacked one after another. The number of nodes per layer decreases with each subsequent layer of the encoder, and increases back in the decoder. Also the decoder is symmetric to the encoder in terms of layer structure.

    \item Loss function: for example we can use mean squared error (mse), but of course other functions are possible too.
    \end{itemize}

  Autoencoders then can then be trained in the same way as artificial neuronal networks.\footnote{https://medium.com/towards-data-science/applied-deep-learning-part-3-autoencoders-1c083af4d798}\\

  Two interesting versions of autoencoders are:

   \begin{itemize}
   	\item Variational Autoencoders: VAEs put a probabilistic spin on the basic autoencoder paradigm, treating their inputs, hidden representations, and reconstructed outputs as probabilistic random variables within a directed graphical model. With this Bayesian perspective, the encoder becomes a variational inference network, mapping observed inputs to (approximate) posterior distributions over latent space, and the decoder becomes a generative network, capable of mapping arbitrary latent coordinates back to distributions over the original data space.\footnote{http://blog.fastforwardlabs.com/2016/08/22/under-the-hood-of-the-variational-autoencoder-in.html}\

   	\begin{center}
   		\includegraphics[width=0.4\textwidth]{Bild4}
   	\end{center}

   	\item Generative Adversarial Networks (GANs): there are two main components of a GAN – Generator Neural Network and Discriminator Neural Network. The Generator Network takes an random input and tries to generate a sample of data. It then generates a data which is then fed into a discriminator network. The task of Discriminator Network is to take input either from the real data or from the generator and try to predict whether the input is real or generated. Now the training of GAN is done as a fight between generator and discriminator.\footnote{https://www.analyticsvidhya.com/blog/2017/06/introductory-generative-adversarial-networks-gans/}

\end{itemize}

\end{document}


\documentclass[../main.tex]{subfiles}
\begin{document}
\subsection{Cryptomorphism: Greedy}
The following theorem appears in Oxley's text\cite{ox_book} which gives us another cryptomorphism of matroids. From this proof we can see that the greedy algorithm is not optimal for hereditary systems unless that system is also a matroid.
\begin{thm}
Let $\mathcal{I}$ be a collection of subsets of a set $E.$ Then $(E,\mathcal{I})$ is a matroid if and only if $\mathcal{I}$ satisfies the following conditions:\\
$(I1)$ $\emptyset \in \mathcal{I}$\\
$(I2)$ If $I \in \mathcal{I}$ and $I' \subset I$ then $I' \in \mathcal{I}$\\
$(G)$ For all weight functions $\omega:E \longrightarrow \mathbb{R^+},$ the greedy algorithm produces a maximal member of $\mathcal{I}$ of maximum weight.
\end{thm}
\begin{proof}
Suppose $(E,\mathcal{I})$ is a matroid. Then $\emptyset \in \mathcal{I}$ and (I2) holds trivially.
And by \textit{theorem 5.2} we know that the greedy algorithm can find a maximal member $B \in \mathcal{B}$ of maximum weight if $(E,\mathcal{I})$ is a matroid.\\
Conversely, suppose $(E,\mathcal{I})$ is a pair satisfying $(I1),(I2)$ and $(G).$ Need to prove $\mathcal{I}$ satisfies $(I3)$ in order to have a matroid.\\
Suppose that (seeking a contradiction), if $I_1,I_2 \in \mathcal{I}$ with $|I_2|>|I_1|$ where there does not exist an $e \in I_2 \setminus I_1$ such that $I_1 \cup \{e\} \in \mathcal{I}.$\\
Now, $|I_1 \setminus I_2| < |I_2 \setminus I_1|$ and $I_1 \setminus I_2$ is non-empty.\\
So we can choose an $\epsilon>0$ such that
\begin{equation}
0 < (1+\epsilon)(|I_1 \setminus I_2|) < |I_2 \setminus I_1|
\end{equation}
Define $\omega:E \longrightarrow \mathbb{R^+}$ by:
\[
\omega(e) = \begin{cases}
               2 ,$ if $ e \in I_1 \cap I_2 \\
               \frac{1}{|I_1 \setminus I_2|} ,$ if $ e \in I_1 \setminus I_2\\
               \frac{1+\epsilon}{|I_2 \setminus I_1|},$ if $ e \in I_2 \setminus I_1 \\
               0, $ otherwise.$
            \end{cases}
\]\\
We need the greedy algorithm to fail for only one weight function to get our contradiction.
\begin{itemize}
\item The greedy algorithm will choose all the elements of $I_1 \cap I_2$ first as they are the heaviest elements.
\item Then it will choose all the elements of $I_1 \setminus I_2.$
\item By assumption, it cannot then pick any element of $I_2 \setminus I_1.$ Thus the remaining elements of $B_G$ will be in $E \setminus (I_1 \cup I_2).$\\
\end{itemize}
Hence,\[
\omega(B_G) = 2|I_1 \cap I_2| + |I_1 \setminus I_2| \bigg( \frac{1}{|I_1 \setminus I_2|} \bigg) \\
= 2|I_1 \cap I_2| + 1
\]\\
But by (I2), $I_2$ is contained in a maximal member $B_2$ of $\mathcal{I}$ and, $I_2 \subset B_2.$\\
\[
\omega(B_2) \geq \omega(I_2) = 2|I_1 \cap I_2| + |I_2 \setminus I_1| \bigg(\frac{1+\epsilon}{|I_2 \setminus I_1|} \bigg)\\
> 2|I_1 \cap I_2| + 1 = \omega(B_G)
\]\\
$\implies \omega(B_2) > \omega(B_G)$\\
Which means the greedy algorithm does not find a solution to our optimisation problem shown by \textit{theorem 5.2}, so the greedy algorithm fails for this weight function. We have a contradiction.\\
$\implies$ (I3) holds.\\
$\implies (E,\mathcal{I})$ is a matroid.
\end{proof}
\end{document}


% Part: first-order-logic
% Chapter: natural-deduction
% Section: quantifier-rules

\documentclass[../../../include/open-logic-section]{subfiles}

\begin{document}

\olfileid{fol}{ntd}{qrl}

\olsection{Quantifier Rules}

\subsection{Rules for $\lforall$}

\begin{defish}
\AxiomC{$!\Subst{!A}{t}{x}$}
\RightLabel{\Intro{\lforall}}
\UnaryInfC{$\lforall[x][!A]$}
\DisplayProof
\hfill
\AxiomC{$\lforall[x][{!A}]$}
\RightLabel{\Elim{\lforall}}
\UnaryInfC{$\Subst{!A}{t}{x}$}
\DisplayProof
\end{defish}

In the rules for~$\lforall$, $t$ is a closed term (a term that does not
contain any variables), and $a$~is a !!{constant} which does not occur in the
conclusion~$\lforall[x][!A]$, or in any assumption which is !!
{undischarged} in the !!{derivation} ending with the premise~$\Subst{!A}{t}
{x}$. We call $a$ the \emph{eigenvariable} of the
\Intro{\lforall} inference.

\subsection{Rules for $\lexists$}

\begin{defish}
\AxiomC{$\Subst{!A}{t}{x}$}
\RightLabel{\Intro{\lexists}}
\UnaryInfC{$\lexists[x][!A]$}
\DisplayProof
\hfill
\AxiomC{$\lexists[x][!A]$}
\AxiomC{[$\Subst{!A}{a}{x}$]$^n$}
\DeduceC{$!C$}
\DischargeRule{\Elim{\lexists}}{n}
\BinaryInfC{$!C$}
\DisplayProof
\end{defish}

Again, $t$ is a closed term, and $a$ is a constant which does not occur in the
premise $\lexists[x][!A]$, in the conclusion~$!C$, or any assumption which
is !!{undischarged} in the !!{derivation}s ending with the two premises
(other than the assumptions $\Subst{!A}{a}{x}$).  We call $a$ the \emph
{eigenvariable} of the \Elim{\lexists} inference.

The condition that an eigenvariable neither occur in the premises nor in
any assumption that is !!{undischarged} in the !!{derivation}s leading
to the premises for the \Intro{\lforall} or \Elim{\lexists} inference is
called the \emph{eigenvariable condition}.

\begin{explain}
We use the term ``eigenvariable'' even though $a$ in the above rules
is a constant. This has historical reasons.

In \Intro{\lexists} and \Elim{\lforall} there are no restrictions, and the
term~$t$ can be any closed term, so we do not have to worry about any
conditions. On the other hand, in the \Elim{\lexists} and
\Intro{\lforall} rules, the eigenvariable condition requires that the
!!{constant}~$a$ does not occur anywhere in the conclusion or in an
!!{undischarged} assumption. The condition is necessary to ensure that
the system is sound, i.e., only !!{derive}s !!{sentence}s from
!!{undischarged} assumptions from which they follow. Without this
condition, the following would be allowed:
\begin{prooftree}
  \AxiomC{$\lexists[x][!A(x)]$}
  \AxiomC{$\Discharge{!A(a)}{1}$}
  \RightLabel{*\Intro{\lforall}}
  \UnaryInfC{$\lforall[x][!A(x)]$}
  \RightLabel{\Elim{\lexists}}
  \BinaryInfC{$\lforall[x][!A(x)]$}
\end{prooftree}
However, $\lexists[x][!A(x)] \Entails/ \lforall[x][!A(x)]$.

In the above derivation we have used $!A(a)$ as a short hand for $\Subst{!A}
{a}{x}$. This is possible since the notation $\lforall[x][!A(x)]$ in the conclusion of the rule indicates that the variable used for the substitution is $x$.
\end{explain}

\begin{explain}
As the elimination rules for quantifiers only allow substituting closed
terms for variables, it follows that any formula that can be derived from a
set of sentences is itself a sentence.
\end{explain}

\end{document}


\chapter{Higher Order Linear ODE's}
% Constant Coefficients
\input{./higherOrder/constCoeffs/constCoeffs.tex}

% Application: Mechanical Vibrations (Homogenous, Linear)
\input{./higherOrder/freeVibrs/freeVibrs.tex}
	% Simple: mx'' + kx = 0
	% Free Damped: mx'' + by' + ky = 0

% Non-Homogenous Linear DiffEq's
\input{./higherOrder/nonHomeg/nonHomeg.tex}
	% Method of Undetermined Coefficients
	% Variation of Parameters

% Forced Vibrations
\input{./higherOrder/forcedVibrs/forcedVibrs.tex}
	% Undamped Forced
		% 1.1 omega != gamma
		% 1.2 omega = gamma
	% Damped Forced


\clearpage
\subsection{Pascal Types} % (fold)
\label{sub:program-creation-pas_types}

\nameref{sub:type}s are used to define how data is interpreted and the operations that can be performed on the data. Table \ref{tbl:program-creation-pas-types} shows the three basic types of data, the associated Pascal type, size in memory, and other related information. Table \ref{tbl:program-creation-pas operators by type} shows the operators that are permitted for each type.

\begin{table}[h]
\begin{minipage}{\textwidth}
\centering
\begin{tabular}{|l|c|c|c|}
\hline
\multicolumn{4}{|c|}{\textbf{Whole Number Types}} \\
\hline
\emph{Name} & \emph{Size} & \multicolumn{2}{c|}{\emph{Range (lowest .. highest)}} \\
\hline
\texttt{Byte} & 1 bytes/8 bits & \multicolumn{2}{c|}{0 .. 255} \\
\texttt{SmallInt} & 2 bytes/16 bits & \multicolumn{2}{c|}{-32,767 .. 32,767} \\
\texttt{Integer} & 4 bytes/32 bits & \multicolumn{2}{c|}{-2147483648 .. 2147483647} \\
\texttt{Int64}    & 8 bytes/64 bits & \multicolumn{2}{c|}{-9,223,372,036,854,775,807 ..} \\
  & & \multicolumn{2}{c|}{9,223,372,036,854,775,807} \\
\hline
\multicolumn{4}{c}{} \\
\hline
\multicolumn{4}{|c|}{\textbf{Real Number Types}} \\
\hline
\emph{Name} & \emph{Size} & \emph{Range (lowest .. highest)} & \emph{Significant Digits} \\
\hline
\texttt{Single} & 4 bytes/32 bits & 1.0e-38 .. 1.0e38 & 6 \\
\texttt{Double} & 8 bytes/64 bits & 2.0e-308 .. 2.0e308 & 10 \\
\texttt{Extended} & 10 bytes/80 bits & 1.9e-4932 .. 1.1e4932 & 20 \\
\hline
\multicolumn{4}{c}{} \\
\hline
\multicolumn{4}{|c|}{\textbf{Text Types}} \\
\hline
\emph{Name} & \emph{Size} & \multicolumn{2}{c|}{\emph{Notes}} \\
\hline
\texttt{Char}  & 1 byte/8 bits & \multicolumn{2}{c|}{} \\
\hline
\texttt{String} & 256 bytes/2048 bits &  \multicolumn{2}{c|}{Supports up to 255 characters by default,} \\
 & & \multicolumn{2}{c|}{compiler options allow longer strings.} \\
\hline
\end{tabular}
\caption{Pascal Data Types}\label{tbl:program-creation-pas-types}
\end{minipage}
\end{table}

\begin{table}[h]
  \centering
  \begin{tabular}{|c|c|l|}
    \hline
    \textbf{Type} & \textbf{Operations Permitted} & \textbf{Notes}\\
    \hline
    Whole Numbers     &   \texttt{( ) + - div / * mod} & \\
       & & \\
    Real Numbers   &   \texttt{( ) + - / *} &    \\
       & & \\
    Text           &   \texttt{( ) + }          & The \texttt{+} performs concatenation.\\
       & & \\
    \hline
  \end{tabular}
  \caption{Pascal Permitted Operators by Type}
  \label{tbl:program-creation-pas operators by type}
\end{table}

\mynote{
\begin{itemize}
  \item The \texttt{Integer} type is the typical whole number type.
  \item The \texttt{Double} type is the typical real number type.
  \item For example values see Table \vref{tbl:program-creation-pas example expresions}.
\end{itemize}
}
% subsection c_types (end)


\documentclass[11pt]{article}
\usepackage{amsfonts, amsmath, amssymb, amsthm, fullpage, mdwlist, fancyhdr}

\setlength{\textheight}{9.25in}
\setlength{\textwidth}{6.5in}
\setlength{\topmargin}{0.0in}
\setlength{\headheight}{0.0in}
\setlength{\headsep}{0.0in}
\setlength{\leftmargin}{0.0in}
\setlength{\oddsidemargin}{0.0in}
\setlength{\parindent}{0pc}

%- arrange by difficulty
%- include some more trivial exercises

\begin{document}
\title{Quadratic Reciprocity}
\author{Alex Zhu}

\maketitle

\section{Useful Facts and Definitions}

\textbf{Definition 1.} Let $p$ be a prime number. An integer $n$ is called a \emph{quadratic residue} mod $p$ if there is a number $x$ such that $x^2 \equiv n \pmod{p}$.
\\\\
\textbf{Definition 2.} Let $p$ be a prime, and let $a$ be an integer. The \emph{Legendre symbol} is a function of $a$ and $p$, denoted by $\left( \frac{a}{p} \right)$, which equals 1 if $a$ is a nonzero quadratic residue mod $p$, -1 if $a$ is a quadratic nonresidue mod $p$, and 0 if $p$ divides $a$.
\\\\
\textbf{Fact 1.} If $a,b$ are integers and $p$ is a prime, then $\left( \frac{a}{p} \right) \left( \frac{b}{p} \right) = \left( \frac{ab}{p} \right)$.
\\\\
\textbf{Fact 2.} (Euler's criterion) If $a$ is an integer and $p$ is an odd prime, then $a^{\frac{p-1}{2}} \equiv \left( \frac{a}{p} \right) \pmod{p}$.
\\\\
\textbf{Fact 3.} (Eistenstein's lemma) If $p$ and $q$ are distinct odd primes, then \[ \left( \frac{q}{p} \right) = (-1)^{\sum_u \lfloor \frac{qu}{p} \rfloor}, \] where $u$ is summed over the even integers $2, 4, \ldots, p-1$.
\\\\
\textbf{Fact 4.} (Law of quadratic reciprocity) If $p$ and $q$ are distinct odd primes, then \[ \left( \frac{q}{p} \right) \left( \frac{p}{q} \right) = (-1)^{\frac{(p-1)(q-1)}{4}}. \]
\\\\
\textbf{Fact 5.} Let $p$ be an odd prime. Then \[ \left( \frac{-1}{p} \right) = (-1)^{\frac{p-1}{2}} \mbox{ and } \left( \frac{2}{p} \right) = (-1)^{\frac{p^2-1}{8}}. \]

\section{Basic Applications}

\begin{enumerate}
  \item Find the order of 3 mod 107.
  \item Find the order of 10 mod 65537.
  \item Determine whether 2013 is a quadratic residue mod 3001.
\end{enumerate}

\section{Problems}

\begin{enumerate}
  \item Partition the $p-1$ nonzero residue classes mod $p$ into sets of the form $\{x, x^{-1}, -x, -x^{-1} \}$. Conclude that $-1$ is a quadratic residue mod 4 if and only if $p \equiv 1 \pmod{4}$.

  \item Let $p$ be a prime. Prove that if $p = 2q + 1$ where $q$ is a prime congruent to 1 mod 4, then 2 is a primitive root mod $p$.

  \item Let $F_n$ be the $n$th Fibonacci number, where $F_0 = 0$, $F_1 = 1$, and $F_{k+1} = F_k + F_{k-1}$ for all $k \geq 1$. Let $p$ be a prime number. Show that:
  \begin{enumerate}
    \item If $p \equiv 1, 4 \pmod{5}$, then $F_p \equiv 1 \pmod{p}$.
    \item If $p \equiv 2, 3 \pmod{5}$, then $F_p \equiv -1 \pmod{p}$.
  \end{enumerate}

  \item Let $m$ and $n$ be positive integers such that $\sqrt{23} > \frac{m}{n}$. Prove that $\sqrt{23} > \frac{m}{n} + \frac{3}{mn}$.

  \item (Bulgaria 1998) Let $m$ and $n$ be natural numbers such that \[ A = \frac{(m+3)^n + 1}{3m} \] is an integer. Prove that $A$ is odd.

  \item (ELMO 2011) Let $p>13$ be a prime of the form $2q+1$, where $q$ is prime. Find the number of ordered pairs of integers $(m,n)$ such that $0\le m<n<p-1$ and
\[3^m+(-12)^m\equiv 3^n+(-12)^n\pmod{p}.\]

  \item Let $a$ be a positive integer, and suppose that $a$ is a quadratic residue modulo all sufficiently large primes. Prove that $a$ is a perfect square.

  \item (APMO 1997) Find an integer $n$, where $100 \leq n \leq 1997$, such that \[ \frac{2^n + 2}{n} \] is also an integer.

  \item Let $a, b, c$ be integers and let $p$ be an odd prime with \[p \not\vert a \;\; \text{and}\;\; p \not\vert b^{2}-4ac.\] Show that \[\sum_{k=1}^{p}\left( \frac{ak^{2}+bk+c}{p}\right) = -\left(\frac{a}{p}\right).\]

  \item (Taiwan 2000) Suppose that for some $m,n \in \mathbb{N}$, we have $\varphi(5^m - 1) = 5^n - 1$, where $\varphi$ denotes the Euler function. Show that $\gcd(m,n) > 1$.

  \item Find all positive integers $n$ such that $2^n - 1$ divides $3^n - 1$.

  \item Does there exist a quadratic polynomial with integer coefficients that is irreducible over $\mathbb{Z}$ but reducible mod $p$ for any prime $p$? What about a polynomial of higher degree?
\end{enumerate}

\end{document}

%  \item Find the number of pairs of positive integers $(a,b)$ with $0 \leq a,b < p$ and $a^2 + b^2 \equiv 1 \pmod{p}$.

%  \item Let $p$ be an odd prime number. Show that the smallest positive quadratic nonresidue of $p$ is smaller than $\sqrt{p}+1$.

%  \item Let $p$ be an odd prime. Prove that there is an integer $x$ such that $p | x^4 + 1$ if and only if $p \equiv 1 \pmod{8}$.

%  \item We will outline here a proof of quadratic reciprocity, due to V.A. Lebesgue. Suppose $p$ and $q$ are odd primes, and let $N$ count the number of solutions to $x_1^2 + x_2^2 + \cdots + x_p^2 \equiv 1 \pmod{q}$.
%  \begin{enumerate}
%    \item Show that the number of solutions to $x^2 \equiv a \pmod{q}$ is $1 + \left( \frac{a}{q} \right)$.
%    \item By performing cyclic shifts on indices (i.e., grouping together solutions of the form $(x_1, x_2, \ldots)$ with $(x_i, x_{i+1}, \ldots)$, etc.), prove that $N \equiv 1 + \left( \frac{p}{q} \right) \pmod{q}$.
%    \item Show that
%    \[ N = \sum_{t_1 + t_2 + \cdots + t_p \equiv 1 \pmod{q}} \left( 1 + \left( \frac{t_1}{q} \right) \right) \left( 1 + \left( \frac{t_1}{q} \right) \right) \cdots \left( 1 + \left( \frac{t_p}{q} \right) \right) \pmod{q} \]
%    \item Show that, when expanded, all the terms in the above sum vanish except \[ 1 + \sum_{t_1 + t_2 + \cdots + t_p \equiv 1 \pmod{q}} \left( \frac{t_1 t_2 \cdots t_p}{q} \right), \] and show that the latter sum evaluates to mod $q$. Deduce the law of quadratic reciprocity.
%  \end{enumerate}

\section{Quadratic Residues}

Let $p$ be a prime number. An integer $n$ is called a \emph{quadratic residue} mod $p$ if there is a number $x$ such that $x^2 \equiv n \pmod{p}$.
\begin{enumerate}
  \item How many quadratic residues are there mod $p$?
  \item Prove that the product of any two quadratic residues is a quadratic residue.
  \item Prove that the product of a quadratic residue and a quadratic nonresidue is a quadratic nonresidue.
  \item With a counting argument, show that the product of a quadratic nonresidue and a quadratic nonresidue is a quadratic residue.
  \item If $p > 2$, prove that -1 is a quadratic residue mod $p$ if and only if $p \equiv 1 \pmod{4}$.
\end{enumerate}

\section{Legendre Symbol and Euler Criterion}

Let $p$ be a prime, and let $a$ be an integer. The \emph{Legendre symbol} is a function of $a$ and $p$, denoted by $\left( \frac{a}{p} \right)$, which equals 1 if $a$ is a nonzero quadratic residue mod $p$, -1 if $a$ is a quadratic nonresidue mod $p$, and 0 if $p$ divides $a$.

\begin{enumerate}
  \item Prove that if $a$ and $b$ are integers and $p$ is a prime, then $\left( \frac{a}{p} \right) \left( \frac{b}{p} \right) = \left( \frac{ab}{p} \right)$.
  \item (Euler's criterion) Prove that $a^{\frac{p-1}{2}} \equiv \left( \frac{a}{p} \right) \pmod{p}$.
  \item Using Euler's criterion, give one-line proofs of every problem in the previous section.
\end{enumerate}


\subsubsection{\tt ipopne}
\label{benchmarkipopne}

{\tt ipopne} is a synthetic AKS benchmark that performs
a character-character matrix product({\apl M\qor\qdot\qne\qtran\0M}),
on arrays of shape {\tt [ 2000, 2000]}.

This benchmark is essentially a variant of the {\tt ipape} benchmark,
so they should both exhibit similar performance characteristics.

We expect \wlf and \awlf to perform equally well on this
benchmark.


\section{Amortized Analysis}

%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Amortized analysis}
  \begin{quote}
	Amortized analysis is \\
	\textcolor{purple}{an algorithm analysis technique} for \\
    \textcolor{red}{analyzing a sequence of operations} \\
	\textcolor{blue}{irrespective of the input} to show that \\
    \textcolor{cyan}{the average cost per operation} is small, even though \\
    \textcolor{teal}{a single operation within the sequence might be expensive}.
  \end{quote}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Methods for amortized analysis: the summation method}
  \begin{gather*}
	o_1, o_2, \ldots, o_n \\[5pt]
	c_1, c_2, \ldots, c_n
  \end{gather*}

  \pause
  \[
	(\sum_{i = 1}^{n} c_i) / n
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Summation method: array doubling revisited}
  \centerline{On any sequence of $n$ \textsc{Insert} ops on an initially empty array.}

  \pause
  \[
    \begin{array}{ccccccccccc}
      o_i: 	& 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 \\
      c_i:  & 1 & 2 & 3 & 1 & 5 & 1 & 1 & 1 & 8 & 1  \\
    \end{array}
  \]

  \pause
  \begin{displaymath}
	c_i = \left\{ \begin{array}{ll}
	  (i-1)+1 = i & \textrm{if $i - 1$ is an exact power of 2}\\
	  1 & \textrm{o.w.}
	\end{array} \right.
  \end{displaymath}

  \pause
  \[
	\sum_{i=1}^{n} c_i \le n + \sum_{j=0}^{\lceil \lg n \rceil - 1} 2^{j} = n +
	(2^{\lceil \lg n \rceil} - 1) \le n + 2n = 3n
  \]

  \pause
  \[
	\forall i, \hat{c_i} = 3
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Methods for amortized analysis: the accounting method}
  \begin{gather*}
	o_1, o_2, \ldots, o_n \\[5pt]
	c_1, c_2, \ldots, c_n \\[5pt]
	a_1, a_2, \ldots, a_n
  \end{gather*}

  \pause
  \[
    \hat{c_i} = c_i + a_i, a_i >=< 0.
  \]

  \pause
  \[
	\forall n, \sum_{i=1}^{n} c_i \le \sum_{i=1}^{n} \hat{c_i} \pause \implies \forall n, \sum_{i=1}^{n} a_i \geq 0
  \]

  \pause
  \begin{alertblock}{Key way of thinking:}
	Put the accounting cost on specific objects.
  \end{alertblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Accounting method: array doubling revisited}
  \[
    \hat{c_i} = 3 \text{\emph{ vs. }} \hat{c_i} = 2
  \]

  \pause
  \[
	\hat{c_i} = 3 = \underbrace{1}_{\textrm{insert}} +
	\underbrace{1}_{\textrm{move itself}} + \underbrace{1}_{\textrm{help move another}}
  \]

  \pause
  \begin{table}
    \begin{tabular}{c|ccc}
	  & $\hat{c_i}$ & $c_i$ (actual cost) & $a_i$ (accounting cost)
	  \\ \hline
	  \textsc{Insert} (normal) & $3$ & $1$ & $2$\\
	  \textsc{Insert} (expansion) & $3$ & $1 + t$ & $-t + 2$
    \end{tabular}
  \end{table}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Array merging (Problem 4.13): the summation method}
  \centerline{\textsc{Create} ($1$); \textsc{Merge} ($2m$)}

  \begin{columns}
	\column{0.40\textwidth}
	  \pause
	  \begin{align*}
		i & \quad c_i \\
		1 & \quad 1  \\
		2 & \quad 1 + 2 \\
		3 & \quad 1 \\
		4 & \quad 1 + 2 + 4 \\
		5 & \quad 1 \\
		6 & \quad 1 + 2 \\
		7 & \quad 1 \\
		8 & \quad 1 + 2 + 4 \\
		\vdots & \quad \cdots
	  \end{align*}
	\column{0.55\textwidth}
	  \pause
	  \[
		\sum_{i=1}^{n} c_i = \sum_{j=0}^{\lfloor \log n \rfloor} \lfloor \frac{n}{2^j} \rfloor 2^j \le n (\lfloor \log n \rfloor + 1)
	  \]

	  \pause
	  \[
		\forall i, \hat{c_i} = 1 + \lfloor \log n \rfloor
	  \]
  \end{columns}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Array merging (Problem 4.13): the accounting method}
  \[
	\hat{c_i} = 1 + \lfloor \log n \rfloor
  \]

  \pause
  \vspace{0.60cm}
  \centerline{What does it mean?}

  \pause
  \vspace{0.80cm}
  \[
	\forall n, \sum_{i=1}^{n} a_i \geq 0
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Two stacks, one queue (Problem 4.14)}
  \begin{algorithm}[H]
    \caption{Simulating a queue using two stacks $S_1, S_2$.}
    \begin{algorithmic}[]
      \Procedure{Enq}{$x$}
		\State \textsl{Push}($S_1, x$)
      \EndProcedure

	  \Statex

      \Procedure{Deq}{\null}
        \If{$S_2 = \emptyset$}
          \While{$S_1 \neq \emptyset$}
            \State \textsl{Push}($S_2$, \textsl{Pop}($S_1$))
          \EndWhile
        \EndIf
        \textsl{Pop}($S_2$)
      \EndProcedure
    \end{algorithmic}
  \end{algorithm}
\end{frame}
%%%%%%%%%%%%%
\begin{frame}{Two stacks, one queue: the summation method}
  \[
	(\sum_{i = 1}^{n} c_i) / n
  \]

  \pause
  \centerline{The operation sequence is NOT known.}
\end{frame}
%%%%%%%%%%%%%
\begin{frame}{Two stacks, one queue: the accounting method}
  \begin{table}
    \begin{tabular}{ccccc}
	  item: & Push into $S_1$ & Pop from $S_1$ & Push into $S_2$ & Pop from $S_2$ \\
	  & $1$ & $1$ & $1$ & $1$
    \end{tabular}
  \end{table}

  % $\#S_1 = t:$
  % \begin{table}
  %   \begin{tabular}{c|ccc}
  %     & $\hat{c_i}$ & $c_i$ (actual cost) & $a_i$ (accounting cost)
  %     \\
  %     \textsc{Enqueue} & 3 & 1 & 2 \\
  %     \textsc{Dequeue} ($S_2 = \emptyset$) & 1 & 1 & 0 \\
  %     \textsc{Dequeue} ($S_2 \neq \emptyset$) & 1 & 1+2t & -2t
  %   \end{tabular}
  % \end{table}

  \pause
  \begin{align*}
	\hat{c}_{\textsc{Enq}} &= 3 \\
	\hat{c}_{\textsc{Deq}} &= 1 \\
  \end{align*}

  \pause
  \[
    \sum_{i=1}^{n} a_i \ge 0 \pause \Longleftarrow \sum_{i=1}^{n} a_i = \#S_1 \times 2
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}[noframenumbering]
  \fignocaption{width = 0.50\textwidth}{figs/thankyou.jpg}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}[noframenumbering]
  \begin{center}
  {\Large 写清楚算法原理，不要只写代码} \\[15pt]
  {\Large 写清楚算法原理，不要只写代码} \\[15pt]
  {\Large 写清楚算法原理，不要只写代码}
  \end{center}
\end{frame}
%%%%%%%%%%%%%%%%%%%%


\chapter{Lecture 32 Jul 18th 2018}%
\label{chp:lecture_32_jul_18th_2018}
% chapter lecture_32_jul_18th_2018

\section{Factorizations in Integral Domains (Continued 2)}%
\label{sec:factorizations_in_integral_domains_continued_2}
% section factorizations_in_integral_domains_continued_2

\subsection{Ascending Chain Condition (Continued)}%
\label{sub:ascending_chain_condition_continued}
% subsection ascending_chain_condition_continued

\begin{thm}[Integral Domain that Satisfies ACCP has a Polynomial Ring that Satisfies ACCP]
\label{thm:integral_domain_that_satisfies_accp_has_a_polynomial_ring_that_satisfies_accp}
  If $R$ is an integral domain satisfying ACCP, so does $R[x]$.
\end{thm}

\begin{proof}
  Suppose not, i.e. $R[x]$ does not satisfy ACCP. Then there exists a chain of principal ideals such that
  \begin{equation}\label{eq:thm95_eq1}
    \lra{f_1} \subsetneq \lra{f_2} \subsetneq \lra{f_3} \subsetneq \hdots \quad \text{in } R[x].
  \end{equation}
  Let $a_i$ be the leading coefficient of $f_i$. Note that $a_i \in R$. From \cref{eq:thm95_eq1}, we have that $f_{i + 1} \, | \, f_i$, and so we must have $a_{i + 1} \, | \, a_i$. Then
  \begin{equation*}
    \lra{a_1} \subseteq lra{a_2} \subseteq \lra{a_3} \subseteq \hdots \;.
  \end{equation*}
  Since $R$ satisfies ACCP, $\exists n \in \mathbb{N}$ such that
  \begin{equation*}
    \lra{a_n} = \lra{a_{n + 1}} = \hdots \;,
  \end{equation*}
  i.e. $a_n \sim a_{n + 1} \sim \hdots$. For $m \geq n$, let $f_m = g f_{m + 1}$ for some $g(x) \in R[x]$. If $b \in R$ is the leading coefficient of $g(x)$, then $a_m = b a_{m + 1}$. Since $a_m \sim a_{m + 1}$. $b$ is a unit in $R$. However, $g(x)$ is not a unit in $R[x]$ since $\lra{f_m} \subsetneq \lra{f_{m + 1}}$. Thus $g(x) \neq b$, which implies $\deg g \geq 1$. Then by \cref{propo:polynomial_ring_is_an_integral_domain},
  \begin{equation*}
    \deg f_m > \deg f_{m + 1},
  \end{equation*}
  which is true for $m \geq n$. By the same argument, we have that
  \begin{equation*}
    \deg f_m > \deg f_{m + 1} > \deg f_{m + 2} > \hdots \;,
  \end{equation*}
  which leads to a contradiction since $\deg f_i \geq 0$ for all $i \in \mathbb{N}$. Thus $R[x]$ must satisfy ACCP.\qed
\end{proof}

\begin{eg}
  Since $\mathbb{Z}$ satisfies ACCP, by \cref{thm:integral_domain_that_satisfies_accp_has_a_polynomial_ring_that_satisfies_accp}, $\mathbb{Z}[x]$ also satisfies ACCP.
\end{eg}

% subsection ascending_chain_condition_continued (end)

\subsection{Unique Factorization Domains and Principal Ideal Domains}%
\label{sub:unique_factorization_domains_and_principal_ideal_domains}
% subsection unique_factorization_domains_and_principal_ideal_domains

\begin{defn}[Unique Factorization Domain (UFD)]\index{Unique Factorization Domain}\index{UFD}
\label{defn:unique_factorization_domain}
  An integral domain $R$ is called a \hlnoteb{unique factorization domain} (UFD) if it satisfies the following conditions:
  \begin{enumerate}
    \item If $0 \neq a \in R$ is a non-unit, then $a$ is a product of irreducible elements in $R$.
    \item If $p_1 p_2 \hdots p_r \sim q_1 q_2 \hdots q_s$ where $p_i$ and $q_i$ are irreducibles, then $r = s$ and (possibly after relabelling) $p_i \sim q_i$ for each $1 \leq i \leq r = s$.
  \end{enumerate}
\end{defn}

\begin{eg}
  Both $\mathbb{Z}$ and $F[x]$, where $F$ is a field, are UFDs.
\end{eg}

\begin{eg}
  Any field is a UFD since all elements in a field are either $0$ or units.
\end{eg}

Recall \cref{propo:primes_are_irreducible}: If $p$ is a prime, then $p$ is irreducible. In comparison, we have the following:

\begin{propo}[Irreducibles are Primes in a UFD]
\label{propo:irreducibles_are_primes_in_a_ufd}
  Let $R$ be a UFD and $p \in R$. If $p$ is irreducible, then $p$ is a prime.\marginnote{This also means that in a UFD, primes and irreducibles are the same.}
\end{propo}

\begin{proof}
  Let $p \in R$ be an irreducible. If $p \, | \, ab \in R$, then $\exists d \in R$ such that $ab = pd$. Since $R$ is a UFD, we can factor $a, b, $ and $d$ into irreducible elements, say
  \begin{align*}
    a &= p_1 p_2 \hdots p_k \\
    b &= q_1 q_2 \hdots q_l \\
    d &= r_1 r_2 \hdots r_m.
  \end{align*}
  where $k, l, m \in \mathbb{N} \cup \{0\}$. Then
  \begin{equation*}
    ab = pd \iff p_1 \hdots p_k q_1 \hdots q_l = p r_1 \hdots r_m.
  \end{equation*}
  Since $p$ is irreducible, by \cref{propo:properties_of_irreducibles}, $p \sim p_i$ or $p \sim q_i$. Therefore $p \, | \, a$ or $p \, | \, b$, which is the definition of a prime.\qed
\end{proof}

\begin{eg}
  Consider $R = \mathbb{Z}[\sqrt{-5}]$ and $p = 1 + \sqrt{-5}$. We proved that $p$ is irreducible but $p$ is not prime. Then by \cref{propo:irreducibles_are_primes_in_a_ufd}, we have that $\mathbb{Z}[\sqrt{-5}]$ is not a UFD.
\end{eg}

\begin{defn}[Greatest Common Divisor]\index{Greatest Common Divisor}
\label{defn:greatest_common_divisor}
  Let $R$ be an integral domain, and $a, b \in R$. We say $d \in R$ is the \hlnoteb{greatest common divisor} of $a, b$, denoted as $\gcd(a, b) = d$, if it satisfies the following conditions:
  \begin{enumerate}
    \item $d \, | \, a$ and $d \, | \, b$;
    \item $e \in R \; e \, | \, a \, \land \, e \, | \, b \implies e \, | \, d$.
  \end{enumerate}
\end{defn}

\begin{propo}
\label{propo:associates_of_the_gcd}
Let $R$ be a UFD and $a, \, b \in R$. If $p_1, ..., p_k$ are the non-associated primes dividing $a$ and $b$, say\marginnote{
\begin{ex}
  Prove \cref{propo:associates_of_the_gcd}.
\end{ex}}
  \begin{gather*}
    a \sim p_1^{a_1} \hdots p_k^{a_k} \\
    b \sim p_1^{b_1} \hdots b_k^{b_k}
  \end{gather*}
  with $a_i, \, b_i \in \mathbb{N}$, then
  \begin{equation*}
    \gcd(a, b) \sim p_1^{\min(a_1, b_1)} \hdots p_k^{\min(a_k, b_k)}
  \end{equation*}
\end{propo}

\begin{proof}
  Let $d = \gcd(a, b)$. It suffices to show that
  \begin{equation*}
    d \, | \, p_1^{\min(a_1, b_1)} \hdots p_k^{\min(a_k, b_k)},
  \end{equation*}
  since $p_1^{\min(a_q, b_1)} \hdots p_k^{\min(a_k, b_k)}$ divides $a$ and $b$ and so it must also divide $d$.

  Suppose that $d \nmid p_1^{\min(a_1, b_1)} \hdots p_k^{\min(a_k, b_k)}$. Then $d \not\sim p_i^{\min(a_i, b_i)}$ for $1 \leq i \leq k$. But that implies that $d = 1$, otherwise $d \nmid \, a$ and $d \nmid b$. However,
  \begin{equation*}
    p_1^{\min(a_1, b_1)} \hdots p_k^{\min(a_k, b_k)} \, \nmid 1
  \end{equation*}
  which contradicts the choice of $d$ as the greatest common divisor.\qed
\end{proof}

\begin{note}
  If $R$ is a UFD with $d, a_1, ..., a_m \in R$, then
  \begin{equation*}
    \gcd(da_1, da_2, ..., da_m) \sim d \gcd(a_1, ..., a_m).
  \end{equation*}
\end{note}

\begin{thm}[UFD and ACCP]
\label{thm:ufd_and_accp}
  Let $R$ be an integral domain. TFAE:
  \begin{enumerate}
    \item $R$ is a UFD;
    \item $R$ satisfies ACCP and $\forall a, \, b \in R$, $\exists d = gcd(a, b) \in R$;
    \item $R$ satsifies ACCP and every irreducible element in $R$ is a prime.
  \end{enumerate}
\end{thm}

\begin{proof}
  $(1) \implies (2)$: By \cref{propo:associates_of_the_gcd}, $\forall a, \, b \in R \quad \exists d = \gcd(a, b) \in R$. Suppose there exists
  \begin{equation*}
    0 \neq \lra{a_1} \subsetneq \lra{a_2} \subsetneq \lra{a_3}\ subsetneq \hdots \; \text{in } R.
  \end{equation*}
  Since $\lra{a_1} \neq R$, $a_1$ is not a unit\sidenote{Otherwise, $1 \in \lra{a_1} \implies \lra{a_1} = R$.} Since $R$ is a UFD, let $a_1 \sim p_1^{k_1} \hdots p_r^{k_r}$, where the $p_i$'s are non-associated primes and $k_i \in \mathbb{N}$, for $1 \leq i \leq r$. Since $a_i \mid a_1$ for $2 \leq i \leq r$, we have that
  \begin{equation*}
    a_i \sim p_1^{d_{i, 1}} p_2^{d_{i, 2}} \hdots p_r^{d_{i, r}}
  \end{equation*}
  where $0 \leq d_{i, j} \leq k_j$ for $1 \leq j \leq r$. This implies that there are only finitely many non-associated choices for $a_i$, which implies that there exists $m \neq n$ such that $a_m \sim a_n \implies \lra{a_m} = \lra{a_n}$, a contradiction. Therefore, $R$ must satisfy ACCP.

  \noindent $(2) \implies (3)$: Let $p \in R$ be an irreducible, and suppose $p \, | \, ab$. By $(2)$, let $d = \gcd(a, p)$. Then $d \mid p$, and by \cref{propo:properties_of_irreducibles}, we have either $p \sim 1$ or $d \sim p$ since $p$ is an irreducible. If $d \sim p$, since $d \mid 1$, we have that $p \mid 1$. If $d \sim 1$, note that we have that
  \begin{equation*}
    \gcd(ab, pb) \sim b \gcd(a, p) \sim b.
  \end{equation*}
  Since $p \mid ab$ and $p \mid pb$, we have $p \mid \gcd(ab, pb)$ and so $p \mid b$.

  \noindent: $(3) \implies (1)$: $R$ satisfies ACCP implies, by \cref{propo:irreducibles_are_primes_in_a_ufd}, every non-unit non-zero $a \in R$ is a product of irreducible elements in $R$. It sufficies to prove that the factorization is unique\sidenote{This would ssatisfy the definition of a UFD.}. Suppose we have
  \begin{equation*}
    p_1 p_2 \hdots p_r \sim q_1 q_2 \hdots q_s
  \end{equation*}
  where $p_i$ and $q_j$ are irreducibles, for $1 \leq i \leq r$ and $1 \leq j \leq s$. Now $p_1 \mid p_1 p_2 \hdots p_r$, and so $p_1 \mid q_1 q_2 \hdots q_s$. By \cref{propo:properties_of_irreducibles} and since $p_1$ is an irreducible, $p_1 \sim q_j$ for some $1 \leq j \leq s$. We may relabel this $q_j$ to be $q_1$. Now since $p_1 \sim q_1$ and $p_1 p_2 \hdots p_r \sim q_1 q_2 \sim q_s$, $\exists a, \, b \in R$ that are units suchthat
  \begin{gather*}
    ap_1 = q_1 \; \text{ and } \; p_1 p_2 \hdots p_r = b q_1 q_2 \hdots q_s = bap_1 q_2 \hdots q_s \\
    \implies p_2 \hdots p_r = ba q_2 \hdots q_s \implies p_2 \hdots p_r \sim q_2 \hdots q_s.
  \end{gather*}
  By repeating the same argument, we have that $r = s$ and $p_i \sim q_i$ for $1 \leq i \leq r$. Therefore the factorization is unique.\qed
\end{proof}

\begin{defn}[Principal Ideal Domain (PID)]\index{Principal Ideal Domain}\index{PID}
\label{defn:principal_ideal_domain}
  An integral domain $R$ is a \hlnoteb{principal ideal domain} (PID) if every ideal is principal.
\end{defn}

\begin{eg}
  A field $F$ is a PID since its only ideals are $\{0\} = \lra{0}$ and $F = \lra{1}$.
\end{eg}

\begin{eg}
  $\mathbb{Z}$ and $F[x]$ are PIDs.
\end{eg}

% subsection unique_factorization_domains_and_principal_ideal_domains (end)

% section factorizations_in_integral_domains_continued_2 (end)

% chapter lecture_32_jul_18th_2018 (end)


\section{Lecture 6 - A \tsolver for \Lra}

\begin{frame}
  \frametitle{A \tsolver for \Lra}

  \scriptsize

  \begin{columns}

  \begin{column}{.4\textwidth}
  \begin{center}
  Tableau
  \end{center}
  $$
  \begin{array}{rcl}
    & \ldots \\
    x_1 & = & 3 x_2 - 4 x_3 + 2 x_4 - x_5 \\
    & \ldots \\
    \\
    \\
  \end{array}
  $$
  \end{column}

  \begin{column}{.3\textwidth}
  \begin{center}
  $lb$~~~~~~~Bounds~~~~~~~$ub$
  \end{center}
  $$
  \begin{array}{rcccl}
     -4 & \leq & x_1 & \leq & 10 \\
      1 & \leq & x_2 & \leq & 3 \\
     -4 & \leq & x_3 & \leq & -1 \\
      1 & \leq & x_4 & \leq & 2 \\
     -1 & \leq & x_5 & \leq & 10 \\
  \end{array}
  $$
  \end{column}

  \begin{column}{.3\textwidth}
  \begin{center}
  $\mu$
  \end{center}
  $$
  \begin{array}{rcl}
  x_1 & \mapsto & \colone{12} \\
  x_2 & \mapsto & 1 \\
  x_3 & \mapsto & -1 \\
  x_4 & \mapsto & 2 \\
  x_5 & \mapsto & -1
  \end{array}
  $$
  \end{column}

  \end{columns}
  \vfill
  which among $\nonbas = \{ x_2, x_3, x_4 \}$ do I choose for pivoting ? Clearly, the value of $\mu(x_1)$
  is too high, I have to decrease it by playing with the values of \nonbas:
  \begin{itemize}
    \item $  3 x_2$ cannot decrease, as $\mu(x_2) = lb(x_2)$ and cannot be moved down
    \item $- 4 x_3$ cannot decrease, as $\mu(x_3) = ub(x_3)$ and cannot be moved up
    \item $  2 x_4$ can decrease, as  $\mu(x_4) = ub(x_4)$, and can be moved down
    \item $-   x_5$ can decrease, as  $\mu(x_5) = lb(x_5)$, and can be moved up
  \end{itemize}
  \vfill
  both $x_4$ and $x_5$ are therefore good candidates for pivoting. To avoid loops,
  choose variable with smallest subscript (Bland's Rule). This rule is not necessarily
  efficient, though

\end{frame}

\begin{frame}
  \frametitle{A \tsolver for \Lra}

  \scriptsize

  There might be cases in which {\bf no suitable variable for pivoting can be found}.
  This indicates unsatisfiability. Consider the following where we have just asserted $x_1 \leq 9$
  \vfill
  \begin{columns}

  \begin{column}{.4\textwidth}
  \begin{center}
  Tableau
  \end{center}
  $$
  \begin{array}{rcl}
    & \ldots \\
    x_1 & = & 3 x_2 - 4 x_3 + 2 x_4 - x_5 \\
    & \ldots \\
    \\
    \\
  \end{array}
  $$
  \end{column}

  \begin{column}{.3\textwidth}
  \begin{center}
  $lb$~~~~~~~Bounds~~~~~~~$ub$
  \end{center}
  $$
  \begin{array}{rcccl}
     -4 & \leq & x_1 & \leq & 9 \\
      1 & \leq & x_2 & \leq & 3 \\
     -4 & \leq & x_3 & \leq & -1 \\
      2 & \leq & x_4 & \leq & 2 \\
     -1 & \leq & x_5 & \leq & -1 \\
  \end{array}
  $$
  \end{column}

  \begin{column}{.3\textwidth}
  \begin{center}
  $\mu$
  \end{center}
  $$
  \begin{array}{rcl}
  x_1 & \mapsto & \colone{12} \\
  x_2 & \mapsto & 1 \\
  x_3 & \mapsto & -1 \\
  x_4 & \mapsto & 2 \\
  x_5 & \mapsto & -1
  \end{array}
  $$
  \end{column}

  \end{columns}
  \vfill
  no variable among $\nonbas = \{ x_2, x_3, x_4 \}$ can be chosen for pivoting. This is because (due to tableau)
  $$x_2 \geq 1\ \swedge\ x_3 \leq -1\ \swedge\ x_4 \geq 4\ \swedge\ x_5 \leq -1\ \ \Rightarrow\ \ x_1 \geq 12\ \ \Rightarrow\ \ \neg (x_1 \leq 9)$$
  \vfill
  Therefore
  $$\{ x_2 \geq 1,\ x_3 \leq -1,\ x_4 \geq 4,\ x_5 \leq -1,\ \neg( x_1 \leq 9 ) \}$$
  is a \tconflict (modulo the row $x_1 = 3 x_2 - 4 x_3 + 2 x_4 - x_5$)

\end{frame}

\begin{frame}
  \frametitle{Lecture 6 - Exercize 1}

  A conflict returned by the Simplex involves a row
  $$
  x_1 = a_2 x_2 + \ldots + a_n x_n
  $$
  and exactly $n$ bounds
  $$
  \{ x_1 \sim_1 b_1, x_2 \sim_2 b_2, \ldots, x_n \sim_n b_n \}
  $$
  with $\sim_i\ \in \{ \leq, \geq \}$.

  This conflict is minimal: we show that we can find a model $\mu$ if we remove a bound.
  W.l.o.g., we remove $x_2 \sim_2 b_2$: then we can set $\mu(x_j) = b_j$ for $j\not=2$.
  Then we can pivot the row
  $$
  x_2 = c_1 x_1 + c_3 x_3 + \ldots + c_n x_n
  $$
  and compute a suitable value for $\mu(x_2) = c_1 \mu(x_1) + \ldots + c_n \mu(x_n)$.\\
  (we can set the value we want to $\mu(x_2)$ because it is unbounded now!)

\end{frame}


% !TEX root=/home/tavant/these/manuscript/src/manuscript.tex

\section{Dispersion relation of the instabilities}
  \label{sec-DR-kinetic}

  The dispersion relations are obtained by coupling the particle dynamics with the electric fields.
  In the case of the kinetic electrostatic dispersion relation, we couple the Vlasov equation with the Poisson equation.
  In our \ac{2D} geometry, we can neglect all the gradients.
  A fixed axial electric field $\vect{E} = E_0 \vect{e_z}$ and a fixed radial magnetic field $\vect{B} = B_0 \vect{e_r}$ are imposed.
  The electrons are drifting in the azimuthal direction due to the $E\times B$ drift, hence
  \[ \vect{u_{e}} = \frac{\vect{E} \times \vect{B}}{\norm{\vect{B}}^2} = \frac{E_0}{B_0}  \vect{e_{\theta}}.    \]
  The ions are accelerated in the axial direction, $ \vect{u_i} = u_i  \vect{e_z}$.
  Using a perturbations as waves of the form \[ \exp \lp i \vect{k} \cdot \vect{x} - i \omega t  \rp, \]
  with $\vect{x}$ the position vector.
  The oscillation wave vector $\vect{k} = k_r \vect{e_r} + k_{\theta} \vect{e_{\theta}}$ is real but its frequency $\omega = \omega_r + i \gamma$ can be complex, where $\gamma$ is the growth rate of the oscillations.

  \vspace{1em}
  The dispersion relation has been studied by \citet{ducrocq2006} in the case of cold ions and Maxwellian electrons in a \ac{2D} geometry.
  A numerical algorithm has been proposed by \citet{cavalier2013} and compared to experimental measurements.
  In \citet{lafleur2016}, the authors added the ion drift velocity to the dispersion relation.

  The \ac{ECDI} dispersion relation obtained presents resonances at the cyclotron frequencies, which broadens when $k_r$ increased.
  The limit when $k_r$ tends to large values is similar to an \ac{IAW}.
  This limit is usually used, even if recently, \ac{2D} \ac{PIC} simulations with a larger domain observed radial structures in the oscillations \citep{janhunen2018,hara2019a}.

  \citet{lafleur2018} removed the Maxwellian hypothesis by using directly the distribution functions measured in the \ac{PIC} simulations in the \ac{IAW} dispersion relation.
  The authors showed that the frequency of the oscillation is almost unperturbed, but the growth rate is significantly reduced, even when the ions were still supposed Maxwellian \citep[Fig. 8]{lafleur2018}.

  Here, we propose to continue the investigation by solving the dispersion relation numerically with the electron and ion distribution function for both the \ac{IAW} and the \ac{ECDI}.

  \subsection{General dispersion relation}
    \label{sec-geneDR}

    We follow the development presented in \citet{ducrocq2006,cavalier2013,lafleur2016}.
    The plasma dielectric function is defined as
    \begin{equation} \label{eq-de}
      \hat\epsilon(\vect{k},\omega) = 1 - \sum_s \chi_s(\vect{k},\omega)
    \end{equation}

    where $\chi_s(\vect{k}, \omega)$ is the susceptibility of the species $s$.
    It is obtained by coupling the Poisson equation with the particles description.
    The dispersion relation is obtained by setting $  \hat\epsilon(\vect{k},\omega) =0$ and solving for $\vect{k},\omega$.

    For the unmagnetized ions, supposing a Maxwellian distribution, the susceptibility is
    \begin{equation} \label{eq-susceptibility}
      \chi_i(\vect{k},\omega) = \frac{\opi^2}{k^2 v^2_{th, i}} Z'\lp \frac{\omega - \vect{k} \cdot \vect{u_{i}}}{k v_{th, i}}  \rp
    \end{equation}
    where $\opi$ the ion plasma pulsation, $k=\norm{\vect{k}}$ and $\vect{u_i}$ is the mean velocity of the ions.
    The function $Z'$ is the derivative of the Fried and Conte function \citep{fried1961}
    \begin{equation} \label{eq-friedandConte}
      Z(\eta) = \frac{1}{\sqrt{\pi}} \int_{-\infty}^{\infty} \frac{\exp{(-t^2)}}{t - \eta} dt.
    \end{equation}
    We use here the Fried and Conte function because of the Maxwellian hypothesis.
    \citet{xie2013} proposes a numerical algorithm to calculate the susceptibility for a general distribution function
    \begin{equation} \label{eq-general}
      Z(\eta, f) = \int_{-\infty}^{\infty} \frac{f(t)}{t - \eta} dt,
    \end{equation}
    with $f$ the velocity distribution function to consider, normalized to one, centered, and of standard deviation $\sigma = 1/2$.
    For the sake of brevity, the generalized dispersion function $Z(\eta,f)$ is noted $Z(\eta)$, and the Fried and Conte function is noted $Z_M(\eta)$.
    The derivative of $Z$ is
    \begin{equation} \label{eq-derivatives}
      Z'(\eta) = \int_{-\infty}^{\infty} \frac{\partial f(t) / \partial t}{t - \eta} dt,
    \end{equation}

    \vspace{1em}
    A general expression for the plasma dielectric function can be obtained for magnetized electrons by making use of the method of characteristics and is given by
    \begin{equation} \label{eq-drECDI}
      \begin{split}
      \hat\epsilon(\vect{k},\omega) =& 1 - \\
       &\frac{\opi^2}{k^2 v^2_{th, i}} Z'\lp \frac{\omega - \vect{k} \cdot \vect{u_{i}}}{k v_{th, i}}  \rp + \\
       & \frac{1}{k^2 \lde^2} \lb 1 + \lp  \frac{\omega - \vect{k} \cdot \vect{u_{e}}}{k v_{th, e}} \rp \sum_{n=-\infty}^{\infty} e^{- \beta} I_n(\beta) Z\lp  \frac{\omega - \vect{k} \cdot \vect{u_{e} - n \oce}}{k_{r} v_{th, e}} \rp  \rb,
    \end{split}
    \end{equation}
    where $I_n$ are the modified Bessel functions of the first kind, and
    \begin{equation} \label{eq-beta}
      \beta = \frac{(k_{\theta}^2 + k_z^2) b^2_{th, e}}{ \oce^2}
    \end{equation}

    \Cref{eq-drECDI} is the dispersion relation for drifting magnetized electrons and unmagnetized ions.
    It will be used to study the \acf{ECDI}.

  \subsection{Modified Ion Acoustic Wave}
    \label{sucsec-IAW}

    The dispersion relation of \cref{eq-drECDI} presents sharp resonances due to cyclotron resonances.
    However, kinetic simulations in \citet{janhunen2018} and \citet{taccogna2019} have shown that after some time (around  $t=0.5\,\micro\second$ and $t=3\,\micro\second$ respectively) the resonances are no longer present, with a progressive decrease of the higher harmonics and the first harmonics becoming the most prominent.
    Without the resonances, the dispersion relation evolves to the nonmagnetic ion-acoustic instability
    \begin{equation} \label{eq-drIAWgene}
      \begin{split}
      \hat\epsilon(\vect{k},\omega) =& 1 - \\
       &\frac{\opi^2}{k^2 v^2_{th, i}} Z'\lp \frac{\omega - \vect{k} \cdot \vect{u_{i}}}{k v_{th, i}}  \rp + \\
       & \frac{1}{k^2 \lde^2}  Z'\lp  \frac{\omega - \vect{k} \cdot \vect{u_{e}}}{k v_{th, e}} \rp ,
    \end{split}
    \end{equation}

    \citet{lafleur2016} and \citet{janhunen2018} show that after some assumptions -- mostly a drifting Maxwellian distribution, cold ions, and a small electron drift velocity compared to thermal speed -- \cref{eq-drIAWgene} can be solved to obtain
    \begin{equation} \label{eq-MIAW}
      \omega = \omega_r + i \gamma = \vect{k} \cdot \vect{u_{i}} \pm \frac{k c_s}{\sqrt{1 + k^2 \lde^2}} \pm i \sqrt{\frac{\pi m_e}{8 m_i}} \frac{\vect{k} \cdot \vect{u_{e}}}{( 1 + k^2 \lde^2)^{3/2}}.
    \end{equation}
    The above equation represents the analytic modified ion-acoustic dispersion relation.
    The wavenumber that corresponds to the maximum growth rate is \citep{lafleur2016}
    \begin{equation} \label{eq-mostk}
      k_{max} = \frac{1}{\sqrt{2} \lde}
    \end{equation}
    which gives, subtituted in \cref{eq-MIAW}
    \begin{equation} \label{eq-mostw}
      \omega_{max} =  \vect{k} \cdot \vect{u_{i}} \pm \frac{\opi}{\sqrt{3}} + i \sqrt{ \frac{\pi m_e}{54 m_i}}\frac{u_e}{\lde}
    \end{equation}
    We can see that the growth rate is proportional to the electron drift velocity, and inversely proportional to the Debye length.

    However, one should note that there is no consensus on the transition to an \ac{IAW}.
    It could be attributed to non-linear resonance broadening, because the electron orbit is distorted, leading to the loss of the phase relation between the electron and the wave \citep{taccogna2019}.
    The electron demagnetization leads to a dispersion relation where the ions and the electrons have the same type of contribution.
    But recently, \citet{janhunen2018a} stated that the demagnetization condition due to nonlinear resonance broadening is not fulfilled.
    \citet{lafleur2017a} use the fact that in \ac{2D} simulations, the finite minimum value for the radial wave vector is responsible for the ion acoustic dispersion relation.


\documentclass{article}

%%%%%% Include Packages %%%%%%
\usepackage{sectsty}
\usepackage{amsmath,amsfonts,amsthm,amssymb}
\usepackage{fancyhdr}
\usepackage{lastpage}
\usepackage{setspace}
\usepackage{graphicx}

%%%%%% Formatting Modifications %%%%%%

\usepackage[margin=2.5cm]{geometry} %% Set margins
\sectionfont{\sectionrule{0pt}{0pt}{-8pt}{0.8pt}} %% Underscore section headers
\setstretch{1.2} %% Set 1.2 spacing

%%%%%% Set Homework Variables %%%%%%

\newcommand{\hwkNum}{5}
\newcommand{\hwkAuthors}{Ben Drucker}

%%%%%% Set Header/Footer %%%%%%

\pagestyle{fancy}
\lhead{\hwkAuthors}
\rhead{Homework \#\hwkNum}
\rfoot{\textit{\footnotesize{\thepage /\pageref{LastPage}}}}
\cfoot{}
\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

%%%%%% Document %%%%%%

\begin{document}

\title{Homework \#\hwkNum}
\author{\hwkAuthors}
\date{}

\maketitle

%%%%%% Begin Content %%%%%%

\section*{4.2}
	\subsection*{20}
		\subsubsection*{a)}
			$F(Y) = \begin{cases}
				0 \leq y \leq 5 &\Rightarrow \int_0^y \frac{y}{25}dy = \frac{y^2}{50} \\
				5 \leq y \leq 10 &\Rightarrow \int_0^y f(y)dy = \int_0^5 f(y)dy + \int_5^y f(y)dy \\
				&= \frac{1}{2} + \int_0^y\left [ \frac{2}{5} - \frac{y}{25} \right ] dy = \frac{2y}{5} - \frac{y^2}{50} -1
			\end{cases}$ \\ \\ \\
			\includegraphics[height=2in]{4-2--20a}
		\subsubsection*{b)}
			$ 0 < p \leq .5 \Rightarrow p=F(y_p) = \frac{y_p^2}{50} \rightarrow y_p = \sqrt{50p} \\
			.5 < p \leq 1 \Rightarrow p = \frac{2y_p}{5} - \frac{y_p}{50}-1 \rightarrow y_p = 10-5\sqrt{2-2p}$
		\subsubsection*{c)}
			$E(Y) =\int_0^5 y\frac{y}{25}dy + \int_5^{10}y \left ( \frac{2}{5} - \frac{y}{25} \right )dy =5 \\
			V(Y) = \left(\int_0^5 \frac{y^3}{25} \, dy+\int_5^{10} y^2 \left(\frac{2}{5}-\frac{y}{25}\right) \, dy\right)-5^2 = \frac{25}{6} $\\
			For a single bus, the values are simply halved. So: $E(X) =2.5, V(X) = \frac{25}{12}$

\section*{4.3}
	\subsection*{40}
		\subsubsection*{a)}
			$P(X \leq 40) = P \left (Z \leq \frac{40-43}{4.5} \right ) \approxeq 0.2546 \\
			P(X >60) = 1- P(Z < \frac{60-43}{4.5}) \approxeq 1- 0.999... $
		\subsubsection*{b)}
			$ P(Z < z) = .75 \rightarrow z = .67 \rightarrow .67 = \frac{x-43}{4.5} \Rightarrow x = 46.015$
	\subsection*{46)}
		\subsubsection*{a)}
			$$P(67 \leq X \leq 75) = P \left ( \frac{67-70}{3} < Z < \frac{75-70}{3}	\right ) \approxeq .953 - .159 = .794$$
		\subsubsection*{b)}
			$Z_{.05/2} = Z_{.025} = 1.96; 1.96 * 3 = 5.88. $
		\subsubsection*{c)}
			$E(RV) = .794 * 10 = 7.94$
		\subsubsection*{d)}
			$P(X \leq 73.84) = 0.89973\\
			P(p = 0.9, n = 10, x = 9) = .387 \\
			P(p = 0.9, n = 10, x = 10) = 0.349 \\
			p = 1-0.387-.349 = .264$
	\subsection*{48}
		\subsubsection*{a)}
		$p(1.72) - p(.55) =  .2485\\
		p(.55)-p(0))+(p(1.72)-p(0))$

%%%%%% End Content %%%%%%
\end{document}

Spring 2013/STAT W1211 - Statistics/Homework/Homework 5/Homework 5.tex


\section{Language Variation Model with continuous time}


\documentclass[a4paper,12pt]{article}
\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{epstopdf}
\usepackage[dvips]{epsfig}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `basename #1 .tif`.png}

\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\ba}{\begin{eqnarray*}}
\newcommand{\ea}{\end{eqnarray*}}
\newcommand{\by}{\begin{array}}
\newcommand{\ey}{\end{array}}
\newcommand{\no}{\nonumber \\}
\newcommand{\expect}[1]{\langle #1 \rangle}
\newcommand{\erfc}{\rm erfc}
\newcommand{\bs}[1]{\boldsymbol #1 }
\newcommand{\wpp}{w_{_+}}
\newcommand{\wmm}{w_{_-}}

\def\slfrac#1#2{{\mathord{\mathchoice   % Bruch mit schraegem Bruchstrich
        {\kern.1em\raise.5ex\hbox{$\scriptstyle#1$}\kern-.1em
        /\kern-.15em\lower.25ex\hbox{$\scriptstyle#2$}}
        {\kern.1em\raise.5ex\hbox{$\scriptstyle#1$}\kern-.1em
        /\kern-.15em\lower.25ex\hbox{$\scriptstyle#2$}}
        {\kern.1em\raise.4ex\hbox{$\scriptscriptstyle#1$}\kern-.1em
        /\kern-.14em\lower.25ex\hbox{$\scriptscriptstyle#2$}}
        {\kern.1em\raise.2ex\hbox{$\scriptscriptstyle#1$}\kern-.1em
        /\kern-.1em\lower.25ex\hbox{$\scriptscriptstyle#2$}}}}}

%define local variable "markanswers"
\newif\ifmarkanswers %default value is "false"

%enable the following line to mark all correct answers
%\markanswerstrue

\textwidth = 16 cm
\textheight = 26 cm
\oddsidemargin = 0.0 cm
\evensidemargin = 0.0 cm
\topmargin = -1.5 cm
\headheight = 0.0 cm
\headsep = 0.5 cm
\parskip = 0.2 cm
\parindent = 0.0 cm

\title{\bf Theoretical Neuroscience II\\Exercise 8\\ Principal Component Analysis (PCA)}
\author{Due date: Thursday, 9 July 2015}

\begin{document}
\maketitle

\section{Motivation}

Principal component analysis is an extremely useful tool for analyzing high-dimensional data.  It identifies correlated sets of variables, so called `principal components' and thereby reduces the dimensionality of the data.

In this exercise, you will analyze multi-unit activity from a network of spiking neurons.  In such networks, quiescent periods with comparatively little activity are interrupted spontaneously by transient periods of intense collective activity (`bursts').  Your task is to investage the origin of these bursts.  How does excitation spread through the network?  Is there a stereotypical propagation path?  Or does each burst start in a unique way?

\section{Principal components analysis}

Given $n$ observations of $m$ variables, each with zero mean, collected in a matrix $\bs X$
\ba
\bs X =\left(  \by{cccc} x_{11} & x_{12} \ldots & x_{1n} \\ \vdots & \ddots & \ddots & \vdots \\ x_{m1} & x_{m2} & \ldots & x_{mn} \ey \right)  \in \mathbb{R}^{m \times n}, \qquad\qquad \sum_j \, x_{ij} = 0, \,\, \forall i
\ea
and the covariance matrix
\ba
\bs C_{\bs X} = \frac{1}{n-1} \, \bs X \, \bs X^T \in \mathbb{R}^{m \times m}
\ea
we seek an orthonormal transformation $\bs P  \in \mathbb{R}^{m \times m}$ such that the transformed observations have diagonal covariance:
\ba
\bs Y = \bs P \, \bs X  \in \mathbb{R}^{m \times n}, \qquad\qquad \bs C_{\bs Y} = \frac{1}{n-1} \, \bs Y \, \bs Y^T = \left(  \by{cccc} y_{11} & 0 &\ldots & 0 \\ 0 & y_{22} & \ldots & \vdots \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots & y_{mm} \ey \right)
\ea

From the `skinny' singular value decomposition $\bs X^T = \bs U \bs S \bs V^T$, we obtain the left singular vectors $\bs U \in  \mathbb{R}^{n \times n}$, the singular values  $\bs S \in  \mathbb{R}^{n \times m}$, and the right singular vectors $\bs V  \in \mathbb{R}^{m \times m}$.

The desired orthonormal transformation is
\ba
\bs P = \bs V^T  \in \mathbb{R}^{m \times m}
\ea
with the {\bf rows} of $\bs P$ being the {\bf principal components}.  The variance captured by each component is revealed by diagonal matrix $\bs S$.  Specifically, element $s_{ii}$ (the $i^{th}$ diagonal element) represents the variance captured by the $i^{th}$ principal component.

\section{Average burst}

Multi-unit-activity is provided in a Matlab-file {\bf MUA\_b\_t\_g}.  After loading this file, you have the number of bursts {\bf Nb}, the number of time points {\bf Nt}, the number of neurons groups {\bf Ng}, the vector of time points {\bf ti}, and the 3d array {\bf MUA\_b\_t\_g} of size {\bf [Nb, Nt, Ng]}, which holds the collective spike rate of each group of neurons for different time points and for different bursts.

Begin by averaging the activity over bursts, to obtain a 2d array {\bf MUA\_g\_t} of size {\bf [Ng, Nt]}!  Note that the variables (neuron groups) do now range over {\it rows}, whereas the observations (time points) now range over columns, in agreement with the matrix $\bs X$ above.

Plot the activity of each group of neurons as a function of time!

Subtract the mean from each row and save the subtracted means for later!

Compute the covariance matrix and plot with Matlab function {\bf pcolor}!

Perform the singular value decomposition (don't forget to transpose the input!), using the Matlab command {\bf [U, S, V] = svd( X' )}!  Plot the diagonal values of {\bf S} with Matlab function {\bf bar}!   How many principal components capture significant variance?

\section{Transformed activity}

Transform the observed activity into the orthonormal space of principal components, beginning with the average over bursts!

Plot the activity of each principal component as a function of time!

Compute the covariance matrix of the transformed activity and plot with Matlab function {\bf pcolor}!  The result should be a diagonal matrix!

Plot the time-dependent activity of the first three principal components, using Matlab function {\bf plot3}!

Zero the activity of all but the first three principal components and project back into the original space of neuron groups!  Plot this `denoised' activity of each group of neurons as a function of time!

\section{Individual bursts}

Transform the activity of individual bursts into the orthonormal space of principal components!

Plot the time-dependent activity of the first three principal components!  Superimpose all bursts in the same plot, by repeatedly using Matlab function{\bf plot3}!

\end{document}


\documentclass[10pt,a4paper]{article}
\usepackage[latin1]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage[\shortlabels]{enumerate}
\author{Michele De Vita}
\title{Solutions of week 1}
\begin{document}
	\section*{Exercise 1:}
	\begin{enumerate}[a)]
		\item 392.36
		\item -23.28
		\item $ SE = \sqrt{Var(\hat{\beta})} = \sqrt{\dfrac{\hat{\sigma}^2}{SSX}} = \dfrac{11.5}{\sqrt{27.07766}} =  2.21$ \\\\
		Confidence interval: $ [-10.15, -1.49] $
		\item $ H_0: \hat{\beta} = 0 $ $ t = \dfrac{\hat{\beta}}{SE} = \dfrac{-5.82}{2.21} = -2.63 $ \\\\
		$ p-value = P(X \leq -2.63 | H_0) + P(X \geq 2.63 | H_0)  = 2 * 0.043 = 0.086$\\\\
		We reject both $ 5\% \text{ and } 1\% $
		\item $ TS = 520.4- 5.82 * 21.4 = 395.852$
		\item The unit measure of the slope is Test Score over Class Size. This mean that for each new student in mean we have a decrease of the test score of 5.82
	\end{enumerate}
	\section*{Exercise 2:}
	\begin{enumerate}[a)]
		\item $ -99.41 +3.94 \cdot 70 = 176.39 | -99.41 +3.94 \cdot 74 = 192.15$
		\item $ (-99.41 +3.94 \cdot (x^* + 1.5))  - (-99.41 +3.94 \cdot x^*) = 3.94\cdot 1.5 = 5.91$
		\item $ SE = 0.31 \,\, - \,\,IC = \hat{\beta} \pm Z_{0.005} \cdot SE = 5.91 \pm  2.58 \cdot 0.31 = [5.1102, 6.7098] $
		\item $ 1 \,\, pound \rightarrow 0.45 \,\, kg  $, $ 1 \,\, inch \rightarrow 2.54 \,\, cm $ \\\\ $ weight = -99.41 \cdot 0.45 + 3.94 \cdot \dfrac{0.45}{2.54}\cdot height = -44.7345 + 0.7 \cdot height$\\
		intercept = -44.7345 \\ slope = 0.7 \\ $ \hat{\sigma} = 10.2 \cdot 0.45 = 4.59$ \\  SE = $ 0.31 \cdot \dfrac{0.45}{2.54} = 0.055  $ \\$ R^2 = 0.81 $
		\item The unit measure of slope is weight (pound) over height (inch). This mean that for each increment by 1 of the height we have an increment of weight by 3.94
	\end{enumerate}
\end{document}


% !TeX spellcheck = en_US
\documentclass[]{report}
\usepackage[margin=1in]{geometry}	% less margin to fit the longer formulas
\usepackage[utf8]{inputenc}
\usepackage{lmodern}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{tikz}
\usepackage{amsmath}
\usepackage{amssymb}

\begin{document}

\section{Full FDTD for the 3D Case}
- again start with maxwells

- currents \( J_x, J_y, J_z \) were added as they are necessary for voltage and current sources

- also anisotropy is possible as permittivity, permeability and conductivity no can vary depending on direction

\begin{align}
	\partial_y E_z - \partial_z E_y &= -\mu_x \partial_t H_x \\
	\partial_z E_x - \partial_x E_z &= -\mu_y \partial_t H_y \\
	\partial_x E_y - \partial_y E_x &= -\mu_z \partial_t H_z \\
	\partial_y H_z - \partial_z H_y &= \varepsilon_x \partial_t E_x + \sigma_x E_x + J_x \\
	\partial_z H_x - \partial_x H_z &= \varepsilon_y \partial_t E_y + \sigma_y E_y + J_y \\
	\partial_x H_y - \partial_y H_x &= \varepsilon_z \partial_t E_z + \sigma_z E_z + J_z \text{ .}
\end{align}
The operator \( \partial_x \) is a shorthand for the partial derivative \( \frac{\partial}{\partial x} \).

- no terms are going to be dropped now

- since form of eqn 1-3 and 4-6 are similar, well only do 1 and 4 and then transfer those results

- the grid will look like this now (add grid and coordinate system indication = left-handed system here)

- node is indicated by a 3-tuple (i,j,k) where x..., y..., z...

- to make the following expressions shorter i,j,k (current node) are taken as default parameters, only the relevant differing parameters are written out, e.g. i+1, j+1, or k+1

- E(i,j,k) becomes E, E(i,j+1,k) would become E(j+1)

- now for equation 1, after discretization
\begin{equation}
	\frac{E_z^{n+1/2}(j+1) - E_z^{n+1/2}}{\Delta y} - \frac{E_y^{n+1/2}(k+1) - E_y^{n+1/2}}{\Delta z} = -\mu_x \frac{H_x^{n+1}-H_x^n}{\Delta t}
\end{equation}

- rearranging to the desired ...

\begin{equation}
	H_z^{n+1} = H_z^n - \frac{\Delta t}{\mu_x \Delta y}\left( E_z^{n+1/2}(j+1) - E_z^{n+1/2}  \right) + \frac{\Delta t}{\mu_x \Delta z}\left( E_y^{n+1/2}(k+1) - E_y^{n+1/2}  \right)
\end{equation}

- in turn for equation 4, after discretization

\begin{equation}
	\frac{H_z^n(j+1)-H_z^n}{\Delta y} - \frac{H_y^n(k+1)-H_y^n}{\Delta z} = \varepsilon_x \frac{E_x^{n+1/2}-E_x^{n-1/2}}{\Delta t} + \sigma_x \frac{E_x^{n+1/2}-E_x^{n-1/2}}{2} + J_x
\end{equation}

- rearranging to ...

\begin{equation}
	E_x^{n+1/2} = E_x^{n+1/2} + \frac{2\Delta t}{2\varepsilon_x + \Delta t \sigma_x}\left( \frac{H_z^n(j+1) - H_z^n}{\Delta y} - \frac{H_y^n(k+1) - H_y^n}{\Delta z} - J_x \right)
\end{equation}

- write out the results for all 6 equations

- cfl condition
\begin{equation}
	\Delta t \leq \frac{1}{c\sqrt{\frac{1}{(\Delta x)^2} + \frac{1}{(\Delta y)^2} + \frac{1}{(\Delta z)^2}}}
\end{equation}

- cell size, depends on simulation wavelength (chosen frequency), as general high-frequency principle no dimension of cell size should be larger than one tenth of wavelength

- example 2.4ghz leads to 12.5cm wavelength

- therefore 1.25 cell size is ok

\begin{equation}
	\Delta t \leq = \frac{1}{c \sqrt{\frac{3}{(1.25cm)^2}}} = \frac{1.25cm}{\sqrt{3}c} \approx 24ps
\end{equation}

\section{Simulations}

- i won't write any matlab scripts for this one, would be a waste of effort

- instead, head to next note on building the grid

- from that point, focus should be spent on implementing a functioning toolchain

- toolchain consists of: parametric modeller, grid builder, simulator, result visualizer

- none of these need to be fancy, parametric modeller only needs to export some boxes for a start, grid builder needs problem configuration and putting in a source, e.g. voltage source, simulator should probably be in grid builder, result visualizer should provide a good way of judging vector fields (plane cuts and 3D view with many lines, thickness/color depends on magnitude or something)

\end{document}


\chapter{Theory}

\section{Feature extraction}

\textbf{by Paul Warkentin} \\

The state of the art method for feature extraction from audio for speech recognition purposes is to use Mel Frequency Cepstral Coefficients (MFCC). They were introduced in 1980 by Davis and Mermelstein, and the European Telecommunications Standards Institute (ETSI) defined a standardised MFCC algorithm to be used in mobile phones.

Based on the underlying assumption, that audio frequencies change very little over very short intervals of time, the signal is divided into 25ms intervals. For each interval, a sequence of transformations is applied, resulting in 13 coefficients. The algorithm is very complex, which is why I will not discuss it here in detail. The steps for the calculation of the MFCCs are the following:
\begin{enumerate}
  \item \textit{Frame the signal into shorter frames.} As described above, we assume that the audio signal is constant over a short time to simplify the complexity of audio signals.
  \item \textit{Calculate the power spectrum of each frame.} This step is motivated by an human organ in the ear, the cochlea. It vibrates at different spots depending on the frequency of detected sounds. The brain is then informed by different nerves on the frequency. Calculating the periodogram estimate of the power spectrum delivers a similar behaviour here by giving information about the different frequencies in the frame.
  \item \textit{Apply mel filterbank to the power spectra.} The periodogram spectral estimate still contains a lot unneeded information. With the mel filterbank, we can sum the energy in various frequency regions for each frame. We are only interested in roughly how much energy occurs at each spot.
  \item \textit{Take the logarithm of all filterbank energies.} Also motivated by the human being, we do not differentiate loudness on a linear scale. That is why we take the logarithm because it also allows us to use cepstral mean subtraction, a channel normalization technique.
  \item \textit{Take the discrete cosine transformation of the log filterbank energies.} At last, compute the DCT of the previously calculated log filterbank energies. Because the filterbanks are all overlapping, the filterbank energies are quite correlated with each other. The discrete cosine transformation decorrelates the energies. Discard all but 13 of the total 26 DCT coefficients, because those represent fast changes in the filterbank energies which has negative influence on speech recognition.
\end{enumerate}

MFCC can be used to identify all the parts of the audio that belong to linguistic content and distinguish these from those parts in the audio signal which carry information about silence, background noise, laughter etc. \\

We decided to use the Python library \textit{python\_speech\_features}\footnote{\url{https://github.com/jameslyons/python_speech_features}} to calculate the MFCC as it is a highly complex and error-prone algorithm.

\section{Classification}

\textbf{by Paul Warkentin} \\

The main part of this project is the classification of extracted audio signals of linguistic content. This is a typical problem in the field of speech recognition, for which convolutional neural networks are a widely used class of deep, feed-forward artificial neural networks. \\

An important feature of convolutional neural networks is that they are translation invariant. The input data for our model are intervals containing MFCC datapoints. As described in chapter \ref{data_preparation}, we do not know where exactly in the interval the features which determine the word being said are localize. Hence, the shift invariance feature of convolutional neural networks is ideal for the speech recognition part in our problem. \\

Combining a convolutional neural network with a LSTM unit for time-series prediction did give us the best result as discussed in chapter \ref{results}.


\chapter{Codes instructions}

\section{Running codes instructions - 1cell system}\label{sec:1code}

We here give a brief explanation on how to run proper test and simulations with codes provided inside the repository at link \url{https://github.com/danieleavitabile/root-simulator}. Regarding singular cell pattern formation there are implemented two main codes:
\begin{itemize}
    \item \verb|1cell_stationaryTest.edp|: contains steps for solving the steady state system through Newton's method described in section \ref{sec:statSys}.
    \item \verb|1cell_semi_implicit.edp|: contains the semi-implicit solver for the full time-dependent problem described in section \ref{sec:SI method}.
    \item \verb|Analysis_Semi_implicitDT.edp|: contains main code to estimate the order of convergence of our semi-implicit method used in \ref{sec:conv}.
\end{itemize}

The first one relies on the separated function \verb|G.edp| computing $G$ function defined in \eqref{eq:G} and function \verb|Jac| computing the corresponding Jacobian matrix defined in \eqref{eq:jac}.The second one include instead a separated piece of code \verb|step_semi_implicit.edp|
in which are defined \verb|varf|,  corresponding type in \verb|FreeFem++| for bilinear form, matrices and \verb|func int step| in order to solve the singular cell system.

To properly run both codes, one need to write the string identifying under which set of parameters one want to solve the system; three main sets of parameters are defined in \verb|PRM.edp| file and each possible group of parameters correspond to the division found in table \ref{tab:setprm}. The choice of parameter set is done as follows:
\begin{lstlisting}[firstnumber = 11, language = C++, caption = Choice of set of parameters]
/// Parameters:
string prm = "Set B"; // "Set A", "Set B", "Set C"
include "PRM.edp";
\end{lstlisting}
In addition, also the mesh has to be settled correctly according to the choice of problem to solve, commenting the not used definition of variable \verb|Th|:
\begin{lstlisting}[firstnumber = 16, language = C++, caption = 1cell\_stationaryTest.edp or 1cell\_semi\_implicit.edp: choice of mesh]
/// Mesh
mesh Th = square(Nx, Ny, [x, y]);  // "Set A", "Set B"
// mesh Th = square(Nx, Ny, [Lx * x, Ly * y]); // "Set C"
\end{lstlisting}
In case "Set A" and "Set B" mesh refinement is characterized by \verb|N_x = N_y = 60|, whereas in case "Set C" by \verb|N_x = 100| and \verb|N_y = 30|.

Finally one need to properly change $\Tilde{c}_1$ and $\Tilde{c}_2$ according to formulas in \eqref{eq:coeffadim} and \eqref{eq:coeffFM}:
\begin{lstlisting}[firstnumber = 16, language = C++, caption = PRM.edp: setting of $\Tilde{c}_1$ and $\Tilde{c}_2$]
// "Set A", "Set B"
func tildec1 = alpha;
func tildec2 = - gamma*alpha;
// "Set C"
// func tildec1 = alpha;
// func tildec2 = - alpha;
\end{lstlisting}

Another important choice for running properly the two solvers is the initialization of the system: in the first case it is the initial guess for the equilibrium to find, in the second case it corresponds to the initial solution at time $t = 0s$ of the ROPs dynamics. Under each result shown is specified which is the initial state from which the solver start; in general one could chose between loading previous solutions setting \verb|ld| parameter equal to 1 or alternatively set custom defined $(u^0, v^0)$:
\begin{lstlisting}[firstnumber = 8, language = C++, caption = choice for initialization]
int ld = 0; // 1: load prev. sol. as initial state
            // 0: use custom initial state
\end{lstlisting}
\begin{lstlisting}[language = C++]
/// Initialization:
if (ld) {
  ifstream fU(path + "U" + tload + ".txt");
  ifstream fV(path + "V" + tload + ".txt");

  fU >> U[];
  fV >> V[];

  plot(U, fill = 1, value = 1, cmm = "Loaded Active ROPS"/*, wait = 1*/);
  plot(V, fill = 1, value = 1, cmm = "Loaded Inactive ROPS"/*, wait = 1*/);
  U[] += pert[];
  V[] += pert[];
}
else
{
  U = u0;
  V = v0;
  U[] += pert[];
  V[] += pert[];
  // srandom(seed);
  // // ranreal 1 generates a random number in [0,1]
  //   for(int ii = 0; ii < Xh.ndof; ii++)
  //   {
  //     U[][ii] += randreal1() / 3e5;
  //     V[][ii] += randreal1() / 3e5;
  //   }
}
\end{lstlisting}

Inside \verb|run.sh| bash file, one need to specify the main to run with \verb|FreeFem++| changing properly \verb|EXEC| variable. Executing this file, the script creates a new folder in \verb|results| directory named with the current date and time, it runs the specified main and saves all the output in \verb|output.txt|. This file together with code files in \verb|.edp| format and solutions files in \verb|.ps|,\verb|.vtk|, \verb|.txt| format are moved inside the directory named with the current date and time, in the corresponding directories \verb|code|, \verb|U| and \verb|V|.

\section{Running codes instructions - 2 and 4 cells systems}\label{sec:code_RR}
% intro/sezione con istruzioni per runnare qua -> descrizione file prm channel e altro ecc
In order to simulate the previously formulated algorithm for a pluricellular system has  been written three main codes:
\begin{itemize}
    \item \verb|2cell_RR.edp|: this main code solves a two cells system with a classic Robin-Robin iterative algorithm, as formulated in sec. \ref{sec:RRclassic}.
    \item \verb|2cell_RRmod.edp|: this main code solves a two cells system with the modified Robin-Robin algorithm described in sec. \ref{sec:RRmodified}.
    \item \verb|4cell_RRmod.edp|: this main code solves a four cells system with the modified Robin-Robin algorithm.
\end{itemize}

For pluricellular systems solver the setting rules for parameters described in sec. \eqref{sec:1code} are still true, excepting those regarding mesh initialization. Indeed, depending on the system to solve, the mesh is properly defined. For example, in a two cells pluricellular system can be defined two possible type of mesh:
\begin{lstlisting}[firstnumber = 24, language = C++, caption = 2cell\_RRmod.edp: choice of mesh]
// 1st type of mesh
border o1(t=Ly*2, 0){x=0; y=t; label=outside;}
border o2(t=0, Lx){x=t; y=0; label=outside;}
border o3(t=0, 2*Ly){x=Lx; y=t; label=outside;}
border o4(t=0, Lx){x=Lx-t; y=2*Ly; label=outside;}
border i1(t=0, Lx){x=Lx - t; y=Ly; label=inside;}
mesh TH = buildmesh(o1(2*Ny) + o2(Nx) + o3(2*Ny) + o4(Nx) + i1(Nx));

reg(0) = TH(Lx/2, Ly - 2).region;
reg(1) = TH(Lx/2, Ly + 2).region;

for(int i = 0; i < N; i++) Th[i] = trunc(TH, region==reg(i));

// 2nd type of mesh
border o1(t=Ly, 0){x=0; y=t; label=outside;}
border o2(t=0, 2 * Lx){x=t; y=0; label=outside;}
border o3(t=0, Ly){x=2*Lx; y=t; label=outside;}
border o4(t=0, 2*Lx){x=2*Lx-t; y=Ly; label=outside;}
border i1(t=0, Ly){x=Lx; y=Ly-t; label=inside;}
mesh TH = buildmesh(o1(Ny) + o2(2*Nx) + o3(Ny) + o4(2*Nx) + i1(Ny));

reg(0) = TH(Lx - 2, Ly/2).region;
reg(1) = TH(Lx + 2, Ly/2).region;

for(int i = 0; i < N; i++) Th[i] = trunc(TH, region==reg(i));
\end{lstlisting}

Bilinear forms characterizing the mathematical formulation are collected in \verb|varf_2cell.edp| and \verb|varf_ncell.edp| files. Inside \verb| varf duidn| for 2 cells system and \verb|duidnj| for 4 cells system one can pass from original Robin-Robin contribute, to not-parallelizable or parallelizable contribute for Robin-Robin modified commenting the right lines specified. (vd. commenti a lato)

Matrices and vectors reassembling procedures for each sub-domain are defined in \verb|assemble_RR|. Here is necessary to comment the alternatives of \verb|if-else| cycle not used for the pluricellular system under study.

Last important file included in the main for model definition is \verb|channelPRM.edp|, where parameters $\alpha_{u/vRR}$ and $\beta_{u/vRR}$ used in our new model to represents communication channels between cells have to be properly settled, together with convergence criterion parameters for the algorithm used:
\begin{itemize}
  \item \verb|toll| represents the tolerance of the normalized residual desired.
  \item\verb|Niter| represents the maximum number of iterations allowed in case tolerance is not reached.
\end{itemize}