\section{Single Variable Statistics}

    \subsection{Intro to Statistics}
    This is a massive shift compared to the rest of the stuff we've done so far. It is a lot of (relatively simple) graphing, and data management, and also includes a lot of spreadsheet work (which I may or may not include). Using a frequency table will help you, as well as attention to detail.

    \subsection{Displaying Frequency}
    To start, we're going to have to (re)define some stuff you have learned in earlier years. This will seem like a lot at first, but most of it is quite basic, and there are no equations to memorize.
    \begin{definition}
        Range: A range, in relation to sets of data, is how far apart the data points are spread. It is calculated by taking the largest number, and subtracting it by the smallest.
    \end{definition}
    \begin{definition}
        Frequency Diagram: A frequency diagram is a graph used to show the frequency of various points, or intervals of data (ex. years, or decades).
    \end{definition}
    \begin{definition}
        Frequency Polygon: A shape that helps the reader understand the overall shape of the diagram or distribution.
    \end{definition}
    \begin{center}
        \begin{tikzpicture}
            \begin{axis}[
                width=10cm,
                height=4cm,
                x tick label style={
                	/pgf/number format/1000 sep=},
                title=Example Frequency Diagram with Polygon,
                ylabel=Frequency,
                xlabel=Event,
                ymin=0,
                ymax=10,
                xmin=1,
                xmax=6,
                enlargelimits=0.05,
                ybar interval=0.8,
            ]
            \addplot
                coordinates {
                    (1,5)
                    (2,8)
                	(3,2)
                	(4,5)
                	(5,3)
                	(6,0.0000)};
            \addplot[sharp plot,mark=square]
                coordinates {
                    (1.5,5)
                    (2.5,8)
                	(3.5,2)
                	(4.5,5)
                	(5.5,3)};
            \end{axis}
        \end{tikzpicture}
    \end{center}
    \begin{definition}
        Cumulative Frequency: This refers to the running total of frequencies. Shown below is an example of once such diagram.
    \end{definition}
    \begin{center}
        \begin{tikzpicture}
            \begin{axis}[
                width=10cm,
                height=4.5cm,
                x tick label style={
                	/pgf/number format/1000 sep=},
                title=Example Cumulative Frequency Diagram,
                ylabel=Cumulative Frequency,
                xlabel=Event,
                ymin=0,
                ymax=25,
                xmin=1,
                xmax=5,
                enlargelimits=0.05,
                ybar interval=1,
            ]
            \addplot[sharp plot,mark=square]
                coordinates {
                    (1,5)
                    (2,13)
                	(3,15)
                	(4,20)
                	(5,23)
                };
            \end{axis}
        \end{tikzpicture}
    \end{center}
    \begin{definition}
        Relative Frequency: This means the frequency expressed as a percentage, found by dividing the frequency by the total number of results.
    \end{definition}
    \begin{center}
        \begin{tikzpicture}
            \begin{axis}[
                width=10cm,
                height=5cm,
                x tick label style={
                	/pgf/number format/1000 sep=},
                title=Example Relative Frequency Diagram,
                ylabel=Frequency(\%),
                xlabel=Event,
                ymin=0,
                ymax=50,
                xmin=1,
                xmax=6,
                enlargelimits=0.05,
                ybar interval=0.8,
            ]
            \addplot
                coordinates {
                    (1,21)
                    (2,34)
                	(3,8)
                	(4,22)
                	(5,13)
                	(6,0.0000)};
            \end{axis}
        \end{tikzpicture}
    \end{center}
    \begin{definition}
        Histogram: Essentially a bar graph, yet with the data in groupings, instead of being separate.
    \end{definition}
    \begin{center}
        \begin{tikzpicture}
            \begin{axis}[
                x tick label style={
                	/pgf/number format/1000 sep=},
                title=Example Histogram,
                width=10cm,
                height=5cm,
                ymin=0,
                ymax=55,
                minor y tick num = 3,
                area style,
            ]
            \addplot+[ybar interval,mark=no] plot coordinates {
                (0, 5)
                (5, 35)
                (10, 50)
                (15, 30)
                (20, 15)
                (25, 0)};
            \end{axis}
        \end{tikzpicture}
    \end{center}
    All of this data and results above can be displayed and organized in something called a frequency table, as defined, and shown below.
    \begin{definition}
        Frequency Table: This is a table that contains all of the above results, as well as tallies of data points.
    \end{definition}
    \begin{center}
        \begin{tabular}{ | l | l | l | l |}
            \hline
            (x) & Frequency & Relative Frequency & Cumulative Frequency \\ \hline
            1 & 5 & $\frac{5}{23}$=21\% & 5 \\ \hline
            2 & 8 & $\frac{8}{23}$=34\% & 13 \\ \hline
            3 & 2 & $\frac{2}{23}$=8\% & 15 \\ \hline
            4 & 5 & $\frac{5}{23}$=22\% & 20 \\ \hline
            5 & 3 & $\frac{3}{23}$=13\% & 23 \\ \hline
        \end{tabular}
    \end{center}

    \subsection{Measures of Central Tendency}
    Measures of Central Tendency deal with the various numbers we use to find data points we can make assumptions based on.
    These three numbers we use are called the \textbf{mean, median and mode}.
    \begin{definition}
        Mean: The mean represents the average of the data, and is found by using the equation below.
        \begin{equation*}
            \mean{x} = \frac{\sum x}{n}
        \end{equation*}
    \end{definition}
    \begin{definition}
        Median: The median represents the middle-most point of data, for example in a set of 5 integers, the median will be the third.
        To find the median of an list of data of which \textbf{n is odd numbered} you can use the equation below.
        \begin{equation*}
            \median{x} = \mbox{\emph{Number in middle of data}}
        \end{equation*}
        if the equation is even numbered, you must use the average of the numbers that the median is in between.
        For example if you have a list of 8 numbers, the median would be the average of 4 and 5.
        The equation for this can be seen below.
        \begin{equation*}
            \median{x} = \frac{\mbox{\emph{Left Number + Right Number}}}{2}
        \end{equation*}
    \end{definition}
    \begin{definition}
        Mode: The mode represents the data point that has been repeated the most often. Can be found by merely looking at the highest repeated data set.
    \end{definition}

    \subsection{Measures of Spread}
    A measure of spread is a measurement that describes how the data is distributed, and how widely it fluctuates.
    To measure these, we divided the spread of the data into something called \emph{quartiles.}
    A quartile, just like its name implies, is merely a quarter of the data.
    The first Quartile (referred to henceforth as Q1) is the median of the \textbf{FIRST HALF} of the data, Q2 is the middle (or median), and Q3 is the median of the \textbf{LAST HALF} of the data.
    If we want to find the most relevant data, we'll typically look at the data between Q1 and Q3, this range of data is called the Inter Quartile Range (IQR).
    \begin{definition}
        Inter Quartile Range: This is the middle-most data, and a grouping you can easily make assumptions from. To find this, we use the equation below.
        \begin{equation*}
            \mbox{IQR} = \mbox{Q3}-\mbox{Q1}
        \end{equation*}
    \end{definition}
    Using this, we can also find other data to make assumptions off of, such as the Semi-Inter Quartile Range.
    \begin{definition}
        Semi Inter Quartile Range: This is $\frac{1}{2}$ of the difference of the IQR, and therefore is less affected by outliers. Can also be referred to as the SIQR, and is found by the equation below.
        \begin{equation*}
            \mbox{SIQR} = \frac{\mbox{IQR}}{2}
        \end{equation*}
    \end{definition}
    We can display these using something called a \textbf{box and whisker plot}.
    This will show Q1, Q2, and Q3, along with the highest and lowest datum.
    An example of this is shown below.
    \begin{center}
        \begin{tikzpicture} [thick, framed]
            \filldraw[fill=green!20] (2.85,0) rectangle (5.7,1);
            \draw (4.45,0)--(4.45,1) node[above]{$\textsc{M}$};
            \draw (5.7,0.5)--(11,0.5);%vandret linie til max
            \draw (2.85,0.5)--(1,0.5);%vandret linie til min
            \draw (11,0.39)--(11,0.61);
            \draw (1,0.39)--(1,0.61);
            \draw [
                postaction={
                    draw,
                    decoration=ticks,
                    segment length=1cm,
                    decorate,
                }
             ] (0,-1) -- (11,-1);
             \foreach \tick in {0,...,11}
                \node at (\tick,-1) [below=1pt] {\tick};
        \end{tikzpicture}
    \end{center}
    \subsection{Intro to Variance}
    To calculate the variance, we must first calculate the deviations.
    A deviation is the difference between an individual value in a data set and the mean of that set.
    To calculate the deviation, we'll need to use the equation below.
    \begin{equation*}
        \mbox{Deviation}= x - \mean{x}
    \end{equation*}
    We use different kinds of deviations for various reasons, such as Mean Deviations ($md$) and Standard Deviations ($\sigma$) They are defined below, though most typically we will use the Standard Deviation. Most of these can be calculated much easier if you use the \emph{STAT} mode on your calculator, but the equations will be below if your calculator doesn't support that.
    \begin{definition}
        Mean Deviation: The mean deviation is the mean of all the distances to the mean. To calculate this, we'll use the formula below to solve for $md$, which represents the mean deviation.
        \begin{equation*}
            md = \frac{\sum|x-\mean{x}|}{n}
        \end{equation*}
    \end{definition}
    \begin{definition}
        Standard Deviation: This is more widely used, and differs from the mean deviation in the fact that it uses square roots and exponents rather than the absolute value function. Will always be bigger than the mean deviation, and is shown as $(\sigma)$.
        \begin{equation*}
            \sigma = \sqrt{\frac{\sum(x-\mean{x})^2}{n}}
        \end{equation*}
    \end{definition}
    Once we know the deviation, we can find the variance of various numbers.
    \begin{definition}
        Variance: This number represents the average of the squares of the deviations. You can find this by simply squaring the Standard Deviation (giving you $\sigma^{2}$), or by using the equation below.
        \begin{equation*}
            Variance = \frac{\sum(x-\mean{x})^{2}}{n}
        \end{equation*}
    \end{definition}

    \subsection{Z\textemdash Scores}
    A Z-Score is the number of Standard Deviations that a datum is from the mean. We can find this using the equation below.
    \begin{equation*}
        z = \frac{x-\mean{x}}{\sigma}
    \end{equation*}
    If $z<0$ the element is less than the mean, if $z=0$, the element is equal to the mean, and if $z>0$, the number is greater than the mean.

    \subsection{Deviations with Intervals Data}
    When data is grouped into intervals, the formula for mean deviation and standard deviation change. Mean Deviation changes to the equation below.
    \begin{equation*}
        md = \frac{\sum f|x-\mean{x}|}{n}
    \end{equation*}
    Standard Deviation also changes into the formula below.
    \begin{equation*}
        \sigma = \sqrt{\frac{\sum f(x-\mean{x})^2}{n}}
    \end{equation*}
    In both of these equations, $f$ represents the frequency of the interval.

    \subsection{Chebyshev's Theorem}
    Chebyshev's Theorem states that given a set of data with a mean (designated $\mean{x}$) and a Standard Deviation ($\sigma$), then \textbf{at least} x\% of the will be within $k$ standard deviations of the mean. We find this using the equation below.
    \begin{equation*}
        x = (1-\frac{1}{k^2})\%
    \end{equation*}

    \subsection{Gathering Data}
    This subsection is entirely definitions, and as such is quite lengthy. It is tested on, but I believe it is multiple choice. As such this is semi-important if any research or actual studies are ever done in your class, or in the real world.
    \begin{definition}
        Population: Refers to individuals who belong to a group being studied.
    \end{definition}
    \begin{definition}
        Sample: A subset of a population
    \end{definition}
    \begin{definition}
        Simple Random Sample: Every member of the population has an equal chance of being selected.
    \end{definition}
    \begin{definition}
        Systematic Sample: Selected by iterating through the population and selecting members at regular intervals.
    \end{definition}
    \begin{definition}
        Stratified Sample: A group of population members with similar characteristics is called a stratum. A stratified sample has the same proportion of members from each stratum.
    \end{definition}
    \begin{definition}
        Cluster Sample: When certain groups are representative of the entire population, this takes a random selection of such groups.
    \end{definition}
    \begin{definition}
        Multistage Sample: Uses several levels of random sampling, randomly sample from groups within groups.
    \end{definition}
    \begin{definition}
        Voluntary Response Sample: Researcher invites any member of population to participate. Results skewed as people who choose to participate not always representative of the population.
    \end{definition}
    \begin{definition}
        Convenience Sample: Easily accessible sample of the population.
    \end{definition}
    \begin{definition}
        Bias: Any factor that favors certain outcomes or responses
    \end{definition}
    \begin{definition}
        Sampling Bias: Occurs when the sampling frame does not reflect the characteristics of the population.
    \end{definition}
    \begin{definition}
        Non-response Bias: Occurs when particular groups are under-represented because they choose to not participate.
    \end{definition}
    \begin{definition}
        Measurement Bias: Occurs when the data collection method consistently misrepresents the characteristics of a population.
    \end{definition}
    \begin{definition}
        Response Bias: Occurs when the participants in a survey deliberately give false/misleading answers.
    \end{definition}


\newpage
\begin{appendices}
\section{Comparisons between Momentum Methods}
\label{appendix:momentum_comp}
The delta rule, at iteration $i+1$, using the gradient descent algorithm with the Classic  heavy-ball Momentum \cite{Polyak1964}, Nesterov \cite{sutskever2013} and Accelerated Nesterov Momentum \cite{nakerst2020gradient} can be summarized in this way:
$$ \Delta^i = \alpha\Delta^{i-1} - \eta\nabla L(w^i +\sigma\Delta^{i-1})$$
$$ w^{i+1} = w^i + \Delta^i$$

When $\sigma = 0$, the delta rule uses the heavy-ball method, and when $\sigma = \alpha$, the Nesterov momentum is used. In \cite{nakerst2020gradient} is shown that can be advantageous to use values of $\sigma$ larger than $\alpha \approx 1$. Instead of using the gradient at an estimated point one step ahead, the gradient is computed at a much further estimated point. This corresponds to an extension of Nesterov and that is why it is called \emph{Accelerated Nesterov Momentum}. Figure \ref{fig:monk1_4_momentum} shows the effect of momentum methods.
\begin{figure}[H]
    \centering
    \includegraphics[scale = 0.8]{Images/monk1_4_momentum.eps}
    \caption{Training Learning curves using Monk 1 as dataset with different momentum methods. To compare the learning curves and correctly observe the convergence speed, all the models have been trained starting from the same initial weights.}
    \label{fig:monk1_4_momentum}
\end{figure}
\newpage

\section{Code Examples}
\label{appendix:code_example}
\begin{listing}[!ht]
\inputminted{python}{code/low_api.py}
\caption{A low level API example on the Monk 1 dataset.}
\label{listing:low_api}
\end{listing}

\begin{listing}[!ht]
\inputminted{python}{code/high_api.py}
\caption{A high level API example on the Monk 1 dataset.}
\label{listing:high_api}
\end{listing}

\begin{listing}[!h]
\inputminted{python}{code/model_selection.py}
\caption{Model Selection API example on the CUP dataset.}
\label{listing:model_selection}
\end{listing}

\section{MONKS}
\label{appendix:monks}
\begin{figure}[H]
    \centering
                \begin{subfigure}{0.9\textwidth}
                    \resizebox{\textwidth}{!}{
                        \includegraphics{Images/monks/monk3_overfitting.eps}
                    }
                    \caption{MONK 3 with no regularization}
                    \label{fig:monk3_app}
                \end{subfigure}
                \begin{subfigure}{0.9\textwidth}
                    \resizebox{\textwidth}{!}{
                        \includegraphics{Images/monks/monk3_over_reg.eps}
                    }
                    \caption{MONK 3 with regularization}
                    \label{fig:monk3_reg_app}
                \end{subfigure}
\caption{Plot of the MSE and accuracy for the MONK 3. In Fig.~\ref{fig:monk3_app}, the model without regularization goes into \emph{overfitting} just before 1000 epochs. Indeed the generalization error measured over the test examples increases, even as the error over the training examples continues to decrease. In Fig.~\ref{fig:monk3_reg_app} is shown how the regularization prevents overfitting.}
\label{fig:monk3_appendix}
\end{figure}

\begin{table}[h]
\small
    \centering
    \begin{tabular}{ |c|c|c|c|c|c|c|c|c|  }
      \hline
       \textbf{Task} &\textbf{Topology}&\textbf{Batch size}& $\boldsymbol{\eta}$ & $\boldsymbol{\alpha}$ & $\boldsymbol{\lambda}$& \textbf{MSE(TR/TS)} & \textbf{Accuracy(TR/TS)($\boldsymbol{\%}$)}\\
     \hline
    MONK 1 &17$\rightarrow$4$\rightarrow$1 & 1 & 0.4 & 0.8  & 0 & 0.00016 / 0.00023 & 100\% / 100\%\\
    \hline
    MONK 1 &17$\rightarrow$10$\rightarrow$1 & 1 & 0.4 & 0.7  & 0 & 0.00012 / 0.00017 & 100\% / 100\%\\
    \hline
    MONK 1 & 17$\rightarrow$4$\rightarrow$1 & 31 & 0.6 & 0.7  & 0 & 0.00115 / 0.00182 & 100\% / 100\%\\
    \hline
    MONK 2 & 17$\rightarrow$3$\rightarrow$1& 1 & 0.1 & 0.4  & 0 & 0.00205 / 0.00235 & 100\% / 100\%\\
    \hline
    MONK 2 & 17$\rightarrow$10$\rightarrow$1& 1 & 0.1 & 0.4  & 0 & 0.00197 / 0.00229 & 100\% / 100\%\\
    \hline
    MONK 2 & 17$\rightarrow$3$\rightarrow$1& 68 & 0.1 & 0.4  & 0 & 0.00301 / 0.00349 & 100\% / 100\%\\
    \hline
    MONK 2 & 17$\rightarrow$3$\rightarrow$1& 169 & 0.01 & 0.9  & 0 & 0.00970 / 0.01217 & 100\% / 100\%\\
    \hline
    MONK 3 (no reg.)&17$\rightarrow$4$\rightarrow$1 &1 & 0.01 & 0.6  & 0 & 0.04519 / 0.04536 & 95\% / 95\%\\
    \hline
    MONK 3 (no reg.)&17$\rightarrow$10$\rightarrow$1 &1 & 0.01 & 0.5  & 0 & 0.04828 / 0.04398 & 95\% / 95\%\\
    \hline
    MONK 3 (no reg.) & 17$\rightarrow$4$\rightarrow$1&61 &0.6 & 0.3 & 0 & 0.04904 / 0.04483 & 94\% / 96\%\\

     \hline
    \end{tabular}
    \caption{Preliminary cases discovered during the screening phase of the MONKS dataset. Normal momentum was used for online and mini-batch, while Nesterov was used for full batch. Also we used the sigmoid as activation function.}
    \label{tab:monk_screening}
\end{table}

\section{Cup Prepocessing Phase}
\label{appendix:preprop}
During this phase, we analyzed the CUP dataset composed of 20 inputs and 2 targets. From the distributions of the variables (Fig.~\ref{fig:dis}) and correlation matrix (Fig.~\ref{fig:corr_mat}) it can be seen that many variables are strongly correlated to each other: A-I, B-U, C-T, D-P, E-L, F-M, G-S, H-R, N-O and Q-V. Therefore, by eliminating the redundant features (A, B, C, D, E, F, G, H, N, Q), those with correlation equal to 1, it is possible to reduce the dataset to 12 attributes (10 inputs and 2 targets).
\begin{figure}[H]
    \centering
    \includegraphics[scale = 0.16]{Images/preprocessing_phase/distributions_2.eps}
    \caption{Distributions of the features, excluded the one representing the id record and those relating to the target.}
    \label{fig:dis}
\end{figure}
\begin{figure}[H]
    \centering
    \includegraphics[scale = 0.5]{Images/preprocessing_phase/correlation_matrix_2.eps}
    \caption{Correlation matrix.}
    \label{fig:corr_mat}
\end{figure}
\end{appendices}


\documentclass[12pt]{cdblatex}
\usepackage{exercises}
\usepackage{fancyhdr}
\usepackage{footer}

\begin{document}

% --------------------------------------------------------------------------------------------
\section*{Exercise 5.1 Swap terms}

\begin{cadabra}
   {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u#}::Indices(position=independent).

   expr := A_{a} (P^{b}+Q^{b}) + C_{a} V^{b}.                            # cdb (ex-0501.100,expr)

   substitute (expr, $A_{a} B?? + C_{a} D?? -> A_{a} D?? + C_{a} B??$)   # cdb (ex-0501.101,expr)

\end{cadabra}

\begin{dgroup*}
   \Dmath*{ \cdb*{ex-0501.100} }
   \Dmath*{ \cdb*{ex-0501.101} }
\end{dgroup*}

\end{document}


\chapter{Singular homology}
Now that we've defined $\pi_1(X)$,
we turn our attention to a second way of capturing the same idea, $H_1(X)$.
We'll then define $H_n(X)$ for $n \ge 2$.
The good thing about the $H_n$ groups is that, unlike the $\pi_n$ groups,
they are much easier to compute in practice.
The downside is that their definition will require quite a bit of setup,
and the ``algebraic'' part of ``algebraic topology'' will become a lot more technical.

\section{Simplices and boundaries}
\prototype{$\partial[v_0, v_1, v_2] = [v_0,v_1]-[v_0,v_2]+[v_1,v_2]$.}
First things first:
\begin{definition}
	The \vocab{standard $n$-simplex}, denoted $\Delta^n$, is defined as
	\[ \left\{ (x_0 , x_1, \dots x_n) \mid x_i \ge 0, x_0+\dots+x_n=1 \right\}. \]
	Hence it's the convex hull of some vertices $[v_0, \dots, v_n]$.
	Note that we keep track of the order $v_0$, \dots, $v_n$ of the vertices,
	for reasons that will soon become clear.

	Given a topological space $X$, a \vocab{singular $n$-simplex} is a map $\sigma : \Delta^n \to X$.
\end{definition}
\begin{example}
	[Singular simplices]
	\listhack
	\label{ex:simplex}
	\begin{enumerate}[(a)]
		\ii Since $\Delta^0 = [v_0]$ is just a point,
		a singular $0$-simplex $X$ is just a point of $X$.
		\ii Since $\Delta^1 = [v_0, v_1]$ is an interval,
		a singular $1$-simplex $X$ is just a path in $X$.
		\ii Since $\Delta^2 = [v_0, v_1, v_2]$ is an equilateral triangle,
		a singular $2$-simplex $X$ looks a ``disk'' in $X$.
	\end{enumerate}
	Here is a picture of all three in a space $X$:
	\begin{center}
		\begin{asy}
			size(8cm);
			bigblob("$X$");
			dot("$\sigma^0$", (-3.7,2.5), dir(-90));

			pen hg = heavygreen;
			pair A0 = (-2.7, 1.1);
			pair A1 = (-2.9, -2.3);
			dot("$v_0$", A0, dir(90), hg);
			dot("$v_1$", A1, dir(-90), hg);
			path s1 = A0..(-3.1,0.7)..A1;
			draw(s1, hg, EndArrow);
			label("$\sigma^1$", s1, dir(10), hg);

			pen rr = red;
			pair B0 = (1, 1.8);
			pair B1 = (-0.7, -1.1);
			pair B2 = (1.9, -1.9);
			dot("$v_0$", B0, dir(90), rr);
			dot("$v_1$", B1, dir(210), rr);
			dot("$v_2$", B2, dir(330), rr);
			pair b0 = (0,0);
			pair b1 = (0,-2);
			pair b2 = (1.6,0.3);
			draw(B0..b0..B1, rr, EndArrow);
			draw(B1..b1..B2, rr, EndArrow);
			draw(B2..b2..B0, rr, EndArrow);
			fill(B0..b0..B1--B1..b1..B2--B2..b2..B0--cycle, lightred+opacity(0.2));
			label("$\sigma^2$", (B0+B1+B2)/3, rr);
		\end{asy}
	\end{center}
	The arrows aren't strictly necessary, but I've included them
	to help keep track of the ``order'' of the vertices;
	this will be useful in just a moment.
\end{example}

Now we're going to do something much like
when we were talking about Stokes' theorem:
we'll put a boundary $\partial$ operator on the singular $n$-simplices.
This will give us a formal linear sums of $n$-simplices $\sum_k a_k \sigma_k$,
which we call an \vocab{$n$-chain}.

In that case,
\begin{definition}
	Given a singular $n$-simplex $\sigma$ with vertices $[v_0, \dots, v_n]$,
	note that for every $i$ we have an $(n-1)$ simplex $[v_0, \dots, v_{i-1}, v_{i+1}, \dots, v_n]$.
	The \vocab{boundary operator} $\partial$ is then defined by
	\[ \partial(\sigma) \defeq \sum_i (-1)^i
		\left[ v_0, \dots, v_{i-1}, v_{i+1}, \dots, v_n \right]. \]
	The boundary operator then extends linearly to $n$-chains:
	\[ \partial\left( \sum_k a_k \sigma_k \right) \defeq \sum a_k \partial(\sigma_k). \]
	By convention, a $0$-chain has empty boundary.
\end{definition}
\begin{example}
	[Boundary operator]
	Consider the chains depicted in \Cref{ex:simplex}. Then
	\begin{enumerate}[(a)]
		\ii $\partial\sigma^0 = 0$.
		\ii $\partial(\sigma^1) = [v_1] - [v_0]$:
		it's the ``difference'' of the $0$-chain corresponding to point $v_1$
		and the $0$-chain corresponding to point $v_0$.
		\ii $\partial(\sigma^2) = [v_0,v_1] - [v_0,v_2] + [v_1, v_2]$;
		i.e.\ one can think of it as the sum of the three oriented arrows
		which make up the ``sides'' of $\sigma^2$.
		\ii Notice that if we take the boundary again, we get
		\begin{align*}
			\partial(\partial(\sigma^2))
			&= \partial([v_0,v_1]) - \partial([v_0,v_2]) + \partial([v_1,v_2]) \\
			&= \left( [v_1]-[v_0] \right) - \left( [v_2]-[v_0] \right) + \left( [v_2]-[v_1] \right)  \\
			&= 0.
		\end{align*}
	\end{enumerate}
\end{example}

The fact that $\partial^2 = 0$ is of course not a coincidence.
\begin{theorem}
	[$\partial^2=0$]
	For any chain $c$, $\partial(\partial(c)) = 0$.
\end{theorem}
\begin{proof}
	Essentially identical to \Cref{prob:partial_zero}:
	this is just a matter of writing down a bunch of $\sum$ signs.
	Diligent readers are welcome to try the computation.
\end{proof}
\begin{remark}
	The eerie similarity between the chains used to integrate differential forms
	and the chains in homology is not a coincidence.
	The de Rham cohomology, discussed much later, will make the relation explicit.
\end{remark}

\section{The singular homology groups}
\prototype{Probably $H_n(S^m)$, especially the case $m = n =1$.}
Let $X$ be a topological space, and let $C_n(X)$ be the free abelian group
of $n$-chains of $X$ that we defined earlier.
Our work above gives us a boundary operator $\partial$, so we have a sequence of maps
\[ \dots \taking\partial C_3(X) \taking\partial C_2(X)
	\taking\partial C_1(X) \taking\partial C_0(X) \taking\partial 0 \]
(here I'm using $0$ to for the trivial group, which is standard notation for abelian groups.)
We'll call this the \vocab{singular chain complex}.

Now, how does this let us detect holes in the space?
To see why, let's consider an annulus, with a $1$-chain $c$ drawn in red:
\begin{center}
	\begin{asy}
		size(6cm);
		pair O = origin;
		filldraw(CR(O, 6), lightblue+opacity(0.2), blue);
		filldraw(CR(O, 1), white, blue);
		label("$X$", 6*dir(45), dir(45));
		pen rr = red;
		pair v0 = 5*dir(100);
		pair v1 = 4*dir(220);
		pair v2 = 2.2*dir(330);
		dot("$v_0$", v0, dir(v0), rr);
		dot("$v_1$", v1, dir(v1), rr);
		dot("$v_2$", v2, dir(v2), rr);
		draw(v0..(3.2*dir(110))..v1, rr, EndArrow);
		draw(v1..(2.1*dir(270))..v2, rr, EndArrow);
		draw(v2..(3.6*dir(45))..v0, rr, EndArrow);
	\end{asy}
\end{center}
Notice that
\[ \partial c = ([v_1]-[v_0]) - ([v_2]-[v_0]) + ([v_2]-[v_1]) = 0 \]
and so we can say this $1$-chain $c$ is a ``cycle'',
because it has trivial boundary.
However, $c$ is not itself the boundary of any $2$-chain,
because of the hole in the center of the space
--- it's impossible to ``fill in'' the interior of $c$!
So, we have detected the hole by the algebraic fact that
\[ c \in \ker\left( C_1(X) \taking{\partial} C_0(X) \right)
	\qquad\text{but}\qquad
	c \notin \img \left( C_2(X) \taking{\partial} C_1(X) \right). \]
Indeed, if the hole was not present then this statement would be false.

We can capture this idea in any dimension, as follows.
\begin{definition}
	Let
	\[ \dots \taking\partial C_2(X) \taking\partial C_1(X) \taking\partial C_0(X) \taking\partial 0 \]
	as above.
	We say that $c \in C_n(X)$ is:
	\begin{itemize}
		\ii a \vocab{cycle} if $c \in \ker\left( C_{n}(X) \taking\partial C_{n-1}(X) \right)$,
		and
		\ii a \vocab{boundary} if $c \in \img \left( C_{n+1}(X) \taking\partial C_n(X) \right)$.
	\end{itemize}
	Denote the cycles and boundaries by $Z_n(X), B_n(X) \subseteq C_n(X)$, respectively.
\end{definition}

\begin{ques}
	Just to get you used to the notation:
	check that $B_n$ and $Z_n$ are themselves abelian groups,
	and that $B_n(X) \subseteq Z_n(X) \subseteq C_n(X)$.
\end{ques}

The key point is that we can now define:
\begin{definition}
	The \vocab{$n$th homology group} $H_n(X)$ is
	defined as \[ H_n(X) \defeq Z_n(X) / B_n(X). \]
\end{definition}
\begin{example}
	[The zeroth homology group]
	Let's compute $H_0(X)$ for a topological space $X$.
	We take $C_0(X)$, which is just formal linear sums of points of $X$.

	First, we consider the kernel of $\partial : C_0(X) \to 0$,
	so the kernel of $\partial$ is the entire space $C_0(X)$:
	that is, every point is a ``cycle''.

	Now, what is the boundary?
	The main idea is that $[b] - [a] = 0$ if and only if
	there's a $1$-chain which connects $a$ to $b$, i.e.\
	there is a path from $a$ to $b$.
	In particular,
	\[ \text{$X$ path connected} \implies H_0(X) \cong \ZZ. \]
\end{example}
More generally, we have
\begin{proposition}[Homology groups split into path-connected components]
	If $X = \bigcup_\alpha X_\alpha$ is a decomposition into path-connected components,
	then we have \[ H_n(X) \cong \bigoplus_\alpha H_n(X_\alpha). \]
	In particular, if $X$ has $r$ path-connected components, then $H_0(X) \cong \ZZ^{\oplus r}$.
\end{proposition}
(If it's surprising to see $\ZZ^{\oplus r}$, remember that
an abelian group is the same thing as a $\ZZ$-module,
so the notation $G \oplus H$ is customary in place of $G \times H$
when $G$, $H$ are abelian.)

Now let's investigate the first homology group.
\begin{theorem}[Hurewicz theorem]
	Let $X$ be path-connected.
	Then $H_1(X)$ is the \emph{abelianization} of $\pi_1(X, x_0)$.
\end{theorem}
We won't prove this but you can see it roughly from the example.
The group $H_1(X)$ captures the same information
as $\pi_1(X, x_0)$: a cycle (in $Z_1(X)$) corresponds to the same thing
as the loops we studied in $\pi_1(X, x_0)$,
and the boundaries (in $B_1(X)$, i.e.\ the things we mod out by)
are exactly the nulhomotopic loops in $\pi_1(X, x_0)$.
The difference is that $H_1(X)$ allows loops to commute,
whereas $\pi_1(X, x_0)$ does not.
\begin{example}
	[The first homology group of the annulus]
	To give a concrete example, consider the annulus $X$ above.
	We found a chain $c$ that wrapped once around the hole of $X$.
	The point is that in fact,
	\[ H_1(X) = \left< c\right> \cong \ZZ \]
	which is to say the chains $c$, $2c$, \dots are all not the same in $H_1(X)$,
	but that any other $1$-chain is equivalent to one of these.
	This captures the fact that $X$ is really just $S^1$.
\end{example}
\begin{example}
	[An explicit boundary in $S^1$]
	\label{ex:S1_c_minus_d}
	In $X = S^1$, let $a$ be the uppermost point and $b$ the lowermost point.
	Let $c$ be the simplex from $a$ to $b$ along the left half of the circle,
	and $d$ the simplex from $a$ to $b$ along the right half.
	Finally, let $\gamma$ be the simplex which represents a loop $\gamma$ from $a$
	to itself, wrapping once counterclockwise around $S^1$.
	We claim that in $H^1(S^1)$ we have
	\[ \gamma = c - d \]
	which geometrically means that $c-d$ represents wrapping once around
	the circle (which is of course what we expect).

	\begin{center}
		\begin{asy}
			size(5cm);
			real r = 0.8;
			pair a = dir(90);
			pair b = dir(-90);

			fill(unitcircle, lightblue+opacity(0.2));
			unfill(CR( -(1-r)/2*a, (1+r)/2 ));

			draw(arc(origin, a, -a), red);
			draw(arc((r-1)*a/2, -a, r*a), red, EndArrow);

			dot("$v_0=a$", r*a, dir(-90));
			dot("$v_1=a$", a, dir(90));
			dot("$v_2=b$", b, dir(-90));
			label("$\gamma$", dir(135), dir(135), red);
			draw(arc((r-1)*a/2, r*a, -a), blue, EndArrow);
			draw(arc(origin, 1, 90, -90), heavygreen, EndArrow);
			label("$c$", (1+r)/2*dir(180), dir(0), blue);
			label("$d$", dir(0), dir(0), heavygreen);
		\end{asy}
	\end{center}

	Indeed this can be seen from the picture above, where we have drawn
	a $2$-simplex whose boundary is exactly $\gamma - c + d$.
	The picture is somewhat metaphorical: in reality $v_0 = v_1 = a$,
	and the entire $2$-simplex is embedded in $S^1$.
	This is why singular homology is so-called: the images of the simplex
	can sometimes look quite ``singular''.
\end{example}

\begin{example}
	[The first homology group of the figure eight]
	Consider $X_8$ (see \Cref{ex:figure8}).
	Both homology and homotopy see the two loops in $X_8$, call them $a$ and $b$.
	The difference is that in $\pi_1(X_8, x_0)$,
	these two loops are not allowed to commute: we don't have $ab \neq ba$,
	because the group operation in $\pi_1$ is ``concatenate paths''
	But in the homology group $H_1(X)$ the way we add $a$ and $b$
	is to add them formally, to get the $1$-chain $a+b$.
	So \[ H_1(X) \cong \ZZ^{\oplus 2} \quad\text{while}\quad \pi_1(X, x_0) = \left< a,b\right>. \]
\end{example}

\begin{example}
	[The homology groups of $S^2$]
	Consider $S^2$, the two-dimensional sphere.
	Since it's path connected, we have $H_0(S^2) = \ZZ$.
	We also have $H_1(S^2) = 0$, for the same reason that $\pi_1(S^2)$ is trivial as well.
	On the other hand we claim that \[ H_2(S^2) \cong \ZZ. \]
	The elements of $H_2(S^2)$ correspond to wrapping $S^2$ in a tetrahedral bag
	(or two bags, or three bags, etc.).
	Thus, the second homology group lets us detect the spherical cavity of $S^2$.
\end{example}
Actually, more generally it turns out that we will have
\[
	H_n(S^m) \cong
	\begin{cases}
		\ZZ & n=m \text{ or } n=0 \\
		0 & \text{otherwise}.
	\end{cases}
\]

\begin{example}
	[Contractible spaces]
	Given any contractible space $X$, it turns out that
	\[
		H_n(X)
		\cong
		\begin{cases}
			\ZZ & n = 0 \\
			0 & \text{otherwise}.
		\end{cases}
	\]
	The reason is that, like homotopy groups, it turns out
	that homology groups are homotopy invariant.
	(We'll prove this next section.)
	So the homology groups of contractible $X$ are the same
	as those of a one-point space, which are those above.
\end{example}

\begin{example}
	[Homology groups of the torus]
	While we won't be able to prove it for a while, it turns out that
	\[
		H_n(S^1 \times S^1)
		\cong
		\begin{cases}
			\ZZ & n = 0,2 \\
			\ZZ^{\oplus 2} & n = 1 \\
			0 & \text{otherwise}.
		\end{cases}
	\]
	The homology group at $1$ corresponds to our knowledge that $\pi_1(S^1 \times S^1) \cong \ZZ^2$
	and the homology group at $2$ detects the ``cavity'' of the torus.
\end{example}

This is fantastic and all, but how does one go about actually computing any homology groups?
This will be a rather long story, and we'll have to do a significant amount of both algebra and geometry
before we're really able to compute any homology groups.
In what follows, it will often be helpful to keep track of which things are purely algebraic
(work for any chain complex), and which parts are actually stating something which is geometrically true.

\section{The homology functor and chain complexes}
As I mentioned before, the homology groups are homotopy invariant.
This will be a similar song and dance as the work we did to
create a functor $\pi_1 : \catname{hTop}_\ast \to \catname{Grp}$.
Rather than working slowly and pulling away the curtain to reveal the category theory at the end,
we'll instead start with the category theory right from the start just to save some time.

\begin{definition}
	The category $\catname{hTop}$ is defined as follows:
	\begin{itemize}
		\ii Objects: topological spaces.
		\ii Morphisms: \emph{homotopy classes} of morphisms $X \to Y$.
	\end{itemize}
	In particular, $X$ and $Y$ are isomorphic in $\catname{hTop}$
	if and only if they are homotopic.
\end{definition}
You'll notice this is the same as $\catname{hTop}_\ast$,
except without the basepoints.

\begin{theorem}[Homology is a functor $\catname{hTop} \to \catname{Grp}$]
	\label{thm:Hn_functor}
	For any particular $n$, $H_n$ is a functor $\catname{hTop} \to \catname{Grp}$.
	In particular,
	\begin{itemize}
		\ii Given any map $f : X \to Y$, we get an induced map $f_\ast : H_n(X) \to H_n(Y)$.
		\ii For two homotopic maps $f, g : X \to Y$, $f_\ast = g_\ast$.
		\ii Two homotopic spaces $X$ and $Y$ have isomorphic homology groups:
		if $f : X \to Y$ is a homotopy then $f_\ast : H_n(X) \to H_n(Y)$ is an isomorphism.
		\ii (Insert your favorite result about functors here.)
	\end{itemize}
\end{theorem}

In order to do this, we have to describe how to take a map $f : X \to Y$
and obtain a map $H_n(f) : H_n(X) \to H_n(Y)$.
Then we have to show that this map doesn't depend on the choice of homotopy.
(This is the analog of the work we did with $f_\sharp$ before.)
It turns out that this time around, proving this is much more tricky,
and we will have to go back to the chain complex
$C_\bullet(X)$ that we built at the beginning.

\subsection{Algebra of chain complexes}
Let's start with the algebra.
First, I'll define the following abstraction of the complex to any sequence of abelian groups.
Actually, though, it works in any category (not just $\catname{AbGrp}$).
The strategy is as follows: we'll define everything that we need completely abstractly,
then show that the geometry concepts we want correspond to this setting.

\begin{definition}
	A \vocab{chain complex} is a sequence of groups $A_n$ and maps
	\[ \dots \taking{\partial} A_{n+1} \taking{\partial}
		A_n \taking{\partial} A_{n-1} \taking{\partial} \dots \]
	such that the composition of any two adjacent maps is the zero morphism.
	We we usually denote this by $A_\bullet$.

	The $n$th homology group $H_n(A_\bullet)$ is defined
	as $\ker(A_n \to A_{n-1}) / \img(A_{n+1} \to A_n)$.
	Cycles and boundaries are defined in the same way as before.
\end{definition}
Obviously, this is just an algebraic generalization of the structure we previously looked at,
rid of all its original geometric context.

\begin{definition}
	A \vocab{morphism of chain complexes} $f : A_\bullet \to B_\bullet$ is
	a sequence of maps $f_n$ for every $n$ such that the diagram
	\begin{diagram}
		\dots &\rTo^{\partial_A} & A_{n+1} & \rTo^{\partial_A} &
		A_n & \rTo^{\partial_A} & A_{n-1} & \rTo^{\partial_A} & \dots \\
		&& \dTo_{f_{n+1}} && \dTo_{f_n} && \dTo_{f_{n-1}} && \\
		\dots & \rTo^{\partial_B} & B_{n+1} & \rTo^{\partial_B} &
		B_n & \rTo^{\partial_B} & B_{n-1} & \rTo^{\partial_B} & \dots \\
	\end{diagram}
	commutes.
	Under this definition, the set of chain complexes becomes a category,
	which we denote $\catname{Cmplx}$.
\end{definition}

Note that given a morphism of chain complexes $f : A_\bullet \to B_\bullet$,
every cycle in $A_n$ gets sent to a cycle in $B_n$, since the square
\begin{diagram}
	A_n & \rTo^{\partial_A} & A_{n-1} \\
	\dTo^{f_n} & & \dTo \\
	B_n & \rTo_{\partial_B} & B_{n-1} \\
\end{diagram}
commutes.
Similarly, every boundary in $A_n$ gets sent to a boundary in $B_n$. Thus,
\begin{moral}
	Every map of $f : A_\bullet \to B_\bullet$ gives a
	map $f_\ast : H_n(A) \to H_n(B)$ for every $n$.
\end{moral}
\begin{exercise}
	Interpret $H_n$ as a functor $\catname{Cmplx} \to \catname{Grp}$.
\end{exercise}

Next, we want to define what it means for two maps $f$ and $g$ to be homotopic.
Here's the answer:
\begin{definition}
	Let $f, g : A_\bullet \to B_\bullet$.
	Suppose that one can find a map $P_n : A_n \to B_{n+1}$ for every $n$ such that
	\[ g_n - f_n = \partial_B \circ P_n + P_{n-1} \circ \partial_A \]
	Then $P$ is a \vocab{chain homotopy} from $f$ to $g$
	and $f$ and $g$ are \vocab{chain homotopic}.
\end{definition}

We can draw a picture to illustrate this (warning: the diagonal dotted arrows do NOT commute
with all the other arrows):
\begin{diagram}
	\dots &\rTo^{\partial_A} & A_{n+1} & \rTo^{\partial_A} &
	A_n & \rTo^{\partial_A} & A_{n-1} & \rTo^{\partial_A} & \dots \\
	&& \dTo~{g-f} & \ldDotted~{P_n} & \dTo~{g-f}
	& \ldDotted~{P_{n-1}} & \dTo~{g-f} && \\
	\dots & \rTo^{\partial_B} & B_{n+1} & \rTo^{\partial_B} &
	B_n & \rTo^{\partial_B} & B_{n-1} & \rTo^{\partial_B} & \dots \\
\end{diagram}
The definition is that in each slanted ``parallelogram'', the $g-f$ arrow is the sum of the two
compositions along the sides.

\begin{remark}
	This equation should look terribly unmotivated right now,
	aside from the fact that we are about to show it does the right algebraic thing.
	Its derivation comes from the geometric context that we have deferred
	until the next section, where ``homotopy'' will naturally give ``chain homotopy''.
\end{remark}

Now, the point of this definition is that
\begin{proposition}[Chain homotopic maps induce the same map on homology groups]
	Let $f, g: A_\bullet \to B_\bullet$ be chain homotopic maps $A_\bullet \to B_\bullet$.
	Then the induced maps $f_\ast, g_\ast : H_n(A_\bullet) \to H_n(B_\bullet)$ coincide for each $n$.
\end{proposition}
\begin{proof}
	It's equivalent to show $g-f$ gives the zero map on homology groups,
	In other words, we need to check that every cycle of $A_n$ becomes
	a boundary of $B_n$ under $g-f$.
	\begin{ques}
		Verify that this is true. \qedhere
	\end{ques}
\end{proof}

\subsection{Geometry of chain complexes}
Now let's fill in the geometric details of the picture above.
First:
\begin{lemma}[Map of space $\implies$ map of singular chain complexes]
	Each $f : X \to Y$ induces a map $C_n(X) \to C_n(Y)$.
\end{lemma}
\begin{proof}
	Take the composition
	\[ \Delta^n \taking{\sigma} X \taking{f} Y. \]
	In other words, a path in $X$ becomes a path in $Y$, et cetera.
	(It's not hard to see that the squares involving $\partial$ commute;
	check it if you like.)
\end{proof}

Now, what we need is to show that if $f , g : X \to Y$ are homotopic,
then they are chain homotopic.
To produce a chain homotopy, we need to take every $n$-simplex $X$
to an $(n+1)$-chain in $Y$, thus defining the map $P_n$.

Let's think about how we might do this. Let's take the $n$-simplex $\sigma : \Delta^n \to X$
and feed it through $f$ and $g$; pictured below is a $1$-simplex $\sigma$ (i.e.\ a path in $X$)
which has been mapped into the space $Y$.
Homotopy means the existence of a map $F : X \times [0,1] \to Y$
such that $F(-,0) = f$ and $F(-,1) = g$, parts of which I've illustrated below with grey arrows
in the image for $Y$.

\begin{center}
	\begin{asy}
		size(14cm);
		bigblob("$Y$");
		pair A = (-3.5,2);
		pair B = (-2.2,-2);
		pair C = (2.2,1.6);
		pair D = (2.0, -1.1);

		pen rr = red;
		dot("$v_0$", A, dir(90), rr); dot("$v_1$", B, dir(-090), rr);
		path v = A..(-2,0)..B;
		draw(v, rr, EndArrow);
		label("$f(\sigma)$", v, dir(180), rr);

		pen hg = heavygreen;
		dot("$w_0$", C, dir(90), hg); dot("$w_1$", D, dir(-90), hg);
		path w = C..(2.3,0)..D;
		draw(w, hg, EndArrow);
		label("$g(\sigma)$", w, dir(0), hg);

		pen gr = grey + dashed;
		draw(A--C, gr, Margins);
		path F = A--D;
		draw(F, gr, Margins);
		label("$F$", F, dir(90), gr);
		draw(B--D, gr, Margins);

		add(shift(12.5,0)*CC());
		pair W = 1.5*dir(135);
		pair X = 1.5*dir(225);
		pair Y = 1.5*dir(45);
		pair Z = 1.5*dir(-45);
		dot("$v_0$", W, dir(W), rr);
		dot("$v_1$", X, dir(X), rr);
		dot("$w_0$", Y, dir(Y), hg);
		dot("$w_1$", Z, dir(Z), hg);

		label("$\Delta^1 \times [0,1]$", 2*dir(90), dir(90));
		pair A1 = (1.5,0);
		pair A2 = (3.5,0);
		pair A3 = (6.25,0);
		pair A4 = (7.5,0);
		draw(A1--A2, EndArrow);
		draw(A3--A4, EndArrow);
		label("$\boxed{X \times [0,1]}$", midpoint(A2--A3), blue);
		label("$F$", A3--A4, dir(90));
		label("$\sigma \times \operatorname{id}$", A1--A2, dir(90));

		draw(W--X, rr, EndArrow);
		draw(Y--Z, hg, EndArrow);
		draw(W--Y, gr);
		draw(X--Z, gr);
		draw(W--Z, gr);
	\end{asy}
\end{center}

This picture suggests how we might proceed:
we want to create a $2$-chain on $Y$ given the $1$-chains we've drawn.
The homotopy $F$ provides us with a ``square'' structure on $Y$,
i.e.\ the square bounded by $v_0$, $v_1$, $w_1$, $w_0$.
We split this up into two triangles; and that's our $2$-chain.

We can make this formal by taking $\Delta^1 \times [0,1]$ (which \emph{is} a square)
and splitting it into two triangles.
Then, if we apply $\sigma \times \id$, we'll get an $2$-chain in $X \times [0,1]$,
and then finally applying $F$ will map everything into our space $Y$.
In our example, the final image is the $2$-chain, consisting of two triangles,
which in our picture can be written as $[v_0, w_0, w_1] - [v_0, v_1, w_1]$;
the boundaries are given by the red, green, grey.

More generally, for an $n$-simplex $\phi = [x_0, \dots, x_n]$
we define the so-called \emph{prism operator} $P_n$
as follows. Set $v_i = f(x_i)$ and $w_i = g(x_i)$ for each $i$.
Then, we let
\[ P_n(\phi) \defeq \sum_{i=0}^n (-1)^i (F \circ (\phi\times\id))
	\left[ v_0, \dots, v_i, w_i, \dots, w_n \right]. \]
This is just the generalization of the construction above to dimensions $n > 1$;
we split $\Delta^n \times [0,1]$ into $n+1$ simplices,
map it into $X$ by $\phi \times \id$ and then push the whole thing into $Y$.
The $(-1)^i$ makes sure that the ``diagonal'' faces all cancel off with each other.

We now claim that for every $\sigma$,
\[ \partial_Y(P_n(\sigma)) = g(\sigma) - f(\sigma) - P_{n-1}(\partial_X\sigma). \]
In the picture, $\partial_Y \circ P_n$ is the boundary of the entire prism
(in the figure, this becomes the red, green, and grey lines, not including diagonal grey,
which is cancelled out).
The $g-f$ is the green minus the red,
and the $P_{n-1} \circ \partial_X$ represents the grey edges of the prism
(not including the diagonal line from $v_1$ to $w_0$).
Indeed, one can check (just by writing down several $\sum$ signs)
that the above identity holds.

So that gives the chain homotopy from $f$ to $g$, completing the proof of \Cref{thm:Hn_functor}.

\section{More examples of chain complexes}
We now end this chapter by providing some more examples of chain complexes,
which we'll use in the next chapter to finally compute topological homology groups.

\begin{example}
	[Reduced homology groups]
	Suppose $X$ is a (nonempty) topological space.
	One can augment the standard singular complex as follows:
	do the same thing as before, but augment the end by adding a $\ZZ$,
	as shown:
	\[ \dots \to C_1(X) \to C_0(X) \taking{\eps} \ZZ \to 0 \]
	Here $\eps$ is defined by $\eps(\sum n_ip_i) = \sum n_i$ for points $p_i \in X$.
	(Recall that a $0$-chain is just a formal sum of points!)
	We denote this \vocab{augmented singular chain complex} by $\wt C_\bullet(X)$.
\end{example}
\begin{ques}
	What's the homology of the above chain at $\ZZ$?
	(Hint: you need $X$ nonempty.)
\end{ques}
\begin{definition}
	\label{def:augment}
	The homology groups of the augmented chain complex are called the
	\vocab{reduced homology groups} $\wt H_n(X)$ of the space $X$.

	Obviously $\wt H_n(X) \cong H_n(X)$ for $n > 0$.
	But when $n=0$, the map $H_0(X) \to \ZZ$ by $\eps$ has kernel $\wt H_0(X)$,
	thus $H_0(X) \cong \wt H_0(X) \oplus \ZZ$.
\end{definition}
This is usually just an added convenience.
For example, it means that if $X$ is contractible,
then all its reduced homology groups vanish,
and thus we won't have to keep fussing with the special $n=0$ case.
\begin{ques}
	Given the claim earlier about $H_n(S^m)$, what should $\wt H_n(S^m)$ be?
\end{ques}

\begin{example}
	[Relative chain groups]
	Suppose $X$ is a topological space, and $A \subseteq X$ a subspace.
	We can ``mod out'' by $A$ by defining
	\[ C_n(X,A) \defeq C_n(X) / C_n(A) \]
	for every $n$. Thus chains contained entirely in $A$ are trivial.

	Then, the usual $\partial$ on $C_n(X)$ generates a new chain complex
	\[ \dots \taking\partial C_{n+1}(X,A) \taking\partial C_n(X,A)
	\taking\partial C_{n-1}(X,A) \taking\partial \dots. \]

	This is well-defined since $\partial$ takes $C_n(A)$ into $C_{n-1}(A)$.
\end{example}
\begin{definition}
	The homology groups of the relative chain complex are the
	\vocab{relative homology groups} and denoted $H_n(X,A)$.
\end{definition}

One na\"{\i}ve guess is that this might equal $H_n(X) / H_n(A)$.
This is not true and in general doesn't even make sense;
if we take $X$ to be $\RR^2$ and $A = S^1$ a circle inside it,
we have $H_1(X) = H_1(\RR^2) = 0$ and $H_1(S^1) = \ZZ$.

Another guess is that $H_n(X,A)$ might just be $\wt H_n(X/A)$.
This will turn out to be true for most reasonable spaces $X$ and $A$,
and we will discuss this when we reach the excision theorem.

\begin{example}
	[Mayer-Vietoris sequence]
	Suppose a space $X$ is covered by two open sets $U$ and $V$.
	We can define $C_n(U+V)$ as follows:
	it consists of chains such that each simplex is either entirely contained in $U$,
	or entirely contained in $V$.

	Of course, $\partial$ then defines another chain complex
	\[ \dots \taking\partial C_{n+1}(U+V) \taking\partial C_n(U+V)
	\taking\partial C_{n-1}(U+V) \taking\partial \dots. \]
\end{example}
So once again, we can define homology groups for this complex;
we denote them by $H_n(U+V)$.
Miraculously, it will turn out that $H_n(U+V) \cong H_n(X)$.

\section\problemhead

\begin{problem}
	For $n \ge 1$ show that the composition
	\[ S^{n-1} \injto D^{n} \taking{F} S^{n-1} \]
	cannot be the identity map on $S^{n-1}$ for any $F$.
	\begin{hint}
		Take the $n-1$st homology groups.
	\end{hint}
	\begin{sol}
		Applying the functor $H_{n-1}$ we get
		that the composition $\ZZ \to 0 \to \ZZ$ is the identity
		which is clearly not possible.
	\end{sol}
\end{problem}
\begin{problem}
	[Brouwer fixed point theorem]
	Use the previous problem to prove that any
	continuous function $f : D^n \to D^n$ has a fixed point.
	\begin{hint}
		Build $F$ as follows:
		draw the ray from $x$ through $f(x)$
		and intersect it with the boundary $S^{n-1}$.
	\end{hint}
\end{problem}


\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own
\providecommand{\mb}[1]{\boldsymbol{#1}}
\providecommand{\mv}[1]{\vec{#1}}
\providecommand{\ve}[1]{\boldsymbol{#1}}
\newcommand{\bpsi}{\ve{\psi}}
\newcommand{\bv}{\mb{v}}
\newcommand{\bx}{\mb{x}}
\newcommand{\bA}{\mathbf{A}}
\newcommand{\bH}{\mathbf{H}}
\newcommand{\bX}{\mathbf{X}}
\newcommand{\bP}{\mathbf{P}}
\newcommand{\bB}{\mathbf{B}}
\newcommand{\bD}{\mathbf{D}}
\newcommand{\bS}{\mathbf{S}}
\newcommand{\bR}{\mathbf{R}}
\newcommand{\bJ}{\mathbf{J}}
\newcommand{\bLambda}{\mathbf{\Lambda}}
\begin{document}
\subsection{Joint Embedding}
We developed a method to jointly embed multiple graphs/networks. Previous spectral embedding techniques work on each graph separately. Our joint embedding approach generalizes Adjacency Spectral Embedding to multiple graphs. Specifically,  the joint embedding method identifies a linear subspace spanned by rank one symmetric matrices and projects adjacency matrices of graphs into this subspace. Given $m$ graphs $\{G_i \} _{i=1}^{m}$ with $\bA_i$ being the corresponding adjacency matrix, the $d$-dimensional joint embedding of graphs $\{G_i \} _{i=1}^{m}$ is given by
 \begin{equation*}
 	\begin{aligned}
 	(\hat{\bH},\hat{\bD}_1,...,\hat{\bD}_m) =	& \underset{\bD_i,\|h_k\|=1}{\operatorname{argmin}}
 		& & \sum\limits_{i=1}^{m} \| \bA_i- \bH \bD_i \bH^T \|  ^2 \\
 		& \text{ subject to}
 		& &  \bD_i \text{ being diagonal.}
 	\end{aligned}
 \end{equation*}
Here, $h_k$ is the $k$th column of matrix $\bH$. The $\hat{\bH}$ are estimated latent positions for vertices, and the diagonal of $\hat{\bD}_i$ can be treat as the feature of graph $i$. We performed theoretical and numerical analysis of the joint embedding. The code and paper can be found \href{https://github.com/shangsiwang/Joint-Embedding}{here}.

We study predicting individual composite creativity index (CCI) through brain connectomes obtained by Multimodal Magnetic Resonance Imaging. In total, $113$ healthy, young adult subjects were scanned and their CCI is assessed by independent judges using the Consensual Assessment Technique. First, we jointly embed brain graphs of all subjects. Figure \ref{fig:cci1} shows a typical graph and $\hat{h}_6 \hat{h}_6^T$ estimated by the joint embedding. Next, we construct a linear regression model by treating the diagonal of $\hat{\bD}_i$ as explanatory variables and CCI as the response variable.

Overall, the regression model for predicting CCI is significant at level $0.05$ compared to the null model. We found CCI positively correlated to overall connectivity of the brain. Furthermore, we found CCI significantly negatively related to $\hat{h}_6 \hat{h}_6^T$. This implies that compared to within hemisphere connectivity across hemisphere connectivity has a larger positive impact on human creativity.

%%%  FIGURE BLOCK
\begin{figure}[!h]
\begin{cframed}
\centering
\includegraphics[scale=0.25]{../../figs/cci_data.png}
\caption{The top panel shows the graph derived from a typical subject. There is much more neural connectivity within each hemisphere. The bottom panel shows the rank one matrix $\hat{h}_6^T\hat{h}_6$, which has positive  connectivity within each hemisphere, but negative  connectivity across hemispheres.}
\label{fig:cci1}
\end{cframed}
\end{figure}
%
\clearpage
\end{document}


  \documentclass{mynotes}

%\geometry{showframe}% for debugging purposes -- displays the margins

\newcommand{\E}{\mbox{E}}
\newcommand{\MSE}{\mbox{MSE}}
\newcommand{\var}{\mbox{var}}
\newcommand{\bx}{\mathbf{x}}

\usepackage{amsmath}
%\usepackage[garamond]{mathdesign}
\usepackage{url}

% Set up the images/graphics package
\usepackage{graphicx}
\setkeys{Gin}{width=\linewidth,totalheight=\textheight,keepaspectratio}
\graphicspath{{graphics/}}

\title[Exercises 4 $\cdot$ SSC 383D]{Exercises 4: Hierarchical Models}
%\author[ ]{ }
\date{}  % if the \date{} command is left out, the current date will be used

% The following package makes prettier tables.  We're all about the bling!
\usepackage{booktabs}

% The units package provides nice, non-stacked fractions and better spacing
% for units.
\usepackage{units}

% The fancyvrb package lets us customize the formatting of verbatim
% environments.  We use a slightly smaller font.
\usepackage{fancyvrb}
\fvset{fontsize=\normalsize}

% Small sections of multiple columns
\usepackage{multicol}

% Provides paragraphs of dummy text
\usepackage{lipsum}

% These commands are used to pretty-print LaTeX commands
\newcommand{\doccmd}[1]{\texttt{\textbackslash#1}}% command name -- adds backslash automatically
\newcommand{\docopt}[1]{\ensuremath{\langle}\textrm{\textit{#1}}\ensuremath{\rangle}}% optional command argument
\newcommand{\docarg}[1]{\textrm{\textit{#1}}}% (required) command argument
\newenvironment{docspec}{\begin{quote}\noindent}{\end{quote}}% command specification environment
\newcommand{\docenv}[1]{\textsf{#1}}% environment name
\newcommand{\docpkg}[1]{\texttt{#1}}% package name
\newcommand{\doccls}[1]{\texttt{#1}}% document class name
\newcommand{\docclsopt}[1]{\texttt{#1}}% document class option name

\newcommand{\N}{\mbox{N}}
\newcommand{\thetahat}{\hat{\theta}}
\newcommand{\sigmahat}{\hat{\sigma}}
\newcommand{\betahat}{\hat{\beta}}

\begin{document}

\maketitle% this prints the handout title, author, and date

\section{Hierarchical models: data-analysis problems}

\subsection{Math tests}

The data set in ``mathtest.csv'' shows the scores on a standardized math test from a sample of 10th-grade students at 100 different U.S.~urban high schools, all having enrollment of at least 400 10th-grade students.  (A lot of educational research involves ``survey tests'' of this sort, with tests administered to all students being the rare exception.)

Let $\theta_i$ be the underlying mean test score for school $i$, and let $y_{ij}$ be the score for the $j$th student in school $i$.  Starting with the ``mathtest.R'' script, you'll notice that the extreme school-level averages $\bar{y}_i$ (both high and low) tend to be at schools where fewer students were sampled.

\begin{enumerate}
\item Explain briefly why this would be.
\item Fit this normal hierarchical model to these data via Gibbs sampling:
\begin{eqnarray*}
y_{ij} &\sim& \mbox{N}(\theta_i, \sigma^2) \\
\theta_i &\sim& \mbox{N}(\mu, \tau^2 \sigma^2)
\end{eqnarray*}
Decide upon sensible priors for the unknown model parameters $(\mu, \sigma^2, \tau^2)$.

\item Suppose you use the posterior mean $\hat{\theta}_i$ from the above model to estimate each school-level mean $\theta_i$.  Define the shrinkage coefficient $\kappa_i$ as
$$
\kappa_i = \frac{ \bar{y}_i - \hat{\theta}_i}{\bar{y}_i} \, ,
$$
which tells you how much the posterior mean shrinks the observed sample mean.  Plot this shrinkage coefficient (in absolute value) for each school as a function of that school's sample size, and comment.

\end{enumerate}

\subsection{Price elasticity of demand}

The data in ``cheese.csv'' are about sales volume, price, and advertisting display activity for packages of Borden sliced ``cheese.'' The data are taken from Rossi, Allenby, and McCulloch's textbook on \textit{Bayesian Statistics and Marketing.} For each of 88 stores (store) in different US cities, we have repeated observations of the weekly sales volume (vol, in terms of packages sold), unit price (price), and whether the product was advertised with an in-store display during that week (disp = 1 for display).

Your goal is to estimate, on a store-by-store basis, the effect of display ads on the demand curve for cheese.  A standard form of a demand curve in economics is of the form $Q = \alpha P^\beta$, where $Q$ is quantity demanded (i.e.~sales volume), $P$ is price, and $\alpha$ and $\beta$ are parameters to be estimated.  You'll notice that this is linear on a log-log scale,
$$
\log Q = \log \alpha + \beta \log P \,
$$
which you should feel free to assume here.  Economists would refer to $\beta$ as the price elasticity of demand (PED).  Notice that on a log-log scale, the errors enter multiplicatively.

There are several things for you to consider in analyzing this data set.
\begin{compactenum}
\item The demand curve might shift (different $\alpha$) and also change shape (different $\beta$) depending on whether there is a display ad or not in the store.
\item Different stores will have very different typical volumes, and your model should account for this.
\item Do different stores have different PEDs?  If so, do you really want to estimate a separate, unrelated $\beta$ for each store?
\item If there is an effect on the demand curve due to showing a display ad, does this effect differ store by store, or does it look relatively stable across stores?
\item Once you build the best model you can using the log-log specification, do see you any evidence of major model mis-fit?
\end{compactenum}
Propose an appropriate hierarchical model that allows you to address these issues, and use Gibbs sampling to fit your model.

\subsection{A hierarchical probit model via data augmentation}

Read the following paper:
\begin{quotation}
``Bayesian Analysis of Binary and Polychotomous Response Data.''  James H. Albert and Siddhartha Chib.  \textit{Journal of the American Statistical Association}, Vol.~88, No.~422 (Jun.,~1993), pp.~669-679
\end{quotation}
The surefire way to get this paper is via access to JStor through the UT Library website.  Let me know if this is an issue for you.

The paper describes a Bayesian treatment of probit regression (similar to logistic regression) using the trick of \textit{data augmentation}---that is, introducing ``latent variables'' that turn a hard problem into a much easier one.  Briefly summarize your understanding of the key trick proposed by this paper.  Then see you if you can apply the trick in the following context, which is more complex than ordinary probit regression.

In ``polls.csv'' you will find the results of several political polls from the 1988 U.S.~presidential election.  The outcome of interest is whether someone plans to vote for George Bush (senior, not junior).  There are several potentially relevant demographic predictors here, including the respondent's state of residence.  The goal is to understand how these relate to the probability that someone will support Bush in the election.  You can imagine this information would help a great deal in poll re-weighting and aggregation (ala Nate Silver).

Use Gibbs sampling, together with the Albert and Chib trick, to fit a hierarchical probit model of the following form:
\begin{eqnarray*}
\mbox{Pr}(y_{ij} = 1) &=& \Phi(z_{ij})  \\
z_{ij} &=& \mu_i + x_{ij}^T \beta_i \, .
\end{eqnarray*}
Here $y_{ij}$ is the response (Bush=1, other=0) for respondent $j$ in state $i$; $\Phi(\cdot)$ is the probit link function, i.e.~the CDF of the standard normal distribution; $\mu_i$ is a state-level intercept term; $x_{ij}$ is a vector of respondent-level demographic predictors; and $\beta_i$ is a vector of regression coefficients for state $i$.

Notes:
\begin{enumerate}
\item There are severe imbalances among the states in terms of numbers of survey respondents. Following the last problem, the key is to impose a hierarchical prior on the state-level parameters.
\item The data-augmentation trick from the Albert and Chib paper above is explained in many standard references on Bayesian analysis.  If you want to get a quick introduction to the idea, you can consult one of these.  A good presentation is in Section 8.1.1 of ``Bayesian analysis for the social sciences'' by Simon Jackman, available as an ebook through lib.utexas.edu.
\item You are welcome to use the logit model instead of the probit model.  If you do this, you'll need to read the following paper, rather than Albert and Chib: Polson, N.G., Scott, J.G. and Windle, J. (2013). Bayesian inference for logistic models using Polya-Gamma latent variables. J. Amer. Statist. Assoc. 108 1339--1349.
\end{enumerate}

%\subsection{Gene expression over time}
%
%In \verb|droslong.csv|, you will find a small subset of a time-course DNA microarray experiment.  The gene-expression profiles of 2000 different genes in the fruit fly (Drosophila) genome are tracked over time during embryogenesis; you are getting data on 14 of these genes, organized in three groups (think of these as marking which cellular pathway that gene influences).  For each gene at each time point, there are 3 ``technical replicates''---that is, three copies of the same biological material from the same fly, run through the same process to measure gene expression.
%
%The question of interest is: how does each gene's expression profile change over time, as the process of embryogenesis unfolds?  Propose a hierarchical model for this data that properly reflects its structure.  Fit this model using Gibbs sampling.
%
%A nice graphics package is the ``lattice'' library.  Install and load this; then try commands such as
%\begin{verbatim}
%xyplot(log2exp~time | gene, data=droslong)
%xyplot(log2exp~time | group, data=droslong)
%\end{verbatim}
%to begin exploring the structure of this data.  If you know ggplot2, you can easily accomplish similar things.

\end{document}


\documentclass[a4paper,11pt]{article}

\input{packages.tex}
\input{tikz.tex}
\input{thmstyle.tex}
\input{macros.tex}

% opening
\title{Notes on compatible composita}
\author{}

\begin{document}

\maketitle

%\begin{abstract}

%\end{abstract}

%\tableofcontents

%\clearpage

In this document, we investigate compatibility questions in the case of the
compositum of two finite fields $\mathbb{F}_{p^m}$ and $\mathbb{F}_{p^n}$, where
the integers $m$ and $n$ are coprime. In that case, the compositum is
$\mathbb{F}_{p^{mn}}$. We denote by $\zeta_m$ (resp. $\zeta_n$) a primitive $m$-th
root of unity (resp. $n$-th). We set $\alpha_m$ a solution of Hilbert 90 in
$\mathbb{F}_{p^m}\otimes\mathbb{F}_{p}(\zeta_m)$ for the root $1\otimes\zeta_m$,
and we similarly set $\alpha_n$. Let now $u, v$ be integers such that
\[
  un+vm = 1.
\]
We now set
\[
  \zeta=\zeta_m^u\zeta_n^v
\]
and
\[
  \alpha=\alpha_m^u\alpha_n^v,
\]
such that $\zeta$ is a primitive $mn$-th root of unity, $\zeta^m=\zeta_n$,
$\zeta^n=\zeta_m$ and $\alpha$ is a
solution of Hilbert 90 in $\mathbb{F}_{p^{mn}}\otimes\mathbb{F}_{p}(\zeta)$ for
the root $1\otimes\zeta$. We also have that
\[
  (\sigma\otimes\Id)(\alpha^n) = (1\otimes\zeta_m)\alpha^n
\]
and
\[
  (\sigma\otimes\Id)(\alpha^m) = (1\otimes\zeta_n)\alpha^m
\]
but not necessarily $\alpha^n=\alpha_m$ and $\alpha^m=\alpha_n$. In fact we have
\begin{align*}
  \alpha^n &= \alpha_m^{un}\alpha_n^{vn} \\
  &= \alpha_m^{1-vm}a_n^v \\
  &= \cfrac{a_n^v}{a_m^v}\alpha_m
\end{align*}
where $a_m=\alpha_m^m\in\mathbb{F}_p(\zeta_m)$ and $a_n =
\alpha_n^n\in\mathbb{F}_p(\zeta_n)$. We also get
\[
  \alpha^m = \cfrac{a_m^u}{a_n^u}\alpha_n
\]
in the same way.
\section{Example with $p=7$, $m=2$, $n=3$}
Let us start with an example where we do not have to worry about tensor
products, because all the needed roots are in the prime field. We take $p=7$,
$m=2$ and $n=3$. Since $p-1=6$ is divisible by both $2$ and $3$, we have
primitive $2$-th and $3$-th roots of unity in $\mathbb{F}_7$. We investigate the
possible embeddings $\mathbb{F}_{7^2}\emb\mathbb{F}_{7^6}$ and
$\mathbb{F}_{7^3}\emb\mathbb{F}_{7^6}$, as shown in Figure~\ref{fig:p7}.
\begin{figure}
  \centering
\begin{tikzpicture}
  \node (1) at (0,0) {$\mathbb{F}_{7}$};
  \node (2) at (-1,1) {$\mathbb{F}_{7^2}$};
  \node (3) at (1,1) {$\mathbb{F}_{7^3}$};
  \node (6) at (0,2) {$\mathbb{F}_{7^6}$};

  \draw (1) -- (2);
  \draw (1) -- (3);
  \draw (2) -- (6);
  \draw (3) -- (6);
\end{tikzpicture}
\phantom{and}
\begin{tikzpicture}
  \node (1) at (0,0) {$1$};
  \node (-1) at (-1,1) {$-1$};
  \node (2) at (0.8,1) {$2$};
  \node (4) at (1.2,1) {$4$};
  \node (3) at (-0.4,2) {$3$};
  \node (5) at (0.4,2) {$5$};

  \draw (1) -- (2);
  \draw (1) -- (-1);
  \draw (1) -- (4);
  \draw (-1) -- (3);
  \draw (-1) -- (5);
  \draw (2) -- (3);
  \draw (4) -- (5);
\end{tikzpicture}
  \caption{Compositum with $p=7$, $m=2$, $n=3$, and the associated roots.}
  \label{fig:p7}
\end{figure}
In the same picture, we see that we have one primitive $2$-th root of unity that
is $-1$, two primitive $3-th$ roots of unity that are $2$ and $4$ and two
primitive $6$-th roots of unity that are $3$ and $5$. We have the compatibility
relations $3^3=5^3=-1$, $3^2=2$ and $5^2=4$.

We define
\[
  \mathbb{F}_{7^2}\cong \mathbb{F}_7[y]/(y^2-3)\cong \mathbb{F}_7(u),
\]
\[
  \mathbb{F}_{7^3}\cong \mathbb{F}_7[x]/(x^3-2)\cong \mathbb{F}_7(t),
\]
and
\[
  \mathbb{F}_{7^6}\cong \mathbb{F}_7[z]/(z^6-5)\cong \mathbb{F}_7(w).
\]
We embed $\mathbb{F}_{7^2}$ in $\mathbb{F}_{7^6}$ by sending $u$ to $w^3$ and we
embed $\mathbb{F}_{7^3}$ in $\mathbb{F}_{7^6}$ by sending $t$ to $3w^4$.

\end{document}


\subsection{Hybrid orbital representation}
\label{sec:spo_hybrid}
The hybrid representation of the single particle orbitals combines a localized atomic basis set around atomic cores and B-splines in the interstitial regions to reduce the memory usage while retaining high speed of evaluation and either retaining or increasing overall accuracy. Full details are provided in Ref.~\cite{Luo2018hyb}.
In practice, we have seen using meshfactor=0.5 is often possible and achieves huge memory saving.
Figure~\ref{fig:hybridrep} illustrates how the regions are assigned. Orbitals within region A are computed as
\[
  \phi^A_n({\bf r})=R_{n,l,m}(r)Y_{l,m}(\hat{r})
\]
Orbitals in region C are computed as the regular B-spline basis described in subsection~\ref{sec:spo_spline} above. The region B interpolates between A and C as
\begin{align}
  \phi^B_n({\bf r}) &= S(r) \phi^A_n({\bf r}) + (1-S(r))\phi^C_n({\bf r}) \\
                S(r) &= \frac{1}{2}-\frac{1}{2}\tanh\left[\alpha\left(\frac{r-r_{\rm A/B}}{r_{\rm B/C}-r_{\rm A/B}}-\frac{1}{2}\right)\right]
\end{align}

\begin{figure}
\centering
\includegraphics[trim={0 152 0 0},clip,width=0.45\columnwidth]{./figures/hybrid_new.jpg}
\qquad
\includegraphics[trim={0 2 0 150},clip,width=0.45\columnwidth]{./figures/hybrid_new.jpg}
\caption{Illustration of regular and hybrid orbital representation. Regular B-spline representation (left panel) contains only one region and a sufficiently fine mesh to resolve orbitals near the nucleus. The hybrid orbital representation (right panel) contains near nucleus (A) regions where spherical harmonics and radial functions are used, buffers (B) or interpolation regions, and an interstitial (C) region where a coarse B-spline mesh is utilized.}
\label{fig:hybridrep}
\end{figure}

To enable hybrid orbital representation, the input XML needs to see the tag \texttt{hybridrep="yes"} shown below.
\begin{lstlisting}[caption=Hybrid orbital representation input example.\label{listing:hybridrep}]
<determinantset type="bspline" source="i" href="pwscf.h5"
                tilematrix="1 1 3 1 2 -1 -2 1 0" twistnum="-1" gpu="yes" meshfactor="0.8"
                twist="0  0  0" precision="single" hybridrep="yes">
  ...
</determinantset>
\end{lstlisting}
Second, the infomation describing the atomic regions is required in the particle set, shown below
\begin{lstlisting}[caption=particleset elements for ions with information needed by hybrid orbital representation.\label{listing:hybridrep_particleset}]
  <group name="Ni">
    <parameter name="charge">          18 </parameter>
    <parameter name="valence">         18 </parameter>
    <parameter name="atomicnumber" >   28 </parameter>
    <parameter name="cutoff_radius" > 1.6 </parameter>
    <parameter name="inner_cutoff" >  1.3 </parameter>
    <parameter name="lmax" >            5 </parameter>
    <parameter name="spline_radius" > 1.8 </parameter>
    <parameter name="spline_npoints">  91 </parameter>
  </group>
\end{lstlisting}

The parameters specific to hybrid representation are listed as
\begin{table}[h]
\centering
\begin{tabularx}{\textwidth}{l l l l l l }
\hline
\multicolumn{6}{l}{\texttt{attrib} element} \\
\hline
\multicolumn{2}{l}{attribute      :} & \multicolumn{4}{l}{}\\
   &   \bfseries name            & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{cutoff\_radius}             &  real            &  $>=0.0$    &  \textit{none}    & The cutoff radius for B/C boundary  \\
   &   \texttt{lmax}         &  integer            &  $>=0$ &  \textit{none} & Largetst angular channel \\
   &   \texttt{inner\_cutoff}             &  real            &  $>=0.0$    &  dep.    & The cutoff radius for A/B boundary  \\
   &   \texttt{spline\_radius}         &  real            &  $>0.0$ &  dep. & radial function radius used in spline \\
   &   \texttt{spline\_npoints}         &  integer            &  $>0$ &  dep. & Number of spline knots \\
  \hline
\end{tabularx}
\end{table}

\begin{itemize}
  \item \texttt{cutoff\_radius} is required for every species. If a species is intended not being covered by atomic regions, setting the value 0.0 will put default values for all the reset parameters. A good value is usually a bit larger than the core radius listed in the pseudopotential file. After a parametric scan, pick the one from the flat energy region with the smallest variance.
  \item \texttt{lmax} is required if $\texttt{cutoff\_radius} > 0.0$. The value usually needs to be at least the highest angular momentum plus 2.
  \item \texttt{inner\_cutoff} is optional and set as $\texttt{cutoff\_radius}-0.3$ by default which is fine in most cases.
  \item \texttt{spline\_radius} and \texttt{spline\_npoints} are optional. By default, they are calculated based on \texttt{cutoff\_radius} and a grid distplacement $0.02$\,bohr.
        If users prefer inputing them, it is required that $\texttt{cutoff\_radius}<=\texttt{spline\_radius}-2\times\texttt{spline\_radius}/(\texttt{spline\_npoints}-1)$.
\end{itemize}

In addition, the hybrid orbital representation allows extra optimization to speed up the non-local pseudopotential evaluation using the batched algorithm listed in Sec.~\ref{sec:nlpp}.


\documentclass{memoir}
\usepackage{notestemplate}

%\logo{~/School-Work/Auxiliary-Files/resources/png/logo.png}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

%\makeindex

\begin{document}

% \maketitle

% Notes taken on

\subsection{Riemann Sphere}
\label{sub:riemann_sphere}

%% ADD RIEMANN SPHERE

Under construction

\subsection{Linear Transformations}
\label{sub:linear_transformations}

\begin{defn}[Linear Fractional Transformation]
	A \textbf{linear fractional transformation} or \textbf{Möbius transformation} is a linear transformation of the complex plane given by
	\begin{align*}
		S(z) := \frac{az+b}{cz+d}
	\end{align*}
	for complex numbers \(a,b,c,d \in \C\) with one of \(c,d\) non-zero.\\

	We extend \(S\) to \(\overline{\C}\) by defining the following values:
\begin{align*}
	S(\infty) = \begin{cases}
		\sfrac{a}{c} & c\neq 0\\
		\infty & c=0
	\end{cases}\\
	S(-\sfrac{d}{c}) = \infty \text{ if }c\neq 0.
\end{align*}
	This extends the mapping so that \(S\) is a topological mapping of the extended plane onto itself, with the topology of distances on the Riemannian sphere.
\end{defn}
This is the generalized form of all linear transformations on \(\overline{\C}\)-- translations, reflections, rotations, and more can be represented as linear fractional transformations.

\begin{anki}
TARGET DECK
Complex Qual::Complex Analysis
START
MathJaxCloze
Text: A **linear fractional transformation** or **Möbius transformation** is a linear transformation of the complex plane given by
{{c1::\(\begin{align*}
        	S(z) := \frac{az+b}{cz+d}
        \end{align*}\)}}
for complex numbers \(a,b,c,d \in \C\) with one of \(c,d\) non-zero.

If we write \(z = \frac{z_1}{z_2}\), we can express \(S(z)\) as a matrix via
{{c1::\(\begin{align*}
		\begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} z_1 \\ z_2 \end{pmatrix} = \begin{pmatrix} w_1 \\ w_2 \end{pmatrix}
	\end{align*}\)}}
where \(w := \frac{w_1}{w_2} = S(z)\). If {{c1::\(ad-bc = 1\)}}, we say the linear fractional transformation is **normalized**.

Extra: We extend \(S\) to \(\overline{\C}\) by defining the following values:
\(\begin{align*}
  	S(\infty) = \begin{cases}
  		\sfrac{a}{c} & c\neq 0\\
  		\infty & c=0
  	\end{cases}\\
  	S(-\sfrac{d}{c}) = \infty \text{ if }c\neq 0.
  \end{align*}\)
This extends the mapping so that \(S\) is a topological mapping of the extended plane onto itself, with the topology of distances on the Riemannian sphere.
Tags: analysis complex_analysis complex_geometry defn
<!--ID: 1624827134946-->
END
\end{anki}

\begin{prop}
	Let \(S\) be a linear fractional transformation
	\begin{align*}
		S(z) = \frac{az+b}{c z + d}
	\end{align*}
	with \(ad-bc\neq 0\). Then
	\begin{align*}
		S:\overline{\C}\to \overline{\C}
	\end{align*}
	is an isomorphism. Furthermore, the inverse of \(S\) is given by
	\begin{align*}
		S^{-1}(z) = \frac{dz -b}{a-cz}.
	\end{align*}
	If \(ad-bc = 1\), then we say the linear fractional transformation is \textbf{normalized}.
\end{prop}

Every linear fractional transformation admits a normalized form. In fact, there are exactly two, which are obtained from each other by changing the sign of the coefficients.

\begin{anki}
START
MathJaxCloze
Text: Let \(S\) be a linear fractional transformation
\(\begin{align*}
  	S(z) = \frac{az+b}{c z + d}
\end{align*}\)
	with \(ad-bc\neq 0\). Then
{{c1::\(\begin{align*}
	S:\overline{\C}\to \overline{\C}
      \end{align*}\)}}
	is {{c1::an isomorphism}}. Furthermore, the inverse of \(S\) is given by
{{c1::\(\begin{align*}
	S^{-1}(z) = \frac{dz -b}{a-cz}.
      \end{align*}\)}}
	If {{c1::\(ad-bc = 1\)}}, then we say the linear fractional transformation is \textbf{normalized}.
Extra: Every linear fractional transformation admits a normalized form. In fact, there are exactly two, which are obtained from each other by changing the sign of the coefficients.
Tags: analysis complex_analysis complex_geometry
<!--ID: 1626152754001-->
END
\end{anki}

\begin{general}[Matrix Formulation of Linear Fractional Transformations]
	Let \(z = \frac{z_1}{z_2}\) be an arbitrary complex number and
	\begin{align*}
		S(z) = \frac{az+b}{cz+d}.
	\end{align*}
	Then we can express this linear fractional translation by
	\begin{align*}
		\begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} z_1 \\ z_2 \end{pmatrix} = \begin{pmatrix} w_1 \\ w_2 \end{pmatrix}
	\end{align*}
	where \(w := \frac{w_1}{w_2} = S(z)\). This representation as matrices is useful as it satisfies the standard matrix operations when it comes to addition, composition, inversion, and so on.\\

	Hence, the set of linear fractional transformations forms a (matrix) group. The identity transformation is given by the identity matrix. The rest of the group properties we leave to be checked by the reader.\\

	In fact, if we restrict ourselves to normalized representations, this matrix group is isomorphic to \(SL(2,\C)\). If furthermore we form an equivalence class on normalized representations, then the group is isomorphic to the \textbf{projective special linear group} \(PSL_2(\C)\).
\end{general}

If
\begin{align*}
	A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}
\end{align*}
then we may denote the linear fractional transformation
\begin{align*}
	S(z) = \frac{az+b}{cz+d}
\end{align*}
simply by \(S_A\).

\begin{prop}[Equivalence of Fractional Linear Transformations]
	Let \(A,A' \in \C^{2\times 2}\) with \(\textrm{det}(A)\neq 0\) and \(\textrm{det}(A') \neq 0\). Then
	\begin{align*}
		S_A = S_{A'} \iff A' = \lambda A
	\end{align*}
	for some \(\lambda \in \C\) non-zero.
\end{prop}

\begin{anki}
START
MathJaxCloze
Text: **Equivalence of Fractional Linear Transformations**
Let \(A,A' \in \mathbb{C}^{2\times 2}\) with \(\textrm{det}(A)\neq 0\) and \(\textrm{det}(A') \neq 0\). Then
{{c1::\(\begin{align*}
        	S_A = S_{A'} \iff A' = \lambda A
        \end{align*} \)}}
	for some \(\lambda \in \C\) non-zero.
Tags: analysis complex_analysis complex_geometry
<!--ID: 1626152754018-->
END
\end{anki}

\begin{defn}[Important Classes of Linear Transformations]
	The linear fractional transformations of the form
	\begin{align*}
		T_\alpha (z) := \begin{pmatrix} 1 & \alpha \\ 0 & 1 \end{pmatrix}
	\end{align*}
	for \(\alpha  \in \C\) are called \textbf{parallel translations}, as they correspond to the transformation \(S(z) = z + \alpha \).\\

	The linear fractional transformations of the form
	\begin{align*}
		R_\alpha (z) := \begin{pmatrix} \alpha  & 0 \\ 0 & 1 \end{pmatrix}
	\end{align*}
	are referred to as \textbf{rotations} if \(\left| \alpha  \right| = 1\), or a \textbf{homothetic transformation} if \(\alpha \) is real with \(\alpha >0\). For arbitrary complex \(\alpha \neq 0\), we can write
	\begin{align*}
		\alpha  = \left| \alpha  \right| \frac{\alpha }{\left| \alpha  \right| }
	\end{align*}
	and hence the general form can be viewed as a composition of a homothetic transformation with a rotation.\\

	The linear fractional transformation
	\begin{align*}
	(z)^{-1} := \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}
	\end{align*}
	is referred to as an \textbf{inversion}, as it represents \(S(z) = \frac{1}{z}\).
\end{defn}
In some sense, we can view these classes of translations as the fundamental linear transformations. If \(S(z)\) is given by
\begin{align*}
	\frac{az+b}{cz+d}
\end{align*}
with \(c\neq 0\), then we can write
\begin{align*}
	S(z) &= \frac{az+b}{c z+d} = \frac{bc-ad}{c^2(z+\frac{d}{c})}+ \frac{a}{c}\\
	&=  T_{\sfrac{a}{c}}\circ R_{\frac{bc-ad}{c^2}}\circ (\cdot)^{-1} \circ T_{\sfrac{d}{c}}
\end{align*}
In the simpler case where \(c=0\), we have
\begin{align*}
	S(z) &= \frac{az+b}{d}\\
	&= T_{\sfrac{b}{d}}\circ R_{\sfrac{a}{d}}
\end{align*}

\begin{anki}
START
MathJaxCloze
Text: The linear fractional transformations of the form
 {{c1::\(\begin{align*}
         	T_\alpha (z) := \begin{pmatrix} 1 & \alpha \\ 0 & 1 \end{pmatrix}
         \end{align*}\)}}
for \(\alpha  \in \C\) are called **parallel translations**, as they correspond to the transformation {{c1::\(S(z) = z + \alpha \)}}.

The linear fractional transformations of the form
 {{c2::\(\begin{align*}
         	R_\alpha (z) := \begin{pmatrix} \alpha  & 0 \\ 0 & 1 \end{pmatrix}
         \end{align*}\)}}
are referred to as **rotations** if {{c2::\(\left| \alpha  \right| = 1\)}}, or a **homothetic transformation** if {{c2::\(\alpha \) is real with \(\alpha >0\)}}. For arbitrary complex \(\alpha \neq 0\), we can write
 {{c2::\(\begin{align*}
        	\alpha  = \left| \alpha  \right| \frac{\alpha }{\left| \alpha  \right| }
        \end{align*}\)}}
and hence the general form can be viewed as {{c2::a composition of a homothetic transformation with a rotation}}.

The linear fractional transformation
 {{c3::\(\begin{align*}
        (z)^{-1} := \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}
        \end{align*}\)}}
is referred to as an **inversion**, as it represents {{c3::\(S(z) = \frac{1}{z}\)}} .
Extra: If \(S(z)\) is given by
\(\begin{align*}
  	\frac{az+b}{cz+d}
  \end{align*}\)
with \(c\neq 0\), then we can write
\(\begin{align*}
  	S(z) &= \frac{az+b}{c z+d} = \frac{bc-ad}{c^2(z+\frac{d}{c})}+ \frac{a}{c}\\
  	&=  T_{\sfrac{a}{c}}\circ R_{\frac{bc-ad}{c^2}}\circ (\cdot)^{-1} \circ T_{\sfrac{d}{c}}
  \end{align*}\)
In the simpler case where \(c=0\), we have
\(\begin{align*}
  	S(z) &= \frac{az+b}{d}\\
  	&= T_{\sfrac{b}{d}}\circ R_{\sfrac{a}{d}}
  \end{align*}\)
Tags: analysis complex_analysis complex_geometry defn
<!--ID: 1624827135038-->
END
\end{anki}

\begin{thm}
	A fractional linear transformation maps straight lines and circles onto straight lines and circles.
\end{thm}
\begin{proof}

\end{proof}

\begin{anki}
START
MathJaxCloze
Text: A fractional linear transformation \(S(z) = \frac{az+b}{c z + d}\), \(ac-bd\neq 0\) maps straight lines and circles onto {{c1::straight lines and circles}}.
Tags: analysis complex_analysis complex_geometry
<!--ID: 1626152754035-->
END
\end{anki}

\begin{lemma}
	Let \(S\) be a fractional linear map. If \(S(\infty) = \infty\), then
	\begin{align*}
		S(z) = az+b
	\end{align*}
	for \(a,b \in \C\).
\end{lemma}

\begin{anki}
START
MathJaxCloze
Text: Let \(S\) be a fractional linear map. If \(S(\infty) = \infty\), then
{{c1::\(\begin{align*}
        	S(z) = az+b
        \end{align*}\)
      	for \(a,b \in \C\)}}.
Tags: analysis complex_analysis complex_geometry
<!--ID: 1626152754052-->
END
\end{anki}

\begin{thm}
	Given any three distinct points \(\left\{ z_1,z_2,z_3 \right\} \) with \(z_i \in \overline{\C}\) and three distinct points \(\left\{ w_1,w_2,w_3 \right\} \) with \(w_i \in \overline{\C}\), there exists a unique fractional linear map \(S\) with
	\begin{align*}
		S(z_i) = w_i.
	\end{align*}
	If \(\left\{ w_1,w_2,w_3 \right\} = \left\{ 0,\infty,1 \right\} \) then the map is given by
	\begin{align*}
		S(z) = \frac{z-z_1}{z-z_2}\cdot \frac{z_3-z_2}{z_3-z_1}.
	\end{align*}
	We can apply this to the general case to get the relation
	\begin{align*}
		\frac{w-w_1}{w-w_2}\cdot \frac{w_3-w_2}{w_3-w_1} = \frac{z-z_1}{z-z_2}\cdot \frac{z_3-z_2}{z_3-z_1}.
	\end{align*}
	We can use this relation to derive an explicit formula for specific values.
\end{thm}

\begin{lemma}
	If \(S\) is a fractional linear map with three fixed points, then \(S\) is the identity map.
\end{lemma}

\begin{proof}

\end{proof}

\begin{anki}
START
MathJaxCloze
Text: Given any three distinct points \(\left\{ z_1,z_2,z_3 \right\} \) with \(z_i \in \overline{\C}\) and three distinct points \(\left\{ w_1,w_2,w_3 \right\} \) with \(w_i \in \overline{\C}\), there exists a {{c1::unique fractional linear map \(S\)::map}} with
{{c1::\(\begin{align*}
        	S(z_i) = w_i.
        \end{align*}\)}}
If \(\left\{ w_1,w_2,w_3 \right\} = \left\{ 0,\infty,1 \right\} \) then the map is given by
 {{c1::\(\begin{align*}
        	S(z) = \frac{z-z_1}{z-z_2}\cdot \frac{z_3-z_2}{z_3-z_1}.
        \end{align*}\)}}
We can apply this to the general case to get the relation
 {{c1::\(\begin{align*}
        	\frac{w-w_1}{w-w_2}\cdot \frac{w_3-w_2}{w_3-w_1} = \frac{z-z_1}{z-z_2}\cdot \frac{z_3-z_2}{z_3-z_1}.
        \end{align*}\)}}
We can use this relation to derive an explicit formula for specific values.
Extra: If \(S\) is a fractional linear map with three fixed points, then \(S\) is the identity map.
Tags: analysis complex_analysis complex_geometry
<!--ID: 1626152754068-->
END
\end{anki}

%% Lots more details can be filled here, TBD based on later content

% \printindex
\end{document}


\documentclass{scrartcl}
\usepackage[a4paper,left=1in,right=1in,top=1.2in,bottom=1in]{geometry}
\usepackage{siunitx}
\usepackage{graphicx}
\usepackage{mathtools}
\setkomafont{disposition}{\normalfont\bfseries}
\newcommand*\diff{\mathop{}\!\mathrm{d}}
\newcommand*\Diff[1]{\mathop{}\!\mathrm{d^#1}}
\newcommand*\colvec[3][]{
    \begin{pmatrix}\ifx\relax#1\relax\else#1\\\fi#2\\#3\end{pmatrix}
}

%title
\title{Exercise 03:\\Development of ocular dominance}
\subtitle{Theoretical Neuroscience II}
\author{Johannes G\"atjen \and Lorena Morton}

%use these for structure/overview
\newcommand\Question{%
  \textbf{Question:}%
}
\newcommand\Answer{%
  \textbf{Answer:}%
}

\begin{document}
\maketitle

\section{Input covariance}

We choose three different values for $\gamma$ ($\gamma_1=0.5, \gamma_2=1, \gamma_3=1.5$) and compute the covariance matrices $\mathbf{C}$. The $c_S$ value does not change with $\gamma$ and is computed as
\begin{align*}
c_S &= \langle u_Ru_R \rangle - \langle u_R \rangle^2 \\
&= \left(1 \cdot 1 \cdot \frac{1}{2} + 0 \cdot 0 \cdot \frac{1}{2}\right) - \left(1 \cdot \frac{1}{2} + 0 \cdot \frac{1}{2}\right)^2 \\
&= \frac{1}{2} - \frac{1}{4} = \frac{1}{4}
\end{align*}
The values for $c_D$ are
\begin{align*}
c_D =& \langle u_Ru_L \rangle - \langle u_R \rangle \langle u_L \rangle \\
\gamma_1: \qquad =& 1 \cdot 1 \cdot \frac{1}{8} - \frac{1}{2} \cdot \frac{1}{2} = - \frac{1}{8} \\
\gamma_2:\qquad =& 1 \cdot 1 \cdot \frac{1}{4} - \frac{1}{2} \cdot \frac{1}{2} = 0 \\
\gamma_3:\qquad =& 1 \cdot 1 \cdot \frac{3}{8} - \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{8}.
\end{align*}
With a large enough sample size the numerical estimate of the covariances corresponds well with the analytical result.

\begin{figure}
\centering
\includegraphics[trim = {1.0cm 0 0.5cm 0.2cm}, width=0.3\textwidth, clip]{../pics/g1v}\includegraphics[trim = {1.0cm 0 0.5cm 0.2cm}, width=0.3\textwidth, clip]{../pics/g2v}\includegraphics[trim = {1.0cm 0 0.5cm 0.2cm}, width=0.3\textwidth, clip]{../pics/g3v}
\caption{Left to right: The eigenvectors of the covariance matrices for $\gamma_1, \gamma_2$ and $\gamma_3$ respectively. With the negative correlation the $[-1, 1]$ eigenvector dominates. With no correlation the two eigenvectors are equally long. With the positive correlation the $[1, 1]$ eigenvector dominates.}
\end{figure}

\section{Development with weight saturation}

Next we develop the input weights $\mathbf{w} = [w_L, w_R]^T$ for a neuron according to the covariance rule
\begin{align*}
\tau \frac{\diff \mathbf{w}}{\diff t} = \mathbf{C} \mathbf{w} \\
\shortintertext{which evaluates to}
\tau \colvec{\frac{\diff w_L}{\diff t}}{\frac{\diff w_R}{\diff t}} = \colvec{w_Lc_S + w_Rc_D}{w_Lc_D + w_Rc_S}
\end{align*}
giving us effectively two differential equations to use for update rules.

Figure \ref{cov} shows the development of the weights for the different values for $\gamma$ and random starting positions. With $\gamma = 1$ the weights develop entirely proportional to the initial condition, until they get to the boundary condition and eventually converge in the state for binocular vision. For $\gamma = 1.5$ the weights always grow roughly in the direction of the principal eigenvector, also converging in the state for binocular vision. With $\gamma = 0.5$ the weights also tend to grow in the direction of the principal eigenvector, but because of the weight saturation rule, whether the final state is monocular or binocular depends on the initial conditions of the weights. Where the initial weights are dissimilar and small enough the weights converge to monocular vision for the side with the higher initial weight. On the other hand, when the initial weights are too large or very similar the system will end in the binocular state.

In conclusion, for monocular neurons to develop, it is necessary, that the inputs are anti-correlated and the initial connection weights are not too similar.

\begin{figure}
\centering
\includegraphics[trim = {1.8cm 0 2.5cm 0.4cm}, width=0.4\textwidth, clip]{../pics/g1t1}
\includegraphics[trim = {1.8cm 0 2.5cm 0.4cm}, width=0.4\textwidth, clip]{../pics/g3t1}
\includegraphics[trim = {1.8cm 0 2.5cm 0.4cm}, width=0.4\textwidth, clip]{../pics/g2t1}
\caption{Development of connection weights with the covariance rule for different starting values in the range of $[0..0.5]$ and three different values for $\gamma$ ($\gamma = 0.5$ top left, $\gamma = 1.5$ top right, $\gamma = 1$ bottom).}
\label{cov}
\end{figure}

\section{Development with dynamic normalization}

Now we modify our update rule to include a dynamic normalization:
\begin{align*}
\tau \frac{\diff \mathbf{w}}{\diff t} &= \mathbf{C} \mathbf{w} - \frac{1}{2} \left( \mathbf{w}^T \mathbf{C} \mathbf{w} \right) \mathbf{w} \\
\text{Let } N &= \mathbf{w}^T \mathbf{C} \mathbf{w} \\
&= \colvec{w_L}{w_R}^T \colvec{w_Lc_S + w_Rc_D}{w_Lc_D + w_Rc_S} \\
&= w_L(w_Lc_S + w_Rc_D) + w_R(w_Lc_D + w_Rc_S) \\
&= w_L^2c_S + 2 w_Rw_Lc_D + w_R^2c_S
\shortintertext{and the update rule evaluates to}
\tau \colvec{\frac{\diff w_L}{\diff t}}{\frac{\diff w_R}{\diff t}} &= \colvec{w_Lc_S + w_Rc_D}{w_Lc_D + w_Rc_S} - \frac{N}{2} \colvec{w_L}{w_R}
\end{align*}

Figure \ref{dyn} shows the same initial conditions and $\gamma$ values as Figure \ref{cov} with the dynamic normalization rule added. In the case of $\gamma = 1$ there is no apparent difference to the development without the dynamic normalization. With $\gamma = 1.5$ the convergence towards the binocular state is stronger than without the dynamic normalization, making no difference for the final state. In the case of the anti-correlated input however, the weights now always end in the state for the monocular neuron where the side with the larger initial value is preferred. This means, that for this model it is sufficient that the inputs are anti-correlated and the initial weights are not exactly equal for monocular neurons to develop.

\begin{figure}
\centering
\includegraphics[trim = {1.8cm 0 2.5cm 0.4cm}, width=0.4\textwidth, clip]{../pics/g1t2}
\includegraphics[trim = {1.8cm 0 2.5cm 0.4cm}, width=0.4\textwidth, clip]{../pics/g3t2}
\includegraphics[trim = {1.8cm 0 2.5cm 0.4cm}, width=0.4\textwidth, clip]{../pics/g2t2}
\caption{Development of connection weights with the covariance rule for different starting values in the range of $[0..0.5]$ and three different values for $\gamma$ ($\gamma = 0.5$ top left, $\gamma = 1.5$ top right, $\gamma = 1$ bottom), including dynamic normalization.}
\label{dyn}
\end{figure}

\end{document}