This chapter describes operators that can be used in TBASIC.

\section{Overview}

There are 37 operators in TBASIC version \tbasver.

\section{Precedence}

Operators has something called \emph{precedence}. It's not something hard despite of its name (especially if you are not a native English speaker, like me). It's just a thing that describes our language programmatically\footnote{in computer language}, which goes like \emph{multiplication comes before the addition and subtraction}.

\begin{tabularx}{\textwidth}{c X}
	\textbf{Order} & \textbf{Operators}
	\\
	\endhead
	1 & :$=$\quad $=$\quad $+=$\quad $-=$\quad *$=$\quad /$=$\quad \%$=$ \\
	2 & OR \\
	3 & AND \\
	4 & | \\
	5 & XOR \\
	6 & \& \\
	7 & $=$$=$\quad !$=$\quad $<$$>$\quad $>$$<$ \\
	8 & $<$$=$\quad $>$$=$\quad $=$$<$\quad $=$$>$\quad $<$\quad $>$ \\
	9 & TO\quad STEP \\
	10 & $>$$>$$>$\quad $<$$<$\quad $>$$>$ \\
	11 & ; \\
	12 & $+$\quad $-$\\
	13 & *\quad /\quad \% \\
	14 & NOT\quad ! \\
	15 & $^\wedge$ \\
	16 & MINUS \\
\end{tabularx}

\section{Precautions}

Because of the way the language is desingned\footnote{don't write like \codebf{1+3*2+4/7}, they look hideous}, you must be careful when using $ - $ operator.

\begin{tabularx}{\textwidth}{l l X}
	\textbf{Code} & \textbf{Evaluation} & \textbf{Remarks}
	\\
	\endhead
	\code{FOO - 1} & subtraction(FOO, 1) & Variable FOO subtracted by 1
	\\
    \code{FOO -1} & FOO(MINUS 1) & Function FOO() with argument -1
    \\
    \code{FOO-1} & FOOMINUS1 & Syntax error
\end{tabularx}

\section{Arithmetic Operators}

\subsection{PLUS} \emph{number} $+$ \emph{number} --- adds two numbers
\subsection{MINUS} \emph{number} $-$ \emph{number} --- subtracts two numbers
\subsection{TIMES} \emph{number} * \emph{number} --- multiplies two numbers
\subsection{DIVIDE} \emph{number} / \emph{number} --- divides two numbers
\subsection{MODULO} \emph{number} \% \emph{number} --- gets modulo (remainder) of two numbers
\subsection{POWEROF} \emph{number a} $^\wedge$ \emph{number b} --- gets \emph{a} to the power of \emph{b}, or $a ^b$. $0 ^0$ will return $1$.

\section{Logical Operators}

\subsection{AND} \emph{condition 1} AND \emph{condition 2} --- returns \textbf{TRUE} if both conditions (boolean value) are true; \textbf{FALSE} otherwise
\subsection{OR} \emph{condition 1} OR \emph{condition 2} --- returns \textbf{TRUE} if one or more conditions (boolean value) are true; \textbf{FALSE} if both conditions are false
\subsection{NOT} NOT \emph{condition} --- negates condition (boolean value)

\section{Relational Operators}

\subsection{EQUALTO} \emph{number} $=$$=$ \emph{number} --- returns \textbf{TRUE} if two numbers represent same quantity; \textbf{FALSE} otherwise
\subsection{NOTEQUALTO} \emph{number} !$=$ \emph{number} --- returns \textbf{TRUE} if two numbers represent different quantity; \textbf{FALSE} otherwise; aliases: $<$$>$, $>$$<$
\subsection{GREATERTHAN} \emph{number a} $>$ \emph{number b} --- returns \textbf{TRUE} if $a > b$ is satisfied; \textbf{FALSE} otherwise
\subsection{LESSTHAN} \emph{number a} $<$ \emph{number b} --- returns \textbf{TRUE} if $a < b$ is satisfied; \textbf{FALSE} otherwise
\subsection{GEQTHAN} \emph{number a} $>$$=$ \emph{number b} --- returns \textbf{TRUE} if $a \geq b$ is satisfied; \textbf{FALSE} otherwise; alias: $=$$<$
\subsection{LEQTHAN} \emph{number a} $<$$=$ \emph{number b} --- returns \textbf{TRUE} if $a \leq b$ is satisfied; \textbf{FALSE} otherwise; alias: $=$$>$

\section{Assignment Operators}

\subsection{ASSIGN} \emph{variable} $=$ \emph{value} --- assign \emph{value} to \emph{variable}; alias: :$=$
\subsection{PLUSASSIGN} \emph{variable} $+=$ \emph{value} --- increment \emph{variable} by \emph{value} and update \emph{variable} to this new value
\subsection{MINUSASSIGN} \emph{variable} $-=$ \emph{value} --- decrement \emph{variable} by \emph{value} and update \emph{variable} to this new value
\subsection{TIMESASSIGN} \emph{variable} *$=$ \emph{value} --- multiply \emph{variable} by \emph{value} and update \emph{variable} to this new value
\subsection{DIVIDEASSIGN} \emph{variable} /$=$ \emph{value} --- divide \emph{variable} by \emph{value} and update \emph{variable} to this new value
\subsection{MODULOASSIGN} \emph{variable} \%$=$ \emph{value} --- modulo \emph{variable} by \emph{value} and update \emph{variable} to this new value

\section{Bitwise Operators}

\subsection{BAND} \emph{integer} \& \emph{integer} --- performs bitwise AND on two integers
\subsection{BOR} \emph{integer} | \emph{integer} --- performs bitwise OR on two integers
\subsection{BNOT} ! \emph{integer} --- performs bitwise NOT on integer
\subsection{BXOR} \emph{integer} XOR \emph{integer} --- performs bitwise XOR on two integers
\subsection{LSHIFT} \emph{integer} $<$$<$ \emph{offset} --- performs left shift on \emph{integer} by \emph{offset}
\subsection{RSHIFT} \emph{integer} $>$$>$ \emph{offset} --- performs signed right shift on \emph{integer} by \emph{offset}; if leftmost bit is 1, bits on the left side are padded with 1
\subsection{URSHIFT} \emph{integer} $>$$>$$>$ \emph{offset} --- performs unsigned right shift on \emph{integer} by \emph{offset}

\section{Miscellaneous Operators}

\subsection{CONCAT} \emph{string} ; \emph{string} --- concatenates two strings
\subsection{MINUS} $-$\emph{number} --- negates \emph{number}, or $-n$
\subsection{TO} \emph{number} TO \emph{number} --- creates integer sequence of specified range; \code{1 TO 4} will create an array of \emph{1, 2, 3, 4}; \code{8 TO 3} will create \emph{8, 7, 6, 5, 4, 3}
\subsection{STEP} \emph{int array} STEP \emph{number} --- filters input array such that every $1+n$th number is remained; \code{1 TO 10 STEP 3} will create an array of {1, 4, 7, 10}


% !TeX root = ../main.tex
\section{Theoretical background of ADC}

\subsection{Second Quantization}
Let $\ket{q}$ be a set of orthonormal one particle basis
\begin{equation}
\begin{aligned}
\langle q | q^{\prime}\rangle &=\delta_{q q^{\prime}} \\ \sum_{q} | q \rangle\langle q |&=1
\end{aligned}
\end{equation}

Define $| q_{1} \ldots q_{N} \rangle$ as the state of $| q_{1} \rangle \dots | q_{N} \rangle$ after antisymmetrization and normalization:
\begin{equation}
| q_{1} \ldots q_{N} \rangle =(N !)^{-\frac{1}{2}} \sum_{P}(-1)^{P} | q_{P(1)} \rangle \ldots | q_{P(N)} \rangle
\end{equation}
where $P$ is permutation of indexes.

In field theory framework, the particle number is subjected to change.
Thus we should consider Fock space rather than Hilbert space.
Fock space is defined by
\begin{equation}
\mathcal{F}=\mathcal{H}_{0} \oplus \mathcal{H}_{1} \oplus \mathcal{H}_{2}^{A} \oplus \cdots
\end{equation}
where only antisymmetric states is included when $N>2$.

Then we can define creation operators by
\begin{equation}
c_{q}^{\dagger} | q_{1} \ldots q_{N} \rangle=| q_{1} \ldots q_{N} q \rangle
\end{equation}
and annihilation operator by its hermitian adjoint.

Thus it's easy to calculate the annihilation operator as
\begin{equation}
\begin{aligned}
	c_{q} | q_{1} \ldots q_{N} \rangle&=\sum_{N^{\prime}} \sum_{q_{1}^{\prime}<q_{2}^{\prime} \cdots<q_{N^{\prime}}^{\prime}} | q_{1}^{\prime} \ldots q_{N^{\prime}}^{\prime} \rangle\langle q_{1}^{\prime} \ldots q_{N^{\prime}}^{\prime}|c_{q}| q_{1} \ldots q_{N}\rangle
	\\
	&=\sum_{N^{\prime}} \sum_{q_{1}^{\prime}<q_{2}^{\prime} \cdots<q_{N^{\prime}}^{\prime}} | q_{1}^{\prime} \ldots q_{N^{\prime}}^{\prime} \rangle\langle q_{1} \ldots q_{N}|c_{q}^{\dagger}| q_{1}^{\prime} \ldots q_{N^{\prime}}^{\prime}\rangle^{*}
\end{aligned}
\end{equation}

Creation operator and annihilation operator are related by anticommutation rule:
\begin{equation}
\begin{aligned}
	&\left(c_{p} c_{q}^{\dagger}+c_{q}^{\dagger} c_{p}\right) | q_{1} \ldots q_{N} \rangle
	\\
	=&\delta_{p q} | q_{1} \ldots q_{N} \rangle-\delta_{p, q_{N}} | q_{1} \ldots q_{N-1} q \rangle+\ldots
	\\
	&+ \delta_{p, q_{N}} | q_{1} \ldots q_{N-1} q \rangle-\delta_{p, q_{N-1}} | q_{1} \ldots q_{N-2} q_{N} q \rangle+\ldots
	\\
	=&\delta_{p q} | q_{1} \ldots q_{N} \rangle
\end{aligned}
\end{equation}

Thus
\begin{equation}
\left\{c_{q}^{\dagger}, c_{p}\right\}=c_{p} c_{q}^{\dagger}+c_{q}^{\dagger} c_{p}=\delta_{p q}
\end{equation}

In the similar way,
\begin{equation}
\left\{c_{p}^{\dagger}, c_{q}^{\dagger}\right\}=0, \quad\left\{c_{p}, c_{q}\right\}=0
\end{equation}

In fact, the anticommutation rule comes from the fermion attribute of electron.
For bosons, the anticommutation should be replaced by commutation rule.

%The advantage of creation and annihilation operator is obvious: we don't need to explicitly antisymmetrize the N-body quantum state, which has a rather complicated expression.
%In fact, we change from the common coordinate or momentum representation to particle number representation.
%Since particles are identical for either fermions and bosons, we can only tell the number of particles on each state instead of the state of each particle.
%Thus, particle number representation and introduction of creation and annihilation operator remove redundant information and generate the anticommutation rule for N-body states automatically.

The introduction of creation and annihilation operators can not only help us with description of quantum states but also with operators.

Generally, a one-particle operator
\begin{equation}
	\hat{W}=\sum_{i=1}^{N} \hat{w}(i)
\end{equation}
can be written as
\begin{equation}
\hat{W}^{\prime}=\sum_{p, q} w_{p q} c_{p}^{\dagger} c_{q}
\end{equation}
%which no longer has a summation over all the particles like the case of quantum mechanics:

%Its expectation value on a N-body quantum state is:
%\begin{equation}
	%\langle q_{1} \ldots q_{N}|\hat{W}^{\prime}| q_{1} \ldots q_{N}\rangle=\sum_{i=1}^{N} w_{q_{i} q_{i}}
%\end{equation}
%which is as expected

%Take the kinetic energy operator as example, since all the electrons are identical, the operator that acts on each single electron must be the same.
%Thus, the quantum mechanics approach of operator cause redundant information, which is removed by second quantization.

%Another important example of one-body operator is particle number operator
%\begin{equation}
	%\hat{N}=\sum_{p} c_{p}^{\dagger} c_{p}, \quad \hat{N} | q_{1} \ldots q_{N} \rangle=N | q_{1} \ldots q_{N} \rangle
%\end{equation}

And the second quantization of two-body operator
\begin{equation}
\hat{V}=\sum_{i<j=1}^{N} \hat{v}(i, j)
\end{equation}
is
\begin{equation}
\hat{V}=\frac{1}{2} \sum_{p, q, r, s} V_{p q r s} c_{p}^{\dagger} c_{q}^{\dagger} c_{s} c_{r}
\end{equation}

%Its expectation value is
%\begin{equation}
%\langle q_{1} \ldots q_{N}|\hat{V}| q_{1} \ldots q_{N}\rangle=\frac{1}{2} \sum_{q_{i}<q_{j}}\left(V_{q_{i} q_{j}\left[q_{i} q_{j}\right]}-V_{q_{j} q_{i}\left[q_{i} q_{j}\right]}\right)=\sum_{q_{i}<q_{j}} V_{q_{i} q_{j}\left[q_{i} q_{j}\right]}
%\end{equation}

%The change of basis will transform creation and annihilation operator just like normal operators:
%\begin{equation}
%\begin{aligned}
%	b_{s}^{\dagger} | q_{1} \ldots q_{N} \rangle&= | q_{1} \ldots q_{N} \tilde{s} \rangle
%	\\
%	&= \sum_{q} | q_{1} \ldots q_{N} q \rangle\langle q | \tilde{s}\rangle
%	\\
%	&=\sum_{q}\langle q | \tilde{s}\rangle c_{q}^{\dagger} | q_{1} \ldots q_{N} \rangle
%\end{aligned}
%\end{equation}
%and thus
%\begin{equation}
%\begin{aligned}
%	b_{s}^{\dagger}&=\sum_{q}\langle q | \tilde{s}\rangle c_{q}^{\dagger}
%	\\
%	b_{s}&=\sum_{q}\langle\tilde{s} | q\rangle c_{q}
%\end{aligned}
%\end{equation}

%If the basis $\ket{q}$ contains both spacial part and spin part with spacial part $x$ and spin part $\sigma$, then it's convenient to define field operators:
%\begin{equation}
%\begin{aligned} \hat{\psi}^{\dagger}(\xi) &=\sum_{q} c_{q}^{\dagger} \psi_{q}^{*}(\xi) \\ \hat{\psi}(\xi) &=\sum_{q} c_{q} \psi_{q}(\xi) \end{aligned}
%\end{equation}
%where $\xi$ is a set of spacial and spin index.
%
%They also have anticommutation rules like creation and annihilation operators:
%\begin{equation}
%\begin{array}{c}{\left\{\hat{\psi}^{\dagger}(\xi), \hat{\psi}\left(\xi^{\prime}\right)\right\}=\delta\left(x-x^{\prime}\right) \delta_{\sigma \sigma^{\prime}}} \\ {\left\{\hat{\psi}(\xi), \hat{\psi}\left(\xi^{\prime}\right)\right\}=0, \quad\left\{\hat{\psi}^{\dagger}(\xi), \hat{\psi}^{\dagger}\left(\xi^{\prime}\right)\right\}=0}\end{array}
%\end{equation}
%
%In addition, normal operators mentioned early can also be written in the form of field operators:
%\begin{equation}
%\begin{aligned} \hat{W} &=\int \mathrm{d} \xi \hat{w}(\xi) \hat{\psi}^{\dagger}(\xi) \hat{\psi}(\xi) \\ \hat{V} &=\frac{1}{2} \iint \mathrm{d} \xi \mathrm{d} \xi^{\prime} \hat{v}\left(\xi, \xi^{\prime}\right) \hat{\psi}^{\dagger}(\xi) \hat{\psi}^{\dagger}\left(\xi^{\prime}\right) \hat{\psi}\left(\xi^{\prime}\right) \hat{\psi}(\xi) \end{aligned}
%\end{equation}

\subsection{Green Function} \label{mathrefs}

In mathematics, a Green's function of an inhomogeneous linear differential operator defined on a domain with specified initial conditions or boundary conditions is its impulse response.
Green function is a important concept in physics, since the expression of green function can easily give the result for an arbitrary source or boundary condition.

Take electromagnetic wave equation of electric potential in classical field theory as an example:
\begin{equation}
\left(\nabla^{2}-\frac{1}{c^2}\frac{\partial^{2}}{\partial t^{2}}\right) \phi(\mathbf{r},t)=-\rho(\mathbf{r},t)
\end{equation}
which give the following solution:
\begin{equation}
\phi(\mathbf{r}, t)=\int \frac{\delta\left(t^{\prime}+\frac{|\mathbf{r}-\mathbf{r}^{\prime}|}{c}-t\right)}{|\mathbf{r}-\mathbf{r}^{\prime}|} \rho\left(\mathbf{r}^{\prime}, t^{\prime}\right) d^{3} r^{\prime} d t^{\prime}
\end{equation}

It's easy to find that $\phi$ is proportional to $\rho$, thus if we define Green function as
\begin{equation}
\left(\nabla^{2}-\frac{1}{c^2}\frac{\partial^{2}}{\partial t^{2}}\right) G(\mathbf{r},t;\mathbf{r^{\prime}},t^{\prime})=-\delta(\mathbf{r}-\mathbf{r^\prime})\delta(t-t^{\prime})
\end{equation}

We should have
\begin{equation}
G(\mathbf{r},t;\mathbf{r^{\prime}},t^{\prime})=
-\frac{\delta\left(t^{\prime}+\frac{|\mathbf{r}-\mathbf{r}^{\prime}|}{c}-t\right)}{|\mathbf{r}-\mathbf{r}^{\prime}|}
\end{equation}
and
\begin{equation}
\phi(\mathbf{r}, t)=
\int G(\mathbf{r},t;\mathbf{r^{\prime}},t^{\prime}) \rho\left(\mathbf{r}^{\prime}, t^{\prime}\right) d^{3} r^{\prime} d t^{\prime}
\end{equation}

In many-body field theory, Green function also acts similar with that of classical field theory in the above example.
While the formal definition looks rather abstract and even forbidding, the benefits afforded by an approach based on the electron propagator should become clear after the theory has been more fully described.

The Hamiltonian in second quantization form is:
\begin{equation}
\hat{H}=\hat{T}+\hat{V}=\sum t_{p q} c_{p}^{\dagger} c_{q}+\frac{1}{2} \sum V_{p q r s} c_{p}^{\dagger} c_{q}^{\dagger} c_{s} c_{r}
\end{equation}

In many-body field theory, Heisenberg picture is the default picture used.
Thus, the time dependence of operator should be
\begin{equation}
O[t]=e^{i \hat{H} (t-t_0)} O[t_0] e^{-i \hat{H} (t-t_0)}
\end{equation}
which also holds for creation and annihilation operators
\begin{equation}
c_{p}^{\dagger}[t]=e^{i \hat{H} t} c_{p}^{\dagger} e^{-i \hat{H} t}, \quad c_{p}[t]=e^{i \hat{H} t} c_{p} e^{-i \hat{H} t}
\end{equation}

The Green function or electron propagator is defined as
\begin{equation} \label{eq:greendef}
G_{p q}\left(t, t^{\prime}\right)=-i \theta\left(t-t^{\prime}\right)\langle\Psi_{0}|c_{p}[t] c_{q}^{\dagger}\left[t^{\prime}\right]| \Psi_{0}\rangle+ i \theta\left(t^{\prime}-t\right)\langle\Psi_{0}|c_{q}^{\dagger}\left[t^{\prime}\right] c_{p}[t]| \Psi_{0}\rangle
\end{equation}
where $\theta(t)$ is step function defined as
\begin{equation}
\theta(t)=\left\{\begin{array}{ll}{1,} & {t>0} \\ {0,} & {t<0}\end{array}\right.
\end{equation}

By defining time ordering operator $\hat{\mathcal{T}}$ as
\begin{equation}
\hat{\mathcal{T}}\left(c_{p}[t] c_{q}^{\dagger}\left[t^{\prime}\right]\right)=\left\{\begin{array}{cc}{c_{p}[t] c_{q}^{\dagger}\left[t^{\prime}\right],} & {t>t^{\prime}} \\ {-c_{q}^{\dagger}\left[t^{\prime}\right] c_{p}[t],} & {t<t^{\prime}}\end{array}\right.
\end{equation}
which put operators with larger times to the left of those with smaller times,
we can write the Green function as
\begin{equation}
G_{p q}\left(t, t^{\prime}\right)=-i\langle\Psi_{0}|\hat{\mathcal{T}}\left(c_{p}[t] c_{q}^{\dagger}\left[t^{\prime}\right]\right)| \Psi_{0}\rangle
\end{equation}

The time ordering operator is necessary since operators with smaller times should be applied to states earlier than those with larger times.

If creation operator is applied before annihilation operator, a middle state with $N+1$ electrons will be produced, otherwise a middle state with $N-1$ electrons will be produced.

Thus It's convenient to divide the Green function to two parts:
\begin{equation}
\boldsymbol{G}\left(t, t^{\prime}\right)=\boldsymbol{G}^{+}\left(t, t^{\prime}\right)+\boldsymbol{G}^{-}\left(t, t^{\prime}\right)
\end{equation}
where $G^{+}$ represents the $N+1$ part and $G^{-}$ represents the $N-1$ part.

In detail,
\begin{equation}
\begin{aligned}
	G_{p q}^{+}\left(t, t^{\prime}\right)
	&=-i \theta\left(t-t^{\prime}\right)\langle\Psi_{0}|e^{i \hat{H} t} c_{p} e^{-i \hat{H} t} e^{i \hat{H} t^{\prime}} c_{q}^{\dagger} e^{-i \hat{H} t^{\prime}}| \Psi_{0}\rangle
	\\
	&=- i \theta\left(t-t^{\prime}\right) e^{i E_{0}\left(t-t^{\prime}\right)}\langle\Psi_{0}|c_{p} e^{-i \hat{H}(t-t)^{\prime}} c_{q}^{\dagger}| \Psi_{0}\rangle
	\\
	&=-i \theta\left(t-t^{\prime}\right) \sum_{n} e^{-i\left(E_{n}^{N+1}-E_{0}\right)\left(t-t^{\prime}\right)}\langle\Psi_{0}|c_{p}| \Psi_{n}^{N+1}\rangle\langle\Psi_{n}^{N+1}|c_{q}^{\dagger}| \Psi_{0}\rangle
\end{aligned}
\end{equation}
where $\ket{\Psi_n^{N+1}}$ is the orthonormal set of eigenstates of Hamiltonian with $N+1$ electrons.

It is similar for the $G^{-}$ part:
\begin{equation}
G_{p q}^{-}\left(t, t^{\prime}\right)=i \theta\left(t^{\prime}-t\right) \sum_{n} e^{i\left(E_{n}^{N-1}-E_{0}\right)\left(t-t^{\prime}\right)}\langle\Psi_{0}|c_{q}^{\dagger}| \Psi_{n}^{N-1}\rangle\langle\Psi_{n}^{N-1}|c_{p}| \Psi_{0}\rangle
\end{equation}

Note that the ionization potentials and electron affinities have appeared, which are exactly what we need:
\begin{equation}
\begin{array}{l}{A_{n}=E_{0}-E_{n}^{N+1}} \\ {I_{n}=E_{n}^{N-1}-E_{0}}\end{array}
\end{equation}

Until now, we are working on time representation, i.e. the parameters of Green function is time.
However, it is useful to switch to energy(frequency) representation by Fourier transformation:
\begin{equation}
G_{p q}(\omega)=\int_{-\infty}^{\infty} e^{i \omega\left(t-t^{\prime}\right)} G_{p q}\left(t, t^{\prime}\right) \mathrm{d}\left(t-t^{\prime}\right)
\end{equation}

However, there will be a problem of singularity during the integration.
Take $G^{+}$ as example, the integration over time-dependent part of $G^{+}$ is
\begin{equation}
\begin{aligned} f_{n}^{+}(\omega) &=\int_{-\infty}^{\infty} e^{i \omega \tau}\left[-i \theta(\tau) e^{-i\left(E_{n}^{N+1}-E_{0}\right) \tau}\right] \mathrm{d} \tau \\ &=-i \int_{0}^{\infty} e^{i\left[\omega-E_{n}^{N+1}+E_{0}\right] \tau} \mathrm{d} \tau \end{aligned}
\end{equation}
which is ill-behaved at the upper limit of time.

A common mathematical trick to solve this singularity problem is to add a infinitely small negative part to the exponent:
\begin{equation}
e^{i\left[\omega-E_{n}^{N+1}+E_{0}\right] \tau} \rightarrow e^{i\left[\omega-E_{n}^{N+1}+E_{0}+i\eta\right] \tau}
\end{equation}
which will give the result
\begin{equation}
f_{n}^{+}(\omega)=\frac{1}{\omega-E_{n}^{N+1}+E_{0}+i \eta}
\end{equation}

This trick works because all what we deal with now are intermediate results and are not any kind of observable.
When we get to observable finally, it will be the time to let $\eta \rightarrow 0$ and will give normal physical quantities.

On the other hand, when we perform inverse Fourier transformation to get Green function back to time representation, residue theorem is used and the sign of $\eta$ will determine the direction of contour integration and thus the sign of time difference.

Similarly, we can also apply the same trick to the $G^{-}$ part.
By including the time-independent part, we obtain the Green function in energy representation:
\begin{equation} \label{spectralrepresentation}
G_{p q}(\omega)=\sum_{n} \frac{\langle\Psi_{0}|c_{p}| \Psi_{n}^{N+1}\rangle\langle\Psi_{n}^{N+1}|c_{q}^{\dagger}| \Psi_{0}\rangle}{\omega+E_{0}-E_{n}^{N+1}+i \eta}+\sum_{n} \frac{\langle\Psi_{0}|c_{q}^{\dagger}| \Psi_{n}^{N-1}\rangle\langle\Psi_{n}^{N-1}|c_{p}| \Psi_{0}\rangle}{\omega+E_{n}^{N-1}-E_{0}-i \eta}
\end{equation}
which is also refered as spectral representation or Lehmann representation.
It is worthy noticing that all the energies that we need are located on the poles in complex plane, while ionization potentials on the upper half and electron affinities on the lower half.

It is also convenient to define spectroscopic factors as:
\begin{equation}
\begin{array}{ll}{x_{p}^{(n)}=\langle\Psi_{0}|c_{p}| \Psi_{n}^{N+1}\rangle,} & { n \in\{N+1\}} \\ {x_{p}^{(n)}=\langle\Psi_{n}^{N-1}|c_{p}| \Psi_{0}\rangle,} & { n \in\{N-1\}}\end{array}
\end{equation}

The spectroscopic factor is closely related to photo-ionization process.
The photo-ionization cross section is
\begin{equation}
\sigma_{n}(\epsilon) \sim \frac{2}{3} \varepsilon|\sum_{p}\langle\varepsilon|\hat{d}| p\rangle x_{p}^{(n)}|^{2}
\end{equation}
where the spectroscopic factors weight the participation of individual orbitals in the final ionic state.

By using the anticommutation rule of creation and annihilation operators, the spectroscopic factors satisfy the following orthonormal relation:
\begin{equation}
\sum_{n \in\{N+1\}} x_{p}^{(n)} x_{q}^{(n) *}+\sum_{n \in\{N-1\}} x_{p}^{(n)} x_{q}^{(n) *}=\delta_{p q}
\end{equation}

The spectral representation can also be written in a more compact form as
\begin{equation}
\begin{aligned} G_{p q}^{+}(\omega) &=\langle\Psi_{0}|c_{p}\left(\omega-\hat{H}+E_{0}+i \eta\right)^{-1} c_{q}^{\dagger}| \Psi_{0}\rangle \\ G_{p q}^{-}(\omega) &=\langle\Psi_{0}|c_{q}^{\dagger}\left(\omega+\hat{H}-E_{0}-i \eta\right)^{-1} c_{p}| \Psi_{0}\rangle \end{aligned}
\end{equation}

From above discussions, we know that Green function can derive spectroscopic factors which can be used to determine photo-ionization cross section.
In addition, Green function can also derive ground state density matrix and thus energy.

The ground state energy is
\begin{equation}
E_{0}=\langle\Psi_{0}|\hat{H}| \Psi_{0}\rangle=\sum t_{r s}\langle\Psi_{0}|c_{r}^{\dagger} c_{r}| \Psi_{0}\rangle+\frac{1}{2} \sum V_{r s u v}\langle\Psi_{0}|c_{r}^{\dagger} c_{s}^{\dagger} c_{v} c_{u}| \Psi_{0}\rangle
\end{equation}

Let us first consider the time derivative of the time-dependent annihilation operator:
\begin{equation}
\begin{aligned}
	i \frac{\partial}{\partial t} c_{p}[t]&=i \frac{\partial}{\partial t}\left(e^{i \hat{H} t} c_{p} e^{-i \hat{H} t}\right)=e^{i \hat{H} t}\left[c_{p}, \hat{H}\right] e^{-i \hat{H} t}
	\\
	&=\sum_{s} t_{p s} c_{s}[t]+\sum_{s, u, v} V_{p s u v} c_{s}^{\dagger}[t] c_{v}[t] c_{u}[t]
\end{aligned}
\end{equation}

Thus we can calculate the time derivative of Green function as
\begin{equation}
\begin{aligned}
i \frac{\partial}{\partial t} G_{p q}\left(t, t^{\prime}\right)
=& \delta\left(t-t^{\prime}\right)\langle\Psi_{0} |\left\{c_{p}[t], c_{q}^{\dagger}\left[t^{\prime}\right]\right\}| \Psi_{0} \rangle
\\
&-i\langle\Psi_{0}|\hat{\mathcal{T}}\left[\left(i \frac{\partial}{\partial t} c_{p}[t]\right) c_{q}^{\dagger}\left[t^{\prime}\right]\right]| \Psi_{0}\rangle
\\
=& \delta_{p q} \delta\left(t-t^{\prime}\right)-i \sum_{s} t_{p s}\langle\Psi_{0}|\hat{\mathcal{T}}\left[c_{s}[t] c_{q}^{\dagger}\left[t^{\prime}\right]\right]| \Psi_{0}\rangle
\\
&- i \sum_{s, u, v} V_{p s u v}\langle\Psi_{0}|\hat{\mathcal{T}}\left[c_{s}^{\dagger}[t] c_{v}[t] c_{u}[t] c_{q}^{\dagger}\left[t^{\prime}\right]\right]| \Psi_{0}\rangle \end{aligned}
\end{equation}

By approaching $t^{\prime}$ to $t^{+}$ and take the trace of the equation above, we obtain
\begin{equation}
\sum_{p} i \frac{\partial}{\partial t} G_{p p}\left(t, t^{+}\right)-i\langle\Psi_{0}|\hat{T}| \Psi_{0}\rangle= 2 i\langle\Psi_{0}|\hat{V}| \Psi_{0}\rangle
\end{equation}

By using residue theorem, the contour integration of $\omega G_{pq}(\omega)$ can be calculate as
\begin{equation}
\begin{aligned}
	\frac{1}{2 \pi i} \oint \omega G_{p q}^{-}(\omega) \mathrm{d} \omega
	&=-\sum_{n}\left(E_{n}^{N-1}-E_{0}\right)\langle\Psi_{0}|c_{q}^{\dagger}| \Psi_{n}^{N-1}\rangle\langle\Psi_{n}^{N-1}|c_{p}| \Psi_{0}\rangle
\\
&=\langle\Psi_{0}|c_{q}^{\dagger}\left[c_{p}, \hat{H}\right]| \Psi_{0}\rangle
\\
	&=\langle\Psi_{0}|\hat{T}| \Psi_{0}\rangle+ 2\langle\Psi_{0}|\hat{V}| \Psi_{0}\rangle
\end{aligned}
\end{equation}

Thus the ground state energy can be obtained as:
\begin{equation} \label{greenenergy}
E_{0}=\frac{1}{4 \pi i} \oint \operatorname{Tr}[(\omega \mathbf{1}+\boldsymbol{T}) \boldsymbol{G}(\omega)] \mathrm{d} \omega=\frac{1}{4 \pi i} \oint \operatorname{Tr}\left[(\omega \mathbf{1}+\boldsymbol{T}) \boldsymbol{G}^{-}(\omega)\right] \mathrm{d} \omega
\end{equation}

This contour integration can be done by residue theorem after we calculate ionization potentials and electron affinities later.

Until now, we have not given any practical approaches to calculate Green function.
We will later see that general Green function can be calculated by Feynman expansion, which is essentially a perturbation theory.
Thus we need to first solve the unperturbed Green function, which is usually refered to as free Green function.
\begin{equation}
\hat{H}_{0}=\sum \varepsilon_{r} c_{r}^{\dagger} c_{r}
\end{equation}

Then we calculate the time dependence of creation and annihilation operators:
\begin{equation}
i \frac{\partial}{\partial t} c_{p}(t)=e^{i \hat{H}_{0} t}\left[c_{p}, \hat{H}_{0}\right] e^{-i \hat{H}_{0} t}=\varepsilon_{p} c_{p}(t)
\end{equation}
thus
\begin{equation}
c_{p}(t)=e^{i \hat{H}_{0} t} c_{p} e^{-i \hat{H}_{0} t}=e^{-i \varepsilon_{p} t} c_{p}
\end{equation}

From the definition of Green function in Eq \ref{eq:greendef}, we have
\begin{equation}
G_{p q}^{0}\left(t, t^{\prime}\right)=-i e^{-i \varepsilon_{p}\left(t-t^{\prime}\right)} \delta_{p q}\left(\theta\left(t-t^{\prime}\right) \overline{n}_{p}-\theta\left(t^{\prime}-t\right) n_{p}\right)
\end{equation}
where
\begin{equation}
n_{p}=
\left\{
\begin{array}{ll}
	{1,} & {p \leq N}
	\\
	{0,} & {p>N}
\end{array}
\right.
\end{equation}
is occupation number of orbital $p$.

The corresponding energy representation is
\begin{equation}
G_{p q}^{0}(\omega)=\delta_{p q}\left(\frac{\overline{n}_{p}}{\omega-\varepsilon_{p}+i \eta}+\frac{n_{p}}{\omega-\varepsilon_{p}-i \eta}\right)
\end{equation}

\subsection{Perturbation Theory for Green Function}

We discussed the definition of Green function and how to derive important physical quantities from Green function in the previously part.
In this part, we will give a perturbation scheme to expand Green function into different orders.

We start from the introduction of M{\o}ller-Plesset partitioning:
\begin{equation}
\hat{H}=\hat{T}+\hat{V}=\hat{H}_{0}+\hat{H}_{I}
\end{equation}
where
\begin{equation}
\hat{H}_{0}=\sum \epsilon_{r} c_{r}^{\dagger} c_{r}
\end{equation}
is the Hartree-Fock Hamiltonian calculated from the solutions of Hartree-Fock equation in second quantization form:
\begin{equation}
t_{r s}+\sum_{k} V_{r k[s k]} n_{k}=\epsilon_{r} \delta_{r s}
\end{equation}
which gives the form of $\hat{H}_I$:
\begin{equation}
\hat{H}_I=\frac{1}{2} \sum V_{p q r s} c_{p}^{\dagger} c_{q}^{\dagger} c_{s} c_{r}-\sum_{k} V_{r k[s k]} n_{k} c_{r}^{\dagger} c_{s}
\end{equation}

For Hartree-Fock ground state:
\begin{equation}
\hat{H}_{0} | \Phi_{0} \rangle=E_{0}^{(0)} | \Phi_{0} \rangle
\end{equation}

Before working on the perturbation theory, we need to transfer to interaction picture.
The basic idea of interaction picture is to remove the rapidly fluctuating exponential factor caused by the trivial unperturbed Hamiltonian before the quantum state:
\begin{equation}
i \frac{\partial}{\partial t} | \Psi(t) \rangle=\hat{H}_{0}+\hat{H}_{1} | \Psi(t) \rangle
\end{equation}

By perturbation assumption, $\hat{H}_1$ is small compared with $\hat{H}_0$, thus the fluctuating factor $e^{-i \hat{H}_0 t}$ caused by unperturbed part dominates the time dependence of $\Psi$.
Thus, the change to interaction picture will help us to focus on the perturbed part of Hamiltonian.

In interaction picture, the quantum state is defined as
\begin{equation}
| \Psi_{I}(t) \rangle=e^{i \hat{H}_{0} t} | \Psi_{S}(t) \rangle
\end{equation}
and thus the Schrodinger equation becomes:
\begin{equation}
i \frac{\partial}{\partial t} | \Psi_{I}(t) \rangle=\hat{H}_{I}(t) | \Psi_{I}(t) \rangle
\end{equation}
where the $\hat{H}_I$ is the perturbation part of Hamiltonian in interaction picture:
\begin{equation}
\hat{H}_{I}(t)=e^{i \hat{H}_{0} t} \hat{H}_{1}(t) e^{-i \hat{H}_{0} t}
\end{equation}

As is expected, the unperturbed part does not exist any more, thus the change of quantum state is slow under the small perturbation assumption.
In fact, the time evolution of quantum state in interaction picture is:
\begin{equation}
| \Psi_{I}(t) \rangle=\hat{U}\left(t, t_{0}\right) | \Psi_{I}\left(t_{0}\right) \rangle
\end{equation}
where
\begin{equation}
\hat{U}\left(t, t_{0}\right)=e^{i \hat{H}_{0} t} e^{-i \hat{H}\left(t-t_{0}\right)} e^{-i \hat{H}_{0} t_{0}}
\end{equation}
is the time evolution operator in interaction picture.

Equation of motion of the evolution operator itself is:
\begin{equation}
i \frac{\partial}{\partial t} \hat{U}\left(t, t_{0}\right)=\hat{H}_{I}(t) U\left(t, t_{0}\right)
\end{equation}

Thus it is possible to expand the evolution operator in series:
\begin{equation}
\hat{U}\left(t, t_{0}\right)=\hat{\mathbb{1}}-i \int_{t_{0}}^{t} \mathrm{d} t_{1} \hat{H}_{I}\left(t_{1}\right)+(-i)^{2} \int_{t_{0}}^{t} \mathrm{dt}_{1} \int_{t_{0}}^{t_{1}} \mathrm{d} t_{2} \hat{H}_{I}\left(t_{1}\right) \hat{H}_{I}\left(t_{2}\right)+\ldots
\end{equation}
and
\begin{equation}
\hat{U}^{(n)}\left(t, t_{0}\right)=
(-i)^{n} \int_{t_n<t_{n-1}<\dots<t_1<t_0<t} \mathrm{d} t_{1} \mathrm{d} t_{2} \ldots \mathrm{d} t_{n} \hat{H}_{I}\left(t_{1}\right) \hat{H}_{I}\left(t_{2}\right) \ldots \hat{H}_{I}\left(t_{n}\right)
\end{equation}
is the general expression for the $n$th order.

Note that the perturbation part of Hamiltonians in interaction picture do not commute for different time arguments, i.e.
\begin{equation}
\left[\hat{H}_{I}(t), \hat{H}_{I}\left(t^{\prime}\right)\right] \neq 0 \text { for } t \neq t^{\prime}
\end{equation}

However, the time arguments themselves can be exchanged since they are intermediate variables.
Thus,
\begin{equation}
\begin{aligned}
	\int_{t_n<t_{n-1}<\dots<t_1<t_0<t} &\mathrm{d} t_{1} \mathrm{d} t_{2} \ldots \mathrm{d} t_{n} \hat{H}_{I}\left(t_{1}\right) \hat{H}_{I}\left(t_{2}\right) \ldots \hat{H}_{I}\left(t_{n}\right)
	\\
	=\int_{t_{P_n}<t_{P_{n-1}}<\dots<t_{P_1}<t_{P_0}<t} &\mathrm{d} t_{1} \mathrm{d} t_{2} \ldots \mathrm{d} t_{n} \hat{H}_{I}\left(t_{P_1}\right) \hat{H}_{I}\left(t_{P_2}\right) \ldots \hat{H}_{I}\left(t_{P_n}\right)
\end{aligned}
\end{equation}
where $P$ is any possible permutation.

Thus each order of evolution operator can be written in the form of time ordering operator:
\begin{equation}
\hat{U}^{(n)}\left(t, t_{0}\right)=\frac{(-i)^{n}}{n !} \int_{t_{0}}^{t} \mathrm{d} t_{1} \ldots \int_{t_{0}}^{t} \mathrm{d} t_{n} \hat{\boldsymbol{T}}\left[\hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right)\right]
\end{equation}

In a symbolic way, the evolution operator can be written as
\begin{equation}
\hat{U}\left(t, t_{0}\right)=\hat{\mathcal{T}} e^{-i \int_{t_{0}}^{t} \mathrm{d} t^{\prime} \hat{H}_{I}\left(t^{\prime}\right)}
\end{equation}

Think about that at time $t=-\infty$, the Hamiltonian of a system is $\hat{H}_0$ and the system is in the ground state.
Then the perturbed Hamiltonian $\hat{H}_1$ is gradually turned on exponentially, and at time $t=0$, the perturbed Hamiltonian is fully turned on:
\begin{equation}
\hat{H}(t)=\hat{H}_{0}+e^{-\epsilon|t|} \hat{H}_{I}
\end{equation}
In the limit of $\epsilon \rightarrow 0$, which is refered to as adiabatic limit, it is expected that the system in the ground state of the total Hamiltonian at $t=0$.

The time evolution operator associated with this process is
\begin{equation}
\hat{U}_{\epsilon}\left(t, t_{0}\right)=\sum_{n=0}^{\infty} \frac{(-i)^{n}}{n !} \int_{t_{0}}^{t} \mathrm{d} t_{1} e^{-\epsilon|t_{1}|} \ldots \int_{t_{0}}^{t} \mathrm{d} t_{n} e^{-\epsilon|t_{n}|} \hat{\mathcal{T}}\left[\hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right)\right]
\end{equation}

However, the situation is not that simple and the evolution operator actually does not converge when $\epsilon \rightarrow 0$.
The divergent is caused by the phase factor of the quantum state, which is not physical.
Thus, it is possible to cancel this unphysical divergence, which leads to the famous Gell-Mann and Low theorem:
\begin{equation}
| \Psi_{0}^{\prime} \rangle=\lim _{\epsilon \rightarrow 0} \frac{\hat{U}_{\epsilon}(0,-\infty) | \Phi_{0} \rangle}{\langle\Phi_{0}|\hat{U}_{\epsilon}(0,-\infty)| \Phi_{0}\rangle}
\end{equation}
is the ground state of $\hat{H}$ with the eigenvalue
\begin{equation}
E_{0}=E_{0}^{(0)}+\lim _{\epsilon \rightarrow 0} \frac{\langle\Phi_{0}|\hat{H}_{I} \hat{U}_{\epsilon}(0,-\infty)| \Phi_{0}\rangle}{\langle\Phi_{0}|\hat{U}_{\epsilon}(0,-\infty)| \Phi_{0}\rangle}
\end{equation}
and the proof is given in the reference \cite{gallmannlow}

Since both the numerator and denominator diverges with $\epsilon \rightarrow 0$, the convergence holds only when one calculate contribution from each order separately and then add them up.
In addition, the normalization rule for the final state $| \Psi_{0}^{\prime}\rangle$ is
\begin{equation}
\langle\Phi_{0} | \Psi_{0}^{\prime}\rangle= 1
\end{equation}
rather than
\begin{equation}
\langle\Psi_{0}^{\prime} | \Psi_{0}^{\prime}\rangle= 1
\end{equation}

And it is also similar for the $t>0$ scheme:
\begin{equation}
| \Psi_{0}^{\prime \prime} \rangle=\lim _{\epsilon \rightarrow 0} \frac{\hat{U}_{\epsilon}(0, \infty) | \Phi_{0} \rangle}{\langle\Phi_{0}|\hat{U}_{\epsilon}(0, \infty)| \Phi_{0}\rangle}=\lim _{\epsilon \rightarrow 0} \frac{\hat{U}_{\epsilon}(0,-\infty) | \Phi_{0} \rangle}{\langle\Phi_{0}|\hat{U}_{\epsilon}(0,-\infty)| \Phi_{0}\rangle}=| \Psi_{0}^{\prime} \rangle
\end{equation}

Then we will calculate the expectation value of any operator $\hat{O}_H$:
\begin{equation}
\begin{aligned}
	\langle\Psi_{0}|\hat{O}_{H}(t)| \Psi_{0}\rangle
	&=\frac{\langle\Psi_{0}^{\prime}|\hat{O}_{H}(t)| \Psi_{0}^{\prime}\rangle}
	{\langle\Psi_{0}^{\prime} | \Psi_{0}^{\prime}\rangle}
	\\
	&=\lim _{\epsilon \rightarrow 0}
	\frac{\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty, 0)\hat{U}_{\epsilon}(0, t) \hat{O}_{I}(t) \hat{U}_{\epsilon}(0, t)\hat{U}_{\epsilon}(t,-\infty)| \Phi_{0}\rangle}
	{\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty,-\infty)| \Phi_{0}\rangle}
	\\
	&=\lim _{\epsilon \rightarrow 0}
	\frac{\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty, t) \hat{O}_{I}(t) \hat{U}_{\epsilon}(t,-\infty)| \Phi_{0}\rangle}
	{\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty,-\infty)| \Phi_{0}\rangle}
	\\
	&=\lim _{\epsilon \rightarrow 0} \sum_{n=0}^{\infty} \frac{(-i)^{n}}{n !} \int_{-\infty}^{\infty} \mathrm{d} t_{1} e^{-\epsilon|t_{1}|} \ldots \int_{-\infty}^{\infty} \mathrm{d} t_{n} e^{-\epsilon|t_{n}|}
	\\
	&\qquad \frac{\langle\Phi_{0}|
	\hat{\mathcal{T}}\left[\hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right) \hat{O}_{I}(t)\right]|
	\Phi_{0}\rangle}
	{\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty,-\infty)| \Phi_{0}\rangle}
\end{aligned}
\end{equation}

Similarly, for expectation of two operators with different time, we have
\begin{equation}
\begin{aligned}
	\langle\Psi_{0}|\hat{\mathcal{T}}\left[\hat{P}_{H}(t) \hat{Q}_{H}\left(t^{\prime}\right)\right]| \Psi_{0}\rangle
	&=\lim _{\epsilon \rightarrow 0} \sum_{n=0}^{\infty} \frac{(-i)^{n}}{n !} \int_{-\infty}^{\infty} \mathrm{d} t_{1} e^{-\epsilon|t_{1}|} \ldots \int_{-\infty}^{\infty} \mathrm{d} t_{n} e^{-\epsilon|t_{n}|}
	\\
	&\qquad =\frac{\langle\Phi_{0}|
	\hat{\boldsymbol{T}}\left[\hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right) \hat{P}_{I}(t) \hat{Q}_{I}\left(t^{\prime}\right)\right]|
	\Phi_{0}\rangle}
	{\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty,-\infty)| \Phi_{0}\rangle}
\end{aligned}
\end{equation}

Thus, it is possible for us to expand Green function to series, which is the original purpose:
\begin{equation} \label{greenexp}
\begin{aligned}
	i G_{p q}\left(t, t^{\prime}\right)
	=&\langle\Psi_{0}|
	\hat{\mathcal{T}}\left[c_{p}[t] c_{q}^{\dagger}\left[t^{\prime}\right]\right]|
	\Psi_{0}\rangle
	\\
	=& \lim _{\epsilon \rightarrow 0} \sum_{n=0}^{\infty} \frac{(-i)^{n}}{n !}
	\int_{-\infty}^{\infty} \mathrm{d} t_{1} e^{-\epsilon|t_{1}|} \ldots \int_{-\infty}^{\infty} \mathrm{d} t_{n} e^{-\epsilon|t_{n}|}
	\\
	& \frac{\langle\Phi_{0}|
	\hat{\boldsymbol{T}}\left[\hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right) c_{p}(t) c_{q}^{\dagger}\left(t^{\prime}\right)\right]|
	\Phi_{0}\rangle}
	{\langle\Phi_{0}|
	\hat{U}_{\epsilon}(\infty,-\infty)|
	\Phi_{0}\rangle}
\end{aligned}
\end{equation}
and the denominator as
\begin{equation}
\begin{aligned}\langle\Phi_{0}|\hat{U}_{\epsilon}(\infty,-\infty)| \Phi_{0}\rangle=\sum_{n=0}^{\infty} \frac{(-i)^{n}}{n !} \int_{-\infty}^{\infty} \mathrm{d} t_{1} e^{-\epsilon|t_{1}|} \ldots \int_{-\infty}^{\infty} \mathrm{d} t_{n} e^{-\epsilon|t_{n}|} \\\langle\Phi_{0}|\hat{\mathcal{T}}\left[\hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right)\right]| \Phi_{0}\rangle \end{aligned}
\end{equation}

As we have seen, field theory approach chooses a totally different way of perturbation expansion, which is using Gell-Mann and Low theorem.
However, the result of perturbation has particular physical meaning, which is the asymptotic behaviour of eigenvalues and eigenstates when magnitude of perturbed Hamiltonian changes.
Thus, all perturbation theory should give the same result.

Generally, M{\o}ller-Plesset perturbation theory gives
\begin{equation} \label{rspt}
| \Psi_{0} \rangle=| \Phi_{0} \rangle+\sum_{n=1}^{\infty}\left[\frac{\hat{Q}_{0}}{E_{0}^{(0)}-\hat{H}_{0}}\left(E_{0}^{(0)}-E_{0}+\hat{H}_{I}\right)\right]^{n} | \Phi_{0} \rangle
\end{equation}
which is proved in reference \cite{rsptproof}.

In fact, we can also write the result from field theory approach into similar expression.
The numerator is:
\begin{equation} \label{expansion}
\begin{array}{l}{\hat{U}_{\epsilon}(0,-\infty) | \Phi_{0} \rangle=} \\ { | \Phi_{0} \rangle+\sum_{n=1}^{\infty}(-i)^{n} \int_{-\infty}^{0} \mathrm{d} t_{1} \int_{-\infty}^{t_{1}} \mathrm{d} t_{2} \ldots \int_{-\infty}^{t_{n-1}} \mathrm{d} t_{n} e^{\epsilon\left(t_{1}+\cdots+t_{n}\right)} \hat{H}_{I}\left(t_{1}\right) \ldots \hat{H}_{I}\left(t_{n}\right) | \Phi_{0} \rangle}\end{array}
\end{equation}

Since
\begin{equation} \label{I2H}
\hat{H}_{I}\left(t_{j}\right)=e^{i \hat{H}_{0} t_{j}} \hat{H}_{I} e^{-i \hat{H}_{0} t_{j}}
\end{equation}
and
\begin{equation}
\hat{H}_{I}\left(t_{j}\right) \hat{H}_{I}\left(t_{j^{\prime}}\right)=e^{i \hat{H}_{0} t_{j}} \hat{H}_{I} e^{-i \hat{H}_{0}\left(t_{j}-t_{j}\right)} \hat{H}_{I} e^{-i \hat{H}_{0} t_{j^{\prime}}}
\end{equation}
, we can expect that there are a lot of terms like $e^{-i \hat{H}_{0}\left(t_{j}-t_{j}\right)}$ after we write every $\hat{H}_I(t_i)$ in Eq \ref{expansion} in terms of Eq \ref{I2H}.

Thus, we change integration variable from $t$ to the following $x$:
\begin{equation}
	\begin{aligned} x_{1} &=t_{1} \\ x_{2} &=t_{2}-t_{1} \\ x_{3} &=t_{3}-t_{2} \\ & \vdots \\ x_{n} &=t_{n}-t_{n-1} \end{aligned}
\end{equation}

Thus
\begin{equation}
	\begin{aligned}
		\hat{U}_{\epsilon}^{(n)}(0,-\infty) | \Phi_{0} \rangle&=(-i)^{n} \int_{-\infty}^{0} \mathrm{d} x_{1} e^{n \epsilon x_{1}} e^{i\left(\hat{H}_{0}-E_{0}^{(0)}\right) x_{1}} \hat{H}_{I}
		\\
		&\qquad {\int_{-\infty}^{0} \mathrm{d} x_{2} e^{(n-1) \epsilon x_{2}} e^{i\left(\hat{H}_{0}-E_{0}^{(0)}\right) x_{2}} \hat{H}_{I} \ldots \int_{-\infty}^{0} \mathrm{d} x_{n} e^{\epsilon x_{n}} e^{i\left(\hat{H}_{0}-E_{0}^{(0)}\right) x_{n}} \hat{H}_{I} | \Phi_{0} \rangle}
		\\
		&={\frac{1}{E_{0}^{(0)}-\hat{H}_{0}+n i \epsilon} \hat{H}_{I}}
		{\frac{1}{E_{0}^{(0)}-\hat{H}_{0}+(n-1) i \epsilon} \hat{H}_{I}} \cdots
		\\
		&\qquad \frac{1}{E_{0}^{(0)}-\hat{H}_{0}+i \epsilon} \hat{H}_{I} | \Phi_{0} \rangle
	\end{aligned}
\end{equation}

However, compared with Eq \ref{rspt}, the expression here misses the important term $Q_0$, which prevent the divergence when $\epsilon \rightarrow 0$.
This is expected, since we have mentioned that the divergent terms appearing in numerator will be canceled out in the denominator.

The denominator is:
\begin{equation}
	\langle\Phi_{0}|\hat{U}_{\epsilon}(0,-\infty)| \Phi_{0}\rangle= 1+\langle\Phi_{0}|\frac{1}{E_{0}^{(0)}-\hat{H}_{0}+i \epsilon} \hat{H}_{I}| \Phi_{0}\rangle+\cdots
\end{equation}

Although it is not easy to prove for all the orders, but it is easy to see that the first order of the total expression gives the same result with M{\o}ller-Plesset perturbation theory.


% include the figures path relative to the master file
\graphicspath{ {./content/method/figures/} }

\section{Methodology}\label{sec:method}

Our approach is based on the original paper of Srinivasan~\textit{et al.} \cite{srinivasan2014fully} to which a feature extraction procedure (\gls{pca}) is added, as proposed by Venhuizen~\textit{et al.} \cite{venhuizen2015automated}.
Our general pipeline is presented on Fig.\,\ref{fig:fig3}.

First the image are preprocessed to remove the noise with the BM3D algorithm developed by Dabov et al.\,\cite{dabov2007image}; they are then flattened and cropped as described by Srinivisan~\textit{et al.} \cite{srinivasan2014fully} (see Fig.\,\ref{fig:fig4}).
\begin{figure}
\centering
\subfigure[Original]{\includegraphics[width = 0.17\textwidth]{Original}}\
\subfigure[Flatten]{\includegraphics[width = 0.17\textwidth]{Flatten}}
\caption{Original and flatten image.}
\label{fig:fig4}
\end{figure}
On the cropped image, we extract \gls{hog} features \cite{dalal2005histograms} as well as \gls{lbp} features either in their standard version \cite{ojala2002multiresolution} or their rotation invariant version \cite{zhao2012rotation} with different neighborhoods.
Furthermore, to consider structures at multiple scale levels, both \gls{hog} and \gls{lbp} features vector are extracted at four levels of the multiscale Gaussian lowpass image pyramid.
For each feature, \gls{hog} or \gls{lbp} the histogram are concatenated into a single feature vector for each image.
Then the obtained histograms are either fed directly to a linear \gls{svm} classifier or reduced with \gls{pca}.
We attempt to combine \gls{hog} and \gls{lbp} feature vectors or use them separately as explain in the next section.


\chapter{Linear elastostatics}

\section{Hooke's law of spring restoring force}
\label{sec:hooke-s-law}

Consider a spring at rest.
One end is fixed.
The other end is movable.

Let \(k\) be the spring's \emph{stiffness}.

Suppose that we pull the movable end,
displacing it by \(x\) from its resting position.

\index{laws named after people!Hooke's law of spring restoring force}%
\index{laws!spring restoring force}%
\index{Hooke!law of spring restoring force}%
\index{statics!Hooke's law of spring restoring force}%
\paragraph{Hooke's law}
For small displacements,
the spring's \emph{restoring force} at the movable end is \( F = - k x \).

It also works for rubbers and metals.%
\footnote{\url{http://www.continuummechanics.org/hookeslaw.html}}

\section{Seeing a rod as a stiff spring}

A solid rod behaves like a very stiff spring.

Let there be one such rod with length \(L\) and cross-section area \(A\).

Let one end be fixed.

If we push the other end,
that is if we apply a force \(F\) that is parallel
to the length of the rod (normal to the cross-section),
then we \emph{compress} the rod.
In this case, the \emph{stress} at the fixed end is \(F/A\).

The many definitions of \emph{strain}%
\footnote{\url{http://www.continuummechanics.org/strain.html}}

\footnote{\url{https://en.wikipedia.org/wiki/Saint-Venant\%27s_Principle}}

\footnote{\url{https://en.wikipedia.org/wiki/Linear_elasticity}}

\footnote{\url{https://en.wikipedia.org/wiki/Continuum_mechanics\#Car_traffic_as_an_introductory_example}}

\footnote{\url{https://en.wikipedia.org/wiki/Continuum_mechanics\#Major_areas}}

\section{Pressure, normal stress, shear stress, and stress}

Both pressure and stress are force per unit area.
But pressure is a scalar and stress is a tensor.%
\footnote{\url{https://physics.stackexchange.com/questions/107824/what-is-the-difference-between-stress-and-pressure}}

\section{Deformation}

A soft object is modeled by a subspace of a Euclidean space.
We call this subspace a \emph{material space}.

A \emph{deformation} assigns a vector to every point in the material space.

\section{Understanding the one-dimensional case}

\subsection{Understanding stress and strain}

Let \(\Delta x\) be the change in length.
Let \(x\) be the rest length.
The \emph{strain} is \( \Delta x / x \).

\subsection{Modeling the tension of a cord}

\subsection{Understanding Young's modulus (elastic modulus)}

Young's modulus (elastic modulus) is?

Relationship between length change and the exerted force.

Force that does not cause acceleration.

\subsection{Modeling continuous deformation using continuum mechanics}

Continuum mechanics?

How do we model stress in several dimensions?

\section{Understanding the three-dimensional case}

\subsection{Understanding Cauchy stress tensor}

The \emph{Cauchy stress tensor} is ...
Example: consider a cube:

% https://en.wikipedia.org/wiki/Stress_(mechanics)#General_stress
% https://en.wikipedia.org/wiki/Cauchy_stress_tensor

\section{Motivating tensor}

% https://en.wikipedia.org/wiki/Hooke%27s_law#General_tensor_form
Generalizing Hooke's law to continuous medium?

Generalizing Ohm's law to continuous medium?


%!TEX root = TTK4215-Summary.tex
\section{Robustness}

\subsection{Parameter drift}
An unknown bounded disturbance added to the measurement $y$ can lead to \emph{parameter drift}, meaning $\theta \to \infty$ as $t \to \infty$. The equilibrium $\tilde{\theta}_e = 0$ can be made u.a.s. by making $u(t)$ PE. (But we don't always decide $u(t)$.)

TTK4215 System identification and adaptive control/sec-robustness.tex


In order to compute solutions to our fractional model problem, the method of eigenfunction expansion requires the computation of the discrete eigenbasis for a given geometry. Since scalability is the primary computational concern for this framework, we build our framework on top of PETSc \cite{petsc-efficient} \cite{petsc-user-ref} and SLEPc \cite{slepc}. PETSc is a parallel and scalable library of routines and data structures for scientific applications. PETSc handles how our matrices are stored in parallel and for operations like matrix multiplication.

Since our framework is built in the method of eigenfunction expansion, we build on top of the SLEPc library for solving the resulting eigenvalue problem. SLEPc has a number of routines available for solving eigenvalue problems and is also built on PETSc. Our approach requires computing the full set of eigenpairs for a system and therefore our framework uses a modified version of spectrum slicing with the independent eigenvalue problems being solved with SLEPc.

{\color{red} \noindent <Introduce Nektar++, expand nektar++ integration description, cite \cite{nektarpp}> }

\subsection{Method}

In order to construct solutions to the fractional poisson problem, we need to discretize the domain of interest. Using the finite element method, geometry can be converted into the symmetric, discrete system $ Kv_k = \lambda_k M v_k $. For this, we have integerated our framework with Nektar++ which is used to convert a given mesh into two PETSc matrices $K$ and $M$.

Equipped with these matrices, we can compute all of the eigenpairs in a given interval using SLEPc's Krylov-Schur solver. Krylov-Schur is an iterative solver that relies on applying $K$ and $M$ repeatedly and solving related systems. If we were to simply give $K$ and $M$ to SLEPc, these sparse  matrices would be divided among the entire set of available processes. Using this naive approach, throwing more processes at the problem will just result in more communication between nodes and will hurt performance. Instead, we use an approach to split the full eigenbasis computation into independent subproblems that can be handled by a single node. This results in a much more scalable approach.

\subsubsection{Spectrum Slicing}

In order to compute the full set of eigenpairs using the available computing resources, we use the method of spectrum slicing. Because we know the eigenvalue problem is symmetric, all of the eigenvalues are real, and because the problem is semidefinite, all of the eigenvalues are contained in the interval $[0,U]$. Instead of solving for all eigenpairs within this interval using all processes, we break it into independent subintervals that can be solved in parallel. This is possible, again, due to the symmetry of the problem.

For a symmetric system, if any two eigenvectors have different eigenvalues, they are inherently orthogonal. Therefore any eigenvectors that are in separate subintervals can be solved for independently without any need for orthogonalization between subintervals.

Each subinterval is solved by a group of processes we refer to as an evaluator. Each evaluator is comprised of a number of nodes that is determined based on problem size and then each node is broken up into its component processes. This communication hierarchy ensures that each evaluator's system is spread out across as few nodes as possible, resulting in minimal communication when applying the discretized operators.

For this approach, we then needed some method for breaking the interval into subintervals that require equal amounts of work to solve. If we were to instead assume that the eigenvalues are equally distributed and then split the interval evenly among evaluators, very bad load imbalance occurs. The worst case of this load imbalance occurs using this naive splitting with two evaluators. For a simple cube domain, the lower half of the interval contains the vast majority of eigenvalues, and therefore the second evaluator has almost no work to do.

One potential way of making this assumption of uniformly distributed eigenvalues work as a splitting method is to split the interval into a large number of subintervals per evaluator and then distribute them in a round-robin fashion. This approach does reduce the load imbalance to some extent but isn't reliable. We did end up using some ideas from this, specifically having many subintervals per evaluator distributed in a round-robin fashion. This idea was helpful so that no single evaluator had all of their subintervals in a region that is very sparse.

\subsubsection{Counting an Interval}

In order to count how many eigenvalues are contained in an interval [a,b], there are two techniques that we explored. Both approaches are based on counting the number of eigenvalues greater than $a$ and then the number greater than $b$ and taking the difference to get the interval count.

Given some matrix $A$ that can be factored such that $A = LDL^T$, where L is a lower triangular matrix and D is a diagonal matrix, the Sylvester inertia theorem states that the inertia of $D$ and $A$ are the same. This means that the number of entries of $D$ that are greater than 0 is also the number of eigenvalues of $A$ that are greater than 0. Therefore if we let $A$ be the shifted eigenvalue problem centered around $a$, specifically that $A = K - aM$, we can compute this factorization to get the number of eigenvalues greater than some given value of $a$.

While this method results in very accurate eigenvalue counts, it becomes prohibitively expensive as the size of the problem increases. Therefore an approximate counting technique was required to find a balance between accuracy and scaling complexity. For this, we use a method called polynomial expansion filtering.

This approach, which is outline in full detail in \cite{eigcount}, relies on two separate approximations that are composed together. The first bit of machinery is that of approximating the trace of a matrix using $n_v$ random vectors. Specifically, this approximation can be seen in equation \eqref{eq:trace}.

\begin{equation}
\label{eq:trace}
\text{tr}(A) \approx \frac{1}{n_v} \sum_{k=1}^{n_v} v_k^TAv_k
\end{equation}

The approximate count of eigenvalues in an interval is computed by approximating the trace of the eigenprojector of $A$, which is referred to as $P$. The second bit of machinery is that of approximating this eigenproject $P$ by polynomial expansion filtering as seen in equation \eqref{eq:polyfilter}. By plugging the matrix $A$ into the chebyshev expansion of degree $p$ of the heaviside function centered at 0, an approximate form of $P$ is constructed.

\begin{equation}
\label{eq:polyfilter}
	P \approx \phi_p(A) = \sum_{j=0}^{p} \gamma_j T_j(A)
\end{equation}

For the generalized eigenvalue problem we are attempting to solve, the count of eigenvalues in an interval $[a,b]$ is then the difference of the counts for the two shifted systems $K-aM$ and $K-bM$. Specifically, this can be seen in equation \eqref{eq:eigapprox}.

\begin{equation}
\label{eq:eigapprox}
	\mu_{[a,b]} = \mu_a - \mu_b \approx \text{tr}[\phi_p (K-aM) - \phi_p (K-bM)]
\end{equation}

\subsection{Results}

One of the main assumptions that determine the load balancing of the spectrum slicing approach is that if two independent subintervals have an equal number of eigenvalues, that they take equal amounts of time to solve. Some of the early experiments with the approximate eigenvalue counting technique resulted in very uneven distribution of eigenvalues per subinterval. However, using the data from these experiments, we plotted each subinterval's solve time and eigenvalue count and can be seen in figure \ref{fig:solvetime_vs_count}. From this it can be seen there is a very distinct linear relationship between the eigenvalue count and solve time.

\begin{figure}
	\caption{Comparison of number of eigenvalues within a subinterval and the amount of time taken to solve for all eigenpairs within the subinterval. Observe the clear linear relationship between the two values.}
	\label{fig:solvetime_vs_count}
	\input{figures/solvetime_vs_count}
\end{figure}

In order to verify that our method results in near-optimal load balancing, we ran the eigenbasis computation for a range of problem sizes and number of evaluators. These experiments were run on the <Institution 1>'s <Cluster 1> computing cluster with 28 processes per node and a single node per evaluator. The results of this scaling experiment can be seen in figure \ref{fig:scaling}.

\begin{figure}
	\caption{The time it takes to compute the full eigenbasis for various problem sizes vs the number of nodes used.}
	\label{fig:scaling}
	\input{figures/scaling_4_log}
\end{figure}

Since the total eigenbasis computation is divided into smaller independent tasks, one way to determine load balancing is to observe the scaling efficiency per number of evaluators. For $p$ evaluators, the scaling efficiency is then equal to $t_1 / (p\cdot t_p) $, where $t_1$ is the time elapsed for a single evaluator and $t_p$ is the maximum time elapsed for all of the $p$ evaluators. The closer this value is to 1, the closer the work is divided perfectly evenly among evaluators. Ideally, this value should be 1 or close to it. The resulting scaling efficiency can then be seen in figure \ref{fig:efficiency}.

\begin{figure}
	\caption{Scaling efficiency vs problem size with respect to number of nodes.}
	\label{fig:efficiency}
	\input{figures/efficiency}
\end{figure}

{\color{blue} \noindent <Comparison of performance and accuracy of exact and approximate interval counting> }

\subsection{Potential Improvements}

Our framework results in close to optimal load balancing for the simple domains we have tested, however, it isn't necessarily guaranteed. We could make better guarantees of the load balancing by designating a single process to maintain a queue of subintervals to be processed. Using this approach, as each evaluator finishes a subinterval, it can report to the task manager and request the next subinterval. This way, the only time any evaluator is starved for work is when there is no more work to be done.

Since the newest versions of PETSc have incorporated GPU acceleration, we could then set up GPU evaluators alongside the previously mentioned CPU evaluators. In this case, a task queue would be essential to properly manage the work among the different types of evaluators. Even if the GPU evaluators are not faster than the CPU evaluators, they would be able to function in parallel, thus resulting in an effective evaluator count greater than the actual number of machines available.


\documentclass[fleqn]{article}

% start meta data
\title{The Effect of Scaling and Mean Centering Prior to a Principal Component Analysis}
\author{Sebastian Raschka\\ \texttt{se.raschka@gmail.com}}
\date{\today}
% end meta data

\usepackage{amsmath}

% start header and footer
\usepackage{fancyhdr}
\pagestyle{fancy}
\lhead{Sebastian Raschka}
\rhead{Centering and scaling variables for PCA}
\cfoot{\thepage} % centered footer
\renewcommand{\headrulewidth}{0.4pt}
\renewcommand{\footrulewidth}{0.4pt}
% end header and footer

\begin{document} % start main document

\maketitle % makes the title page from meta data

\noindent A frequently asked question is whether data needs to be centered prior to dimensionality reduction via \emph{Principal Component Analysis (PCA)} or not. Here, the assumption is that the PCA is computed based on the \emph{covariance matrix}, i.e., the $k$ principal components are the eigenvectors of the covariance matrix that correspond to the $k$ largest eigenvalues.

\section{Mean Centering Does not Affect the Covariance Matrix}
\label{meancenteringdoesnotaffectthecovariancematrix}

Under the assumption that the PCA is obtained from covariance matrix, the resulting principal components will be the same regardless of whether mean centering was performed or not as long as the covariance matrix stays the same. The following equations will show that \emph{mean centering} does not affect the covariance matrix.
\noindent Let  $\bf{x}$ and $\bf{y}$  be two random variables so that  the covariance between the attributes is calculated as

\begin{equation} \sigma_{xy} = \frac{1}{n-1} \sum_{i}^{n} (x_i - \bar{x})(y_i - \bar{y})   \end{equation}

\noindent The centered variables can be written as

\begin{equation} x' = x - \bar{x} \text{ and } y' = y - \bar{y} \end{equation}

\noindent And the centered covariance can be calculated as follows:

\begin{equation} \sigma_{xy}' = \frac{1}{n-1} \sum_{i}^{n} (x_i' - \bar{x}')(y_i' - \bar{y}')   \end{equation}

\noindent Mean centering results in samples means of 0, $\bar{x}' = 0$ and $\bar{y}' = 0$, thus

\begin{equation} \sigma_{xy}' = \frac{1}{n-1} \sum_{i}^{n} x_i' y_i'   \end{equation}

\noindent This is equal original covariance matrix, which can be shown by resubstitution:

\begin{equation} x' = x - \bar{x} \text{ and } y' = y - \bar{y} \end{equation}

\noindent Even the centering of only one variable,  e.g., $\bf{x}$,  would leave the covariance matrix unaffected.

\begin{equation} \sigma_{\text{xy}} = \frac{1}{n-1} \sum_{i}^{n} (x_i' - \bar{x}')(y_i - \bar{y})   \end{equation}
\begin{equation}  =  \frac{1}{n-1} \sum_{i}^{n} (x_i' - 0)(y_i - \bar{y})   \end{equation}
\begin{equation}  =  \frac{1}{n-1} \sum_{i}^{n} (x_i - \bar{x})(y_i - \bar{y})   \end{equation}

\section{Scaling Does Affect the Covariance Matrix}
\label{scalingofvariablesdoesaffectthecovariancematrix}

However, in contrast to mean centering, \emph{scaling} (e.g, pounds into kilogram where 1 pound = 0.453592 kg) \textbf{does} have an effect on the covariance matrix and therefore influences the results of a PCA.

\noindent Let $c$ be the scaling factor for $\bf{x}$. Given that the ``original'' covariance is calculated as

\begin{equation} \sigma_{xy} = \frac{1}{n-1} \sum_{i}^{n} (x_i - \bar{x})(y_i - \bar{y})   \end{equation}

\noindent the covariance after scaling is calculated as follows:

\begin{equation} \sigma_{xy}' = \frac{1}{n-1} \sum_{i}^{n} (c \cdot x_i - c \cdot  \bar{x})(y_i - \bar{y})   \end{equation}
\begin{equation} =  \frac{c}{n-1} \sum_{i}^{n} (x_i -   \bar{x})(y_i - \bar{y})   \end{equation}

\begin{equation} \Rightarrow \sigma_{xy} = \frac{\sigma_{xy}'}{c} \end{equation}
\begin{equation} \Rightarrow \sigma_{xy}' = c \cdot \sigma_{xy} \end{equation}

\noindent Therefore, the scaling of one attribute by a constant $c$ will result in a rescaled covariance $c \sigma_{xy}$. E.g., if an attribute $\bf{x}$ measured in pounds is rescaled  into kilograms, the covariance between $\bf{x}$ and $\bf{y}$ will be 0.453592 times less the original value.

\section{Standardizing Does Affect the Covariance Matrix}
\label{standardizingaffectsthecovariance}

Standardization of features also has an effect on the outcome of a PCA since standardization can be understood as a scaling the covariance between every pair of variables by the product of the standard deviations of each pair of variables.

\noindent The equation for standardization of a variable is written as

\begin{equation} z = \frac{x_i - \bar{x}}{\sigma} \end{equation}

The ``original'' covariance matrix:

\begin{equation} \sigma_{xy} = \frac{1}{n-1} \sum_{i}^{n} (x_i - \bar{x})(y_i - \bar{y})   \end{equation}

And after standardizing both variables:

\begin{equation} x' = \frac{x - \bar{x}}{\sigma_x} \text{ and } y' =\frac{y - \bar{y}}{\sigma_y} \end{equation}

\begin{equation} \sigma_{xy}' =  \frac{1}{n-1} \sum_{i}^{n} (x_i' - 0)(y_i' - 0)   \end{equation}

\begin{equation}  =  \frac{1}{n-1} \sum_{i}^{n} \bigg(\frac{x - \bar{x}}{\sigma_x}\bigg)\bigg(\frac{y - \bar{y}}{\sigma_y}\bigg)   \end{equation}

\begin{equation}   = \frac{1}{(n-1) \cdot \sigma_x \sigma_y} \sum_{i}^{n} (x_i - \bar{x})(y_i - \bar{y})   \end{equation}

\begin{equation} \Rightarrow \sigma_{xy}' = \frac{\sigma_{xy}}{\sigma_x \sigma_y} \end{equation}

\end{document} % end main document


 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage {algorithmic}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 1} }
\vspace {1em}
\begin {Instruction}

Adapted From Virginia Williams’ lecture notes. Additional credits: J. Su, W. Yang, Gregory Valiant, Mary Wootters, Aviad Rubinstein, Sami Alsheikh.
\end {Instruction}

\begin{centering}
\section*{Karatsuba's Algorithm \& Pseudocode Practice}
\end{centering}

\section{Karatsuba Integer Multiplication Cont.}

\subsection{Divide and Conquer}

The ``Divide and Conquer'' algorithm design paradigm is a very useful and widely applicable technique. We will see a variety of problems to which it can be fruitfully applied. The high-level idea is just to split a given problem up into smaller pieces, and the solve the smaller pieces, often recursively.

How can we apply Divide and Conquer to integer multiplication? Lets try splitting up the numbers. For example, if we were multiplying 1234 × 5678, we could express this as ($(12 \cdot 100) + 34) \cdot ((56 \cdot 100) + 78$). In general, if we are multiplying two $n$-digit numbers $x$ and $y$, we can write $x = 10^{n/2} \cdot a + b$ and $y = 10^{n/2} \cdot c + d$. So

$$
x \cdot y = (10^{n/2}a + b) \cdot (10^{n/2}c + d) = 10^nac + 10^{n/2}(ad + bc) + bd.
$$

Now we can split this problem into four subproblems, where each subproblem is similar to the
original problem, but with half the digits. This gives rise to a recursive algorithm.

Interestingly enough, \textbf{this algorithm isn’t actually better!} Intuitively this is because if we
expand the recursion, we still have to multiply every pair of digits, just like we did before. But
in order to prove this formally, we need to formally define the runtime of an algorithm, and
prove that these algorithms are not very different in runtime.

\subsection{Recurrence Relation}
We can analyze the runtime of the algorithm as follows.

Let $T(n)$ be the runtime of the algorithm, given an input of size $n$ (two $n$-digit numbers).
Because we are breaking up the problem into four subproblems with half the digits, plus some
addition with linear cost, we have the equation $T(n) = 4T(n/2) + O(n)$. (Don’t worry if you
haven’t seen big-O notation before; we’ll go over this in detail in the coming lectures.)

Although \textbf{in general you should pay attention to the $O(n)$ term}, today we will just ignore
it because the term doesn’t matter in this case.

By repeatedly breaking up the problem into subproblems, we find that

$$
T(n) = 4T(n/2) = 16T(n/4) = \cdots = 2^{2t}T\left(\frac{n}{2^t}\right) = n^2 T(1)
$$

Since $T(1)$ is the time it takes to multiply two digits, we see that the above suggestion does not reduce the number of 1-digit operations.

\textbf{Note:} In the lecture slides, we’ll consider a slightly different argument, which analyzes a
recursion tree. It’s a good exercise to understand both arguments! Again, we’ll discuss both
techniques more in coming lectures.

\subsection{Divide and conquer (take 2)}

Karatsuba found a better algorithm (in 1960, published in 1962) by noticing that we only need the sum of $ad$ and $bc$, not their actual values. So he improved the algorithm by computing $ac$ and $bd$ as before, and computing $(a+b)\cdot (c +d)$. It turns out that if $t = (a+b) \cdot (c +d)$, then $ad + bc = t - ac - bd$. Now instead of solving four subproblems, we only need to solve three! This idea goes back to Gauss, who found a similar efficient way to multiply two complex numbers.

Sure, we need to do more additions, but again it turns out that additions are pretty cheap. To do a quick-and-dirty analysis of the number of operations required by Karatsuba multiplication, first assume that $n = 2^s$ for some integer $s$. (Note that we can always add $0$’s to the front of a number until the length is a power of two, so this assumption holds without loss of generality.) Letting $T(n)$ denote the number of multiplications of pairs of 1-digit numbers required to compute the product of two $n$-digit numbers, Karatsuba’s algorithm gives $T(n) = 3T(n/2)$, since we’ve divided the problem into three recursive calls to multiplication of length $n/2$ numbers. [Note, we are cheating a bit here, since $(a + b)$ and $(c + d)$ might actually be $n/2 + 1$ digit numbers, but lets ignore this for now $\cdots$] Hence we have the following:

$$
T(n) = 3T(n/2) = 3^2T(n/4) = \cdots = 3^sT(n/2^s)
$$

Since we assumed that $n = 2^s$, we have that $T(n/2^s) = T(1) = 1$ since multiplying two 1-digit
numbers counts as 1 basic operation. Hence $T(n) = 3s$, where $n = 2^s$. Solving for $s$ yields
$s = \log_2 n$, and hence we get

$$
T(n) = 3^{\log_2 n} = 2^{(\log_2 3)(\log_2 n)} = n^{\log_w 3} \leq n^{1.6}
$$

We were pretty sloppy with the above argumentation in a lot of ways. However, we’ll see a much more principled way of analyzing the runtime of recursive algorithms in the coming classes, so we won’t sweat about it too much now. The point is that (even if you do it correctly) the running time of this algorithm scales like $n^{1.6}$. Thus is \textbf{much} better than the $n^2$ algorithm that we learned in grade school!

\subsection{Can we do better}

Progress on efficient algorithms for multiplication of $n$-digit numbers compared continued beyond Karatsuba's algorithm. Although you don’t need to know these algorithms for CS 161, it is interesting to review the history of progress on this problem. Toom and Cook (1963) developed an algorithm that ran in time $O(n^{1.465}$) by showing how a single $n$-sized problem could be broken up into five $n/3$-sized problems. Schönage and Strassen (1971) developed an algorithm that runs in time $O(n \log(n) \log \log(n))$. More than 35 years later, Fürer (2007) developed an algorithm that ran in time $n \log(n)2\log^*(n)$. In case you are wondering what that weird function $\log^*(n)$ (read “log star”) is, it is the number of times you have to apply the logarithm function $\log()$ iteratively to $n$ in order to get down to something $\leq 1$. For all values of $n$ less than the estimated number of atoms in the universe, the value of $\log^*(n)$ (with base 2) is less than 5. So $\log^*(n)$ is a really really really slowly growing function of n. Finally, Harvey and van der Hoeven (2019) gave an algorithm that runs in time $O(n \log n)$. This is conjectured to be optimal. It is quite amazing that the seemingly simple (and old) question of multiplying two numbers has proved to be so mysterious and has seen new research advances as recently as 2019. This is what makes the study of algorithms so exciting!

\section{Practice with Pseudo-code}

Let's now shift gears a bit from algorithms, and practice some C++. It's important for use to be able to take our ideas and convert them into code. Generally speaking, the following approach to solving problems is critical.

\begin{itemize}
    \item Read the problem carefully and make sure you understand what you're trying to solve. This might require that you run through a few examples, by hand, just like we did in lecture.
    \item Try to figure out a \textit{correct} solution to your problem. Usually, this comes down to figuring out how you would do it by hand, and then trying to convert this into a step-by-step process: an algorithm.
    \item Write down your proposed solution in pseudo-code. We will go through a few examples, but generally speaking, pseudo-code is where you can work through edge-cases, figure out what steps you're missing, and really just jot down everything you need.
    \item Convert your pseudo-code to code. This means you can convert your code to Python, C++, etc. Generally, you'll be converting your code to one of a few languages, but it could be any!
\end{itemize}

\subsection{Finding the minimum element in a collection of elements}

Let's start with a simple problem. We want to find the smallest element in an arbitrary collection. This is what the pseudo-code might look like for this problem:

\begin{verbatim}
algorithm findMinOfVector
  input: a vector of integers nums
  output: smallest integer within nums
  minSoFar = infinity
  for each element num in nums
    minSoFar = min(minSoFar, num)
  return minSoFar
\end{verbatim}

This is a very simple algorithm, so we're pretty sure it's right. We'll jump straight to the last step above then, were we can to convert this function. This is what the corresponding pseudo-code looks like:

\begin{lstlisting}
#include <vector>
#include <limits>

int findMinOfVector(const std::vector<int>& nums) {
 int minSoFar = std::numeric_limits<int>::max();
 for(int i = 0; i < nums.size(); i++) {
   minSoFar = std::min(minSoFar, nums[i]);
 }
 return minSoFar;
}
\end{lstlisting}

You should be comfortable going back-and-forth between the options above. A few things to keep in-mind in the above code:

\begin{itemize}
    \item auto
    \item range-based for-loops
    \item size\_t (though not applicable with range-based for loop)
\end{itemize}

Also, check-out the course website C++ resources for more.

\subsection{Practice Spotting Bugs in Pseudo-code}

Another reason why pseudo-code is useful, is that it can be a low-commitment way to find bugs in your code. Before you actually get around to writing your code (which can take a lot of time, and requires you know the syntax for your language), it's very useful to read your pseudo-code and see if there are any bugs.

Let's try it out.

\begin{itemize}
    \item What does the algorithm below say it does?
    \item What issues prevent the algorithm from being correct?
\end{itemize}

\begin{verbatim}
algorithm findSecondSmallest
  input: list of integers nums, where size of nums > 1
  output: the second smallest element in nums
  firstSmallest = -infinity
  secondSmallest = -infinity
  for each element num in nums
    if num < firstSmallest
      firstSmallest = num
      secondSmallest = firstSmallest
    else if num < secondSmallest
      secondSmallest = num
  return num
\end{verbatim}

The first issue is with how we're setting the initial values for \texttt{firstSmallest} and \texttt{secondSmallest}. They should be set to a value which is trivially \textbf{larger} than all other values (eg, \texttt{infinity}).

The second issue has to do with the order in which we're updating the \texttt{firstSmallest} and \texttt{secondSmallest} when we find an element \texttt{num} that's smaller. We need to first set our \texttt{secondSmallest} variable to our \texttt{firstSmallest} (since we found a smaller number, our previous smallest is now the second smallest), and \textbf{then} set our \texttt{firstSmallest} to the number we just found.

The last issue with this code with the our return statement. First of all, \texttt{num} is likely our of scope at this point, and is not that we want to return. What we want to return is \texttt{secondSmallest}.

With the fixes discussed above, our pseudocode should look like this:

\begin{verbatim}
algorithm findSecondSmallest
  input: list of integers nums, where size of nums > 1
  output: the second smallest element in nums
  firstSmallest = infinity
  secondSmallest = infinity
  for each element num in nums
    if num < firstSmallest
      secondSmallest = firstSmallest
      firstSmallest = num
    else if num < secondSmallest
      secondSmallest = num
  return secondSmallest
\end{verbatim}

\subsection{Planning an Algorithm with Pseudocode}

The next useful thing about pseudocode is that it helps us plan algorithms. We can plan and write out our pseudocode without ever having to think about syntax, or issues that we'll likely run into when actually trying to write the code. Let's work through an example.

\textbf{Problem:} Kanye West wants to make sure he’s the only person named Kanye West in the world fast. Write a function that takes in both a vector of all names in the world sorted in alphabetical order along with a target name (e.g. “Kanye West”), and returns whether or not the target name appears no more than once in the vector.

\begin{itemize}
    \item What are inputs and outputs?
    \item How would you approach this at a high-level?
\end{itemize}

You might write something like this:

\begin{verbatim}
algorithm findIfNameIsUnique
  input: sorted vector allNames, targetName
  output: true if targetName appears once, false otherwise

for each element name in allNames
  if name == targetName
    if next name == targetName
            return false
    else
            return true
return true
\end{verbatim}

Once you have the initial version of the pseudo-code written, you might want to refine it further, by writing a bit more. Eventually, it looks almost like real code (see below), at which point, you can probably get started on actually writing out the code.

\begin{verbatim}
algorithm findIfNameIsUnique
  input: sorted vector allNames, targetName
  output: true if targetName appears once, false otherwise

start = 0
end = allNames.size() - 1
while start <= end
  midIndex = (start + end) / 2
  if allNames[midIndex] == targetName
    if neighboring name(s) are not targetName
return true
    else
return false
  else if allNames[midIndex] < targetName
    start = midIndex + 1
  else
    end = midIndex - 1
return true  // does not appear

\end{verbatim}
\end{document}