\documentclass[11pt]{article}
\usepackage{amsmath,amssymb}
\addtolength{\textwidth}{1.5in}
\addtolength{\hoffset}{-1in}
\addtolength{\textheight}{1.5in}
\addtolength{\voffset}{-1in}

\title{STAT3401 (Multivariate Methods) Coursework}
\author{To be submitted by 9am, Monday 7th November 2005}
\date{}
\begin{document}
\setlength{\parindent}{0pt}
\setlength{\parskip}{12pt}
\sffamily
\maketitle

\textbf{Please read the following notes before attempting the coursework:}

\begin{enumerate}

\item You should attempt all exercises
\item \textbf{The first section is intended to be completed individually.   Students suspected of copying from others will be penalised}
\item The deadline for this coursework is 9am Monday 7th November, and is to be submitted to the office in 2-5 Kirkby Place.   If you have problems getting to the University by 9am you should hand it in the previous week.   Posting through the letter box is done at your own risk.
\item A late piece of coursework will receive no marks unless accompanied by an acceptable Extenuating Circumstances Form.
\item Please show all workings, if you have checked your calculations with a computer include all relevant computer output.
\item This coursework accounts for 15\% of your final mark on this module
\end{enumerate}

\textbf{Learning outcomes of this coursework:}

Distance and covariance matrices, matrix manipulation and linear combinations form the basis of most multivariate techniques.   Understanding how these work provides a good foundation for learning about various multivariate techniques.   By the end of this coursework you should have gained experience in:

\begin{itemize}
\item Producing an exploratory data analysis in relation to multivariate data
\item Conducting eigen analysis of covariance, correlation and partitioned matrices
\item Exploring a number of properties of these matrices and decompositions
\item Constructing your own distance matrix
\end{itemize}

\section{Matrix operations - Individual work}

The first part of the coursework is to be completed individually.   It is intended to consolidate your ability to deal with matrices and will provide the foundation for later work in the term looking at specific multivariate techniques.

\subsection{Introductory Exercise - 5 marks}

Consider the following three matrices:

\begin{displaymath}
\boldsymbol{A} = \left( \begin{array}{ll} 4 & 4 \\ 4 & 4\end{array} \right),
\boldsymbol{B} = \left( \begin{array}{ll} 4 & 4.001 \\ 4.001 & 4.002 \end{array} \right),
\boldsymbol{C} = \left( \begin{array}{ll} 4 & 4.001 \\ 4.001 & 4.002001 \end{array} \right)
\end{displaymath}

You will notice that the matrices are very similar, and that the differences could easily be lost in rounding.

\begin{itemize}
\item Find the determininants of the three matrices, i.e. $|\boldsymbol{A}|, |\boldsymbol{B}|, |\boldsymbol{C}|$.

\item Where possible, find the inverse of these matrices, i.e. $\boldsymbol{A}^{-1}, \boldsymbol{B}^{-1}, \boldsymbol{C}^{-1}$ (an inverse might not exist for every matrix).

\item Comment on the sensitivity of the inverses to small differences between the three matrices $\boldsymbol{A}$,  $\boldsymbol{B}$ and  $\boldsymbol{C}$.

\end{itemize}

\subsection{Hotelling's data on reading and writing}

The next part of the coursework is based upon Hotelling (1935), and are measures of reading speed $x_{1}$, reading power $x_{2}$, arithmetic speed $x_{3}$ and arithmetic power $x_{4}$ observed on 140 children.   You each have been given your own set of data in the student portal (called yourname.csv, placed in the coursework folder) which are similar to Hotelling's data, but not identical.

\subsubsection{Exploratory data analysis - 25 marks}

\begin{itemize}
\item You are to present a suitable exploratory data analysis of these data, including whatever figures and tables you think are informative.   This should not take more than four pages of A4.
\end{itemize}

%sigma <- matrix(c(1.0, 0.6328, 0.2412, 0.0586, 0.6328,1,-0.0553, 0.0655,
%0.2412, -0.0553, 1, 0.4248, 0.0586, 0.0655, 0.4248, 1),4,4)
%var <- c(5,2,3,1)
%spot <- mvrnorm(140, mu = c(40,55,63,62), Sigma = sigma * outer(var,var))
%cor(spot)
%var(spot)
%spot <- as.data.frame(spot)
%names(spot) <- c("ReadSpeed", "ReadPower", "ArithSpeed", "ArithPower")
%write.table(spot, "u:/teaching/mvm/hotelling.csv", sep = ",", col.names =TRUE)

\subsubsection{Matrix Operations on the covariance and correlation matrices - 25 marks}

This part of the coursework allows us to revise the matrix operations which underly principal components, a topic we will look at later this term.   Whilst you can use a computer to help you with the calculations, you are to include sufficient workings in your coursework to indicate that you understand the derivations of the various answers.

\begin{itemize}
\item Estimate the covariance $\boldsymbol{\hat{\Sigma}}$ and the correlation $\boldsymbol{R}$ matrices for your data.

\item For both the covariance matrix $\boldsymbol{\hat{\Sigma}}$ and the correlation matrix $\boldsymbol{R}$ , find the associated eigenvalues $\lambda_{1} > \lambda_{2} > \lambda_{3}  > \lambda_{4}$ and the corresponding eigenvectors $\boldsymbol{a_{1}}, \boldsymbol{a_{2}}, \boldsymbol{a_{3}}, \boldsymbol{a_{4}}$.  \textbf{Comment on noteworthy differences between the two eigen decompositions}.

\item Find $trace(\boldsymbol{\hat{\Sigma}})$and $trace(\boldsymbol{R})$.   What is the sum of the eigenvalues from the two decompositions, and how do these compare with the traces of the respective covariance and correlation matrices?   \textbf{What information does the trace of the covariance matrix contain?}

\item Find the determinant of $\boldsymbol{\hat{\Sigma}}$  (i.e. $|\boldsymbol{\hat{\Sigma}}|$).   How does this compare with the product of the eigenvalues obtained from the covariance matrix (i.e. $\prod_{i=1}^{4} \lambda_{i}$).

\item Use the first two eigenvectors ($\boldsymbol{a_{1}}$ and $\boldsymbol{a_{2}}$ corresponding to the two largest eigenvalues) to derive the linear combinations $z_{1}$ and $z_{2}$ as follows:

\begin{displaymath}
z_{1i} = \boldsymbol{a_{1}}^{T}\boldsymbol{x} = a_{11}x_{1i} + a_{21}x_{2i} + a_{31}x_{3i} + a_{41}x_{4i}
\end{displaymath}

\begin{displaymath}
z_{2i} = \boldsymbol{a_{2}}^{T}\boldsymbol{x} = a_{12}x_{1i} + a_{22}x_{2i} + a_{32}x_{3i} + a_{42}x_{4i}
\end{displaymath}

\item The eigenvalues provide information on the variation that is captured by each of these linear combinations.   Therefore $\frac{\lambda_{i}}{\Sigma_{i=4}^{4} \lambda_{i}}$ gives an indication as to the total proportion of variation ``explained'' by each linear combination.   \textbf{What is the total proportion of variation explained by the two linear combinations $\boldsymbol{z_{1}}$ and $\boldsymbol{z_{2}}$?   Produce a plot which you feel depicts the proportion of variation ``explained'' by each of the four linear combinations.}

\item One of the goals of this kind of analysis is that these linear combinations explain the data in two dimensions almost as well as having to work in four dimensions.   Produce a scatter plot of $z_{1}$ against $z_{2}$.   \textbf{How does this compare with a full $4 \times 4$ pairwise scatter plot?}

\end{itemize}

\subsubsection{Matrix operations on the partitioned correlation matrix - 25 marks}

This part of the coursework revises those matrix operations which underly a technique called canonical correlation, which we will study in more detail next term.

The correlation matrix obtained from Hotelling's original data is given below.  We have seen how it is possible to partition a matrix.   Consider partitioning the $4 \times 4$ correlation matrix $\boldsymbol{R}$ into four $2 \times 2$ sub-matrices:

\begin{displaymath}
\boldsymbol{R} = \left( \begin{array}{l|l} \boldsymbol{R_{11}} & \boldsymbol{R_{12}} \\ \hline    \boldsymbol{R_{21}} & \boldsymbol{R_{22}} \end{array} \right) =
\left( \begin{array}{ll|ll} 1.0 & 0.6328 & 0.2412 & 0.0586 \\
0.6328 & 1.0 & -0.553 & 0.0655 \\
\hline
0.2412 & -0.553 & 1.0 & 0.4248 \\
0.0586 & 0.0655 & 0.4248 & 1.0
\end{array} \right)
\end{displaymath}

So for example

\begin{displaymath}
\boldsymbol{R_{11}} = \left( \begin{array}{ll} 1.0 & 0.6328\\
0.6328 & 1.0
\end{array}
\right)
\end{displaymath}

The object of Hotelling's analysis was to see whether there was some kind of correlation between the two measures (speed and power) recorded on reading, and the two measures (speed and power) recorded on arithmetic.

One way of conducting such an analysis is to carry out an eigen decomposition of a $2 \times 2$ matrix formed from the various sub-matrices.   Your coursework is to demonstrate that you can carry out these matrix operations, which are detailed below.   You are to show all workings, although you are obviously free to check each stage with a computer.

%\subsection{Tasks}

You are to use the \emph{correlation matrix you obtained from your own data} set.

\begin{itemize}

\item Find the matrix $\boldsymbol{A}$, where:

\begin{displaymath}
\boldsymbol{A} = \boldsymbol{R_{12}^{-1}R_{21}^{T}R_{11}^{-1}R_{21}}
\end{displaymath}

\item For matrix $\boldsymbol{A}$, find the associated eigenvalues $\lambda_{1} > \lambda_{2}$ and the corresponding eigenvectors $\boldsymbol{b_{1}}, \boldsymbol{b_{2}}$.

We will find out later that these eigenvalues are the squares of the correlation between the linear combinations we identify, and that $\boldsymbol{b}$ are the loadings on the second set of variables.

\item We now need to find the coefficients for the first set of variables (reading).   Find
$a_{1} = \boldsymbol{R_{11}^{-1} R_{21} b_{1}}$ and $a_{2} = \boldsymbol{R_{11}^{-1} R_{21} b_{2}}$.   Again, show your workings.

\item Similar to a previous exercise, find the linear combinations:

\begin{displaymath}
u_{1i}  = a_{11}x_{1i} + a_{21}x_{2i}
\end{displaymath}

\begin{displaymath}
v_{1i} =  b_{12}x_{3i} + b_{22}x_{4i}
\end{displaymath}

and produce a scatterplot of $u_{1}$ against $v_{1}$.

\item Estimate the correlation between $u_{1}$ and both $x_{1}$ and $x_{2}$.   Also estimate the correlation between $v_{1}$ and both $x_{3}$ and $x_{4}$.   \textbf{Do you think $u_{1}$ and $v_{1}$ provide an adequate representation of their two respective observed variables?}

\end{itemize}

\section{Distances and similarities - Group work - 20 marks}

This part of the coursework may be completed as part of a group.   You are to work in groups of no more than five.   I will be happy to provide \emph{appropriate} assistance (for example if you need help finding the numbers in various languages).

We are going to investigate the relationship between languages (for example Italian and French are thought to belong to one family of languages, English and German to another, Polish and Czech to another).   In order to do this, we need to devise some way of measuring similarities or differences between languages.   To ensure this is a feasible task, you are to consider the numbers $1$ to $10$ in seven or eight different languages (any languages will do).

You are to devise a scheme for measuring either the similarity OR the difference between corresponding numbers in different languages.   You may use any method you choose, for example you could award a point for similarity if two languages have the same number of syllables, another point if they have a similar vowel sound.   Equally, you may decide to make a more subjective assessment on a scale of 1 to 10 as to whether the numbers sound alike.   You may also wish to split the job, so that each member of the group works with two numbers.

\subsection{Tasks}

\begin{itemize}
\item Produce a similarity (or distance) matrix for each number showing how similar or different your eight languages are.  \textbf{You are to explain your methodology in no more than five sentences.}

\item Produce a single similarity (or distance) matrix for all ten numbers.   You are required to consider different methods of pooling results.   For example, if a number shows a lot of variability should you standardise the measures before you pool them?   If you split the task among your group, did some members get better results and should you produce a weighted sum to reflect this?   \textbf{You are to explain your method of pooling results, and to identify at least two advantages and two disadvantages of that particular method.}
\end{itemize}

As you may have guessed, we will use these distance matrices next term when we consider cluster analysis.

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Carlos Santos             %
% ECE 351-51                %
% Lab 5                     %
% 03/3/2020                 %
%                           %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass[12pt]{article}

% Language and font encoding
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{caption}
\usepackage{float}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{rotating}
\usepackage{setspace}

% Sets page size and margins
\usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}

% Useful packages
\usepackage[colorinlistoftodos]{todonotes}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\usepackage{listings}
\usepackage{gensymb}
\usepackage{graphicx} %package to manage images
\usepackage{listings}
\usepackage{color}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=Python,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

%Line Spacing
\setstretch{1.5}

%Info for Title Page
\title{ECE 351 Lab 4 Report \\ Section 52}
\date{March 3, 2020}
\author{Carlos Santos}
\begin{document}

%Make a Title Page
\vspace{\fill}
\maketitle
\vspace{\fill}
\clearpage

%Introduction
\section*{Introduction}

This lab was focused on being able to program the "Sine Method" taught by Dr. Sullivan.
It was also focused on using built in functions to check our work.

%Equations
\section*{Equations}

Transfer Function for the RLC circuit:
\begin{equation}
    H(S) = \frac{(s/RC)}{(S^2 + S/RC + 1/LC)}
\end{equation}

Equations for the  Sine  Method:
\begin{equation}
    s = \frac{-b+\sqrt{b^2 - 4ac}}{2a}
\end{equation}

\begin{equation}
    p = \alpha + j\omega
\end{equation}

\begin{equation}
    g = \frac{1}{RC} * s
\end{equation}
where s = p

\begin{equation}
    |g| = \sqrt{\alpha ^ 2 + \beta ^ 2}
\end{equation}

\begin{equation}
    <g = tan^-1 (\frac{\beta}{\alpha})
\end{equation}

Final Value Theorem:
\begin{equation}
    \lim_{t\to\infty} f(t) \xrightarrow{} \lim_{s\to 0}sF(S)
\end{equation}

\begin{equation}
    \lim_{s\to\infty} sF(s) = \lim_{s\to 0} \frac{\frac{1}{RC}}{s^2 + \frac{1}{RC}s + \frac{1}{LC}} = 0
\end{equation}

%Methodology
\section*{Methodology}
 \begin{enumerate}
     \item Find the H(s) equation for the RLC circuit given in the prelab handout.
     \item Find "Sine Method" equations
     \begin{enumerate}
         \item Next, translate that into code.
         \item The code looks like this:
        \\ \begin{lstlisting}
    def sine_method(R,L,C,t):
    y = np.zeros(t.shape) #t.shape of whatever is inputted in
    alpha = -1/(2*R*C)
    omega = 0.5*np.sqrt(1/(R*C))**2 - 4*(1/(np.sqrt(L*C)))**2 + (0 * 1j)
    p = alpha + omega
    g = 1/(R*C)*p
    g_mag = np.abs(g)
    g_rad = np.angle(g)
    g_deg = g_rad*(180/np.pi)
    y = ((g_mag/np.abs(omega))*np.exp(alpha*t)
        *np.sin(np.abs(omega)*t + g_rad))*u(t)
    return y
        \end{lstlisting}
    \item Next, graph the h(t) function.
    \item Now, using the sig.impulse graph the impulse response
    \item Again, use the sig.step to graph the step response.
    \item Verify plots with the "Final Value Theorem".
     \end{enumerate}
 \end{enumerate}

%Results
\section*{Results}

\begin{figure}[H]
\caption{RCL Transfer Equation}
\centering
\includegraphics[width=.8\textwidth]{h_t.png}
\end{figure}

\begin{figure}[H]
\caption{Impulse Response}
\centering
\includegraphics[width=.8\textwidth]{sig_impulse.png}
\end{figure}

\begin{figure}[H]
\caption{Step Response}
\centering
\includegraphics[width=.8\textwidth]{sig_step.png}
\end{figure}

\\
During this lab I found out that it was hard to input the whole sine method at once.
Instead the best solution is to break it up into parts and let python do the work for me.
Also, the "Final Value Theorem" results in "0" as the solution which matches the graphs.
\\
For future reference the code below is how we can declare our transfer function in python. From my understanding we allocate two vectors. One for the numerator and anther for the denominator. num = [s,1] den = [$s^2$,s,1]

\begin{lstlisting}
num = [1/(R*C),0] # numerator
den = [1,1/(R*C),(1/np.sqrt(L*C))**2] # denomerator
\end{lstlisting}

%Questions
\section*{Questions}
\begin{enumerate}
    \item Explain the result of the Final Value Theorem from Part 2 Task 2 in terms of the physical
circuit components.
    \begin{enumerate}
        \item In terms of hardware the Final Value Theorem shows what the final state of a group of components will be. Looking at our circuit we know the capacitor charges up and then discharges. The inductor then charges up and discharges. The Final Value Theorem shows that eventually the system will loose all energy through the resister and be "off".
    \end{enumerate}
    \item Leave any feedback on the clarity of the expectations, instructions, and deliverables.
    \begin{enumerate}
        \item Not much to say, the lab was straight forward.
    \end{enumerate}
\end{enumerate}

%Conclusion
\section*{Conclusion}

\end{document}


\section{Inner Product}
\label{innerproduct}

Iverson generalized linear algebra's inner
product.~\cite{Iverson62,KEIverson:tot}
In his groundbreaking book, he said, in regard to
{\em generalized matrix product}:

\begin{quote}
This formulation emphasizes the fact that matrix multiplication
incorporates two elementary operations (${+}, {\times}$) and
suggests that they be displayed explicitly. The ordinary
matrix product will therefore be written as:
$X \stackrel{+}{\times} Y$.
\end{quote}

For \ibmapl, that notation had to be linearized to allow entry
and display on the typewriter-like terminals of that era.
Hence, the ordinary
inner product is now written as: {\apl X\qplus\qdot\qtimes\0Y},
and the general case as: {\apl X f\qdot\0g Y}, where {\apl f}
and {\apl g} are any suitable verbs.
The generalized inner product has many uses: one step
of the Boolean-Boolean transitive-closure computation on a connectivity matrix
can be computed as: {\apl M\qor\qdot\qand\0M}.
A weaving on a loom can be modeled as a single inner product: a matrix,
{\apl tieup}, specifies which harness(es) should be raised
when a given treadle is pushed; {\apl tread} specifies the
treadling order; {\apl thread} specifies which threads are
in the weft:

\medskip
{\apl tieup\qlbr\0tread\qsemic\qrbr\qor\qdot\qeq\0thread}
\medskip

\fixme{Give example}

The classical algorithm for inner product is painfully slow for all
data types, but for Boolean arrays, it is excruciatingly so.
In the early 1970s, the author heard of an inner-product algorithm
that had been used by colleagues at I.P. Sharp Associates,
who were developing the STAR APL interpreter for the CDC STAR-100,
a memory-to-memory supercomputer that required stride-1
algorithms for acceptable performance. The STAR APL algorithm
worked by interchanging the loop ordering. The classic inner product
can be written as:

\input{ipclassic}

\noindent By comparison, the STAR APL algorithm can be written
in APL this way, with one less level of visible loop
nesting:\footnote{The mere removal of one level of explicit
looping makes the STAR APL model execute about 90 times
faster than the Classic model, on 200x200 matrices of
real numbers. Of course, that is still 30 times slower than
the APL derived function.}

\input{ipstar}

The STAR APL algorithm has a number of advantages over the classical
one: First, it references {\tt Xel}, an element of {\tt X}, only once, and
does so in stride-1 (ravel) order. Second, any type conversion or
analysis of {\apl Xel} is amortized over
the inner loop, so its cost is usually negligible.
Third, the multiplies ({\apl g}) in the inner loop are stride-1,
scalar-vector computations, so
they make good use of cache and are trivially vectorizable.
Fourth, the sum ({\apl f}) reduction in the inner loop is also stride-1
and vector-vector, offering similar performance benefits.
Fifth, the latter two vector operations can be loop-fused.
Finally, in the Boolean domain, an entire word from {\apl X} can
be quickly checked to see if it is all zeros or all ones.
This often permits even more work to be elided.

The author realized that the STAR algorithm should work extremely well
with APL's Boolean arrays, for the above reasons, and because
it could perform the inner loop a word or a matrix row at a time,
producing, by itself, a substantial speedup.
David Allen, then working in the APL Development Group, AKA ``The Zoo",
at I.P. Sharp Associates Limited, implemented our algorithm in
the SHARP APL interpreter. The computational complexity
of inner product allowed us to obtain
final performance improvements on Boolean arrays of
about three orders of magnitude (32 times 32).

The STAR APL algorithm offers other benefits.
Looking at the STAR APL inner loop, we see that it computes a
scalar-vector function on {\apl g}:

\medskip
{\apl Xel \qtimes~Y\qlbr\0J\qsemic\qrbr}\\

\noindent We exploited some
simple algebra to eliminate some or all of the work done
in each of these computations.~\cite{RBernecky:apex}
Consider {\apl 0 1 \qor\qdot\qand~Y}: In the first iteration,
{\apl Xel} is {\apl 0}, {\apl f} is {\apl \qor}, and {\apl g} is {\apl \qand}.
Hence, the {\apl g} computation is:

{\apl 0~\qand~Y\qlbr0\qsemic\qrbr}

\noindent This will always generate a vector of zeros of the same
length as a row of {\apl Y}. Knowing this lets us skip
the {\apl g} computation for that element.
Furthermore, since zero is the identity element for {\apl \qor},
we can also skip the reduction part of the computation on {\apl f}.

The author, while adapting the STAR APL algorithm for use
in the SAC back end of the APEX APL compiler, developed
a further performance enhancement, along these same
lines.\cite{RBernecky:apex}
Some time later, Hui independently rediscovered both of these
enhancements.\cite{RKWHui:innerproduct}
The STAR APL and STARLIKE algorithms are now used in
the APEX APL compiler, Dyalog APL, the J interpreter, and
in Walter Fil's Plural compiler.

That enhancement, which we dubbed {\em STARLIKE},
recognized, essentially, that
in the second iteration, {\apl Xel}
is {\apl 1}, which is the identity element for {\apl \qand}.
This lets us, again, skip the {\apl g}-computation entirely,
and then perform the {\apl f}-reduction using the current row of Y.

An added advantage of the STAR algorithm is that it works well
on all data types, providing a crude form of sparse array support
for elements of {\apl X}.


\section{Application to synthetic data}\label{sec:synthetic}

\subsection{Simple model test}

We have simulated a lopolithic intrusion, a funnel-shaped source \cite[]{cawthorn-2018}, with simple geometry (blue prisms in Figs \ref{fig:simple_model} and \ref{fig:simple_results}), which extends from $z_0=0$ m to $1\,600$ m along depth and satisfies most of the constraints described in subsection \ref{sec:constraints}. It is formed by $L=8$ prisms, all of them with the same number of vertices $ V = 20 $, thickness $ dz = 200 $ m and horizontal coordinates $ (x_0^k, y_0^k) = (0, 0) $ m of the origins $O^k$, $k=1,\dots,L$. The radii of all vertices are equal to each other within the same prism and decrease linearly with depth, varying from $r_j^0=1\,920$ m, at the shallowest prism, $r_j^L=800$ m, at the deepest prism, $j=1,\dots, V$. All prisms have the same total-magnetization direction with inclination $ -50^\circ $, declination $ 9^\circ $ and intensity $ m_0 = 9 $ A/m. We calculated the total-field anomaly produced by this simple model, in an area of $ 100 $ km$^2$ area, by simulating an airborne survey composed of $ 21 $ flight lines that are equally spaced $ 500 $ m apart along the $ y $-axis, at a constant vertical coordinate $ z = -150 $ m. At each line, there are $ 101 $ observation points spaced $100$ m apart along $ x $-axis.
The main geomagnetic field direction simulated was $ -21.5^\circ $ and $ -18.7^\circ $ for the inclination and declination, respectively.
The total-field anomaly is corrupted with a pseudorandom Gaussian noise having mean $ \mu_0=0 $ nT and standard deviation $ \sigma_0 = 5 $ nT (Fig. \ref{fig:simple_model}a).

We have inverted the noise-corrupted total-field anomaly (Fig. \ref{fig:simple_model}a) produced by the simulated lopolith-like body (blue prisms in Fig. \ref{fig:simple_model}b) and obtained 36 different estimates.
Each estimate was obtained by using six different pairs of depth to the top $ z_0 $ and total-magnetization intensity $ m_0 $.
The ranges of $ z_0 $ and $ m_0 $ are shown in Fig. \ref{fig:simple_model}(c).
Fig. \ref{fig:simple_model}(d) shows the RTP anomaly obtained from the noise-corrupted
total-field anomaly (Fig. \ref{fig:simple_model}a).
Note that the RTP anomaly exhibits predominantly positive values and decays to zero.
As previously mentioned, the RTP anomaly is used to set up the initial approximation.
The blue circle in Fig. \ref{fig:simple_model}(d) represents the horizontal projection
of the initial approximation $\hat{\mathbf{p}}_{(0)}$ whose shape is a vertical cylinder.
All estimates were generated by using the true values of total-magnetization inclination and declination, the same interpretation model formed by $ L = 5 $ prisms,
each one with $ V = 20 $ vertices, and the same weights for the constraining functions:
$\tilde{\alpha}_1 = 10^{-5}$,
$\tilde{\alpha}_2 = 10^{-4}$,
$\tilde{\alpha}_3 = 10^{-4}$,
$\tilde{\alpha}_4 = 0$,
$\tilde{\alpha}_5 = 0$,
$\tilde{\alpha}_6 = 10^{-7}$, and
$\tilde{\alpha}_7 = 10^{-6}$.
In all inversions, the initial approximation $\hat{\mathbf{p}}_{(0)}$ (red prisms in Fig. \ref{fig:simple_results}b) has the same constant radii $ r^k_j = 2\,000 $ m,
$ k = 1, \dots, L $, $ j = 1, \dots, V $, the same prism thicknesses $ dz = 350 $ m and the same origins $(x^k_0, y^k_0) = (0, 0) $ m for all prisms.

In a field data application, the unknown complexity of the real source can not be fully represented by the set of prisms forming the interpretative model. To approximate the synthetic test to a field data application, we define the initial approximation $\hat{\mathbf{p}}_{(0)}$ with a smaller number of parameters $ P $ than the true source. We perform that by decreasing the number of prisms $ L $ and/or the number of vertices $ V $. This is also a strategy applied to all synthetic tests to evaluate the robustness of the method.

Fig. \ref{fig:simple_model}(c) shows the discrete mapping of the goal functional
$\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ (Eq. \ref{eq:gamma}) on the plane of the total-magnetization intensity ($ m_0 $) versus depth to the top ($ z_0 $).
The true values of depth to the top $ z_0 $ and total-magnetization intensity $ m_0 $ (represented by the red triangle in Fig. \ref{fig:simple_model}c) produce
the smallest value of goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$
(Eq. \ref{eq:gamma}) and are taken as the optimum values.
This estimated model (red prisms in Figs \ref{fig:simple_results}c and d)
not only fits the noise-corrupted data (Fig. \ref{fig:simple_results}a),
but also retrieves the geometry of the true model (blue prisms in Figs \ref{fig:simple_results}b-d).
The inset in Fig. \ref{fig:simple_results}(a) shows that the residuals follow a normal distribution with mean $ \mu $ and standard deviation $ \sigma $ compatible with those values used to generate the synthetic noise.
The estimated thickness of each prism is $ dz = 306.09 $ m resulting in a depth-to-bottom estimate ($ 1\,530.42$ m) very close to the true one ($ 1\,600 $ m).
These results illustrate the good performance of our method in an ideal case.

\subsection{Dipping model test}

Fig. \ref{fig:dipping_model}(a) shows the noise-corrupted total-field anomaly produced by an outcropping low-angle dipping volcanic duct (blue prisms in Figs \ref{fig:dipping_model}c and d) embedded in nonmagnetic host rocks.
The simulated magnetic data are contaminated with a pseudorandom Gaussian noise having mean $ \mu_0=0 $ nT and standard deviation $ \sigma_0 = 5 $ nT.
We simulated airborne magnetic survey whose flight heights ranging from 0 to $ 720 $ m
(Fig. \ref{fig:dipping_model}b) and resulting in a total of $ 1\,694 $ measurements.
The simulated dipping source has total-magnetization direction with inclination
$ -50^\circ $, declination $ 9^\circ $ and intensity $ m_0 = 12 $ A/m.
The main geomagnetic field direction has inclination of $ -21.5^\circ $ and declination of $ -18.7^\circ $.
To set up the simulated dipping source (blue prisms in Figs \ref{fig:dipping_model}c and d),
we used $L=8$ prisms, all of them with the same number of vertices $ V = 20 $ and thickness
$ dz = 380 $ m.
The horizontal coordinates of the center of the shallowest prism that composes the
simulated dipping source are $ (x_0^1, y_0^1) = (-300, 600) $ m.
The simulated dipping source has a top 0 m deep ($z_0 = 0$) and a base $ 3\,040 $ m deep.

To set up the initial approximation and other variables in the inversion, we calculate the RTP anomaly (Fig. \ref{fig:dipping_rtp}a) of the noise-corrupted total-field anomaly (Fig. \ref{fig:dipping_model}a).
Fig. \ref{fig:dipping_rtp}(b) shows the goal function (Eq. \ref{eq:gamma}) on the plane
$z_0 \times m_0 $ produced by 36 estimates obtained with a grid of $6 \times 6$ tentative values of depth to the top $z_0$ and total-magnetization intensity $m_0$.

All $ 36 $ estimates were generated by using the true values of total-magnetization inclination and declination, the same interpretation model formed by $ L = 5 $ prisms,
each one with $ V = 20 $ vertices, and the same weights for the constraining functions:
$\tilde{\alpha}_1 = 10^{-3}$,
$\tilde{\alpha}_2 = 10^{-3}$,
$\tilde{\alpha}_3 = 10^{-6}$,
$\tilde{\alpha}_4 = 10^{-6}$,
$\tilde{\alpha}_5 = 10^{-6}$,
$\tilde{\alpha}_6 = 10^{-6}$, and
$\tilde{\alpha}_7 = 10^{-5}$.
Similar results were obtained by setting $\tilde{\alpha}_4$ and $\tilde{\alpha}_5$ to zero (not shown).

The optimum solution is the one that produces the smallest value of the discrete mapping
of the goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ (Fig.
\ref{fig:dipping_rtp}b).
The white diamond in Fig. \ref{fig:dipping_rtp}(b) pinpoints the minimum of the goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ which is achieved when the estimated body has a top $ 20 $ m deep and a total-magnetization intensity of $ 12 $ A/m.
Note that the depth to the top of the estimated body is deeper than the true one;
however, the total-magnetization intensity was retrieved correctly.

Fig. \ref{fig:dipping_results}(a) shows the magnetic-data residuals, defined as the difference
between the synthetic noise-corrupted magnetic data in Fig. \ref{fig:dipping_model}(a)
and the predicted data (not shown).
Fig. \ref{fig:dipping_results}(b) shows the simulated dipping source (blue prisms) and
the cylinder-like initial approximation (red prisms) with thickness $ dz = 800 $ m and
and the same origins $(x^k_0, y^k_0) = (-200, 0) $ m for all prisms.
Note that the estimated source (red prisms in Figs.\ref{fig:dipping_results}c and d)
with $z_0 = 20$ m and $m_0 = 12$ A/m
(pinpointed as the white diamond in Fig. \ref{fig:dipping_rtp}b)
retrieved the shape of the simulated dipping source (blue prisms) reasonably, although
the retrieved depth to the bottom of $ 3\,452.97 $ m was deeper than the true one.
The retrieved deep-bottomed dipping source is probably influenced by a depth to the top $ z_0 $ deeper than the true one.

\subsection{Dipping model in the presence of a regional field test}

We analyze the ability of the proposed method to retrieve the geometry of the source
in a simulated real-world scenario, where the observed total-field anomaly is produced not only by
the magnetized source but also by a regional field.
Here, the magnetized source to be retrieved has the same shape (blue prisms in Figs \ref{fig:dipping_model}c and d) and magnetization direction of the previous test.
In this test, the noise-corrupted total-field anomaly to be inverted
(Fig. \ref{fig:dipping_regional_model}a) is obtained by adding
a simulated regional field (Fig. \ref{fig:dipping_regional_model}b),
represented by a first-order polynomial,
to a simulated residual field (Fig. \ref{fig:dipping_model}a) due to outcropping low-angle dipping volcanic duct (blue prisms in Figs \ref{fig:dipping_model}c and d).

To remove the regional field from the original data (Fig. \ref{fig:dipping_regional_model}a),
we perform a regional-residual separation by fitting a first-order polynomial
to the original data using the least-squares method.
Fig. \ref{fig:dipping_regional_model}(c) shows the residual total-field anomaly after subtracting from the original anomaly a regional total-field anomaly
(Fig. \ref{fig:dipping_regional_model}d) obtained by a least-squares polynomial fitting.

We invert the anomaly over the area delimited by the magenta rectangle shown in Fig. \ref{fig:dipping_regional_model}(c).
This anomaly to be inverted is called residual total-field anomaly and it is shown in
Fig. \ref{fig:dipping_regional_rtp}(a).
Because we expect that this anomaly is due to the dipping model (blue prisms in Figs. \ref{fig:dipping_model}c and d), we calculate the
differences (Fig. \ref{fig:dipping_regional_rtp}b) between the total-field anomaly produced by the dipping model (Fig. \ref{fig:dipping_model}a) and the residual total-field anomaly
(Fig. \ref{fig:dipping_regional_rtp}a) after a regional-residual separation using a least-squares polynomial fitting.
Note that over the source the differences are less than $ 0.8\%$ of the amplitude of the data due to the dipping model.
Fig. \ref{fig:dipping_regional_rtp}(c) shows the RTP anomaly of the residual total-field anomaly shown in \ref{fig:dipping_regional_rtp}(a) that is used to set up the
cylinder-like initial approximation.
We perform $ 36 $ inversions by setting the same control variables of the previous test and similar results were found by setting $\tilde{\alpha}_4$ and $\tilde{\alpha}_5$ to zero (not shown).
The $ 36 $ estimates yield the discrete mapping of the goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ (Eq. \ref{eq:gamma})
on the plane $z_0 \times m_0 $ shown in Fig. \ref{fig:dipping_regional_rtp}(d).
The smallest value of this goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ (white diamond) pinpoints
the optimum pair of $z_0$ and $m_0$.
By comparing the retrieved values of $z_0$ and $m_0$ (white diamond) and
the true ones (red triangle), we can note an ambiguity involving these variables.
In this test, the retrieved depth to the top of $ 40 $ m is deeper than the true one and the
retrieved total-magnetization intensity of $ 15 $ A/m is higher than the true one.

Although there is an ambiguity involving $z_0$ and $m_0$, the small residuals (Fig. \ref{fig:regional-results}a) indicate a reasonable fit of the inverted total-field anomaly
(Fig. \ref{fig:dipping_regional_rtp}a).
In all 36 inversions, we used the same cylinder-like initial approximation (red prisms in Fig. \ref{fig:regional-results}b).
By assiging $z_0 = 40$ m and $m_0 = 15$ A/m (pinpointed as the white diamond in Fig. \ref{fig:dipping_regional_rtp}d), the estimated source (red prisms in Figs.\ref{fig:regional-results}c and d) recovers the shape of the simulated dipping source
(blue prisms in Figs.\ref{fig:regional-results}b-d) reasonably.

\subsection{Complex model test}

Here, we have simulated a complex high-angle dipping intrusion (blue prisms in Figs \ref{fig:complex_model} and \ref{fig:complex_result}) inspired by an alkaline vertical dipping intrusion.
The simulated intrusion extends from $z_0=130$ m to $6\,130$ m along depth.
It is formed by $ L = 10 $ prisms, all of them with the same number of vertices $ V = 30 $ of thickness $ dz = 600 $ m.
The horizontal coordinates of the origins $ O^k $ vary linearly from $ (x_0^0, y_0^0) = (-250, 250) $ m, at the shallowest prism, to $ (x_L^0, y_L^0) = (250, -750) $ m, at the deepest prism.
This ensemble of 10 vertically juxtaposed prisms, exhibiting horizontal displacements between them, form a dyke-like intrusion, dipping at a high angle to the northwest (blue prisms in Figs \ref{fig:complex_model} and \ref{fig:complex_result}).
The radii defining the vertices of these prisms, $ r^k_j, k = 1, \dots, L$, $j = 1,\dots, V$,
vary from $ 240 $ m to $ 1\,540 $ m and also differ from each other within the same prism.
All prisms have a constant total magnetization with inclination $ -50^\circ $, declination $ 9^\circ $ and intensity $ m_0 = 12 $ A/m.
We stress that the shape of the simulated complex model (blue prisms in Figs \ref{fig:complex_model} and \ref{fig:complex_result}) violates severely two constraints described in subsection \ref{sec:constraints}: $\varphi_{1}(\mathbf{p})$ (Eq. \ref{eq:phi1})
and $\varphi_{2}(\mathbf{p})$ (Eq. \ref{eq:phi2}).
The violation of these constraints can be viewed by the nonsmoothness feature of the adjacent radii defining the horizontal section of each vertical prism and by the nonsmoothness feature of the adjacent radii of the vertically adjacent prisms.

We calculate the total-field anomaly produced by this complex model, in an area of $ 100 $ km$^2 $, by simulating an airborne survey composed of 18 north-south flight lines distributed from $ -5\,000 $ m to $ 5\,000 $ m along the $ y $-axis and a single east-west tie line approximately located at $ x = 0 $ m.
The data points are located on the undulated surface shown in Fig. \ref{fig:complex_model}(a). Notice that both flight and tie lines are not perfectly straight.
To compute the synthetic total-field anomaly, we consider a constant main geomagnetic field with inclination $ -21.5^\circ $ and declination $ -18.7^\circ $.
Finally, we have contaminated the synthetic total-field anomaly with an additive pseudorandom Gaussian noise having mean and standard deviation equal to $0$ nT and $5$ nT, respectively (Fig. \ref{fig:complex_model}a).

To set up the cylinder-like initial approximation, we calculate the RTP anomaly (Fig. \ref{fig:complex_rtp}a) of the noise-corrupted total-field anomaly
(Fig. \ref{fig:complex_model}a) .
We have inverted the noise-corrupted total-field anomaly produced by the complex model by using 36 different pairs of depth to the top $ z_0 $ and total-magnetization intensity
$ m_0 $ (Fig. \ref{fig:complex_rtp}b).
Differently from the previous simulations, in this test, the grid of $ m_0 $ and $ z_0 $
does not contain the true ones (represented by the red triangle in Fig. \ref{fig:complex_rtp}b).
All the $ 36 $ estimates, obtained considering a grid of 6 $\times$ 6 tentative values of $z_0$ and $m_0$ (shown in Fig. \ref{fig:complex_rtp}b), use: i) the true magnetization direction of the simulated complex model (i.e., $ -50^\circ $, declination $ 9^\circ $), ii) the same interpretation model formed by $ L = 8 $ prisms, each one with $ V = 20 $ vertices, iii) the same cylinder-like initial approximation, and iv) the same weights for the constraining functions:
$\tilde{\alpha}_1 = 10^{-5}$,
$\tilde{\alpha}_2 = 10^{-4}$,
$\tilde{\alpha}_3 = 10^{-4}$,
$\tilde{\alpha}_4 = 0$,
$\tilde{\alpha}_5 = 0$,
$\tilde{\alpha}_6 = 10^{-7}$, and
$\tilde{\alpha}_7 = 10^{-6}$.

Fig. \ref{fig:complex_rtp}(b) shows the goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ (Eq. \ref{eq:gamma}), with different total-magnetization intensity $ m_0 $ and depth-to-the-top $z_0$, on the plane ($ m_0 \times z_0 $).
We note that a well-defined minimum region the goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ (dark blue region) contains the true pair of $z_0 = 130$ m and $m_0 = 12 $ A/m (red triangle) and the optimum pair of
$z_0 = 150$ m and $m_0 = 12.6 $ A/m (white diamond), which yields the smallest value of the goal function $\Gamma (\hat{\mathbf{p}}_{(f)}, m_{0}, z_{0})$ and
retrieves the estimated model.

Fig. \ref{fig:complex_result}(a) shows the residuals defined as the difference
between the synthetic noise-corrupted magnetic data in Fig. \ref{fig:complex_model}(a)
and the predicted data (not shown).
The cylinder-like initial approximation ($ \hat{\mathbf{p}}_{(0)} $), shown
in Fig. \ref{fig:complex_result}(b) by the red prisms, is composed by a set of eight prisms, has the same constant radii $ r^k_j = 800 $ m, $ k = 1, \dots, 8 $, $ j = 1, \dots, 20 $,
the same thickness $ dz = 650 $ m and the same origin
$ (x^k_0, y^k_0) = (-300, 300) $ m for all prisms of the interpretation model.

Figs. \ref{fig:complex_result}(c) and (d) show the estimated model (red prisms)
obtained by using $z_0 = 150$ m and $m_0 = 12.6 $ A/m
(white diamond in Fig. \ref{fig:complex_rtp}b).
Note that the this estimated model fits the noise-corrupted data
(small residuals in Fig. \ref{fig:complex_result}a)
and also retrieves the geometry of the true source
(blue prisms in Figs. \ref{fig:complex_result}b-d).
The inset in Fig. \ref{fig:complex_result}(a) shows that the residuals follow a normal distribution with mean $ \mu $ and standard deviation $ \sigma $ compatible with those values used to generate the noise-corrupted data.
The estimated of depth to the bottom ($ 5\,904.45 $ m) and volume ($ 11.19 $ km$^3 $) are underestimated, but still close to the true values ($ 6\,130 $ m and $ 12.60 $ km$^3 $).
These results show that our method can also be very useful to interpret complex sources, even if they do not perfectly satisfy the constraints imposed to solve the nonlinear inverse problem.


%
\documentclass[12pt,a4paper]{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[affil-it]{authblk}

% Math
\usepackage{amsmath}

% Links
\usepackage[pdftex]{hyperref}%
\hypersetup{%
colorlinks=true,%
urlcolor=black,%
citecolor=black,%
linkcolor=black,%
pdfauthor="T. Whyntie"}

% Front matter formatting
\renewcommand\Affilfont{\fontsize{9}{10.8}\itshape}

\title{Maxwell's equations with and without magnetic charge}

\author[1,2]{T.~Whyntie}

\affil[1]{School of Physics \& Astronomy, Queen Mary University of London, UK}
\affil[2]{Institute for Research in Schools, UK}

% Omit the date.
\date{}

\begin{document}

\input{defs}

\maketitle

\begin{abstract}
Maxwell's equations of classical electromagnetism~\cite{Maxwell1865}
are presented,
in Gaussian units\footnote{
The equations are written in the \emph{Gaussian unit}
system, and not SI units, for simplicity;
in this system the electric field $\electricfield$
and the magnetic field $\magneticfield$ have the same units.},
without (\ref{eq:1},~\ref{eq:2},~\ref{eq:3},~\ref{eq:4})
and
with (\ref{eq:1m},~\ref{eq:2m},~\ref{eq:3m},~\ref{eq:4m})
magnetic charge.
Equations~\ref{eq:1m} to~\ref{eq:4m} are
taken from the front cover of the
Monopole and Exotics Detector at the LHC (MoEDAL)
Technical Design Report~\cite{MoEDAL2009},
the Large Hadron Collider's seventh major experiment
and the latest venture to be undertaken in the search for
Dirac's hypothesised magnetic monopole~\cite{Dirac1931}.
\end{abstract}

\begin{eqnarray}
\label{eq:1} \divvec{\electricfield} & = & 4 \pi \electricchargedensity\\ [10pt]
\label{eq:2} \divvec{\magneticfield} & = & 0\\ [10pt]
\label{eq:3} - \curlvec{\electricfield} & = & \myfrac{1}{\speedoflight} \, \timediff{\magneticfield}\\ [10pt]
\label{eq:4}   \curlvec{\magneticfield} & = & \myfrac{1}{\speedoflight} \, \timediff{\electricfield} + \myfrac{4 \pi}{\speedoflight} \, \electriccurrentdensity
\end{eqnarray}

\begin{eqnarray}
\label{eq:1m} \divvec{\electricfield} & = & 4 \pi \electricchargedensity\\ [10pt]
\label{eq:2m} \divvec{\magneticfield} & = & 4 \pi \magneticchargedensity\\ [10pt]
\label{eq:3m} - \curlvec{\electricfield} & = & \myfrac{1}{\speedoflight} \, \timediff{\magneticfield} + \myfrac{4 \pi}{\speedoflight} \, \magneticcurrentdensity\\ [10pt]
\label{eq:4m}   \curlvec{\magneticfield} & = & \myfrac{1}{\speedoflight} \, \timediff{\electricfield} + \myfrac{4 \pi}{\speedoflight} \, \electriccurrentdensity
\end{eqnarray}

\section*{Acknowlegdements}
This work was supported by the
UK Science and Technology Facilities Council (STFC)
through grant ST/N00101X/1.
The author wishes to thank A. Pontzen for many
fruitful discussions.

\bibliographystyle{elsarticle-num}
\bibliography{bibfile}

\end{document}
%


\XtoCBlock{PILimit}
\label{block:PILimit}
\begin{figure}[H]\includegraphics{PILimit}\end{figure}

\begin{XtoCtabular}{Inports}
In & Control error input\tabularnewline
\hline
Init & Value which is loaded at initialization function call\tabularnewline
\hline
max & Maximum output value\tabularnewline
\hline
min & Minimum output value\tabularnewline
\hline
Enable & Enable == 0: Deactivation of block; Out set to 0

Enable 0->1: Preload of integral part

Enable == 1: Activation of block\tabularnewline
\hline
\end{XtoCtabular}

\begin{XtoCtabular}{Outports}
Out & \tabularnewline
\hline
\end{XtoCtabular}

\begin{XtoCtabular}{Mask Parameters}
Kp & Proportional Factor\tabularnewline
\hline
Ki & Integral Factor\tabularnewline
\hline
ts\_fact & Multiplication factor of base sampling time (in integer format)\tabularnewline
\hline
\end{XtoCtabular}

\subsubsection*{Description:}
PI controller with output limitation:

    G(s) = Kp + Ki/s

% include optional documentation file
\InputIfFileExists{\XcHomePath/Library/Control/Doc/PILimit_Info.tex}{\vspace{1ex}}{}

\subsubsection*{Implementations:}
\begin{tabular}{l l}
\textbf{FiP8} & 8 Bit Fixed Point Implementation\tabularnewline
\textbf{FiP16} & 16 Bit Fixed Point Implementation\tabularnewline
\textbf{FiP32} & 32 Bit Fixed Point Implementation\tabularnewline
\textbf{Float32} & 32 Bit Floating Point Implementation\tabularnewline
\textbf{Float64} & 64 Bit Floating Point Implementation\tabularnewline
\end{tabular}

\XtoCImplementation{FiP8}
\index{Block ID!3232}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP8 \tabularnewline
\textbf{ID} & 3232 \tabularnewline
\textbf{Revision} & 2.0 \tabularnewline
\textbf{C filename} & PILimit\_FiP8.c \tabularnewline
\textbf{H filename} & PILimit\_FiP8.h \tabularnewline
\end{tabular}
\vspace{1ex}

8 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & Integral coefficient\tabularnewline
\hline
b1 & Proportional coefficient\tabularnewline
\hline
sfrb0 & Shift factor for PI coefficient b0\tabularnewline
\hline
sfrb1 & Shift factor for PI coefficient b1\tabularnewline
\hline
i\_old & Integrator value of previous cycle\tabularnewline
\hline
enable\_old & Enable value of previous cycle\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int8          *In;
     int8          *Init;
     int8          *max;
     int8          *min;
     int8          *Enable;
     int8          Out;
     int8          b0;
     int8          b1;
     int8          sfrb0;
     int8          sfrb1;
     int16         i_old;
     int8          enable_old;
} PILIMIT_FIP8;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_PILimit_FiP8.tex}{}{}
\fi
\XtoCImplementation{FiP16}
\index{Block ID!3233}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP16 \tabularnewline
\textbf{ID} & 3233 \tabularnewline
\textbf{Revision} & 2.0 \tabularnewline
\textbf{C filename} & PILimit\_FiP16.c \tabularnewline
\textbf{H filename} & PILimit\_FiP16.h \tabularnewline
\end{tabular}
\vspace{1ex}

16 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & Integrall coefficient\tabularnewline
\hline
b1 & Proportional coefficient\tabularnewline
\hline
sfrb0 & Shift factor for PI coefficient b0\tabularnewline
\hline
sfrb1 & Shift factor for PI coefficient b1\tabularnewline
\hline
i\_old & Integrator value of previous cycle\tabularnewline
\hline
enable\_old & Enable value of previous cycle\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int16         *In;
     int16         *Init;
     int16         *max;
     int16         *min;
     int8          *Enable;
     int16         Out;
     int16         b0;
     int16         b1;
     int8          sfrb0;
     int8          sfrb1;
     int32         i_old;
     int8          enable_old;
} PILIMIT_FIP16;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_PILimit_FiP16.tex}{}{}
\fi
\XtoCImplementation{FiP32}
\index{Block ID!3234}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP32 \tabularnewline
\textbf{ID} & 3234 \tabularnewline
\textbf{Revision} & 2.0 \tabularnewline
\textbf{C filename} & PILimit\_FiP32.c \tabularnewline
\textbf{H filename} & PILimit\_FiP32.h \tabularnewline
\end{tabular}
\vspace{1ex}

32 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & Integrall coefficient\tabularnewline
\hline
b1 & Proportional coefficient\tabularnewline
\hline
sfrb0 & Shift factor for PI coefficient b0\tabularnewline
\hline
sfrb1 & Shift factor for PI coefficient b1\tabularnewline
\hline
i\_old & Integrator value of previous cycle\tabularnewline
\hline
enable\_old & Enable value of previous cycle\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int32         *In;
     int32         *Init;
     int32         *max;
     int32         *min;
     int8          *Enable;
     int32         Out;
     int32         b0;
     int32         b1;
     int8          sfrb0;
     int8          sfrb1;
     int64         i_old;
     int8          enable_old;
} PILIMIT_FIP32;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_PILimit_FiP32.tex}{}{}
\fi
\XtoCImplementation{Float32}
\index{Block ID!3235}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & Float32 \tabularnewline
\textbf{ID} & 3235 \tabularnewline
\textbf{Revision} & 2.0 \tabularnewline
\textbf{C filename} & PILimit\_Float32.c \tabularnewline
\textbf{H filename} & PILimit\_Float32.h \tabularnewline
\end{tabular}
\vspace{1ex}

32 Bit Floating Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & Integral coefficient\tabularnewline
\hline
b1 & Proportional coefficient\tabularnewline
\hline
i\_old & Integrator value of previous cycle\tabularnewline
\hline
enable\_old & Enable value of previous cycle\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     float32       *In;
     float32       *Init;
     float32       *max;
     float32       *min;
     int8          *Enable;
     float32       Out;
     float32       b0;
     float32       b1;
     float32       i_old;
     int8          enable_old;
} PILIMIT_FLOAT32;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_PILimit_Float32.tex}{}{}
\fi
\XtoCImplementation{Float64}
\index{Block ID!3236}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & Float64 \tabularnewline
\textbf{ID} & 3236 \tabularnewline
\textbf{Revision} & 2.0 \tabularnewline
\textbf{C filename} & PILimit\_Float64.c \tabularnewline
\textbf{H filename} & PILimit\_Float64.h \tabularnewline
\end{tabular}
\vspace{1ex}

64 Bit Floating Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & Integral coefficient\tabularnewline
\hline
b1 & Proportional coefficient\tabularnewline
\hline
i\_old & Integrator value of previous cycle\tabularnewline
\hline
enable\_old & Enable value of previous cycle\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     float64       *In;
     float64       *Init;
     float64       *max;
     float64       *min;
     int8          *Enable;
     float64       Out;
     float64       b0;
     float64       b1;
     float64       i_old;
     int8          enable_old;
} PILIMIT_FLOAT64;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_PILimit_Float64.tex}{}{}
\fi


\documentclass[../main]{subfiles}
\begin{document}

\chapter{Perfectly secret encryption}

\section{Perfect secrecy}

\begin{definition}[Perfect Secrecy]
    An encryption scheme (\textit{Gen, Enc, Dec}) is perfectly secret if for every message $m \in{} \mathcal{M}$ and every ciphertext $c \in{} \mathcal{C}$ for which $Pr(C=c)>0$ we have that $$Pr(M=m \mid{} C=c) = Pr(M=m)$$
\end{definition}

\begin{lemma}[Flips previous equation]
    \label{lemma:perfect-secrecy-first-characterization}
    An encryption scheme is perfectly secure if and only if for every distribution on $m \in{} \mathcal{M}$ and for every ciphertext $c \in{} \mathcal{C}$ such that $Pr(C=c) > 0$, we have that $$Pr(C=c \mid{} M=m) = Pr(C=c)$$
\end{lemma}

\paragraph{Proof}
We want to prove the following statement:
$$Pr(C=c) > 0, Pr(M=m \mid C=c) = Pr(M=m) \Leftrightarrow{} Pr(C=c \mid M=m) = Pr(C=c)$$
We will do it by the splitting the proof in two parts (the two implications):
\begin{itemize}
    \item[$\Leftarrow$]
        \begin{align*}
            Pr(C=c \mid M=m) &= Pr(C=c) \\
            Pr(C=c) &= Pr(C=c \mid M=m) && \text{(Use the Bayes theorem)} \\
            \cancel{Pr(C=c)} &= \frac{Pr(M=m \mid C=c) \cdot{} \cancel{Pr(C=c)}}{Pr(M=m)} && \text{(Pr(C=c) $>$ 0 can be canceled)} \\
            1 \cdot{} Pr(M=m) &= \frac{Pr(M=m \mid C=c)}{\cancel{Pr(M=m)}} \cdot{} \cancel{Pr(M=m)} \\
        \end{align*}
        As a consequence:
        \begin{align*}
            Pr(M=m \mid C=c) &= Pr(M=m)
        \end{align*}
    \item[$\Rightarrow$]
        \begin{align*}
            Pr(M=m \mid C=c) &= Pr(M=m) && \text{(Use the Bayes theorem)}\\
            \frac{Pr(C=c \mid M=m) \cdot{} \cancel{Pr(M=m)}}{Pr(C=c)} &= \cancel{Pr(M=m)} && \text{(Pr(M=m) $>$ 0 can be canceled)}\\
            \frac{Pr(C=c \mid M=m)}{\cancel{Pr(C=c)}} \cdot{} \cancel{Pr(C=c)} &= 1 \cdot{} Pr(C=c)
        \end{align*}
        As a consequence:
        \begin{align*}
            Pr(C=c \mid M=m) &= Pr(C=c)
        \end{align*}
\end{itemize}

We have therefore proved \ref{lemma:perfect-secrecy-first-characterization}.
%----------------------------------------------------------------------------------
\begin{lemma}[Same probability for every pair of messages]
    \label{lemma:perfect-secrecy-second-characterization}
    An encryption scheme is perfectly secure if for every distribution on $\mathcal{M}$ and for every pair of messages $m_0, m_1 \in{} \mathcal{M}$ and for every ciphertext $c \in{} \mathcal{C}$, it holds that $$Pr(C=C \mid{} M=m_0) = Pr(C=c \mid{} M = m_1)$$
\end{lemma}

\newpage

\paragraph{Proof}
    \begin{enumerate}
        \item[$\Rightarrow{}$]
        This implication is easy to be proved:
            \begin{align*}
                Pr(C=c \mid M=m_0) &= Pr(C=c) \\
                &= Pr(C=c \mid M=m_1)
            \end{align*}
        We are applying the previous lemma because the underlying encryption scheme is perfectly secure.
        \item[$\Leftarrow{}$]
        We can assume that given $c \in \mathcal{C}$, every term in the form
        \[
            Pr(C=c \mid M=m_i)
        \]
        have the same value. Call this value $p_c \in \mathbb{R}_{[0, 1]}$ we can then write that
            \begin{align*}
                Pr(C=c) &= \sum_{m_i \in M} Pr(C=c \mid M=m_i) \\
                &= \sum_{m_i \in M} Pr(C=c \mid M=m_i) \cdot Pr(M=m_i) && \text{(equal by prob theory)} \\
                &= \sum_{m_i \in M} p_c \cdot Pr(M=m_i) \\
                &= p_c \cdot \sum_{m_i \in M} Pr(M=m_i) && \text{(sum is equal to 1)} \\
                &= p_c \cdot 1 \\
                &= p_c \\
                &= Pr(C=c \mid M=m)
            \end{align*}
    \end{enumerate}

    We have therefore proved \ref{lemma:perfect-secrecy-second-characterization}.
%----------------------------------------------------------------------------------
\section{One-Time Pad (aka Vernam's Cipher)}
\begin{definition}[OTP]
    $\mathcal{M} = \mathcal{C} = \mathcal{K} = \{0, 1\}^n$ where
    \begin{align*}
        & Gen \leftarrow{} \{0, 1\}^n \\
        & Enc(k,m) = m \oplus k \\
        & Dec(k,c) = c \oplus k \\
        & \text{Gen returns 000, 001, $\cdots$, 111} && \text{(n=3, each with prob $\frac{1}{8}=\frac{1}{2^n}$)} \\
        & Enc(100, 001) = 101 \\
        & Dec(110, 011) = 101
    \end{align*}
\end{definition}

\begin{theorem}[OTP perfectly-secret]
    \label{theorem:otp-perfectly-secure}
    The OTP is perfectly secure (perfectly-secret).
\end{theorem}

\paragraph{Proof}
    Let's fix a distribution on $\mathcal{M}$ and let's also fix a message $m \in{} \mathcal{M}$ and a ciphertext $c \in{} \mathcal{C}$.
    Consider now the probability of $C=c$ given that $M=m$: we would like to prove that this does \underline{not} depend on $m$.
    \begin{align*}
        Pr(C=c \mid{} M=m) &= Pr(M \oplus{} K=c \mid{} M=m) && \text{(substitute C as rand var)} \\
        &= Pr(m \oplus{} K=c) && \text{(M and K are independent)} \\
        &= Pr(m \oplus{} (m \oplus{} K) = m \oplus{} c) && \text{($m \oplus{} x = m \oplus{} y$ if $x=y$)} \\
        &= Pr((m \oplus{} m)\oplus K = m \oplus{} c) && \text{($m \oplus m$ is $0^n=1$)} \\
        &= Pr(K=m \oplus{} c) \\
        &= \frac{1}{2^n}
    \end{align*}
    We are happy now because we can write $$Pr(C=c \mid{} M=m_0) = \frac{1}{2^n} = Pr(C=c \mid{} M=m_i)$$
    As a consequence, the OTP is perfectly secure and therefore \ref{theorem:otp-perfectly-secure} is proved.
%----------------------------------------------------------------------------------
\section{Shannon's Theorem}

\begin{theorem}[Key set is greater or equal of message set]
    \label{theorem:cardinality-key-message-set}
    If an encryption scheme is perfectly secure, then $\mid{} \mathcal{K}\mid{} \geq \mid{} \mathcal{M}\mid{} $.
\end{theorem}

\paragraph{Proof}
    \begin{itemize}
        \item By way of contradiction. Let us assume that some concrete perfectly secure encryption scheme exists such that $\mid{} \mathcal{K}\mid{} < \mid{}\mathcal{M}\mid{}$, call it $\Pi$
        \item From the fact that $\Pi$ is perfectly secure, we know that it must satisfy the definition of perfect security for every distribution on $\mathcal{M}$. So, suppose that every $m \in{} \mathcal{M}$ is equally likely to occur. Let's also pick one ciphertext $c \in{} \mathcal{C}$ such that $Pr(C=c) > 0$
        \item Let us define $\mathcal{M}(c) \hat{=} \{\hat{m} \mid{} \hat{m} = Dec(k,c)$ $\forall k \in{} \mathcal{K}$\} and let's evaluate $\mid{} \mathcal{M}(c) \mid{}$:
        \begin{itemize}
            \item[$\circ$] $\mid{}\mathcal{M}(c)\mid{} \leq \mid{}\mathcal{K}\mid{} < \mid{}\mathcal{M}\mid{}$
        \end{itemize}
        \item In other words, there must be a message $m' \in{} \mathcal{M}$ such that $m' \notin{} \mathcal{M}(c)$, $$Pr(M=m' \mid C=c) = 0 \text{(because it's impossible)} < Pr(M=m')$$
    \end{itemize}
    Since $\Pi$ was supposed to be perfectly secure, we have reached a contradiction and therefore \ref{theorem:cardinality-key-message-set} is proved.
%----------------------------------------------------------------------------------
\newpage
\section{Indistinguishability}

This characterization doesn't rely on probability theory, it's more computational.
We can consider a program called $PrivK^{eav}_{A, \Pi}$ where:
\begin{itemize}
    \item the abbreviation $eav$ stands for eavesdropper (ciphertext-attack);
    \item $A$ is the adversary that tries to force the scheme;
    \item the scheme $\Pi$ is defined as usual with the triple $(Gen, Enc, Dec)$.
\end{itemize}
In this game the adversary $A$ ask for a pair of messages, it generates a key calling $Gen$ and then generate a random bit called $b$ that takes value 0 or 1 with probability $\frac{1}{2}$ each. After, we encrypt the message $m_b$ according to $k$ key and we ask to adversary to guess which one of the two messages was encrypted. The result is 1 if the adversary correctly guess the value of $b$ or 0 otherwise.\\
$PrivK^{eav}_{A,\Pi}$:\\
$(m_0,m_1) \; \leftarrow{} \; A$\\
$k \; \leftarrow{} \; Gen$\\
$b \; \leftarrow{} \; \{0,1\}$\\
$c \; \leftarrow{} \; Enc(k,m_b)$\\
$b^* \; \leftarrow{} \; A(c)$\\
\textbf{Result}: $\neg(b \oplus{} b^*)$

\begin{definition}[Indistinguishable encryption]
    An encryption scheme $\Pi = (Gen, Enc, Dec)$ has indistinguishable encryptions if and only if for every adversary $A$ we have that:$$Pr(PrivK^{eav}_{A,\Pi} = 1) = \frac{1}{2}$$
\end{definition}

\begin{theorem}[Strong characterization]
    $\Pi$ is perfectly secret if and only if $\Pi$ has indistinguishable encryptions.
\end{theorem}

\end{document}


\documentclass[12pt]{article}
\usepackage[usenames]{color} %used for font color
\usepackage{amssymb} %maths
\usepackage{amsmath} %maths
\usepackage[utf8]{inputenc} %useful to type directly diacritic characters
\usepackage{graphicx}
\usepackage [english]{babel}
\usepackage [autostyle, english = american]{csquotes}
\MakeOuterQuote{"}
\graphicspath{ {./} }
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\prob}{\mathbb{P}}

\author{Tianshuang (Ethan) Qiu}
\begin{document}
\title{Homework 3}
\maketitle
\newpage

\section{Ross 8.2}

\subsection{a}
When $n>0$, both the numerator and the denominatormator is greater than 0, so $a_n > 0 \forall n>0$. Now consider $x_n = \frac{n}{n^2} = \frac{1}{n}$, the denominatormator is less than $a_n$, so $x_n > a_n \forall n>0$.
\newline
Obviously 0 converges to 0, and we have proven in class that $\frac{1}{n}$ converges to 0 (pick $N = \frac{1}{\epsilon}$). $a_n \to 0$ by squeeze theorem.

\subsection{c}
This sequence converges to $\frac{4}{7}$. We will try to show that $x_n = |c_n-\frac{4}{7}| \to 0$.
\newline
$$|c_n-\frac{4}{7}| = |\frac{28n+21}{49n-35} - \frac{4(7n-5)}{49n-35}|$$
$$= \frac{28n+21-28n+20}{49n-35}$$
$$= \frac{41}{49n-35}$$
$$\lim x_n = 41 \times \lim (\frac{1}{49n-35})$$
Once again, we squeeze this sequence with 0 and $\frac{1}{n}$. Since the denominatormator is positive $x_n > 0 \forall n>0$, and $49n-35 > n \forall n>\frac{35}{48}$, so $x_n < \frac{1}{n}$ when $n$ is large. Therefore $\frac{1}{49n-35} \to 0$ by squeeze theorem.
\newline
Therefore $\lim x_n = 49 \times 0 = 0$, and $c_n \to \frac{4}{7}$

\subsection{e}
The sequence converges to 0. We define $x_n = \frac{1}{n}, y_n = \frac{-1}{n}$. Since $-1 \leq \sin n \leq 1$, $y_n \leq s_n \leq x_n$.
\newline
We have shown above that $\frac{1}{n} \to 0$. $\frac{-1}{n}$ can be shown with the same reasoning to converge $\forall n>\frac{1}{\epsilon}$. Therefore $s_n \to 0$ by squeeze theorem.
\newpage

\section{Q2}
\subsection{a}
We will prove this via induction.
\newline
Base case: n = 1, $LHS = 1+a; RHS = \frac{1-a^{2}}{1-a}$. $1-a^2 = (1+a)(1-a)$, therefore LHS = RHS. Base case holds.
\newline
Inductive hypothesis, let the claim hold for an arbitrary $n \in \N$.
$$1+a+...+a^n+a^{n+1} = \frac{1-a^{n+1}}{1-a}+a^{n+1}$$
$$=\frac{1-a^{n+1}+a^{n+1}-a^{n+2}}{1-a}$$
$$=\frac{1-a^{n+2}}{1-a}$$
Thus we have proven the inductive hypothesis. Q.E.D.

\subsection{b}
Since $|s_{n+1}-s_n|<1/2^n \forall n$, $|s_{n+2}-s_n| \leq 1/2^n + 1/2^{n+1}$ by the triangle inequality. We can further extend this so $|s_{n+k}-s_n| \leq 1/2^n + 1/2^{n+1} + ... +1/2^{n+k-1}$
\newline
Let $\epsilon > 0$, we pick $N = \max \{ \log_2 (1/\epsilon), 1\}$. Pick $j,k > N$.
$$|s_j - s_k| \leq \sum ^{j+k-1}_{i=j} \frac{1}{2^i} = \frac{1}{2^j}(1+1/2+...+1/2^{k-1})$$
$$=\frac{1}{2^j}(2(1-1/2^{k})) = \frac{1-1/2^k}{2^{j-1}} \leq \frac{1}{2^j}$$
Since $j> \log_2 (1/\epsilon$), $|s_j - s_k|\leq \frac{1}{2^j} < \epsilon$.
\newline
Therefore the sequence is cauchy. Q.E.D.
\newpage

\section{Ross 10.7}
We essentially want to slowly pick items in $S$ that is closer to $\sup S$.
\newline
Consider $x_n = \sup S - 1/n$. Since $x_n < \sup S, \exists s_n \in S s.t. x_n \leq s_n < \sup S$. Both $x_n$ and $\sup S$ converge to $\sup S$, therefore $s_n \to \sup S$ by squeeze theorem.
\newpage

\section{Q4}
Since $s_n \to s$ by our assumption, let $\epsilon > 0, \exists N s.t. \forall n>N, |s_n - s| < \epsilon$.
\newline
Therefore $s-\epsilon < s_n < s + \epsilon$. Since $s_n$ is between these two end points, and since the $\lim \inf$ is non-decreasing, we have $\lim \inf s_n \geq s-\epsilon$.
\newline
Since $\epsilon$ can be arbitrarily small, we must have $\lim \inf s_n \geq s$. Q.E.D.
\newpage

\section{Q5}
\subsection{a}
We will attempt to prove this via induction.
\newline
Base case: $n = 1, s_n = 2, 2^2-2 >0$
\newline
Inductive case: assume that $s_n^2>2$, then $$s_{n+1}^2-2 = \frac{s_n^2}{4} + \frac{1}{s_n^2} + 2$$. Since $s_n^2 > 2, \frac{s_n^2}{4}>0, \frac{1}{s_n^2}>0$. So $s_{n+1}^2 > 2$.
\newline
$s_n^2 -2> 0 \forall n \in \N$

\subsection{b}
We first prove that $s_n > 0 \forall n \in \N$. $n = 1, s_n = 2, 2>0$.
\newline
In the inductive case, assume $s_n > 0$, so $s_n/2 > 0, 1/s_n>0$, therefore $s_n{n+1}>0$. We have proven the inductive case.
\newline
To prove that it is monotone will again use induction.
\newline
Base case: $n = 1, s_n = 2, s_{n+1} = 2/2 + 1/2 = 3/2$, $s_{n+1}<s_n$. The base case holds.
\newline
Inductive case: assume that $s_n < s_{n-1}$. Consider $s_n-s_{n+1} = s_n - \frac{s_n}{2} - \frac{1}{s_n}$
$$\frac{s_n}{2} - \frac{1}{s_n} = \frac{s_n^2-2}{2s_n}$$
We have shown that $s_n^2 -2> 0 $ and that $s_n>0$, so this fraction is positive, therefore $s_n > s_{n+1}$. This sequence is monotonically decreasing.
\newline
Furthermore, since $0<s_n<2$, the sequence is bounded above and below. Therefore it converges. Q.E.D.
\newpage

\section{Q6}
\subsection{a}
$\lim s_n = \sqrt2$.
\newline
We have shown that the sequence converges, so it is also cauchy. Let $\epsilon > 0$, there exists $N$ such that for $ i> N, |s_i - s_{i+1}| < \epsilon$.
\newline
Since $\epsilon$ can get arbitrarily small and that it converges, at very large values of $n$, $s_n$ should be very close the the limit and to $s_{n+1}$. We should have $\lim_{n \to \infty} s_n= \lim_{n \to \infty} s_{n+1}$. Therefore $\lim s_n$ should satisfy $$\lim s_n = \frac{\lim s_n}{2}+ \frac{1}{\lim s_n}$$
$$(\lim s_n)^2 = \frac{\lim s_n^2}{2} + 1$$
$$\frac{1}{2}(\lim s_n)^2 = 1$$
$$(\lim s_n)^2 = 2$$
$$\lim s_n= \sqrt2$$

\subsection{b}
The sequence $s_n$ has $s_n \in \Q$ because each term is the sum of two rationals. Since it is convergent, the sequence is cauchy, but $s_n \to \sqrt 2$, which is an irrational.
\newpage

\section{Q7}
\subsection{a}
Since the sequence is $-1$ when $n$ is odd and $1$ otherwise, it oscillates ad infinitum. $-1 \leq s_n \leq 1$, so $\lim \inf s_n = -1, \lim \sup s_n = 1$
\newline
Let $x_n = \{ s_n | n \bmod 2 \equiv 1\} = \{-1, -1, -1 ...\}$. Since $x_n = -1 \forall n \in \N$, $x_n \to -1 = \lim \inf$
\newline
Let $y_n = \{ s_n | n \bmod 2 \equiv 0\} = \{1, 1, 1 ...\}$. Since $y_n = 1 \forall n \in \N$, $x_n \to 1 = \lim \sup$

\subsection{b}
We claim that $s_n \to 0$.
\newline
Let $\epsilon > 0$, choose $N = \log_2(1/\epsilon)$, $\forall n > N, |s_n-0| = |s_n| = \frac{1}{2^n}$
$$\frac{1}{2^n} < \frac{1}{2^N} = \epsilon$$
Therefore the sequence converges to 0.
\newline
Since we have proven that a sequence converges $iff$ its $\lim \inf = \lim \sup$, we have $\lim \inf s_n= \lim \sup s_n = 0$
\newline
Define $x_n = \{ s_n | n \bmod 2 \equiv 1\}$. Since we are only taking odd $n$'s from the set, $x_n < 0$, and since a subsequence of a convergent sequence also converges to its limit, $x_n \to \lim \inf$
\newline
By the same logic we define Define $y_n = \{ s_n | n \bmod 2 \equiv 0\}$. We can see that $y_n > 0$, and by the convergence theorem stated above we have $y_n \to \lim \sup s_n= 0$

\subsection{c}
Using the limit theorem we can break this down into $\lim (-1)^n$ and $\lim 1/n$. From the work above and in class proofs we have $\lim \inf s_n = -1 + 0 = -1$ and $\lim \sup s_n = 1 + 0 = 1$
\newline
Similar to (a), we define $x_n = \{ s_n | n \bmod 2 \equiv 1\} = (-1)^{2n-1} + 1/(2n-1)$. $\lim x_n = \lim (-1)^{2n-1} + \lim 1/(2n-1)$. The former is constant while the latter is a subsequence of $1/n$ so it converges to 0. Moreover, this sequence only decreasing due to $1/n$ so it is monotone. Finally we have $\lim x_n = \lim \inf s_n = -1$
\newline
Then we define $y_n = \{ s_n | n \bmod 2 \equiv 0\} = (-1)^{2n} + 1/(2n) = 1+1/(2n)$. By the same logic as above the first term converges to 1 while the second converges to 0. It is also monotonically decreasing for the same reason. Finally we have $\lim y_n = \lim \sup s_n = 1$

\end{document}


\subsection{Oriented Gaussians dataset}

\begin{figure}
  \includegraphics[width=\columnwidth]{experiment-Gaussians.pdf}
  \vspace{2mm}
  \includegraphics[width=\columnwidth]{colormap.pdf}
  %
  \caption{%
  %
  {\bfseries \sffamily Oriented Gaussians dataset (results)}
  %
  Randomized SOM made of $1024$ neurons with a $2$-nearest neighbors induced topology. Model has been trained for $25,000$ epochs on oriented Gaussian datasets. \textbf{A} Map topology in neural space. \textbf{B} Map topology in data space. \textbf{C to H} Receptive field of the map for six samples.
  %
  }
  \label{fig:gaussians:results}
\end{figure}


\section{Basics}

For more information, please refer to \cite{AurelienGeron2019} Part 1.

\subsection{Classification}

In \cindex{supervised learning} the dataset is a collection of $\set{(X_i, y_i)}_{i=1}^N$. $X$ is called \cindex{feature vector}, and $y$ is called \cindex{label}. If $y$ is discrete, it is \cindex{classification}. If $y$ is continuous, it is called \cindex{regression}. In \cindex{unsupervised learning} the dataset is a collection of $\set{X}_{i=1}^N$.

Some useful definitions are:
\begin{enumerate}
    \item \cindex{batch learning}:
        \begin{itemize}
            \item use all available data.
            \item cannot be modified after production launch.
            \item \cindex{offline learning}
        \end{itemize}
    \item \cindex{online learning}:
        \begin{itemize}
            \item sequentially take data inputs in \cindex{mini-batch}.
            \item \cindex{out-of-core} learning: handle huge dataset that is too big for the machine memory.
            \item \cindex{learning rate}: the speed of learning.
                \begin{itemize}
                    \item If it is too low, the system stop improvement.
                    \item If it is too high, sensitive to data noise. So need to monitor system performance and do early stop.
                \end{itemize}
        \end{itemize}
\end{enumerate}

The result of learning could be measured using \cindex{utility} which is a positive measure, or a \cindex{cost function} which is a negative measure.

\subsection{Prepare Data}

\subsubsection{Background}

Very large samples can still be non-representative if the sampling method is flawed. It is called \cindex{sampling error}. The most famous one is in year 1936 US presidential election between Landon and Roosevelt: the Literary Digest poll. Literary Digest use phone directory to select the candidate, which favours wealthy people. Only 25\% answered, so they may not care about politics, or do not like Literary Digest.

The process of selecting relevant feature is called \cindex{feature engineering}, which includes feature selection and feature extraction.

If the model \cindex{overfit}, try to simplify the model using fewer parameters or using regularization, or increase the training data size. If the model \cindex{underfit}, make the model more powerful, or reduce regularization.

\subsubsection{Select Test Data}

The data will be split into three disjoint sets: training set, validation set and test set. It is common to use $80\%$ for training and $20\%$ for test. However if the data set is very huge, $1\%$ for testing is ok.

Test data selection process (\cindex{stratified sampling}):
\begin{enumerate}
    \item divide all data into homogeneous subgroups (\cindex{strata}), such as by gender, income, etc.
    \item select a random percentage as test data.
\end{enumerate}

\subsubsection{Data Cleaning}

After test set is selected, the training set need to be cleaned. There are few useful data cleaning methods:
\begin{enumerate}
	\item scaling:
		\begin{enumerate}
			\item min-max scaling: scale to $[0,1]$.
			\item standardization: scale to zero mean and unit variance.
		\end{enumerate}
	\item string handling:
		\begin{enumerate}
			\item ordinal encoding: convert string to enumeration.
			\item one-hot encoder: convert string to a $0$-$1$ matrix.
		\end{enumerate}
	\item NAN handling: replace NAN by median.
	\item polynomial change: change number using polynomial function.
\end{enumerate}

The same cleaning result need to be applied to validation and test set after the training is done.

If the category is too huge, the one-hot encoder may generate a wide matrix. In this case, an embedding to low dimentional vector could be learned using representation learning.

\subsubsection{Training with Data}

The training process now has these steps:
\begin{enumerate}
	\item run different models on \cindex{training set} with different \cindex{hyperparameter} to train parameters.
	\item run these models on \cindex{validation set} (or called \cindex{development set}, or \cindex{dev set}). Or using \cindex{cross validation}:
        \begin{enumerate}
            \item split the training data into $k$ chunks.
            \item run the model $k$ times. Each time select one chunk as validation set and the remaining as training set.
            \item take the average of validation error as the final error.
        \end{enumerate}
	\item select best model and its hyperparameter based on validation error.
	\item run the best model on $\set{\text{training set + validation set}}$ to train its parameters.
	\item calculate \cindex{generalization error} based on \cindex{test set}.
\end{enumerate}

If the training error is low on training set but the generalization error is high on test set, the model is overfitting.

\subsection{Prediction Output Format}

There are 4 different output formats:
\begin{enumerate}
    \item binary: one output that is in $\set{ 0, 1}$. such as \cindex{SVM}.
    \item \cindex{multiclass}: only one output, but could be of $n$ different values. There are two ways to train the model:
        \begin{enumerate}
            \item \cindex{one-versus-the-rest} (\cindex{OvR}): train $n$ classifiers. Each classifier run against the rest values. Preferred for most classifiers.
            \item \cindex{one-versus-one} (\cindex{OvO}): train $\binom{n}{2}$ classifiers. Each classifier run one value against another value. SVM is slow for large data set so usually prefer OvO.
        \end{enumerate}
    \item \cindex{multilabel}: there are multiple categories of binary output result.
    \item \cindex{multioutput}: there are multiple categories of multiclass output result.
\end{enumerate}

\subsection{Gradient Descent}

\subsubsection{Batch Gradient Descent}

\cindex{Gradient descent} is used to minimize cost function. All features need to have a similar scale in order to increase converge speed.

In GD learning, $\eta$ is the learning rate, $\theta$ is model's parameter vector, $f$ (such as MSE) is the cost function. Formula (\ref{gdlearning}) will try to minimize cost function recursively until $|\nabla_\theta f| < \varepsilon$ ($\varepsilon$ is called \cindex{tolerance})

\begin{equation}\label{gdlearning}
	\theta \gets \theta - \eta \nabla_\theta f
\end{equation}

Each iteration will use all input because the cost function is the sum of error over all inputs. So it is called \cindex{batch} and slow on very large data set. It takes $\displaystyle O\left(\frac{1}{\varepsilon}\right)$ iteration to reach the $\varepsilon$ tolerance.

\subsubsection{Stochastic Gradient Descent}

A random instance is choose at every step to compute the gradient. The SGD has the chance of finding global minimum. The learning rate needs to be reduced gradually in order to let the training converge, which is called \cindex{simulated annealing} process with \cindex{learning schedule}. Each round of learning schedule is called \cindex{epoch}.

If instances are chosen randomly, there is chance that some instances are never chosen. One solution is to randomly sort the test set and iterate all samples, and then sort it again.

\subsubsection{Mini-batch Gradient Decent}

Mini-batch gradient descent is in the middle between SGD and GD. Each iteration it will take a small random set of instance from training set called \cindex{mini-batch}.

\subsubsection{Early Stopping}

It means stop training when the validation error reaches minimum. The validation error curve for gradient descent will decrease and then goes up which means the model starts to overfit the training data. One solution is to continue run for a while and roll back to previous minimum, which means the previous minimum needs to be saved. For mini-batch and stochastic GD, the curve is not smooth so the decision would be observe for sometime and then decide whether to rollback.

\subsection{Learning Curve}

\cindex{Learning curve} is a plot of error against training set size for both training set and validation set.

At the beginning the error for training set is low and gradually increases, and for validation set it is high and gradually decreases. The error of training set could be $0$ even for the first few test samples. The gap between two sets could be decomposed into 3 components:
\begin{enumerate}
	\item bias: the model makes wrong assumption about the data. usually \cindex{underfit} the training data.
	\item variance: the model is sensitive to small variation to the training data. usually \cindex{overfit} the training data.
	\item irreducible error: data is noisy. need to clean data.
\end{enumerate}

Increasing the model complexity will typically increase the variance and reduce the bias, so it is a trade-off.

\subsection{Error Measure}

There are various distance measure available:
\begin{enumerate}
    \item \cindex{$l_1$ norm}: $\displaystyle \frac{1}{m} \sum_{i=1}^m \absolutevalue{h(X^{(i)}) - y^{(i)}}$. Also called \cindex{Manhattan norm}, or \cindex{mean absolute error}, or \cindex{MSE}.
    \item \cindex{$l_2$ norm}: $\displaystyle \sqrt{\frac{1}{m} \sum_{i=1}^m \left(h(X^{(i)}) - y^{(i)} \right)^2}$. Also called \cindex{root mean square error}, or \cindex{RMSE}.
    \item In general, the \cindex{$l_k$ norm} is $\norm{X - y}_k$.
\end{enumerate}

Higher norm focuses on large number. So $l_2$ is sensitive to outliers. But if outlier is rare, the $l_2$ is preferred.

\subsection{Regularization}

\subsubsection{Ridge Regression}

\cindex{Ridge Regression} also called \cindex{Tikhonov regularization} or $l_2$ . It add $l_2$ norm to the cost function during training ($\theta_0$ is not added):

\begin{equation}
	\alpha \sum_{i=1}^n \theta_i^2
\end{equation}

The extra cost should only be added during training and removed in evaluation. It is common to use different cost function in training and testing. The training cost function needs to be optimization friendly.

Ridge regression is very sensitive to data scale, so do the data scaling before the training.

\subsubsection{Lasso Regression}
\cindex{Lasso Regression} (Least absolute shrinkage and selection operator regression) will add $l_1$ norm to the cost function ($\theta_0$ is not added):
\begin{equation}
	\alpha \sum_{i=1}^n \absolutevalue{\theta_i}
\end{equation}

Unimportant features will have $0$ weight, so it will output a \cindex{sparse model}. Reduce the learning rate gradually to avoid bouncing around optimal.

\subsubsection{Elastic Net}
\cindex{Elastic Net} is the middle between \cindex{ridge regression} and \cindex{Lasso regression}. It will add the following to cost function:

\begin{equation}
	\frac{1-r}{2} \alpha \sum_{i=1}^n \theta_i^2 + r \alpha \sum_{i=1}^n \absolutevalue{\theta_i}
\end{equation}

Choose Elastic Net or Lasso Regression over Ridge Regression.

\subsection{Tune Model}
two different model tuning methods:
\begin{enumerate}
	\item grid search: generate a combination of parameters and search
	\item randomized search
\end{enumerate}

\subsection{Performance Measure}

\begin{figure}[H]
\centering
\begin{tikzpicture}
	\node at (1,3) {predicted};
	\node at (0.5,2.5) {$0$};
	\node at (1.5,2.5) {$1$};
	\node at (0.5,2) {TN};
	\node at (1.5,2) {FP};
	\node at (0.5,1.5) {FN};
	\node at (1.5,1.5) {TP};
	\node at (-0.25,2) {$0$};
	\node at (-0.25,1.5) {$1$};
	\node at (-1,1.75) {actual};
\end{tikzpicture}

\caption{confusion matrix}
\end{figure}

\begin{figure}[H]
\centering
\begin{tikzpicture}
	\node [circle,fill] (s) [label=$0$] at (0,0) {};
	\node [circle,draw] (s) [label=$0.1$] at (1,0) {};
	\node [circle,draw] (s) [label=$0.2$] at (2,0) {};
	\node [circle,fill] (s) [label=$0.3$] at (3,0) {};
	\node [circle,draw] (s) [label=$0.4$] at (4,0) {};
	\node [circle,fill] (s) [label=$0.5$] at (5,0) {};
	\node [circle,draw] (s) [label=$0.6$] at (6,0) {};
	\node [circle,fill] (s) [label=$0.7$] at (7,0) {};
	\node [circle,fill] (s) [label=$0.8$] at (8,0) {};
	\node [circle,draw] (s) [label=$0.9$] at (9,0) {};
	\node [circle,fill] (s) [label=$1.0$] at (10,0) {};

	% draw the threshold
	\draw [dashed] (5.5,3) -- (5.5,-4.5);

	% draw precision
	\node [circle,draw] (s) at (6,2) {};
	\node [circle,fill] (s) at (7,2) {};
	\node [circle,fill] (s) at (8,2) {};
	\node [circle,draw] (s) at (9,2) {};
	\node [circle,fill] (s) at (10,2) {};

	\draw (5.75,2.25) -- (10.25,2.25);

	\node [circle,fill] (s) at (7,2.5) {};
	\node [circle,fill] (s) at (8,2.5) {};
	\node [circle,fill] (s) at (10,2.5) {};

	\node at (11.25,2.25) {= precision};

	% draw recall
	\node [circle,fill] (s) at (0,-2) {};
	\node [circle,fill] (s) at (3,-2) {};
	\node [circle,fill] (s) at (5,-2) {};
	\node [circle,fill] (s) at (7,-2) {};
	\node [circle,fill] (s) at (8,-2) {};
	\node [circle,fill] (s) at (10,-2) {};

	\draw (-0.25,-1.75) -- (10.25,-1.75);

	\node [circle,fill] (s) at (7,-1.5) {};
	\node [circle,fill] (s) at (8,-1.5) {};
	\node [circle,fill] (s) at (10,-1.5) {};

	\node at (11.25,-1.75) {= recall};

	% ROC

	\node [circle,draw] (s) at (1,-4) {};
	\node [circle,draw] (s) at (2,-4) {};
	\node [circle,draw] (s) at (4,-4) {};
	\node [circle,draw] (s) at (6,-4) {};
	\node [circle,draw] (s) at (9,-4) {};

	\draw (0,-3.75) -- (10.25,-3.75);

	\node [circle,draw] (s) at (6,-3.5) {};
	\node [circle,draw] (s) at (9,-3.5) {};

	\node at (12,-3.75) {=$1 - \text{specificity}$};
\end{tikzpicture}
\caption{PR and ROC curve}
\end{figure}

\begin{equation}
    \begin{aligned}
	\text{precision} &= \frac{TP}{TP + FP} \\
	\text{recall} &= \frac{TP}{TP + FN} \\
	\text{false positive rate} &= \frac{FP}{FP + TN} \\
    F_1 &= \frac{2}{\displaystyle \frac{1}{\text{precision}} + \frac{1}{\text{recall}}}
    \end{aligned}
\end{equation}

\cindex{confusion matrix} (\cindex{PR curve}) is a plot between \cindex{precision} and \cindex{recall} (precision is the $y$ axis). There is a trade-off between precision and recall. Given a fixed model and prediction, when the threshold increases, the precision might increase (it could decrease) and the recall will always decrease. Sometimes you may care more about precision, such as classifying safe video for kids. Sometimes you care more about recall, such as the surveillance image classification. For two models, select the one that could embed the other PR curve.

\cindex{receiver operating characteristic} (ROC) is a plot between \cindex{recall} and $1 - \text{specificity}$ (recall is the $y$ axis). The \cindex{area under the curve} (\cindex{AUC}) measures how good a model is. A random classifier will have $\text{AUC} = 0.5$. Compared with ROC curve, PR curve is preferred when:
\begin{enumerate}
	\item positive rate is rare.
	\item care more about false positive than false negative.
\end{enumerate}


\documentclass{article}
\usepackage[margin=1in]{geometry}
\setlength{\parindent}{0in}

\usepackage[utf8]{inputenc}

\usepackage{latexsym,amsfonts,amssymb,amsthm,amsmath}

\DeclareMathOperator{\Tr}{Tr}
\DeclareMathOperator{\Id}{Id}

\usepackage{braket}
\usepackage{mathrsfs}

\newcommand{\norm}[1]{\left\lVert#1\right\rVert}

\title{Quantum Computing - Assignment 3}
\author{Kishlaya Jaiswal}

\begin{document}

\maketitle

\subsection*{Exercise 1}
Let $\ket{\psi}_{AB} \in H_A \otimes H_B$ be a pure state. Assume wlog $dim(H_A) \geq dim(H_B) = m$. By Schmidt decomposition there exist orthonormal bases $\{u_i \mid i \leq m\} \subset H_A$ and $\{v_i \mid i \leq m\} \subset H_B$ such that $$\ket{\psi}_{AB} = \sum_i \alpha_i \ket{u_i} \otimes \ket{v_i}$$

\textbf{Claim} $\ket{\psi}_{AB}$ is entangled iff more than one Schmidt coefficients $\{\alpha_i \mid i \leq m\}$ are non-zero
\begin{proof}
Firstly, it is clear that atleast one Schmidt coefficient is non-zero because otherwise $\ket{\psi}_{AB} = 0$ is not a valid pure state. \\

So suppose exactly one Schmidt coefficient $\alpha_i$ is non-zero, then
$$\ket{\psi}_{AB} = \alpha_i \ket{u_i}_A \otimes \ket{v_i}_B$$ is a product state.

Conversely, suppose more than one Schmidt coefficient is non-zero, say $\alpha_i$ and $\alpha_j$ where $i \neq j$. We need to show that $\ket{\psi}_{AB}$ is entangled. For the sake of contradiction assume that $\ket{\psi}_{AB} = \ket{\phi}_A \otimes \ket{\varphi}_B$ is a product state. Then we have
\begin{align*}
    \ket{\phi}_A \otimes \ket{\varphi}_B &= \sum_i \alpha_i \ket{u_i} \otimes \ket{v_i} \\
    \implies \big(\bra{u_i} \otimes \bra{v_i}\big) \big(\ket{\phi}_A \otimes \ket{\varphi}_B\big) &= \big(\bra{u_i} \otimes \bra{v_i}\big) \left(\sum_i \alpha_i \ket{u_i} \otimes \ket{v_i}\right) \\
    \implies \braket{u_i | \phi} \braket{v_i | \varphi} &= \alpha_i
\end{align*}

Similarly, $\braket{u_j | \phi} \braket{v_j | \varphi} = \alpha_j$. Furthermore,
\begin{align*}
    \ket{\phi}_A \otimes \ket{\varphi}_B &= \sum_i \alpha_i \ket{u_i} \otimes \ket{v_i} \\
    \implies \big(\bra{u_i} \otimes \bra{v_j}\big) \big(\ket{\phi}_A \otimes \ket{\varphi}_B \big) &= \big(\bra{u_i} \otimes \bra{v_j} \big) \left(\sum_i \alpha_i \ket{u_i} \otimes \ket{v_i}\right) \\
    \implies \braket{u_i | \phi} \braket{v_j | \varphi} &= 0
\end{align*}

Thus, we get $\alpha_i \alpha_j = \braket{u_i | \phi} \braket{v_i | \varphi} \braket{u_j | \phi} \braket{v_j | \varphi} = 0$. That is both $\alpha_i$ and $\alpha_j$ can't be non-zero, leading us to a contradiction.

\end{proof}

\subsection*{Exercise 2}
\begin{proof}
We will prove that Trace is a commutative operator, that is $\Tr(AB)= \Tr(BA)$
\begin{align*}
    \Tr(AB) &= \sum_i \bra i AB \ket i \\
    &= \sum_i \bra i A \left(\sum_j \ket j \bra j \right)B \ket i \\
    &= \sum_i \sum_j \bra i A \ket j \bra j B \ket i \\
    &= \sum_j \sum_i \bra j B \ket i \bra i A \ket j \\
    &= \sum_j \bra j B \left( \sum_i \ket i \bra i \right) A \ket j \\
    &= \sum_j \bra j BA \ket j \\
    &= \Tr(BA)
\end{align*}

Now $\Tr(A(BC)) = \Tr((BC)A)$ and $\Tr(B(CA)) = \Tr((CA)B)$. Since matrix multiplication is associative we get $\Tr(ABC) = \Tr(BCA) = \Tr(CAB)$ \\

\begin{align*}
    \Tr_B (\ket{x_1} \bra{x_2}_A \otimes \ket{y_1} \bra{y_2}_B) &= \sum_i I_A \otimes \bra{i}_B \bigg( \ket{x_1} \bra{x_2}_A \otimes \ket{y_1} \bra{y_2}_B \bigg) I_A \otimes \ket{i}_B
\end{align*}

Using the rule $(A \otimes B).(C \otimes D) = AB \otimes CD$, we get
\begin{align*}
    \Tr_B (\ket{x_1} \bra{x_2}_A \otimes \ket{y_1} \bra{y_2}_B) &= \sum_i \ket{x_1} \bra{x_2}_A \otimes \bra{i}_B \ket{y_1} \bra{y_2}_B \ket{i}_B \\
    &= \ket{x_1} \bra{x_2}_A \otimes \sum_i \bra{i}_B \ket{y_1} \bra{y_2}_B \ket{i}_B \\
    &= \ket{x_1} \bra{x_2}_A \otimes \Tr (\ket{y_1} \bra{y_2}_B) \\
    &= \ket{x_1} \bra{x_2}_A \otimes \Tr (\braket{y_2 | y_1}) \text{  (Trace is commutative)} \\
    &= \ket{x_1} \bra{x_2}_A \braket{y_2 | y_1}
\end{align*}
\end{proof}

\subsection*{Exercise 3}
Let $N : \mathscr{L}(H) \to \mathscr{L}(H)$, $N(\rho) = (1-p)\rho + p Z \rho Z$. We will show that $N$ is a quantum channel, that is $N$ is a linear, trace-preserving and completely positive operator. \\

\textbf{Claim} $N$ is linear
\begin{proof}
\begin{align*}
    N(\rho_1 + \lambda \rho_2) &= (1-p)(\rho_1 + \lambda \rho_2) + p Z (\rho_1 + \lambda \rho_2) Z \\
    &= (1-p)(\rho_1) + (1-p)(\lambda \rho_2) + p Z (\rho_1) Z + pZ(\lambda \rho_2) Z \\
    &= ((1-p)\rho_1 + p Z \rho_1 Z) + \lambda ((1-p)\rho_2 + p Z \rho_2 Z) \\
    &= N(\rho_1) + \lambda N(\rho_2)
\end{align*}
\end{proof}

\textbf{Claim} $N$ is trace preserving
\begin{proof}
\begin{align*}
    \Tr (N(\rho)) &= \Tr ((1-p)\rho + p Z \rho Z) \\
    &= (1-p) \Tr (\rho) + p \Tr (Z \rho Z) \\
    &= (1-p) \Tr (\rho) + p \Tr (Z^2 \rho) \\
    &= (1-p) \Tr (\rho) + p \Tr (\rho) \\
    &= \Tr (\rho)
\end{align*}
\end{proof}

\textbf{Claim} $\forall E$, $\Id_E \otimes N : \mathscr{L}(H_E \otimes H) \to \mathscr{L}(H_E \otimes H)$ is a positive operator
\begin{proof}

Let $\theta \in \mathscr{L}(H_E \otimes H)$ be any PSD matrix. Want to show that $(\Id_E \otimes N)(\theta)$ is also PSD matrix.
Since $\theta$ is a PSD matrix, we have $\forall x$ $\braket{x|\theta|x} \geq 0$

Identify $\mathscr{L}(H_E \otimes H) \cong \mathscr{L}(H_E) \otimes \mathscr{L}(H)$ and fix any basis $\{\sigma_i \mid i \leq m\}$ for $\mathscr{L}(H_E)$ and $\{\rho_j \mid j \leq n\}$ for $\mathscr{L}(H)$, then $\theta = \sum_{i,j} c_{ij} (\sigma_i \otimes \rho_j)$ for some $c_{ij} \in \mathbb{C}$.

$$(\Id_E \otimes N)(\theta) = \sum_{i,j} c_{ij} (\Id_E \otimes N)(\sigma_i \otimes \rho_j) = \sum_{i,j} c_{ij} \sigma_i \otimes N(\rho_j)$$

\begin{align*}
    \braket{x | \Id_E \otimes N(\theta) | x} &= \sum_{i,j} c_{ij} \braket{x | \sigma_i \otimes N(\rho_j) | x} \\
    &= \sum_{i,j} c_{ij} (1-p) \braket{x | \sigma_i \otimes \rho_j | x} + \sum_{i,j} c_{ij} p \braket{x | \sigma_i \otimes Z\rho_j Z | x} \\
\end{align*}

But we can re-write $$\sigma \otimes Z \rho Z = (I_E \otimes Z)(\sigma \otimes \rho)(I_E \otimes Z)$$ $$\implies \braket{x|\sigma \otimes Z \rho Z|x} = \braket{y|\sigma \otimes \rho|y}$$
where $\ket{y} = (I_E \otimes Z) \ket{x}$ (because $I_E \otimes Z$ is Hermitian). Hence we get
$$\braket{x | \Id_E \otimes N(\theta) | x} = (1-p) \sum_{i,j} c_{ij} \braket{x | \sigma_i \otimes \rho_j | x} + p \sum_{i,j} c_{ij} \braket{y | \sigma_i \otimes \rho_j | y} = (1-p)\braket{x|\theta|x} + p\braket{y|\theta|y} \geq 0$$

that is $N$ is a completely positive operator.
\end{proof}

% First we show that $N$ is a positive map. Let $v$ be any vector and $\rho$ be a PSD matrix, i.e. $\braket{v|\rho|v} \geq 0$, then
% \begin{align*}
%     \braket{v|N(\rho)|v} &= (1-p)\braket{v|\rho|v} + p \braket{v|Z \rho Z|v} \\
%     &= (1-p)\braket{v|\rho|v} + p \braket{w| \rho |w} \text{  where } w = Z\ket{v}\\
%     &\geq (1-p)0 + p0 = 0
% \end{align*}

% Now consider any elementary PSD matrix $(\sigma \otimes \rho)$ and vector $\ket{u} \otimes \ket{v}$, then
% \begin{itemize}
%     \item $(\bra{u} \otimes \bra{v})(\sigma \otimes \rho)(\ket{u} \otimes \ket{v}) = \bra{u} \sigma \ket{u}  \bra{v} \rho \ket{v} \geq 0$
%     \item and similarly, $\bra{u} \sigma \ket{u}  \bra{w} \rho \ket{w} \geq 0$ where $w = Z\ket{v}$
% \end{itemize}

% \begin{align*}
%     \braket{u \otimes v|\Id_E \otimes N(\sigma \otimes \rho)|u \otimes v} &= \braket{u \otimes v|\Id_E(\sigma) \otimes N(\rho)|u \otimes v} \\
%     &= \braket{u \otimes v|\sigma \otimes N(\rho)|u \otimes v} \\
%     &= \braket{u | \sigma | u} \braket{v | N(\rho) | v} \\
%     &= (1-p) \braket{u | \sigma | u} \braket{v|\rho|v} + p \braket{u | \sigma | u} \braket{w| \rho |w} \text{  where } w = Z\ket{v}\\
%     &\geq (1-p)0 + p0 = 0
% \end{align*}

% For any general PSD matrix $\theta \in H_E \otimes H$ and any vector $x$, using Schmidt decomposition, we can write
% \begin{itemize}
%     \item $\theta = \sum_i \alpha_i \sigma_i \otimes \rho_i$ where $\alpha_i \geq 0$
%     \item $\ket{x} = \sum_i \beta_i \ket{u_i} \otimes \ket{v_i}$ where $\beta_i \geq 0$
% \end{itemize}

% \begin{align*}
%     \braket{x | \Id_E \otimes N (\theta) | x} &= \sum_i \alpha_i \braket{x | \Id_E \otimes N(\sigma_i \otimes \rho_i) | x} \\
%     &= \sum_i \alpha_i \sum_j \sum_k \beta_j \beta_k \braket{u_j \otimes v_j | \Id_E \otimes N(\sigma_i \otimes \rho_i) | u_k \otimes v_k} \\
%     &= \sum_{i,j,k} \alpha_i \beta_j \beta_k \bigg(\braket{u_j \otimes v_j | \Id_E \otimes N(\sigma_i \otimes \rho_i) | u_k \otimes v_k}\bigg) \\
%     &\geq 0
% \end{align*}

% Hence $\Id_E \otimes N$ is a positive map for every $E$. That is $N$ is a completely positive map.

We begin by noting the following relations:
$$ZXZ = -X, ZYZ = -Y$$
because
\begin{itemize}
    \item $ZXZ\ket0 = ZX\ket0 = Z\ket1 = - \ket1 = -X\ket0$ and $ZXZ\ket1 = -ZX\ket1 = -Z\ket0 = - \ket0 = -X\ket1$
    \item $ZYZ\ket0 = ZY\ket0 = iZ\ket1 = - i\ket1 = -Y\ket0$ and $ZYZ\ket1 = -ZY\ket1 = iZ\ket0 = i\ket0 = -Y\ket1$ \\
\end{itemize}

Thus, we get $N(X) = (1-p)X + pZXZ = (1-2p)X$ and $N(Y) = (1-p)Y + pZYZ = (1-2p)Y$ and $N(Z) = (1-p)Z + pZZZ = Z$. Hence
\begin{align*}
    N\left(\frac12 (I + r_x X + r_y Y + r_z Z)\right) &= \frac12 (N(I) + r_x N(X) + r_y N(Y) + r_z N(Z)) \\
    &= \frac12 (I + r_x (1-2p)X + r_y (1-2p) Y + r_z Z)
\end{align*}

\subsection*{Exercise 4}
\textbf{Claim} Let $A$ be any linear operator over a $\mathbb{C}-$vector space, then we can write $A = B+iC$ for some $B,C$ Hermitian operators
\begin{proof}
Consider $B = (A+A^\dagger)/2$ and $C = (A-A^\dagger)/2i$ then $A = B+iC$ is clear.

Moreover, $B^\dagger = (A^\dagger + A)/2 = B$ and $C^\dagger = -(A^\dagger-A)/2i = C$, as desired.
\end{proof}

\textbf{Claim} Let $A$ be a Hermitian operator then $\forall v$, $\braket{v|A|v} \in \mathbb{R}$
\begin{proof}
Using spectral theorem, we know that there exists an orthonormal eigenbasis $\{v_i \mid i \leq n\}$ such that $Av_i = \lambda_i v_i$ where $\lambda_i \in \mathbb{R}$, $\forall i \leq n$

Now for any $v$, write $v = \sum_i c_i v_i$ and then
$$\braket{v|A|v} = \sum_i |c_i|^2 \lambda_i \in \mathbb{R}$$
\end{proof}

\textbf{Claim} Let $A$ be any linear operator over a $\mathbb{C}-$vector space such that $\forall v$, $\braket{v|A|v} \in \mathbb{R}$. Then $A$ is a  Hermitian operator.
\begin{proof}
We write $A = B+iC$ with $B,C$ Hermitian.

Since $B,C$ are Hermitian, $\braket{v|B|v} \in \mathbb{R}$ and $\braket{v|C|v} \in \mathbb{R}$. Therefore $\forall v$

$$\braket{v|A|v} = \braket{v|B|v} + i\braket{v|C|v} \in \mathbb{R} \implies \braket{v|C|v} = 0$$

Hence $C = 0 \implies A = B$ which is Hermitian.
\end{proof}

Finally, let $A$ be a positive operator, which means $\forall v$, $\braket{v|A|v} \geq 0$. That is $\forall v$, $\braket{v|A|v} \in \mathbb{R}$. Using above claim, we get that $A$ is Hermitian.
\end{document}