%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------
\documentclass[paper=a4, fontsize=11pt]{scrartcl} % A4 paper and 11pt font size

\usepackage[T1]{fontenc} % Use 8-bit encoding that has 256 glyphs
\usepackage{fourier} % Use the Adobe Utopia font for the document - comment this line to return to the LaTeX default
\usepackage[english]{babel} % English language/hyphenation
\usepackage{amsmath,amsfonts,amsthm,amssymb} % Math packages
\usepackage{mathrsfs}
\usepackage{algorithm, algorithmic}
\renewcommand{\algorithmicrequire}{\textbf{Input:}} %Use Input in the format of Algorithm
\renewcommand{\algorithmicensure}{\textbf{Output:}} %UseOutput in the format of Algorithm
\usepackage{listings}
\lstset{language=Matlab}
\usepackage{enumerate}
\usepackage{graphicx}

\usepackage{lipsum} % Used for inserting dummy 'Lorem ipsum' text into the template
\usepackage{sectsty} % Allows customizing section commands
\allsectionsfont{\centering \normalfont\scshape} % Make all sections centered, the default font and small caps
\usepackage{fancyhdr} % Custom headers and footers
\pagestyle{fancyplain} % Makes all pages in the document conform to the custom headers and footers
\fancyhead{} % No page header - if you want one, create it in the same way as the footers below
\fancyfoot[L]{} % Empty left footer
\fancyfoot[C]{} % Empty center footer
\fancyfoot[R]{\thepage} % Page numbering for right footer
\renewcommand{\headrulewidth}{0pt} % Remove header underlines
\renewcommand{\footrulewidth}{0pt} % Remove footer underlines
\setlength{\headheight}{13.6pt} % Customize the height of the header

\numberwithin{equation}{section} % Number equations within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)
\numberwithin{figure}{section} % Number figures within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)
\numberwithin{table}{section} % Number tables within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)

\setlength\parindent{0pt} % Removes all indentation from paragraphs - comment this line for an assignment with lots of text

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------
\newcommand{\horrule}[1]{\rule{\linewidth}{#1}} % Create horizontal rule command with 1 argument of height

\title{
\normalfont \normalsize
\textsc{Shanghai Jiao Tong University, UM-SJTU JOINT INSTITUTE} \\ [25pt] % Your university, school and/or department name(s)
\horrule{0.5pt} \\[0.4cm] % Thin top horizontal rule
\huge Introduction to Numerical Analysis \\ Project2 \\ % The assignment title
\horrule{2pt} \\[0.5cm] % Thick bottom horizontal rule
}

\author{Yu Cang \\ 018370210001} % Your name

\date{\normalsize \today} % Today's date or a custom date

\begin{document}

\maketitle % Print the title

\section{Task 1}
	Note the symmetry of $x_1$, for any solution $(a, b)$, $(-a, b)$ is also valid.\\
	If no special claim, the norm function for a vector is taken as the 2-norm.
	\begin{enumerate}[(a)]
		\item
			For fixed-point iteration, the iteration function $G$ is taken as
			\begin{equation}
				G \triangleq
				\begin{bmatrix}
					\sqrt{x_2}\\
					\sqrt{1-x_1^2}
				\end{bmatrix}
				=
				\begin{bmatrix}
					x_1\\
					x_2\\
				\end{bmatrix}
				=x
			\end{equation}
			The iteration converges to $(0.7862, 0.6180)$ with intial guess $x_{init} = [\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}]^T$ and tolerance set to $10^{-10}$.
		\item
			The newton iteration is implemented as mentioned in the project sheet, each time the linear system
			\begin{equation}
				J_F(x_k) w_k = y_k
			\end{equation}
			is solved with the built-in function 'linsolve' inside matlab.\\
			The iteration converges to $(0.7862, 0.6180)$ with intial guess $x_{init} = [\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}]^T$ and tolerance set to $10^{-10}$.
		\item
			The input of the Broyden's method is a little bit different from that in newton or fixed-point. The initial Jacobi matrix should be provided, denoted as $A_0$. In my implementation, it is given as
			\begin{equation}
				A_0 = J_F(x_0)
			\end{equation}
			An extra calculation on $x_1$ should be done before the iteration loop as the iteration process requires both $x_0$ and $x_1$. It is approximated through newton's method in my implementation as
			\begin{equation}
				x_1 \approxeq x_0 - A_0^{-1}F(x_0)
			\end{equation}
			After all the preparation work, the iteration loop can be carried out as mentioned in the project sheet, and the key part is the application of Sherman-Morrison's formula to calculated the inverse of $A_k$ iteratively as
			\begin{equation}
				A_k^{-1} = A_{k-1}^{-1} + \frac{(s_k^T A_{k-1}^{-1} y_k)s_k^T A_{k-1}^{-1}}{s_k^T A_{k-1}^{-1} y_k}
			\end{equation}
			where $s_k = x_k - x_{k-1}$ and $y_k = F(x_k) - F(x_{k-1})$. And the iteration of $x_k$ is given as
			\begin{equation}
				x_{k+1} = x_k - A_k^{-1}F(x_k)
			\end{equation}
			The iteration converges to $(0.7862, 0.6180)$ with intial guess $x_{init} = [\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}]^T$ and tolerance set to $10^{-10}$.
		\item
			With numerical tests, newton's method out-performs the other two in terms of time cost.

	\end{enumerate}

\section{Task 2}
	With trival inequality analysis, the exact solution is given as $x=[0.5, 0, -\frac{\pi}{6}]$, which can be used for verification.
	\subsection{Solve directly}
		Applying the fixed-point iteration, newton's method and the broyden's method to this non-linear system respectively, with initial guess given as $x_0 = [0.1 0.1 0.1]^T$, and tolerance set to $10^{-10}$, the iteration process all converge to the exact solution as
		\begin{equation}
			\begin{aligned}
				x_1 & = 0.5\\
				x_2 & = 3.5440\times 10^{-11}\\
				x_3 & = -0.5236\\
			\end{aligned}
		\end{equation}
		The iteration function for fixed-point iteration is given as
		\begin{equation}
			G =
			\begin{bmatrix}
				\frac{0.5+cos(x_2 x_3)}{3}\\
				\frac{\sqrt(0.25-x_1^2)}{25}\\
				\frac{\frac{3-10 \pi}{3}-e^{-x_1 x_2}}{20}
			\end{bmatrix}
			= x
		\end{equation}

	\subsection{Downhill simplex algorithm}
		In general, the initial value for such a non-linear system is hard to know, even the range of each component is hard to estimate. In this case, the downhill simplex method can be applied to produce a initial guess for further calculation.

		The downhill simplex algorithm was implemented in my code. The kernel idea of this algorithm is to minimize the performance of the worst vertex gradually.

		Inside the iteration loop, serval operations guide the simplex to move towards the minimum:
		\begin{enumerate}
			\item reflection
			\item expansion
			\item contraction
			\item shrink
		\end{enumerate}

		 These operations are designed for different ranges respectively. The whole range of the function is divided by 3 points:
		 \begin{enumerate}
		 	\item  the best
		 	\item  the last-but-one
		 	\item  the worst
		 \end{enumerate}
 		 Details of the implement will not be discussed here and can be referenced clearly from the code.

	\subsection{Solve with initialization}
		Before using the downhill simplex to find a fine initial guess, a targeting function $f: \mathbb{R^n} \rightarrow \mathbb{R}$ is taken as
		\begin{equation}
			f(X) = ||F(x)||_2
		\end{equation}
		inside the matlab code as the evaluating function for the algorithm.\\
		Numerical experiments were done to identify the difference in terms of time cost.

		\begin{center}
			\begin{tabular}{ccc}
				\hline
				\quad       & Without initialization & With initialization\\
				\hline
				fixed-point & $4.033 ms$             & $3.970 ms$\\
				neweon      & $4.661 ms$             & $0.679 ms$\\
				broyden     & $4.447 ms$             & $0.799 ms$\\
				\hline
			\end{tabular}
		\end{center}

		It can be observed from the table that the downhill simplex initialization process does help a lot to reduce time cost for further newton or broyden calculation as it provide a better initial guess. \\
		However, the time cost is almost equivalent for the fixed-point iteration, which indicates that it does not benefit from the downhill simplex initialization.

\section{Task 3}
	\begin{enumerate}[(a)]
		\item
			Denote $y_1(t) \triangleq n(t)$, $y_2(t) \triangleq x(t)$ and
			\begin{equation}
				\Phi \triangleq
				\begin{bmatrix}
					\dot{y_1}\\
					 \dot{y_2}
				\end{bmatrix}
				=
				\begin{bmatrix}
					\gamma - \alpha y_2^2 y_1\\
					\alpha y_2^2 y_1 - \frac{y_1}{1+y_1}
				\end{bmatrix}
			\end{equation}
			Then, the synthetic data for $n$ and $x$ can be calculated iteratively. To make it more accurate, the Runge-Kutta 4-stage method was adopted. The matlab built-in function 'ode45' was used in my implementation. \\
			Plot of $n(t)$ and $x(t)$ were generated as follows
			\begin{figure}[htbp]\centering
				\begin{minipage}{7.5cm}
					\centering
					\includegraphics[height=7cm,width=7cm]{../pic/n_and_ns.eps}
				\end{minipage}%
				\begin{minipage}{7.5cm}
					\centering
					\includegraphics[height=7cm,width=7cm]{../pic/x_and_xs.eps}
				\end{minipage}
			\end{figure}

			Plot of $n(t)$ and $x(t)$ with noise were also generated here to make a clear comparsion.
		\item
			At each point (except the two boundary points), the 1st-order derivative of $n(t)$ and $x(t)$ can be approximated using central difference as
			\begin{equation}
				\dot{n_k} = \frac{n_{k+1}-n_{k-1}}{2 \Delta t} \label{si_eq1}
			\end{equation}
			\begin{equation}
				\dot{x_k} = \frac{x_{k+1}-x_{k-1}}{2 \Delta t} \label{si_eq2}
			\end{equation}
			Thus a pair of $\alpha$ and $\gamma$ can be solved at each point as
			\begin{equation}
				\alpha_k = \frac{\dot{x_k} + \frac{x_k}{1+x_k}}{n_k^2x_k}
			\end{equation}
			\begin{equation}
				\gamma_k = \dot{x_k} + \dot{n_k} + \frac{x_k}{1+x_k}
			\end{equation}
			for $k=2,3,...,N-1$.\\
			Finally, the estimated $\alpha$ and $\gamma$ can be calculated as
			\begin{equation}
				\hat{\alpha} = \frac{\sum_{k=2}^{N-1}\alpha_k}{N-2}
			\end{equation}
			\begin{equation}
				\hat{\gamma} = \frac{\sum_{k=2}^{N-1}\gamma_k}{N-2}
			\end{equation}
			Numerical experiment shows that $\hat{\alpha} = 0.099824$ and $\hat{\gamma} = 0.199699$, which implies that $\hat{\alpha}$ and $\hat{\gamma}$ agree well with $\alpha$ and $\gamma$.
		\item
			The noise is generated with the built-in function 'normrnd' of matlab in my code, plots are given in part(a).

		\item
			Considering the mathmatical expect of $Eq(\ref{si_eq1})$ and $Eq(\ref{si_eq2})$ as
			\begin{equation}
				\begin{aligned}
					E(\dot{n}) & = \gamma - \alpha E(n^2 x)\\
					E(\dot{x}) & = \alpha E(n^2 x) - E(\frac{x}{1+x})
				\end{aligned}
			\end{equation}
			Then $\alpha$ and $\gamma$ can be resolved as
			\begin{equation}
				\begin{aligned}
					\hat{\alpha} & = \frac{E(\dot{x}) + E(\frac{x}{1+x})}{E(n^2 x)}\\
					\hat{\gamma} & = E(\dot{x}) + E(\dot{n}) + E(\frac{x}{1+x})
				\end{aligned}
			\end{equation}
			But the signal we have is not clean, what we can obtain from the data are quantities with noise like $E(\dot{n^*})$, $E(\dot{x^*})$, $E({n^*}^2 x^*)$, and $E(\frac{x^*}{1+x^*})$. These clean quantities like $E(\dot{n})$, $E(\dot{x})$, $E(n^2 x)$, and $E(\frac{x}{1+x})$ should be approximated from these polluted quantities so that $\hat{\alpha}$ and $\hat{\gamma}$ can be resolved.\\
			As $\epsilon \sim N(0, 0.01)$ and $\nu \sim N(0, 0.001)$, then $E(\epsilon) = 0$, $E(\nu) = 0$ and $E(\epsilon^2) = 0.01$, $E(\nu^2)=0.001$.
			Since $n^* = n + \epsilon$ and $x^* = x + \nu$, then
			\begin{equation}
				\begin{aligned}
					E(n^*) & = E(n) + E(\epsilon) = E(n)\\
					E(x^*) & = E(x) + E(\nu) = E(x)
				\end{aligned}
			\end{equation}
			As $\epsilon$ and $\nu$ are independent, thus
			\begin{equation}
				\begin{aligned}
					E({n^*}^2 x^*) & = E(n^2 x + 2nx\epsilon + x\epsilon^2 + n^2\nu + 2n\epsilon\nu + \epsilon^2 \nu)\\
					               & = E(n^2 x) + E(x)E(\epsilon^2)
				\end{aligned}
			\end{equation}
			and
			\begin{equation}
				E(\frac{x^*}{1+x^*}) = E(\frac{x}{1+x^*}) \approxeq E(\frac{x}{1+x})
			\end{equation}
			For derivatives, similiar relation can be concluded as
			\begin{equation}
				\begin{aligned}
					E(\dot{n_k^*}) & = \frac{E(n_{k+1}^*) - E(n_{k-1}^*)}{2 \Delta t} = \frac{E(n_{k+1}) - E(n_{k-1})}{2 \Delta t} = E(\dot{n_k})\\
					E(\dot{x_k^*}) & = \frac{E(x_{k+1}^*) - E(x_{k-1}^*)}{2 \Delta t} = \frac{E(x_{k+1}) - E(x_{k-1})}{2 \Delta t} = E(\dot{x_k})
				\end{aligned}
			\end{equation}
			Hence these clean quantities can be approximated as
			\begin{equation}
				\begin{aligned}
					E(n) & = E(n^*)\\
					E(x) & = E(x^*)\\
					E(\dot{n}) & = E(\dot{n^*}) = \frac{\sum_{k=2}^{N-1}\dot{n_k^*}}{N-2}\\
					E(\dot{x}) & = E(\dot{x^*}) = \frac{\sum_{k=2}^{N-1}\dot{x_k^*}}{N-2}\\
					E(n^2 x) & = E({n^*}^2 x^*) - E(x)E(\epsilon^2)\\
					E(\frac{x}{1+x}) & = E(\frac{x^*}{1+x^*})
				\end{aligned}
			\end{equation}
			where the properties of the noise is utilized.\\
			In this way, $\hat{\alpha}$ and $\hat{\gamma}$ were resolved in numerical experiment as
			\begin{equation}
				\begin{aligned}
					\hat{\alpha} & = 0.102744\\
					\hat{\gamma} & = 0.201082
				\end{aligned}
			\end{equation}
			The result has been validated for about 20 times, although the noise is different at each time, as well as $\hat{\alpha}$ and $\hat{\gamma}$, but each time the result was quite stable.
		\item
			If the properties of noise are not available, some approximations or assumptions can not be made finely.\\
			At first, a distinct attempt is to treat $n^*$ and $x^*$ like what we have done in part(b).
			\begin{equation}
				\hat{\alpha} = \frac{\sum_{k=2}^{N-1}\alpha_k^*}{N-2}
			\end{equation}
			\begin{equation}
				\hat{\gamma} = \frac{\sum_{k=2}^{N-1}\gamma_k^*}{N-2}
			\end{equation}
			In this way, the only difference is the input data sequence, as the input in part(b) is clean but here is polluted. \\
			Numerical experiments were done for 200 times to illustrate the initial guess(Fig\ref{coarse_ag}) and the variance(Fig\ref{err_ag}) of $\hat{\alpha}$ and $\hat{\gamma}$.
			\begin{figure}\centering
				\includegraphics[height=8cm]{../pic/coarse_estimation.eps}
				\caption{Coarse estimation}
				\label{coarse_ag}
			\end{figure}
			\begin{figure}\centering
				\includegraphics[height=8cm]{../pic/coarse_estimation_err.eps}
				\caption{Error of estimation}
				\label{err_ag}
			\end{figure}

			It can be seen from Fig\ref{err_ag} that the error of $\hat{\gamma}$ is about 3\% relative to $\bar{\gamma}$, which is quite nice. But the error of $\hat{\alpha}$ is about 200\% relative to $\bar{\alpha}$, which means that the estimation of $\alpha$ fails in this manner.
			Hence, more sophisticated estimation need to be adopted, although this coarse approach gives relative precise estimation of $\gamma$.\\
			Considering the approach in part(e), where $E(n^2 x)$ is estimated utilizing the info of the noise, similiar approximations can be made as
			\begin{equation}
				E(n^2 x) = E({n^*}^2 x^*) = \frac{\sum_{k=1}^{N}{n_k^*}^2 x_k^*}{N}
			\end{equation}
			and
			\begin{equation}
				E(\dot{n}) = E(\dot{n^*}) = \frac{\sum_{k=2}^{N-1}\dot{n_k^*}}{N-2}
			\end{equation}
			Here, no assumption of the distribution of the noise was made. With $\hat{\gamma}$ given by the coarse estimation, a relative finer estimation of $\hat{\alpha}$ can be calculated as
			\begin{equation}
				\hat{\alpha} = \frac{\hat{\gamma} - E(\dot{n^*})}{E({n^*}^2 x^*)}
			\end{equation}
			Numerical experiment was carried to verifiy the approximations. This time, the relative error of $\hat{\alpha}$ is satisfying, and the estimation result is
			\begin{equation}
				\begin{aligned}
					\hat{\alpha} & = 0.100805\\
					\hat{\gamma} & = 0.198847
				\end{aligned}
			\end{equation}

			Further, we can apply the scheme described in part(e) without tuning  $E(n^2 x)$ finely, just take it as $E({n^*}^2 x^*)$, which also gives similiar result. But these trival stuff will not be discussed here as it has been clear enough for this problem...
	\end{enumerate}

\end{document}


\section{Sketch of proof}
\label{sec:appb}
Let $T_{n}$ denote the space of trigonometric polynomials
$\{ e^{-int}, \ldots e^{int} \}$.
Let $\mathcal{K}$ denote the Helmholtz double layer integral operator
given by
\begin{equation}
\mathcal{K}[\phi] = \frac{i}{2}\int_{\Gamma} k H_{1}^{1}(k|x(t)-y(t)|) \frac{(x(t)-y(t))\cdot n(y(t))}{|x(t)-y(t)|} \phi(y(t)) dS_{y} \,
\end{equation}
where $k$ the Helmholtz parameter is assumed to be a dirichlet eigenvalue of the simply connected
domain $\Omega$ whose boundary is given by $\Gamma$, $(x(t),y(t)): [0,2\pi] \to \Gamma$ is
an arclength parameterization of the boundary, and $H_{1}^{1}$ is the Hankel function
of the first kind of order $1$.
It is well known that if $\Gamma$ is of class $C^{\ell,\alpha}$, then the
integral operator $\mathcal{K}: C^{\ell-1,\alpha} \to C^{\ell-1,\alpha}$ is a compact operator.
Furthermore, as shown in Zhao and Barnett, at the Dirichlet eigenvalues, the operator
$I + \mathcal{K}$ is not invertible and has a $p-$dimensional null-space, where
$p$ is the multiplicity of the eigenvalue.
By Fredholm theory, the adjoint $I + \mathcal{K}^{\ast}$ also has a $p$-dimensional null space.
For convenience, we restrict our attention to the case $p=1$. This
part of the analysis extends to non-simple eigenvalues as well.
Let $\phi_{0}$ denote
the null vector.
Let $\psi_{0}$ denote the null vector of $I+\mathcal{K}^{\ast}$.

In order to prove the regularity of the eigenfunctions, we need to analyze
the problem in $C^{\ell-1,\alpha}$ space.
However in order to obtain a bound for the magnitude of the Fredholm
determinant, it is more convenient to work in $L^{2}$. It is well known
that the integral operator $I + \mathcal{K}$ is a second kind
Fredholm operator from $L^{2} \to L^{2}$ as well.
For the remainder of the section, we will work with $L^{2}$.

Let $K_{n}: T_{n} \to T_{n}$
denote a discretization of $\mathcal{K}$
using kress quadrature.
Let $P_{n}$ denote the orthogonal
projection operator from $L^{2} \to T_{n}$.
Let $P_{n,\phi_{0}^{\perp}}: T_{n} \to T_{n}$ denote the orthogonal
projection on to the orthogonal complement of $P_{n} \phi_{0}$.
Let $P_{n,\psi_{0}^{\perp}}: T_{n} \to T_{n}$ denote the
analogous operator for $\psi_{0}$.

The matrix $I+K_{n}$ can be rewritten as a map from the
following two bases for $T_{n}: \{ P_{n} \phi_{0}, \text{basis for }
P_{n}\phi_{0}^{\perp} \}$, and for the range the basis:
$\{ P_{n} \psi_{0}, \text{basis for } P_{n} \psi_{0}^{\perp} \}$.
In these bases, the linear operator $I_{n} + K_{n}$ is given by
\begin{equation}
\begin{bmatrix}
A_{n} & B_{n} \\
C_{n} & D_{n}
\end{bmatrix} \, .
\end{equation}

The operator $A_{n}$
is the restriction of $I_{n} + K_{n}$ as a map from $P_{n} \phi_{0}$
to $P_{n} \psi_{0}$. Thus, it is just a number and it is given by
\begin{equation}
A_{n} = \left(P_{n} \psi_{0}, (I_{n} + K_{n}) P_{n} \phi_{0} \right) \,.
\end{equation}

The operator $D_{n}$ is the restriction of $I_{n} + K_{n}$
as a map from $P_{n} \phi_{0}^{\perp}$
to $P_{n} \psi_{0}^{\perp}$.
The following statement requires a more careful proof.
{\color{red} As $n \to \infty$, $D_{n}$ converges pointwise to the restriction
of $I + K: \phi_{0}^{\perp} \to R(I+K)$. Moreover, the collection
of operators $K_{n}$ are collectively compact, and hence their restrictions
are also collectively compact.
Since the operator $I+K: \phi_{0}^{\perp} \to R(I+K)$ is an invertible
operator, by standard estimates, for $n$ large enough $D_{n}$ is also
an invertible operator.}

The operator $B_{n}$ is the restriction of $I_{n} + K_{n}$ as a map from
$P_{n} \phi_{0} ^{\perp} \to P_{n} \psi_{0}$ and is given by
\begin{equation}
B_{n} \phi = \left( P_{n} \psi_{0}, (I_{n} + K_{n})
(P_{n} \phi -( P_{n} \phi, P_{n} \phi_{0}) P_{n} \phi_{0}) \right) P_{n} \psi_{0}  \, .
\end{equation}

And finally the operator $C_{n}$ is the restriction of $I_{n} + K_{n}$ as a map from
$P_{n} \phi_{0} \to P_{n} \psi_{0}^{\perp}$ and is given by
\begin{equation}
C_{n}P_{n} \phi_{0} = (I_{n} + K_{n}) P_{n} \phi_{0} - \left( (I_{n} + K_{n}) P_{n}\phi_{0}, P_{n} \psi_{0} \right)P_{n} \psi_{0}
\end{equation}

%Thus for large enough $n$,
%$f_{n} = P_{n} \phi_{0} - (P_{n} \phi_{0}, P_{n} \psi_{0}) P_{n} \psi_{0} \neq 0$
%and $f_{n} \in P_{n} \psi_{0}^{\perp}$.
%Similarly $g_{n} = P_{n} \psi_{0} - (P_{n}\phi_{0}, P_{n} \psi_{0}) P_{n} \phi_{0} \neq 0$
%and $g_{n} \in P_{n} \phi_{0}^{\perp}$.

By block linear algebra, the determinant of the matrix above is also given by
\begin{equation}
\det{
\begin{bmatrix}
A_{n} & B_{n} \\
C_{n} & D_{n}
\end{bmatrix}
}
= \det{D_{n}} \cdot (A_{n} - B_{n} D_{n}^{-1} C_{n}) \,.
\end{equation}

First, we observe that $D_{n}$ is the restriction of an operator in trace class
with a kernel whose leading order singularity is $r^{2} \log{r}$.
{\color{red} Thus, $D_{n}$ is itself in trace class, and extending the proof of Bornemann to this case,
we observe that the discrete determinant of $D_{n}$ converges to the Fredholm determinant
of $D$ at a rate of at least $O(1/n)$. This in particular implies that
$\sup_{n} |\det{D_{n}}| <\infty$}.

Using Cauchy-Schwarz, we get that
\begin{equation}
\begin{aligned}
|A_{n}| &\leq |P_{n} \psi_{0}| |(I_{n} + K_{n})P_{n} \phi_{0}| \\
|B_{n}D_{n}^{-1} C_{n}| &\leq |B_{n}||D_{n}^{-1}|(1+ |P_{n} \psi_{0}|^2) |(I_{n} + K_{n})P_{n} \phi_{0}| \,.
\end{aligned}
\end{equation}
$B_{n}$ is a discretized block of a bounded operator and hence is bounded,
and $D_{n}^{-1}$ is a bounded operator by the discussion above.

Combining these results together, we get that
\begin{equation}
\left| \det{\begin{bmatrix} A_{n} & B_{n} \\ C_{n} & D_{n} \end{bmatrix}} \right| \leq C |(I_{n} + K_{n})\phi_{n}| \, ,
\end{equation}
where $C = \det{D_{n}} |B_{n}||D_{n}^{-1}| (1 + |P_{n} \psi_{0}|^2)$.


\documentclass{article}

\input{../../preamble.tex}

\pagestyle{main}
\renewcommand{\leftmark}{Lab Report 1 (Wave Motion)}

\begin{document}

\section{Wave Amplitude vs. Velocity}
\textbf{Materials}: I used the \href{https://phet.colorado.edu/sims/html/wave-on-a-string/latest/wave-on-a-string_en.html}{Wave on a String PhET Simulation}.\par
\textbf{Constants}: I used pulse mode with no damping, a pulse width of $\SI{0.50}{\second}$, high tension, and a fixed end. The length of the string, as measured with the ruler feature, is $\SI{7.5}{\centi\meter}$.\par
\textbf{Experiment}: To see if wave speed is affected by the amplitude of the wave pulse, I measured the wave speed $v$ at amplitudes $\SI{0.25}{\centi\meter}$, $\SI{0.75}{\centi\meter}$, and $\SI{1.25}{\centi\meter}$. Since waves travel with a constant wave speed, I minimized error in my timing by letting the wave pulse travel back and forth on the string five times (a total distance of $5\cdot 2\cdot\SI{7.5}{\centi\meter}=\SI{75}{\centi\meter}$ per trial). For timing, I used the in-simulation timer, which I set to start as soon as I clicked the button on the wave pulse generator and manually stopped as best I could when the leading edge returned to the wave-pulse-generator end after the fifth cycle.\par
\textbf{Data}:
\begin{table}[h!]
    \centering
    \renewcommand{\arraystretch}{1.4}
    \begin{tabular}{c|ccc}
                                  & Trial 1 & Trial 2 & Trial 3\\
        \hline
        $\SI{0.25}{\centi\meter}$ & 12.07   & 11.97   & 12.07\\
        $\SI{0.75}{\centi\meter}$ & 12.07   & 12.03   & 11.97\\
        $\SI{1.25}{\centi\meter}$ & 12.03   & 12.05   & 12.01\\
    \end{tabular}
    \caption{Wave amplitude vs. time (to travel $\SI{75}{\centi\meter}$).}
    \label{fig:exp1data}
\end{table}\par
\textbf{Analysis}:\\
$A=\SI{0.25}{\centi\meter}$: We know that
\begin{equation*}
    v = \frac{\Delta x}{\Delta t}
\end{equation*}
Since $\Delta x\approx\SI{75}{\centi\meter}$ and
\begin{equation*}
    \Delta T \approx \bar{t} = \frac{12.07+11.97+12.07}{3} = 12.04
\end{equation*}
we have that
\begin{equation*}
    v \approx \frac{75}{12.04} = \SI{6.23}{\centi\meter\per\second}
\end{equation*}
More specifically, since $\delta x=0.1$ from the ruler gradations and
\begin{equation*}
    \delta t = \sqrt{\frac{(12.07-12.04)^2+(11.97-12.04)^2+(12.07-12.04)^2}{3}} = 0.05
\end{equation*}
we have that
\begin{equation*}
    \delta v = \sqrt{\left( \frac{0.1}{75} \right)^2+\left( \frac{0.05}{12.04} \right)^2}\cdot 6.23 = 0.03
\end{equation*}
Therefore, we conclude that
\begin{equation*}
    v = 6.23\pm\SI{0.03}{\centi\meter}\tag{$A=\SI{0.25}{\centi\meter}$}
\end{equation*}\\
By an analogous method to the above, we can conclude for the remaining two amplitudes that
\begin{gather*}
    v = 6.24\pm\SI{0.02}{\centi\meter}\tag{$A=\SI{0.75}{\centi\meter}$}\\
    v = 6.23\pm\SI{0.01}{\centi\meter}\tag{$A=\SI{1.25}{\centi\meter}$}
\end{gather*}
Since all measurements are within their uncertainties of each other, we can conclude that the velocity did not significantly change based on the amplitude.

\section{Standing Wave Diagram}
\begin{figure}[h!]
    \centering
    \begin{tikzpicture}
        \footnotesize
        \draw [orx,thick,dashed] (0,0) -- (2*pi,0);

        \filldraw [semithick,left color=gray,right color=gray,middle color=white] (0.05,-0.1) rectangle (-0.1,-1.4);
        \filldraw [semithick,fill=gax!40] (0,-1.5) circle (4mm);
        \fill [gray] (-0.3,-1.5) circle (2pt);
        \fill [gax!20] (0,-1.5) circle (2pt);
        \draw
            (0,-1.5) circle (2pt)
            (-2pt,-1.5) -- ++(4pt,0)
        ;

        \draw [
            rex,very thick,postaction={decorate},
            decoration={
                markings,
                mark=between positions 0.2 and 0.8 step 0.2 with \fill [grx] circle (1mm);
            }
        ] plot[domain=0:2*pi,smooth] (\x,{-sin(\x r)});

        \shade [left color=gax,right color=white] (2*pi,-1.5) rectangle ++(0.2,3);
        \draw [semithick] (2*pi,-1.5) -- ++(0,3);
        \fill [grx]
            (2*pi,0.1) arc[start angle=90,end angle=270,radius=1mm]
            (0,0) circle (1mm)
        ;

        \node at (pi/2,2) {antinode}
            edge [out=180,in=90,->,shorten >=4pt] (0,0)
            edge [out=-90,in=90,->,shorten >=2pt] ({pi/2},-1)
            edge [out=0,in=110,->,shorten >=2pt] ({3*pi/2},1)
        ;
        \node at ({3*pi/2},-2) {node}
            edge [out=180,in=-90,->,shorten >=3pt] (pi,0)
            edge [out=0,in=-110,->,shorten >=8pt] (2*pi,0)
        ;
    \end{tikzpicture}
    \caption{Nodes and antinodes on a standing Wave.}
    \label{fig:standingWaveNodes}
\end{figure}
As can be seen in the right side of Figure \ref{fig:standingWaveNodes} (where the wave evens out), the distance between adjacent nodes and the distance between adjacent antinodes is, in both cases, equal to $\lambda/2$. The distance between a node and its nearest antinode (and vice versa) is $\lambda/4$.

\section{Five Questions}
\begin{enumerate}
    \item In general, what are the boundary conditions of a standing wave?
    \begin{itemize}[label={--}]
        \item Fixed at one end; free or fixed at the other (consider the case in Figure \ref{fig:standingWaveNodes} versus the case of a plucked string on a string instrument).
    \end{itemize}
    \item What is the type of a soundwave (longitudinal or transverse)?
    \begin{itemize}[label={--}]
        \item Longitudinal compression wave.
    \end{itemize}
    \item What is the boundary condition of this specific setup?
    \begin{itemize}[label={--}]
        \item Open-closed.
    \end{itemize}
    \item What is the type of the wave in the spring?
    \begin{itemize}[label={--}]
        \item Longitudinal compression wave.
    \end{itemize}
    \item What is the boundary condition of this setup?
    \begin{itemize}[label={--}]
        \item Open-closed.
    \end{itemize}
\end{enumerate}

\end{document}


\documentclass[../main]{subfiles}
\pagestyle{fancy}

\begin{document}

\chapter{Forcing properties}
\label{apx.forcing}
\thispagestyle{fancy}

\section{Lifting Criterion}
When we are working with an elementary embedding $\pi\colon\M\to\N$ between sets in the universe, we would sometimes like to lift such an embedding to a generic extension, meaning that given a forcing notion $\mathbb P\in\M$ and an $\M$-generic $g\subset\mathbb P$, we're interested in when we can lift $\pi$ to an embedding
\eq{
  \pi^+\colon\M[g]\to\N[h],
}

where $h\subset\pi(\mathbb P)$ is $\N$-generic. The \textbf{lifting criterion} shows exactly when this is possible. The following is Proposition 9.1 in \cite{handbook-cummings}.

\prop[The Lifting Criterion][prop.lifting-criterion]{
  Let $\pi\colon\M\to\N$ be an elementary embedding between weak $\kappa$-models. Fix a forcing notion $\mathbb P\in\M$, an $\M$-generic $g\subset\mathbb P$ and an $\N$-generic $h\subset\pi(\mathbb P)$. Then the following are equivalent:
  \begin{itemize}
    \item $\pi[g]\subset h$;
    \item There exists an elementary $\pi^+\colon\M[g]\to\N[h]$ such that $\pi^+(g)=h$ and $\pi^+\restr\M = \pi$.
  \end{itemize}
}
\proof{
  $(ii)\Rightarrow(i)$ is clear, so assume $(i)$. Define $\pi^+\colon\M[g]\to\N[h]$ as $\pi^+(\dot\tau^g) := \pi(\tau)^h$. To see that $\pi^+$ is well-defined fix $\dot\sigma,\dot\tau\in\M^{\mathbb P}$ such that $\dot\sigma^g=\dot\tau^g$, and fix $p\in g$ such that $p\forces\dot\sigma=\dot\tau$. By elementarity $\pi(p)\forces\pi(\dot\sigma)=\pi(\dot\tau)$, so since $\pi(p)\in h$ by $(i)$ we get that $\pi(\dot\sigma)^h=\pi(\dot\tau)^h$.

  \qquad To show elementarity, note that for $x\in\M$ it holds that $\pi(\check x) = \check{\pi(x)}$, implying $\pi^+(x)=\pi^+(\check x^g) = \pi(\check x)^h = \pi(x)$. Further, letting $\dot g\in\M^{\mathbb P}$ be the standard $\mathbb P$-name for $g$, then $\pi(\dot g)$ is the standard $\pi(\mathbb P)$-name for $h$ and therefore $\pi^+(g)=h$.
}

\section{Countable Embedding Absoluteness}
A key folklore lemma which we will frequently need when dealing with elementary embeddings existing in generic extensions is the following.

\lemm[Countable Embedding Absoluteness][lemm.ctblabs]{
  Let $\M,\N$ be sets, $\P$ a transitive class with $\M,\N\in\P$, and let $\pi\colon\M\to\N$ be an elementary embedding. Assume that $\P\models\zf^-+\dc+\godel{\text{$\M$ is countable}}$ and fix any finite $X\subset\M$.

  \qquad Then $\P$ contains an elementary embedding $\pi^*\colon\M\to\N$ which agrees with $\pi$ on $X$. If $\pi$ has a critical point and if $\M$ is transitive then we can also assume that $\crit\pi=\crit\pi^*$.\footnote{We are using transitivity of $\M$ to ensure that the \textit{ordinal} $\crit\pi$ exists.}
}
\proof{
  Let $\{a_i\mid i<\omega\}\in\P$ be an enumeration of $\M$ and set $\M\restr n:=\{a_i\mid i<n\}$. Then, in $\P$, build the tree $\T$ of all partial isomorphisms between $\M\restr n$ and $\N$ for $n<\omega$, ordered by extension. Then $\T$ is illfounded in $V$ by assumption, so it's also illfounded in $\P$ since $\P$ is transitive and $\P\models\zf^-+\dc$. The branch then gives us the embedding $\pi^*$, and if $\crit\pi$ exists then we can ensure that it agrees with $\pi$ on the critical point and finitely many values by adding these conditions to $\T$.
}

\section{Preservation of sequence closure}
The following lemma is from \cite{lucke-schlicht-notes} and gives a useful condition on when sequence closure is preserved when moving to generic extensions.

\lemm[][lemm.lucke-schlicht]{
  Let $\lambda$ be an infinite cardinal, $\M\models\zf^-$ a transitive model, $\mathbb P\in\M$ a $\lambda^+$-cc forcing notion and $g\subset\mathbb P$ an $\M$-generic filter. Then $V\models{^\lambda}\M\subset\M$ implies that $V[g]\models{^\lambda}\M\subset\M$.
}
\proof{
  Work in $V[g]$. Let $c:=\bra{c_\alpha\mid\alpha<\lambda}$ be a $\lambda$-sequence such that $c_\alpha\in\M[g]$ for every $\alpha<\lambda$. Fix for every $\alpha<\lambda$ a $\mathbb P$-name $\dot c_\alpha$ such that $\dot c_\alpha^g=c_\alpha$. Also let $\dot a$ be a $\mathbb P$-name with $\dot a^g=\bra{\dot c_\alpha\mid\alpha<\lambda}$ and choose $p\in g$ such that $V\models\godel{p\forces\forall\alpha<\check\lambda\colon\dot a(\alpha)\in\M^{\mathbb P}}$.

  \qquad Now, working in $V$, there is for each $\alpha<\lambda$ a maximal antichain $A_\alpha$ below $p$ such that every $q\in A_\alpha$ decides $\dot a(\alpha)$; i.e., $q\forces\godel{\dot a(\alpha)=\check x}$ for some $x\in\M$. Define now
  \eq{
    \sigma := \{((\alpha,x),q)\mid\alpha\in\lambda\land q\in A_\alpha\land q\forces\godel{\dot a(\alpha)=\check x}\}.
  }

  Then $p\forces\godel{\sigma=\dot a}$. Note that $\abs{\sigma}\leq\lambda$, since $\abs{A_\alpha}\leq\lambda$ for each $\alpha<\lambda$. Thus $\sigma\in\M$. Now it holds that
  \eq{
    V[g]\models\godel{\bra{\dot c_\alpha\mid\alpha<\lambda}=\dot a^g=\sigma^g\in\M[g]},
  }

  and we can compute $c=\bra{c_\alpha\mid\alpha<\lambda}=\bra{\dot c_\alpha^g\mid\alpha<\lambda}$ from $\bra{\dot c_\alpha\mid\alpha<\lambda}$ and $g$, so $c\in\M[g]$ by Replacement.
}

\end{document}


\section{Experiment}

In this section we explore the results of a Python implementation of $\beta$-VAE on a toy EMNIST (extension to MNIST) dataset of handwritten digits and characters. The code can be found here: \url{https://github.com/benlevyx/beta-vae}.

\subsection{Model architecture and dataset}

We use the EMNIST dataset, which is an image dataset similar to MNIST, except that it also includes handwritten letters \cite{cohen2017emnist}. Each digit is $28\times 28$ pixels with a single grayscale channel. The dataset includes handwritten digits from 0 to 9 and letters from a to z, including lowercase and uppercase. For simplicity, we take a subset of the data comprising the letters A through D (uppercase and lowercase).

The model encoder and decoder both consist of feed-forward neural networks. The 2D images are flattened to a vector of 784, which are then fed into the network in batches of 256 for computational efficiency. Since the goal of this experiment is to be able to plot the low-dimensional representations of the data for different levels of $\beta$, we use a latent dimension $d=2$. We train models with $\beta=0.0, 1.0, 5.0$ for 15 epochs each using the Adam optimizer with learning rate $10^{-3}$.

\begin{figure}[h!]
    \centering
    \includegraphics[width=0.7\textwidth]{beta-vae-experiment}
    \caption{Results from experiment with different levels of $\beta$. Each row represents a model trained with a different level of $\beta$. The left column plots the 2-dimensional representation of the evaluation data, colour-coded according to whether the point is an `A' or `Z'. The right column shows a traversal of the latent space in the fixed range $(-4,4)$ for both latent dimensions.}
    \label{fig:beta-vae-experiment}
\end{figure}

\subsection{Effect of $\beta$ on latent space}

In figure \ref{fig:beta-vae-experiment}, we can see the effect of $\beta$ on the latent space. The starkest difference is between $\beta=0$ and the cases where $\beta > 0$. In the former, we see that the latent space is extended along a line such that each point is essentially assigned its own location in that latent space. This resembles the latent space for a non-Bayesian autoencoder, since the model is only incentivized to reconstruct the data faithfully and is not encouraged to learn a smooth latent space.

As $\beta$ increases, we see that the latent space becomes closer to a standard Gaussian distribution. This is a double-edged sword: althouh the latent space is smoother and perhaps generalizes better, we completely lose the ability to discriminate between the two classes.

Of course, this is an extreme case meant to illustrate the effect of strong and weak regularization on the geometry of the latent space. In the real world, we would implement a $\beta$-VAE with a larger bottleneck dimension (2 is quite extreme), which would also allow for more fine-tuning of $\beta$.