\section{Gaussian process submodels}

This package represents a Gaussian process $f\sim \textup{GP}(M,C)$ as a \texttt{GaussianProcess} object which, as you might expect, is a PyMC \texttt{stochastic} whose value is a \texttt{Realization} object. It is not feasible to endow a full \texttt{GaussianProcess} with a \texttt{logp} attribute, so \texttt{GaussianProcess} objects cannot be handled by PyMC's standard MCMC machinery.

However, the evaluation $f(x_*)$ on a mesh $x_*$ is a simple multivariate normal random variable, which can be handled by the standard machinery. If $f(x_*)$ is incorporated in the model as a variable, a minor extension to the standard machinery (section \ref{sec:step-methods}) makes it possible to handle $f$ itself as well.

Pairs of $f$ and $f(x_*)$ variables are created by \texttt{GPSubmodel} objects, which are containers for PyMC variables. \texttt{GaussianProcess} objects can only be incorporated in PyMC probability models via \texttt{GPSubmodel} objects.

\section{Example: nonparametric regression with unknown mean and covariance parameters}\label{sub:BasicMCMC}

A GP submodel is created in \file{examples/gp/PyMCmodel.py} with the following call:
\begin{verbatim}
    sm = gp.GPSubmodel('sm',M,C,fmesh)
\end{verbatim}
There are two stochastic variables in the submodel: \texttt{smf} and \texttt{smf_eval}. The first is the actual Gaussian process $f$: a stochastic variable valued as a \texttt{Realization} object. The second is $f(x_*)$, where $x_*$ is the input argument \texttt{fmesh}.

Once the GP submodel is created, we can create other variables that depend on $f$ and $f(x_*)$. In \file{PyMCmodel.py}, the observation $d$ depends on $f(x_*)$:
\begin{verbatim}
    d = pymc.Normal('d',mu=smf_eval, tau=1./V, value=init_val, observed=True)
\end{verbatim}
The full probability model is shown as a directed acyclic graph in figure \ref{fig:unobservedModel}. It illustrates the dependency relationships between the variables in a GP submodel.

The file \file{examples/gp/MCMC.py} fits the probability model created in \file{PyMCmodel.py} using MCMC. The `business part' of the file is very simple:
\begin{verbatim}
    GPSampler = MCMC(PyMCmodel)
    GPSampler.isample(iter=5000,burn=1000,thin=100)
\end{verbatim}
Most of the code in the file is devoted to plotting, and its output is shown in figure \ref{fig:MCMCOutput}. Note that after the MCMC run \texttt{GPSampler.trace('sm_f')[:]} yields a sequence of \texttt{Realization} objects, which can be evaluated on new arrays for plotting. GP realizations can even be tallied on disk using the HDF5 backend (see PyMC's user guide).

\begin{figure}
    \centering
        \epsfig{file=figs/unobservedModel.pdf, width=15cm}
    \caption{The PyMC-generated directed acyclic graph representation of the extended nonparametric regression model created by \sffamily{`examples/gp/PyMCModel.py'}. Ellipses represent \texttt{Stochastic} objects (variables whose values are unknown even if their parents' values are known), triangles represent \texttt{Deterministic} objects (variables whose values can be determined if their parents' values are known), and rectangles represent \texttt{Stochastic} objects with the \member{isdata} flag set to \member{True} (data). Rectangles represent potentials Arrows point from parent to child. The submodel contains the Gaussian process \texttt{sm\_f} and its evaluation \texttt{sm\_f\_eval} on input array \texttt{sm\_mesh}. It also contains the mean \texttt{sm\_M\_eval} of \texttt{sm\_f\_eval} and the lower-triangular Cholesky factor \texttt{sm\_S\_eval} of its covariance matrix, and a potential \texttt{sm_fr_check} that forces that covariance matrix to remain positive definite. The actual covariance evaluation \texttt{sm\_C\_eval} is not needed by the model, but it is exposed for use by Gibbs step methods.}
    \label{fig:unobservedModel}
\end{figure}

\begin{figure}
    \centering
        \epsfig{file=figs/gibbsSamples.pdf,width=10cm}
        % \epsfig{file=figs/metroSamples.pdf,width=10cm}
    \caption{The output of \sffamily{`examples/gp/MCMC.py'}. The left-hand panel shows all the samples generated for the Gaussian process $f$, and the right-hand panel shows the trace of $f(0)$.}
    \label{fig:MCMCOutput}
\end{figure}

\section{Step methods}
\label{sec:step-methods}
Since $f$ has no \texttt{logp} attribute, the Metropolis-Hastings family of step methods cannot be used to update $f$, $f(x_*)$ or any of their mean or covariance parameters. This package uses a relatively simple work-around that will be described here. Throughout this section, the parents of $f$ are denoted $P$ and the children $K$.

\subsection{Step methods that handle parents of Gaussian processes}
If we could come up with a probability density function for $f$, the Metropolis-Hastings acceptance ratio for a proposed value $P_p$ of the parents \emph{and} a proposed value $\tilde f$ for $f$ would be:
\begin{eqnarray*}
    \frac{p(K|f_p)\ p(f_p|P_p)\ q(P)}{p(K|f)\ p(f|P)\ q(P_p)}
\end{eqnarray*}
where $q$ denotes the proposal density. Now, suppose we proposed a value for $f$ conditional on the proposed values for the parents $P$ \emph{and} conditional on $f(x_*)$. The new acceptance ratio would become
\begin{eqnarray*}
    \frac{p(K|f_p)\ p(f_p|f(x_*), P_p)\ p(f(x_*) | P_p)\ q(f_p|f(x_*),f_p, P_p)\ q(P)}{p(K|f)\ p(f|f(x_*), P)\ p(f(x_*) | P)\ q(f_p|f(x_*),f,P)\ q(P_p)}
\end{eqnarray*}
 We want to avoid computing all terms with $f$ or $f_p$ in the consequent position:
\begin{eqnarray*}
    p(f_p|f(x_*), P),\\ q(f|f(x_*),f_p,P),\\ p(f|f(x_*), P),\\ q(f_p|f(x_*),f,P_p),
\end{eqnarray*}
but all other terms are fine. We can make the problem terms cancel by choosing our proposal distribution as follows:
\begin{eqnarray*}
    q(f_p|f(x_*),f,P_p) = p(f_p|f(x_*), P).
\end{eqnarray*}
In other words, if we propose $f$ from its prior distribution conditional on $f(x_*)$ and its parents whenever we propose $f(x_*)$, we don't have to worry about computing the intractable terms. This argument can be made more rigorous by replacing $f$ with its evaluation at all points where we will ever want to know it.

By choosing the same proposal distribution for $\tilde f$ as above, we again avoid having to compute the intractable terms. In other words, every time a value is proposed for a \texttt{GP}'s parent, a value must be proposed for the \texttt{GP}  conditional on its value's evaluation on its mesh, and the prior probability of the \texttt{GP}'s children must be included in the acceptance ratio.

\bigskip
To summarize, any Metropolis-Hastings step method can handle the parents of $f$, as well as $f(x_*)$, if it proposes values for $f$ jointly with its target variable as outlined above.

This minor alteration can be done using the function \texttt{wrap_metropolis_for_gp_parents}, which takes a subclass of \texttt{Metropolis} as an argument and returns a new step method class with altered \texttt{propose} and \texttt{reject} methods. The function automatically produces modified versions of all Metropolis step methods in PyMC's step method registry (\texttt{Metropolis}, \texttt{AdaptiveMetropolis}, etc.). The modified step methods are automatically assigned to parents of Gaussian processes.

\subsection{Choosing a mesh}

The mesh points $x_*$ are the points where Metropolis-Hastings step methods can `grab' the value of $f$ to moderate the variance of its proposal distribution. If $x_*$ is an empty array, $f$'s value will be proposed from its prior, and rejection rates are likely to be quite large. If $x_*$ is too dense, on the other hand, computation of the log-probability of $f(x_*)$ will be expensive, as it scales as the cube of the number of points in the mesh.This continuum is illustrated in figure \ref{fig:meshpropose}. Finding the happy medium requires some experimentation.

Another important point to bear in mind is that if $f$'s children depend on its value only via its evaluation on the mesh, the likelihood terms $p(K|f_p)$ and $p(K|f)$ will cancel. In other words, if the mesh is chosen so that $p(K|f)=p(K|f(x_*))$ then the proposed value of $f$ will have no bearing on the acceptance probability of the proposed value of $f(x_*)$ or of the parents $P$. This is the situation in \file{PyMCModel.py}. Such a mesh choice will generally improve the acceptance rate.

\begin{figure}
    \centering
        \epsfig{file=figs/nomeshpropose.pdf,width=9cm}
        \epsfig{file=figs/lightmeshpropose.pdf,width=9cm}
        \epsfig{file=figs/densemeshpropose.pdf,width=9cm}
    \caption{Several possible proposals of $f$ (curves) given proposed values for $f(x_*)$ (heavy dots) with no mesh (top), a sparse mesh (middle), and a dense mesh (bottom). Proposal distributions' envelopes are shown as shaded regions, with means shown as broken lines. With no mesh, $f$ is being proposed from its prior and the acceptance rate will be very low. A denser mesh permits a high degree of control over $f$, but computing the log-probability will be more expensive.}
    \label{fig:meshpropose}
\end{figure}

\subsection{The \texttt{GPEvaluationGibbs} class}
If all of $f$'s children $K$, depend on it as follows:
\begin{eqnarray*}
    K_i|f \stackrel{\textup{\tiny ind}}{\sim} \textup{Normal}(f(x_{*i}),V_i)
\end{eqnarray*}
then $f(x_*)$ can be handled by the \texttt{GPEvaluationGibbs} step method. This step method is used in \file{MCMC.py}:
\begin{verbatim}
    GPSampler.use_step_method(gp.GPEvaluationGibbs, GPSampler.submod, GPSampler.V, GPSampler.d)
\end{verbatim}
The initialization arguments are the Gaussian process submodel that contains $f$, the observation variance of $f$'s children, and the children, in this case the vector-valued normal variable $d$.

\texttt{GPEvaluationGibbs} covers the standard submodel encountered in geostatistics, but there are many conjugate situations to which it does not apply. If necessary, special step methods can be written to handle these situations. If \texttt{GPEvaluationGibbs} is not assigned manually, $f(x_*)$ will generally be handled by a wrapped version of \texttt{AdaptiveMetropolis}.

\section{Geostatistical example}\label{sub:geostat}
\begin{figure}
    \centering
        \epsfig{file=figs/elevmean.pdf, width=8cm}
        \epsfig{file=figs/elevvar.pdf, width=8cm}
    \caption{The posterior mean and variance surfaces for the $v$ variable of the Walker lake example. The posterior variance is relatively small in the neighborhood of observations, but large in regions where no observations were made.}
    \label{fig:walker}
\end{figure}
\begin{figure}
    \centering
        \epsfig{file=figs/elevdraw0.pdf, width=8cm}
        \epsfig{file=figs/elevdraw1.pdf, width=8cm}
    \caption{Two realizations from the posterior distribution of the $v$ surface for the Walker Lake example. Elevation is measured in meters.}
    \label{fig:walkerreal}
\end{figure}
Bayesian geostatistics is demonstrated in the folder \file{examples/gp/more_examples/Geostatistics}. File \file{getdata.py} downloads the Walker Lake dataset of Isaaks and Srivastava \cite{isaaks} from the internet and manipulates the $x$ and $y$ coordinates into the array format described in section \ref{sec:highdim}. File \file{model.py} contains the geostatistical model specification, which is

\begin{eqnarray*}
    d|f \sim \textup{Normal}(f(x),V)\\
    f|M,C \sim \textup{GP}(M,C) \\
    M:x\rightarrow m\\
    C:x,y,\mathtt{amp},\mathtt{scale},\mathtt{diff\_degree}\rightarrow \mathtt{matern.euclidean}(x,y;\mathtt{amp},\mathtt{scale},\mathtt{diff\_degree})\\
    p(m)\propto 1\\
    \mathtt{amp}\sim \textup{Exponential}(7e-5) \\
    \mathtt{scale}\sim \textup{Exponential}(4e-3) \\
    \mathtt{diff\_degree}\sim \textup{Uniform}(.5,2)\\
    V\sim \textup{Exponential}(5e-9)\\
\end{eqnarray*}

File \file{mcmc.py} fits the model and produces output maps.  The output of \file{mcmc.py} is shown in figures \ref{fig:walker} and \ref{fig:walkerreal}. Figure \ref{fig:walker} shows the posterior mean and variance of the $v$ variable of the dataset, which is a function of elevation (see Isaaks and Srivastava \cite{isaaks}, appendix A). The mean and variance surfaces are generated conveniently from the trace as follows:
\begin{verbatim}
    Msurf = zeros(dplot.shape[:2])
    E2surf = zeros(dplot.shape[:2])
    for i in xrange(n):
        WalkerSampler.remember(i)
        Msurf_i = WalkerSampler.walker_v.M_obs.value(dplot)
        Msurf += Msurf_i / n
        E2surf += (WalkerSampler.walker_v.C_obs.value(dplot) + Msurf_i**2) / n
\end{verbatim}
The call \texttt{WalkerSampler.remember(i)} resets all the variables in the model to their values at frame $i$ of the MCMC trace. The surfaces $E[v]$ (\texttt{Msurf}) and $E[v^2]$ (\texttt{E2surf}) are easily accumulated by stepping through the trace using \texttt{remember} and evaluating the \texttt{M_obs} and \texttt{C_obs} attributes of the Gaussian process submodel on input array \texttt{dplot}. The standard deviation surface is generated using the standard formula:
\begin{verbatim}
    Vsurf = E2surf - Msurf**2
    SDsurf = sqrt(Vsurf)
\end{verbatim}

Figure \ref{fig:walkerreal} shows two realizations from the posterior predictive distribution of the $v$ surface. They are much rougher than their mean surface, and also much more expensive to compute. However, they are equally convenient to evaluate:
\begin{verbatim}
    indices = random_integers(n,size=2)
    for j,i in enumerate(indices):
        WalkerSampler.remember(i)
        R = WalkerSampler.walker_v.f.value(dplot)
\end{verbatim}
Again, \texttt{remember} is used to reset the model to a randomly-selected frame of the trace, and the Gaussian process $f$ is evaluated on \texttt{dplot} in that frame.

\section{Biological example: Munch, Kottas and Mangel's stock-recruitment study}\label{sub:MMKMCMC}

% \begin{figure}
%     \centering
%     % FIXME: Re-render this graph
%         \epsfig{file=figs/MKMsalmon.pdf, width=10cm}
%     \caption{A PyMC-generated directed acyclic graph representation of the probability model in (\ref{eqn:MMKModel}), which is implemented in file \sffamily{`examples/gp/more_examples/MKMSalmon/salmon_sampler.py'}. A very similar probability model was used by Munch, Kottas and Mangel to infer stock-recruitment functions for three salmonid species. The label `\texttt{pink.SR}' indicates that this particular model corresponds to the pink salmon (\emph{Onchorhynchus gorbuscha}) data. Note that two coordinate transformations are implemented using PyMC deterministic variables. This technique can help lazy programmers avoid transforming priors by hand, and in less trivial cases it can save computation by caching the transformed parameters.}
%     \label{fig:MMKsalmonmodel}
% \end{figure}

\begin{figure}
    \centering
        \epsfig{file=figs/pinkfpost.pdf, width=10cm}
    \caption{The posterior of the stock-recruitment function for pink salmon (\emph{Onchorhynchus gorbuscha}). The data are shown as heavy black dots. The centered 95\%, 50\% and 25\% posterior probability intervals are shown as shaded regions. Three draws from the posterior are plotted.}
    \label{fig:pinkfpost}
\end{figure}

\begin{figure}
    \centering
        \epsfig{file=figs/pinkbeta0post.pdf, width=5cm}
        \epsfig{file=figs/pinkbeta1post.pdf, width=5cm}
        \epsfig{file=figs/pinkamppost.pdf, width=5cm}
        \epsfig{file=figs/pinkdiffdegreepost.pdf, width=5cm}
        \epsfig{file=figs/pinkscalepost.pdf, width=5cm}
        \epsfig{file=figs/pinkVpost.pdf, width=5cm}
    \caption{The marginal posterior distributions of the mean and covariance parameters of the stock-recruitment function for pink salmon (\emph{Onchorhynchus gorbuscha}).}
    \label{fig:pinkparams}
\end{figure}

We now have the tools to duplicate Munch, Kottas and Mangel's \cite{mmk} stock-recruitment results. The full probability we'll use follows. It is like Munch, Kottas and Mangel's probability model, but we will use a Mat\`ern covariance function. Here \texttt{SR} is the stock-recruitment function, C is its covariance and M is its mean.
\begin{equation}
    \label{eqn:MMKModel}
    \begin{array}{ll}
        \texttt{frye}[i] \stackrel{\tiny{\textup{ind}}}{\sim} \textup{N}(exp(\mathtt{SR}(\log(\mathtt{abundance}[i]))), V),& i=1\ldots n\\
        \texttt{SR}\sim \textup{GP}(M,C)& \\
        M:x\rightarrow \beta_0+\beta_1 \log(x)&\\
        C:x,y\rightarrow \texttt{matern.euclidean}(x,y;\ \texttt{diff_degree},\texttt{amp},\texttt{scale})&\\
        V,\ \beta_0,\ \beta_1,\ \texttt{diff_degree},\ \texttt{amp},\ \texttt{scale} \sim \texttt{priors}.&
    \end{array}
\end{equation}
The priors can be found by reading their paper (except the prior on \texttt{diff_degree}, which I chose) or by reading the file \file{examples/more_examples/MMKSalmon/salmon_sampler.py}. This file contains a PyMC \texttt{MCMC} subclass called \texttt{SalmonSampler}, which reads in the data, creates a probability model incorporating the data, and provides some plotting methods.

Note that this model differs from the model in section \ref{sub:MMKregression} in that we're putting a Gaussian process prior on the stock-recruitment function in log-log space, so conditioning its value at zero isn't an option.

% The probability model \texttt{SalmonSampler} creates is visualized as a directed acyclic graph in figure \ref{fig:MMKsalmonmodel}.

Munch, Kottas and Mangel specify priors for $\texttt{amp}^{-2}$ and $V^{-1}$, and I've used PyMC deterministic variables to conveniently implement the transformations rather than changing variables manually. The posterior distribution of \texttt{SR} for the pink salmon (\emph{Onchorhynchus gorbuscha}) is shown in figure \ref{fig:pinkfpost}, and the posterior of the mean and covariance parameters for the same species are shown in figure \ref{fig:pinkparams}.

% \chapter{Old stuff}
%
% \section{The \texttt{GP} class}
% \texttt{GP} is a subclass of the PyMC \texttt{Stochastic} class whose value attribute is a \texttt{Realization} instance. Its init method takes the following arguments:
% \begin{description}
%     \item[$M$:] A mean object or mean-valued deterministic variable.
%     \item[$C$:] A covariance object or covariance-valued deterministic variable.
%     \item[\texttt{mesh=None}:] An array or an array-valued variable.
%     \item[\texttt{init_mesh_vals=None}:] An optional vector of initial values for the value attribute's evaluation on \texttt{mesh}.
%     \item[\texttt{mesh_eval_isdata=False}:] If \texttt{True}, the value attribute's evaluation on \texttt{mesh} is fixed at \texttt{init_mesh_vals}.
%     \item[\texttt{doc, name, trace, cache_depth, verbose}:] Optional arguments that are passed directly to \code{Stochastic.\_\_init\_\_}. See PyMC's documentation.
% \end{description}
%
% \texttt{GP} instances have a log-probability attribute like any PyMC stochastic variable, but there is an important difference: If the value attribute is a realization called $f$, the log-probability attribute gives $p(f\texttt{(mesh))}|M,C)$. In other words, the log-probability attribute only cares about $f$'s evaluation on \texttt{mesh}. The reason is simple: it would be expensive and difficult to assign something like a log-density to entire realizations in most cases.
%
% \section{The \texttt{GPMetropolis} and \texttt{GPParentMetropolis} step methods}
%
% \section{GPMetropolis}
% The Metropolis-Hastings acceptance ratio for a proposed value $f_p$ can be written as follows:
% \begin{eqnarray*}
%     \frac{p(K|\tilde f_p, f_p(\mathtt{mesh}))\ p(\tilde f_p|f_p(\mathtt{mesh}), P)\ p(f_p(\mathtt{mesh}) | P)\ q(\tilde f|f(\mathtt{mesh}))\ q(f(\mathtt{mesh}))}{p(\mathtt{K}|\tilde f, f(\mathtt{mesh}))\ p(\tilde f|f(\mathtt{mesh}), P)\ p(f(\mathtt{mesh}) | P)\ q(\tilde f_p|f(\mathtt{mesh}))\ q(f_p(\mathtt{mesh}))},
% \end{eqnarray*}
%
% \texttt{GPMetropolis} reports its competence to handle \texttt{GP} instances as \texttt{3}, so it will be chosen as the default handler. Its init method takes the following arguments:
% \begin{description}
%     \item[\texttt{stoch}:] The \texttt{GPMetropolis} instance to be handled.
%     \item[\texttt{scale=.1}:] $f(\texttt{mesh})$ will be proposed from a random-walk multivariate normal distribution with covariance equal to \texttt{C(mesh, mesh * scale * adaptive_scale_factor)}, where C is the covariance-valued parent and \texttt{adaptive_scale_factor} is the adaptive scaling factor, which is updated when \texttt{self.tune()} is called.
%     \item[\texttt{verbose = 0}:] An integer from 0 to 3 indicating the preferred verbosity level.
% \end{description}

% \section{The \texttt{GPNormal} step method}
% As we've already seen in other contexts, a \texttt{GP}'s full conditional distribution is a Gaussian process if its Markov blanket is described by the following probability model:
% \begin{eqnarray*}
%     K_i |f \sim \textup{N}(f(o_i), V_i) & i=0\ldots n-1\\
%     f|M,C\sim\textup{GP}(M,C)
% \end{eqnarray*}
% In other words, each of its children $K_i$ is normally distributed with variance $V_i$ and mean equal to $f(o_i)$, where $o_i$ is an arbitrary mesh.
%
% In this case, a \texttt{GP} instance $f$ can be handled by the Gibbs step method \texttt{GPNormal}, which will have much better mixing properties that \texttt{GPMetropolis}. Its init method takes the following parameters:
% \begin{description}
%     \item[$f$:] The \texttt{GP} instance to be handled.
%     \item[\texttt{obs_mesh:}] An array or array-valued variable giving the observation mesh $o$.
%     \item[\texttt{obs_V}:] An array or array-valued variable giving the observation variance $V$.
%     \item[\texttt{obs_vals}:] An array or array-valued variable giving the concatenation of the values of the children $K_i$.
% \end{description}
%
% \texttt{GPNormal} doesn't register itself, so it will never be assigned as a default handler. This may change eventually.
%
% \texttt{GPNormal} doesn't care about $f$'s mesh, because it never has to evaluate $f$'s log-probability. However, a good choice of mesh is important because it will used by the \texttt{GPParentMetropolis} instances that handle $f$'s mean and covariance parameters.
%
%
% % \section{Example: Ellner, Seifu and Smith's blowfly study}\label{sub:ESSMCMC}
% %
% % \textbf{I haven't been able to get this one to mix yet.} The model is:
% % \begin{eqnarray*}
% %     \textup{data}_t \stackrel{\tiny{\textup{ind}}}{\sim}(A_t, V)\\
% %     A_t = B(A_{t-\tau})\psi_{t-\tau} - D(A_{t-1})\phi_{t-1}, & t>\tau\\
% %     B \sim \textup{GP}(sx\exp(-rx), C_B) \\
% %     D \sim \textup{GP}(mx, C_D) \\
% %     \psi_t \stackrel{\tiny{\textup{iid}}}{\sim}\textup{lognorm}(\mu_\phi,V_\phi) \\
% %     \phi_t \stackrel{\tiny{\textup{iid}}}{\sim}\textup{lognorm}(\mu_\phi,V_\phi)\\
% %     C_B = \textup{Mat\`ern}(\sigma_B,\nu_B,\phi_B) \\
% %     C_D = \textup{Mat\`ern}(\sigma_D,\nu_D,\phi_D)   \\
% %     A_{1\ldots\tau},r,s,m,\textup{covariance parameters},\mu_\phi,\mu_\psi,V_\phi,V_\psi,V \sim \textup{priors},
% % \end{eqnarray*}
% % and I assumed $\tau$ fixed for purposes of the demo. The DAG schematic is shown in figure \ref{fig:ESSblowflymodel}.
% %
% % What happens is the following: when $B$ and $D$ have mean zero, the model burns in and mixes badly. When they have the mean functions parametrized above, the amplitude of their covariance params get very small; the model is preferring the simpler parametric version, since it can explain the data as well as the nonparametric version. This would make a great demo for both the importance of a flexible mean function and the natural parsimony of Bayesian statistics, but I can't even get the parametric version mixing and don't have time for week-long runs. We'll see what happens with this.
% %
% % \begin{figure}
% %     \centering
% %         \epsfig{file=figs/ESSblowfly.pdf,width=15cm}
% %     \caption{The model for Ellner, Seifu and Smith's \cite{ess} blowfly data.}
% %     \label{fig:ESSblowflymodel}
% % \end{figure}
% %
% % % \begin{figure}
% % %     \centering
% % %         \epsfig{file=figs/ESSBPosterior.pdf,width=10cm}
% % %         \epsfig{file=figs/ESSDPosterior.pdf,width=10cm}
% % %     \caption{caption}
% % %     \label{fig:ESSBD}
% % % \end{figure}
% % %
% % % \begin{figure}
% % %     \centering
% % %         \epsfig{file=figs/ESSmuphiPosterior.pdf,width=7cm}
% % %         \epsfig{file=figs/ESSVphiPosterior.pdf,width=7cm}
% % %         \epsfig{file=figs/ESSmupsiPosterior.pdf,width=7cm}
% % %         \epsfig{file=figs/ESSVpsiPosterior.pdf,width=7cm}
% % %     \caption{caption}
% % %     \label{fig:ESSphipsi}
% % % \end{figure}
% % %
% % % \begin{figure}
% % %     \centering
% % %         \epsfig{file=figs/ESSblowflyfit.pdf,width=15cm}
% % %     \caption{caption}
% % %     \label{fig:ESSfit}
% % % \end{figure}
% %


\subsection{Prototypical Networks for Small Datasets}
\label{models:protonet}

Prototypical Networks \cite{protonet} is a meta-learning model for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each new class. The ability of a algorithm to perform few-shot learning is typically measured by its performance on n-shot, k-way classification tasks. First a model is given a query sample belonging to a new, previously unseen class. Then, it’s also given a support set, S, consisting of n examples, each from k different unseen classes. Finally, the algorithm then has to determine which of the support set classes the query samples belong to.
Schemes for few shot classification tasks like Prototypical Networks can also be of use for training small datasets where all classes are known.

Prototypical Networks applies a compelling inductive bias in the form of class prototypes to achieve impressive few-shot performance. The key assumption is made is that there exists an embedding in which samples from each class cluster around a single prototypical representation which is simply the mean of the individual samples. This idea streamlines n-shot classification in the case of $n > 1$ as classification is simply performed by taking the label of the closest class prototype.

\subsection{DenseNet}

As our state of the art model we selected DenseNet because it can handle small datasets with low error rate\cite{pmlr-v80-pham18a}.

DenseNet \cite{densenet} works by concatenating the feature-maps of a convolutional block to the feature-maps of all the previous convolutional blocks and using this value as input for the next convolutional block. This way each convolutional block receives all the collective knowledge of the previous layers maintaining the global state of the network which can be accessed.

Convolutional networks construct informative features by fusion both spatial and chanel-wise information within local receptive fields at each layer. Squeeze and excitation blocks (SE block) \cite{Hu2017SqueezeandExcitationN} focus on the chanel-wise information used in the convolutional layers. SE blocks improve the quality of representations produced by the network by modeling the interdependency between channels to perform feature recalibration. SE blocks can be included in any model that uses convolutional layers to improve its performance at low computational cost. We added SE blocks to our DenseNet model to improve its performance.

\subsection{Data Augmentation}

Image data augmentation is a set of techniques that aim at artificially augmenting the amount of data that can be obtained from the images in the dataset. These techniques modify the images in the dataset  with a set of predefined operations to create new images that can be used to train a model.  In this manner, we can compensate  for the lack of  variability in a small dataset\cite{cubuk2019autoaugment}.


\section{Introduction}

An \emph{affine} transformation is a transformation of the form
\[
  T(x) = Ax + b
\]
The {\tt{}affine} module provides routines to create, apply, and
manipulate affine transformations on Euclidean 3-space.

\section{Interface}

\nwfilename{affine.nw}\nwbegincode{1}\sublabel{NWaff9-aff8-1}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-aff8-1}}}\moddef{affine.h~{\nwtagstyle{}\subpageref{NWaff9-aff8-1}}}\endmoddef
#ifndef AFFINE_H
#define AFFINE_H

\LA{}exported functions~{\nwtagstyle{}\subpageref{NWaff9-expI-1}}\RA{}

#endif /* AFFINE_H */
\nwnotused{affine.h}\nwendcode{}\nwbegindocs{2}\nwdocspar

We represent affine transformations by a 3-by-4 double
precision matrix in column-major format.  The leading
3-by-3 submatrix represents the linear part ($A$),
and the trailing column vector represents the translation
($b$).

The {\tt{}identity} function initializes the matrix to the
representation of the identity transform; $A = I$, $b = 0$.

\nwenddocs{}\nwbegincode{3}\sublabel{NWaff9-expI-1}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-expI-1}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NWaff9-expI-1}}}\endmoddef
void affine_identity(double* T);

\nwalsodefined{\\{NWaff9-expI-2}\\{NWaff9-expI-3}\\{NWaff9-expI-4}}\nwused{\\{NWaff9-aff8-1}}\nwendcode{}\nwbegindocs{4}\nwdocspar

There are several functions which modify an existing
transformation $T$ by composing it with a simple transformation
$S$ to get a new transformation $ST$.
In the {\tt{}translate} and {\tt{}translate{\char95}xyz} functions, $S$ is
a simple translation ($A = 0$, $b$ is specified by the arguments).
The {\tt{}rotate{\char95}ox}, {\tt{}rotate{\char95}oy}, and {\tt{}rotate{\char95}oz} functions
apply rotations by some fixed number of radians about the $x$,
$y$, and $z$ axes, respectively.  The rotations are oriented
according to the right hand rule.  The {\tt{}scale}, {\tt{}scale{\char95}x},
{\tt{}scale{\char95}y}, and {\tt{}scale{\char95}z} apply a scaling, either of all
variables or just of the individual components.

\nwenddocs{}\nwbegincode{5}\sublabel{NWaff9-expI-2}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-expI-2}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NWaff9-expI-1}}}\plusendmoddef
void affine_translate    (double* T, double* c);
void affine_translate_xyz(double* T, double x, double y, double z);
void affine_rotate_ox    (double* T, double ox);
void affine_rotate_oy    (double* T, double oy);
void affine_rotate_oz    (double* T, double oz);
void affine_scale        (double* T, double s);
void affine_scale_x      (double* T, double s);
void affine_scale_y      (double* T, double s);
void affine_scale_z      (double* T, double s);
\nwendcode{}\nwbegindocs{6}\nwdocspar

The {\tt{}compose} function composes two transformations
to get $S := TS$.

\nwenddocs{}\nwbegincode{7}\sublabel{NWaff9-expI-3}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-expI-3}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NWaff9-expI-1}}}\plusendmoddef
void affine_compose(double* T, double* S);

\nwendcode{}\nwbegindocs{8}\nwdocspar

There are several versions of the {\tt{}apply} transformation.
The basic {\tt{}apply} transformation computes $x := T(x)$,
but there is also {\tt{}apply{\char95}A} to apply just the linear
part of the transformation ($x := Ax$), and {\tt{}apply{\char95}AT}
to apply the transpose of the linear part ($x := A^T x$).
The basic {\tt{}apply} functions assume that the argument vector
is laid out contiguously in memory, but there are also
versions of the functions which assume the elements are
separated by a constant stride.

\nwenddocs{}\nwbegincode{9}\sublabel{NWaff9-expI-4}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-expI-4}}}\moddef{exported functions~{\nwtagstyle{}\subpageref{NWaff9-expI-1}}}\plusendmoddef
void affine_apply          (double* T, double* x);
void affine_apply_stride   (double* T, double* x, int stride);
void affine_apply_A        (double* T, double* x);
void affine_apply_A_stride (double* T, double* x, int stride);
void affine_apply_AT       (double* T, double* x);
void affine_apply_AT_stride(double* T, double* x, int stride);

\nwendcode{}\nwbegindocs{10}\nwdocspar

\section{Implementation}

\nwenddocs{}\nwbegincode{11}\sublabel{NWaff9-aff8.2-1}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-aff8.2-1}}}\moddef{affine.c~{\nwtagstyle{}\subpageref{NWaff9-aff8.2-1}}}\endmoddef
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include "affine.h"

\LA{}functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}\RA{}
\nwnotused{affine.c}\nwendcode{}\nwbegindocs{12}\nwdocspar

\nwenddocs{}\nwbegincode{13}\sublabel{NWaff9-fun9-1}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\endmoddef
void affine_identity(double* T)
\{
    memset(T, 0, 12*sizeof(double));
    T[0] = 1;
    T[4] = 1;
    T[8] = 1;
\}

\nwalsodefined{\\{NWaff9-fun9-2}\\{NWaff9-fun9-3}\\{NWaff9-fun9-4}\\{NWaff9-fun9-5}\\{NWaff9-fun9-6}\\{NWaff9-fun9-7}\\{NWaff9-fun9-8}\\{NWaff9-fun9-9}\\{NWaff9-fun9-A}\\{NWaff9-fun9-B}\\{NWaff9-fun9-C}}\nwused{\\{NWaff9-aff8.2-1}}\nwendcode{}\nwbegindocs{14}\nwdocspar

Note that the {\tt{}translate} function overwrites $T$ with
the composition of translation with $T$, not with the
composition of $T$ with the translation.  The same holds
for the rotations.

\nwenddocs{}\nwbegincode{15}\sublabel{NWaff9-fun9-2}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-2}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_translate(double* T, double* c)
\{
    T[9 ] += c[0];
    T[10] += c[1];
    T[11] += c[2];
\}

\nwendcode{}\nwbegindocs{16}\nwdocspar

\nwenddocs{}\nwbegincode{17}\sublabel{NWaff9-fun9-3}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-3}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_translate_xyz(double* T, double x, double y, double z)
\{
    T[9 ] += x;
    T[10] += y;
    T[11] += z;
\}

\nwendcode{}\nwbegindocs{18}\nwdocspar

The rotation functions obey the right-hand rule, which usually
confuses me.  That is why there is a sign reversal on the angle
for the rotation about the $y$ axis.  Since these are purely
linear transformations, we just need to transform each column
of $T$ in turn.

\nwenddocs{}\nwbegincode{19}\sublabel{NWaff9-fun9-4}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-4}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_rotate_ox(double* T, double ox)
\{
    double c = cos(ox);
    double s = sin(ox);
    int i;

    for (i = 0; i < 4; ++i) \{
        double y =  c*T[1] + s*T[2];
        double z = -s*T[1] + c*T[2];
        T[1] = y;
        T[2] = z;
        T += 3;
    \}
\}

\nwendcode{}\nwbegindocs{20}\nwdocspar

\nwenddocs{}\nwbegincode{21}\sublabel{NWaff9-fun9-5}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-5}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_rotate_oy(double* T, double oy)
\{
    double c =  cos(oy);
    double s = -sin(oy);
    int i;

    for (i = 0; i < 4; ++i) \{
        double x =  c*T[0] + s*T[2];
        double z = -s*T[0] + c*T[2];
        T[0] = x;
        T[2] = z;
        T += 3;
    \}
\}

\nwendcode{}\nwbegindocs{22}\nwdocspar

\nwenddocs{}\nwbegincode{23}\sublabel{NWaff9-fun9-6}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-6}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_rotate_oz(double* T, double oz)
\{
    double c = cos(oz);
    double s = sin(oz);
    int i;

    for (i = 0; i < 4; ++i) \{
        double x =  c*T[0] + s*T[1];
        double y = -s*T[0] + c*T[1];
        T[0] = x;
        T[1] = y;
        T += 3;
    \}
\}

\nwendcode{}\nwbegindocs{24}\nwdocspar

The {\tt{}scale} function scales everything in sight by the given constant.

\nwenddocs{}\nwbegincode{25}\sublabel{NWaff9-fun9-7}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-7}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_scale(double* T, double s)
\{
    int i;
    for (i = 0; i < 12; ++i)
        *T++ *= s;
\}

\nwendcode{}\nwbegindocs{26}\nwdocspar

The {\tt{}scale{\char95}x}, {\tt{}scale{\char95}y}, and {\tt{}scale{\char95}z} functions only scale
the appropriate rows.

\nwenddocs{}\nwbegincode{27}\sublabel{NWaff9-fun9-8}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-8}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_scale_x(double* T, double s)
\{
    int row = 0;
    \LA{}scale row~{\nwtagstyle{}\subpageref{NWaff9-sca9-1}}\RA{}
\}

void affine_scale_y(double* T, double s)
\{
    int row = 1;
    \LA{}scale row~{\nwtagstyle{}\subpageref{NWaff9-sca9-1}}\RA{}
\}

void affine_scale_z(double* T, double s)
\{
    int row = 2;
    \LA{}scale row~{\nwtagstyle{}\subpageref{NWaff9-sca9-1}}\RA{}
\}

\nwendcode{}\nwbegindocs{28}\nwdocspar

\nwenddocs{}\nwbegincode{29}\sublabel{NWaff9-sca9-1}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-sca9-1}}}\moddef{scale row~{\nwtagstyle{}\subpageref{NWaff9-sca9-1}}}\endmoddef
int i;
for (i = 0; i < 4; ++i, T += 3)
    T[row] *= s;
\nwused{\\{NWaff9-fun9-8}}\nwendcode{}\nwbegindocs{30}\nwdocspar

The {\tt{}compose} function does \emph{not} necessarily do a purely
linear transformation.  The composition $TS$ will have the form
\begin{eqnarray*}
  TS(x) & = & A_T (A_S x + b_S) + b_T \\
        & = & A_T A_S x + T(b_S)
\end{eqnarray*}
So we apply $A_T$ to the first three columns of $S$, and then
apply $T$ to the last column of $S$.

\nwenddocs{}\nwbegincode{31}\sublabel{NWaff9-fun9-9}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-9}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_compose(double* T, double* S)
\{
    int i;
    for (i = 0; i < 3; ++i) \{
        double y[3];

        y[0] = T[0]*S[0] + T[3]*S[1] + T[6]*S[2];
        y[1] = T[1]*S[0] + T[4]*S[1] + T[7]*S[2];
        y[2] = T[2]*S[0] + T[5]*S[1] + T[8]*S[2];

        S[0] = y[0];
        S[1] = y[1];
        S[2] = y[2];

        S += 3;
    \}

    affine_apply(T, S);
\}

\nwendcode{}\nwbegindocs{32}\nwdocspar

The {\tt{}apply}, {\tt{}apply{\char95}A}, and {\tt{}apply{\char95}AT} functions are
implemented as simple wrappers around their strided
counterparts.

\nwenddocs{}\nwbegincode{33}\sublabel{NWaff9-fun9-A}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-A}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_apply(double* T, double* x)
\{
    affine_apply_stride(T, x, 1);
\}

void affine_apply_stride(double* T, double* x, int stride)
\{
    double x0 = x[0*stride];
    double x1 = x[1*stride];
    double x2 = x[2*stride];

    x[0*stride] = T[0]*x0 + T[3]*x1 + T[6]*x2 + T[9 ];
    x[1*stride] = T[1]*x0 + T[4]*x1 + T[7]*x2 + T[10];
    x[2*stride] = T[2]*x0 + T[5]*x1 + T[8]*x2 + T[11];
\}

\nwendcode{}\nwbegindocs{34}\nwdocspar

\nwenddocs{}\nwbegincode{35}\sublabel{NWaff9-fun9-B}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-B}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_apply_A(double* T, double* x)
\{
    affine_apply_A_stride(T, x, 1);
\}

void affine_apply_A_stride(double* T, double* x, int stride)
\{
    double x0 = x[0*stride];
    double x1 = x[1*stride];
    double x2 = x[2*stride];

    x[0*stride] = T[0]*x0 + T[3]*x1 + T[6]*x2;
    x[1*stride] = T[1]*x0 + T[4]*x1 + T[7]*x2;
    x[2*stride] = T[2]*x0 + T[5]*x1 + T[8]*x2;
\}

\nwendcode{}\nwbegindocs{36}\nwdocspar

\nwenddocs{}\nwbegincode{37}\sublabel{NWaff9-fun9-C}\nwmargintag{{\nwtagstyle{}\subpageref{NWaff9-fun9-C}}}\moddef{functions~{\nwtagstyle{}\subpageref{NWaff9-fun9-1}}}\plusendmoddef
void affine_apply_AT(double* T, double* x)
\{
    affine_apply_AT_stride(T, x, 1);
\}

void affine_apply_AT_stride(double* T, double* x, int stride)
\{
    double x0 = x[0*stride];
    double x1 = x[1*stride];
    double x2 = x[2*stride];

    x[0] = T[0]*x0 + T[1]*x1 + T[2]*x2;
    x[1] = T[3]*x0 + T[4]*x1 + T[5]*x2;
    x[2] = T[6]*x0 + T[7]*x1 + T[8]*x2;
\}

\nwendcode{}

\nwixlogsorted{c}{{affine.c}{NWaff9-aff8.2-1}{\nwixd{NWaff9-aff8.2-1}}}%
\nwixlogsorted{c}{{affine.h}{NWaff9-aff8-1}{\nwixd{NWaff9-aff8-1}}}%
\nwixlogsorted{c}{{exported functions}{NWaff9-expI-1}{\nwixu{NWaff9-aff8-1}\nwixd{NWaff9-expI-1}\nwixd{NWaff9-expI-2}\nwixd{NWaff9-expI-3}\nwixd{NWaff9-expI-4}}}%
\nwixlogsorted{c}{{functions}{NWaff9-fun9-1}{\nwixu{NWaff9-aff8.2-1}\nwixd{NWaff9-fun9-1}\nwixd{NWaff9-fun9-2}\nwixd{NWaff9-fun9-3}\nwixd{NWaff9-fun9-4}\nwixd{NWaff9-fun9-5}\nwixd{NWaff9-fun9-6}\nwixd{NWaff9-fun9-7}\nwixd{NWaff9-fun9-8}\nwixd{NWaff9-fun9-9}\nwixd{NWaff9-fun9-A}\nwixd{NWaff9-fun9-B}\nwixd{NWaff9-fun9-C}}}%
\nwixlogsorted{c}{{scale row}{NWaff9-sca9-1}{\nwixu{NWaff9-fun9-8}\nwixd{NWaff9-sca9-1}}}%
\nwbegindocs{38}\nwdocspar

\nwenddocs{}


 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 % @File    : c:\Users\Administrator\Desktop\GNN\tutorial\sections\Graph-Structure.tex
 % @Date    : 2020-12-08 17:23:06
 % @Author  : RankFan
 % @Email   : 1917703489@qq.com
 % -----
 % Last Modified: 2020-12-14 16:03:43
 % Modified By: Rank_fan
 % -----
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Graph Types \& Structures}
     在CNN中，可以将图片转换为RGB的像素矩阵，这种数据可视为时序数据，其他的时序数据还包括语音数据等类型，同时现实中也存在多种信息网络数据，社交网络、金融关系、交通流量网络包括生物学中的蛋白质分子结构，都会涉及大量的高度关联数据。这些数据构成庞大的图，图数据库就是呈现和查询这些关联的做好的方式。

     从数学定义上来说，图是{\bf 节点(vertex，或者 node) }和{\bf 边(Edge)}的集合，在一张图中，一个节点代表一个实体的特征，边，就是关联这些节点的关系 (relation) ，通常这些节点的关系我们会想办法去用数字描述。

     \begin{mydef}[Graph]
        A graph G is a tuple  $ G = (V, E)  $ of vertices $V$ and edges $E$. For  undirected graphs, edges are subsets of cardinality two of the vertices, so each edge is of the form $(u, v)$ with $u, v \in V$. For directed graphs, the order of the edges in the tuple $(u, v) $is
        relevant to indicate the direction of the edge. If not mentioned otherwise, we will assume
        that a graph is undirected and has no self-loops, i.e. edges for which $u = v$.
     \end{mydef}

     \begin{mydef}[Degree]
        The degree of a vertex $v$ of an undirected graph $G = (V, E)$ is the number of vertices that are connected to $v$ by means of an edge, i.e. $ deg (v) :
        \left |  \left \{ u|\left ( u,v \right )  \in E,u \neq v \right \} \right |  $ For directed graphs, a vertex has an {\bf in-degree} and an {\bf out-degree},
        depending on the direction of the edges.
     \end{mydef}

     A graph G = (V, E) may also contain labels or attributes, for example in the form of node labels, edge labels, or edge weights. This is known as an attributed graph.

     \begin{mydef}[Attributed graph]
        {\rm{An attributed graph}} is a graph that has either labels or attributes on the nodes and/or edges. When present, labels are each assumed to be defined
        over a common alphabet, $ \sum_V $ for nodes and $\sum_E$ for edges, with a function, $I_V$ for nodes and $I_E$ for edges, to assign each entity its label. We thus have

        \begin{equation}
            \textstyle
            \begin{split}
                I_{V} : V \longrightarrow \textstyle  \sum_{V} \\
                I_{E} : E \longrightarrow \textstyle  \sum_{E}
            \end{split}
        \end{equation}

        and both of these functions need to be total. A graph with additional attributes for verticesand or edges has attribute functions
        \begin{equation}
            \textstyle
            \begin{split}
                \mathcal{A_{V}}  : V \longrightarrow \textstyle  \mathbb{R}^{d} \\
                \mathcal{A_{V}} : E \longrightarrow \textstyle  \mathbb{  R } ^{d}
            \end{split}
        \end{equation}

        that are typically assumed to be real-valued, i.e. d = 1. Scalar-valued edge attributes areoften also referred to as weights, with the tacit assumption that the values refer to the strength of a specific connection.

        Since a graph defines a connectivity for each of its vertices, its edges induce sequence. for visiting them. These sequences have specific names, depending on their properties.
     \end{mydef}

     \begin{mydef}[Walks, paths, and cycles]
        A sequence of k nodes  $ v_1 , v_2 , \cdots , v_k $ of the vertices of a graph G is called a walk of length k − 1 if the edge between two consecutive vertices exists.
        Vertices of a walk are allowed to repeat.
        If, however, node repetition is not allowed, one typically refers to the node sequence as a path, the
        adjective directed often being added in the case of directed graphs.
        Special consideration is given to cycles, i.e. walks of length k − 1 for which $v_1 = v_k$ .
     \end{mydef}

     \begin{mydef}[Random walks]
        A walk in a graph is referred to as a random walk if the next vertex (or edge) is picked in a probabilistic manner.
        Having picked a start node at random,a typical choice, for example, would be to pick any outgoing edge of the node with uniform
        probability (in case of unweighted graphs), or with a probability proportional to its weight.
     \end{mydef}

     \begin{mydef}[Connected graph]
        A graph is said to be connected if a walk between all pairs of nodes exists.
        \begin{myRemark}
            Specifically, in a fully connected graph, each pair of nodes is connected by an edge. If a graph is not connected, the set of its nodes can be partitioned using an equivalence relation u ~ v if and only if a walk between u and v exists. The equivalence classes under this relation are called connected components.

            Likewise, paths can also be used to assess distances in a graph. This viewpoint is oftenhelpful when approximating high-dimensional manifolds through graphs, and it is possible to give bounds on the dissimilarity of graph-based distances and geodesic distances of the manifold .
        \end{myRemark}
     \end{mydef}

     \begin{mydef}[Shortest paths and distances]
        Given two vertices u and v of a graph that are in the same connected component, among all the paths connecting them, there is at least one shortest path that has the minimum number of vertices out of all other paths connecting the two vertices.
     \end{mydef}

     \begin{mydef}[k-hop neighbourhood of a vertex]
        Given a vertex v of a graph G and $ k \in \mathbb{N}_{>0} $ its k-hop neighbourhood  $N^{(k)} (v)$ is defined as all the vertices in G that are reachable
        in at most k steps, which includes v, assuming uniform edge weights.   For example, $N^{(1)} (v)$ is just the set of vertices that are connected to v by an edge and v.
        \begin{myRemark}
            This definition can be connected to the idea of a “ball” in metric spaces by observing that each k-hop neighbourhood of a vertex v induces a subgraph of the original graph G. For increasing values of k, these induced subgraphs are nested—and for k sufficiently large, the original graph G is obtained. This concept will play an important role later on when we define graph kernels that operate at multiple scales,
        \end{myRemark}
     \end{mydef}

     \begin{mydef}[Adjacency matrix A]
        A graph is often represented through its adjacencymatrix A. A graph with n vertices will thus be represented by an n × n binary matrix whose entry Aij = 1 if the ith and jth vertex of the graph are connected by an edge.
        \begin{myRemark}
            The adjacency matrix is symmetrical for undirected graphs, whereas for directed graphs $A_{ij}$ and $A_{ji}$ can be different depending on the edge structure. Furthermore, if edge weights are available,
            i.e. $\mathcal{A_{E} (\cdot)} $ exists and d = 1, it is also possible to derive a weighted variant of the adjacency
            matrix by setting Aij to the corresponding edge weight.
        \end{myRemark}
     \end{mydef}

     \begin{mydef}[Graph Laplacian]
        Let A be the adjacency matrix of a graph G = (V, E).If weights are available, each entry Aij thus consists of the weight of the corresponding edge. Furthermore, let D be the degree matrix of G. This diagonal matrix contains the degree of each vertex $v \in  V $in the unweighted case.  For weighted graphs, each entry consists of the sum of all edge weights of all edges that are incident on the corresponding vertex.Laplacian ,matrix will be a symmetric positive semi-definite matrix for undirected graphs.

        \begin{equation}
            L := D − A
        \end{equation}
     \end{mydef}
\begin{figure}[ht!]
    \centering
    \includegraphics[scale = 0.5]{def_graph.png}
\end{figure}
\begin{figure}[htb!]
    \centering
    \includegraphics[scale = 0.5]{label_property.png}
    \caption{属性图}
\end{figure}


\documentclass[main.tex]{subfiles}
\begin{document}

\section{Multipolar expansion and STF formalism}

\marginpar{Monday\\ 2021-6-7, \\ compiled \\ \today}

Suppose we have a static potential, which obeys \(\triangle \phi = 4 \pi \rho \) for some static source with energy density \(\rho \) which is localized within a radius \(R\).
The procedure for solving this kind of problem, like in electromagnetism, is through Green's functions.
If we take a function \(G\) such that \(\triangle G (x-y) = 4\pi \delta (x-y)\), then \(G(x-y) = 1 / \abs{x - y}\); then
%
\begin{align}
\phi = \triangle^{-1} \rho = \int \dd[3]{y} G(x-y) \rho (y) = \int \dd[3]{y} \frac{\rho (y)}{\abs{x-y}}
\,.
\end{align}

We can make this formal solution clearer by introducing spherical harmonics and expressing the solution as a series of multipoles of the source:
%
\begin{align}
L^2 Y_{\ell m} = - \ell (\ell + 1) Y_{\ell m}
\,
\end{align}
%
are eigenfunctions of \(L^2\), where
%
\begin{align}
\triangle = r^{-2} \partial_{r} \qty(r^2 \partial_{r}) + r^{-2} L^2 (\theta , \varphi )
\,.
\end{align}

In terms of these,
%
\begin{align}
\frac{1}{\abs{x-y}} = \sum _{\ell = 0} \sum _{m=- \ell}^{\ell}
\frac{1}{2 \ell + 1} \frac{r^{\prime \ell}}{r^{\ell + 1}}
Y^{*}_{\ell m} (\theta ', \varphi ') Y_{\ell m} (\theta , \varphi )
\,,
\end{align}
%
where \(r' = \abs{y}\) and \(r = \abs{x}\), and similarly for the angles.

For the exterior solution, a function in the form \(Y_{\ell m} r^{- (\ell +1)}\) is a solution of the equation \(\triangle (Y_{\ell m} r^{- (\ell +1)}) = 0\).
Therefore, the exterior solution reads
%
\begin{align}
\phi^{\text{ext}} = \sum _{\ell= 0} \sum _{m=-\ell}^{\ell} \frac{Q_{\ell m}}{2 \ell + 1} \frac{Y_{\ell m}}{r^{\ell + 1}}
\,,
\end{align}
%
a linear combination with coefficients \(Q_{\ell m}\).

The coefficients are determined by \textbf{matching} the interior solution:
%
\begin{align}
\triangle^{-1} \rho = \sum _{\ell} \sum _{m}
\int \dd[3]{y} \frac{r^{\prime \ell}}{2 \ell + 1}
\rho (y) Y^{*}_{\ell m} (\theta ', \varphi ') \frac{Y_{\ell m}}{r^{\ell + 1}}
\,.
\end{align}

Therefore, the coefficients
%
\begin{align}
Q_{
    \ell m
}
= \int \dd[3]{y}
r^{\prime \ell} Y^{*}_{\ell m} \rho
\,
\end{align}
%
are the \textbf{multipoles}.
For example, with \(\ell = m = 0\) we recover \(Q_{00} = \int \dd[3]{y} \rho \), the ``mass''.

This procedure can be applied in Cartesian coordinates, which leads us to introduce the STF tensors.

We start by Taylor expanding the Green function in \(y\):
%
\begin{align}
\frac{1}{\abs{x-y}} = f(y) = f(0)
+ y^{i} \eval{\pdv{f}{y^{i}}}_{0}
+ y^{i}y^{j} \eval{\pdv{f}{y^{i}}{y^{j}}}_{0} + \order{y^3}
\,.
\end{align}

Now,
%
\begin{align}
\eval{\pdv{f}{y^{i}}}_{y=0} = \pdv{y^{i}} \eval{\qty( \frac{1}{\abs{x-y}})}_{y=0}
= - \pdv{x^{i}} \eval{\qty( \frac{1}{\abs{x-y}})}_{y=0}
\,,
\end{align}
%
which generalizes to
%
\begin{align}
\frac{\partial^{L} f}{y^{i_1} \dots y^{i_L}} = (-)^{L} \frac{\partial^{L}}{x^{i_1}\dots x^{i_L}} \frac{1}{\abs{x}}
\,,
\end{align}
%
so we can rewrite \(1/ \abs{x-y}\) in terms of derivatives of \(1/ \abs{x}\):
%
\begin{align}
\frac{1}{\abs{x-y}} = \sum _{\ell=0}
\frac{(-)^{\ell}}{\ell!} y^{i_1} \dots y^{i_L} \underbrace{\pdv{x^{i_1}} \dots
\pdv{x^{i_L}} \frac{1}{\abs{x}}}_{\text{STF}}
\,,
\end{align}
%
and the highlighted terms are STF in the sense that they are
\begin{enumerate}
    \item symmetric in the indices \(i_1 \dots i_L\);
    \item trace-free, in that the contraction of that object with \(\delta^{i_a i_b}\) is a bunch of derivatives acting on \(\triangle (1/\abs{x}) = 0\).
\end{enumerate}

If we plug this into the initial equation, we get
%
\begin{align}
\triangle^{-1} \rho &=
\int \dd[3]{y} \rho
\sum _{\ell} \frac{(-)^{\ell}}{\ell!}
y^{i_1} \dots y^{i_L}\pdv{x^{i_1}} \dots
\pdv{x^{i_L}} \frac{1}{\abs{x}}  \\
&=
\sum _{\ell=0} \frac{(-)^{\ell}}{\ell!}
\underbrace{\int \dd[3]{y} \qty( \rho y^{i_1} \dots y^{i_L})}_{\text{Cartesian multipoles of \(\rho \)}}
\pdv{x^{i_1}} \dots \pdv{x^{i_L}} \frac{1}{\abs{x}}
\,.
\end{align}

These multipoles read
%
\begin{align}
Q_{i_1 \dots i_L} = \int \dd[3]{y} \rho y^{<i_1} \dots y^{i_L>}
\,,
\end{align}
%
where the index brackets \(<>\) denote trace-free symmetrization --- this can be done since any traceful or antisymmetric part vanishes by contraction with the STF object.

There must be a parallel between the spherical harmonics \(Y_{\ell m}\) and this object \(y^{<i_1} \dots y^{i_L>}\)!

We can construct such an object like
%
\begin{align}
T^{<ij>} = T^{(ij)} - \frac{1}{3} T \delta^{ij}
\,.
\end{align}

In order to get the relation we seek, we need the relation
%
\begin{align}
\partial_{i} r = \partial_{i} (x^{j} x_j)^{1/2}
= \frac{x_{i}}{r} = n_i
\,,
\end{align}
%
while
%
\begin{align}
\partial_i n_j = \frac{1}{r} (\delta_{ij} - n_i n_j)
\,.
\end{align}

With these, we find that
%
\begin{align}
\partial_{i} \frac{1}{r} = - \frac{n_i}{r^2}
\qquad \text{and} \qquad
\partial_{i} \partial_{j} \frac{1}{r} = - \frac{3}{r^3} n^{<i} n^{j>}
\,,
\end{align}
%
and in general
%
\begin{align}
\partial_{i_1} \dots \partial_{i_L} \frac{1}{r}
= (-)^{\ell} \frac{(2 \ell -1)!!}{r^{\ell+1}} n^{<i_1} \dots n^{i_\ell>}
\,,
\end{align}
%
therefore
%
\begin{align}
y^{i} = r' n^{\prime i}
\,,
\end{align}
%
so
%
\begin{align}
\frac{1}{\abs{x-y}} = \sum _{\ell} \frac{(2 \ell - 1)!!}{\ell!} \frac{r^{\prime \ell}}{r^{\ell + 1}} n^{\prime i_1} \dots n^{\prime i_\ell}
n^{<i_1} \dots n^{i_\ell>}
\,.
\end{align}

This means that the Taylor expansion above can also be written as
%
\begin{align}
\triangle^{-1} \rho =
\sum _{\ell=0} \frac{(2 \ell - 1)!!}{\ell!} \frac{1}{r^{\ell + 1}}
Q_{i_1 \dots i_\ell} n^{<i_1} \dots n^{i_\ell>}
\,.
\end{align}

Why do we use unit vectors?
We want a connection between the spherical coordinates and the Cartesian ones.

It is a theorem that the functions
%
\begin{align}
F_{\ell} (\hat{n}) = Q_{i_1 \dots i_\ell} n^{<i_1} \dots n^{i_\ell>}
\,
\end{align}
%
are eigenfunctions of \(L^2\) (the angular part of \(\triangle\)) with eigenvalues \(\Lambda = - \ell (\ell + 1)\).
The proof of this can be found in the notes --- it is a direct calculation.

This means that there exist some cartesian tensors
%
\begin{align}
\mathcal{Y}^{\ell m}_{i_1 \dots i_\ell}
\,
\end{align}
%
such that
%
\begin{align}
Y_{\ell m} (\theta , \varphi ) = \mathcal{Y}^{ \ell m}_{i_1 \dots i_\ell} n^{<i_1} \dots n^{i_\ell>}
\,.
\end{align}

The \(\theta , \varphi \) dependence is contained inside of \(\hat{n} = \hat{n} ( \theta , \varphi )\).

What we have found is an alternative way of representing spherical harmonics and rotations.

It is a theorem that the functions \(\mathcal{Y}^{\ell m}_{i_1 \dots i_\ell}\) form a basis for rank-\(\ell\) STF tensors:
%
\begin{align}
T_{i_1 \dots i_\ell} = \sum _{m= - \ell}^{\ell} T_{\ell m}
\mathcal{Y}^{\ell m}_{i_1 \dots i_\ell}
\,.
\end{align}

The components \(T_{\ell m}\) are called \textbf{spherical components} of \(T_{i_1 \dots i_2}\).

Some useful properties are:
%
\begin{align}
T_{i_1 \dots i_\ell} n^{i_1} \dots n^{i_\ell} =
\sum _{m=-\ell}^{\ell} T_{\ell m} Y_{\ell m}
\,;
\end{align}
%
the components are calculated like
%
\begin{align}
T_{\ell m} = \frac{4 \pi \ell!}{(2 \ell + 1)!!} T_{i_1 \dots i_\ell}
\qty(\mathcal{Y}^{i_1 \dots i_\ell}_{\ell m})^{*}
\,.
\end{align}

Under rotations, \(T_{\ell m}\) transform like \(Y_{\ell m}^{*}\).
This is because a change in \(Y_{\ell m}\) must be compensated.

In summary, rank \(\ell\) STF tensors have \(2 \ell + 1\) components and are an irreducible representation of \(SO(3)\).

In multi-index notation we have
%
\begin{align}
\partial_{L} \frac{1}{r} \propto \frac{1}{r^{\ell + 1}} n^L
\,.
\end{align}

See the notes for the generalization to the full \(h_{ij}\) in linear theory.
Do the green exercise:

In order to move to GR from linear theory we need to substitute \(T_{\mu n }\) with the Isaacson tensor \(\tau_{\mu \nu }\).

% TODO: find and replace < with \langle

\end{document}


\documentclass{scrartcl}

\usepackage[ngerman]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\usepackage{framed}
\usepackage{listings}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{tikz-qtree}
\usetikzlibrary{shapes,automata,arrows}
\usepackage{pgfplots}
\usepackage{color}

\begin{document}

%---

\subsection{Picture}

{
\setlength{\unitlength}{1cm}
\begin{picture}(2,2)
    \put(0,0){\vector(4,0){3.3}}
    \put(0,0){\vector(0,4){3.3}}
    \put(3.4,-0.1){$x$}
    \put(-0.1,3.4){$y$}
    \multiput(0,0)(1,0){4}{\line(0,-1){0.1}}
    \multiput(0,0)(0,1){4}{\line(-1,0){0.1}}

    \put(0,2){\vector(1,-1){2}}
    \put(2,0){\vector(1,1){1}}
    \put(2,2){\circle*{1}}
    \put(2,2){\circle{2}}
\end{picture}
}

%----

\subsection{Tikz}

\begin{tikzpicture}
\draw[->] (0, 0) -- (0, 4);
\draw[->] (0, 0) -- (4, 0);
\node at (0, 4.2) {$y$};
 \node at (4.125, -0.25) {$x$};
 \foreach \x in {0, 1, 2, 3}
 {
  \draw (\x,0) -- (\x,-0.2);
  \node at (\x,-0.75) {$\x$};
  \draw (0, \x) -- (-0.2, \x);
  \node at (-0.5, \x-0.25) {$\x$};
 }
\coordinate (m) at (2,2);
\draw (m) circle (0.5);
\draw[fill] (m) circle (0.25);
\draw[->] (0, 2) -- (2, 0);
\draw[->] (2, 0) -- (3, 1);
\end{tikzpicture}

%---

\subsection{Automata}

%\usepackage{tikz}
%\usetikzlibrary{automata, arrows}

Graph des regul\"aren Ausdrucks \texttt{(ab*|b|ca)}

\begin{tikzpicture}[shorten >=1pt,
    node distance=2.2cm, >=stealth, thick, auto, bend angle=45]
  \tikzstyle{every state}=[draw=blue!50, very thick, fill=blue!20]

  \node[state, initial]      (A)                                    {$S_0$};
  \node[state, accepting]    (B) [below of=A]                       {$S_1$};
  \node[state, accepting]    (C) [right of=B, node distance=1.25cm] {$S_2$};
  \node[state]               (D) [below of=B]                       {$S_3$};

  \path[->] (A) edge                             node {$a$} (B)
            (B) edge [loop below]                node {$b$} ()
            (A) edge                             node {$b$} (C)
            (A) edge [bend right, bend angle=45] node {$c$} (D)
            (D) edge [bend right]                node {$a$} (C);
\end{tikzpicture}

%\usepackage{tikz}
%\usetikzlibrary{shapes,arrows}

\subsection{Automata 2}

\tikzstyle{block} = [rectangle, draw,
    text width=8em, text centered, rounded corners, minimum height=4em]
	\tikzstyle{line} = [draw, thick, -> ] %, -latex']
 	\tikzstyle{cloud} = [ellipse, draw, node distance=4cm,
     minimum height=2em]

	\begin{tikzpicture}[node distance = 2cm, auto]
    % Place nodes
    	\node [cloud] (inhalt) {\small \texttt{inhalt.tex}};
     	\node [block, left of=inhalt, above of=inhalt] (handout)
            {\small \texttt{handout.tex}};
    	\node [block, right of=inhalt, above of=inhalt] (pres)
            {\small \texttt{presentation.tex}};
    	\path [line] (inhalt) -- (pres);
    	\path [line] (inhalt) -- (handout);
	\end{tikzpicture}

%---

\subsection{trees}

\begin{tikzpicture}
\Tree
[.S
    [.NP [.Det the ] [.N cat ] ]
    [.VP [.V sat ]
        [.PP
            [.P on ]
            [.NP [.Det the ] [.N mat ] ]
        ]
    ]
]
\end{tikzpicture}

%---

\subsection{Pythagoras}

\begin{tikzpicture}
    % for a, A(1, 1.5) top right, side length = 1
    \fill [red!15]   (0,0) rectangle (1,-1);
    % for b, B(0, 0) bottom left, side length = 1.5
    \fill [green!15] (1,0) rectangle (2.5,1.5);

    % length of c, C(1, 0) bottom right
    \pgfmathsetmacro{\lengthC}{(sqrt(1^2+1.5^2))}
    % angle between a and c
    \pgfmathsetmacro{\myAngle}
        {90 - acos((\lengthC^2 + 1^2 - 1.5^2) / (2*1.8*1))}
    \begin{scope}[rotate=-\myAngle]
        \fill [blue!15] (0,0) rectangle (-\lengthC,\lengthC); % for c
    \end{scope}

    % labels
    \draw [gray,decorate,decoration={brace,amplitude=5pt}]
          (1,0) -- (0,0)
    node [black,midway,below=3pt] {\footnotesize $a$};
    \draw [gray,decorate,decoration={brace,amplitude=5pt}]
          (1,1.5) -- (1,0)
    node [black,midway,right=3pt] {\footnotesize $b$};
    \draw [gray,decorate,decoration={brace,amplitude=5pt}]
          (0,0) -- (1,1.5)
    node [black,midway,left=1.5pt,yshift=6pt] {\footnotesize $c$};
\end{tikzpicture}

\subsection{Trigonometrie}

\begin{tikzpicture}[scale=2,cap=round]
  % Local definitions
  \def\costhirty{0.8660256}
  % Colors
  \colorlet{anglecolor}{orange}
  \colorlet{sincolor}{red}
  \colorlet{tancolor}{green!80!black}
  \colorlet{coscolor}{blue}
  % Styles
  \tikzstyle{axes}=[]
  \tikzstyle{important line}=[very thick]
  \tikzstyle{information text}=[]
  % The graphic
  \draw[style=help lines,step=0.5cm] (-1.4,-1.4) grid (1.4,1.4);
  \draw (0,0) circle (1cm);
  \begin{scope}[style=axes]
    \draw[->] (-1.5,0) -- (1.5,0) node[right] {$x$} coordinate(x axis);
    \draw[->] (0,-1.5) -- (0,1.5) node[above] {$y$} coordinate(y axis);
    \foreach \x/\xtext in {-1, 1}
       \draw[xshift=\x cm] (0pt,1pt) -- (0pt,-1pt) node[below,fill=white] {$\xtext$};
    \foreach \y/\ytext in {-1, 1}
       \draw[yshift=\y cm] (1pt,0pt) -- (-1pt,0pt) node[left,fill=white] {$\ytext$};
  \end{scope}
  \filldraw[fill=orange!20,draw=anglecolor] (0,0) -- (3mm,0pt) arc(0:30:3mm);
  \draw (15:2mm) node[anglecolor] {$\alpha$};
  \draw[style=important line,sincolor]
    (30:1cm) -- node[left=1pt,fill=white] {$\sin \alpha$} (30:1cm |- x axis);
  \draw[style=important line,coscolor]
    (30:1cm |- x axis) -- node[below=2pt,fill=white] {$\cos \alpha$} (0,0);
  \draw[style=important line,tancolor] (1,0) -- node[right=1pt,fill=white] {
    $\displaystyle \tan \alpha \color{black}=
    \frac{{\color{sincolor}\sin \alpha}}{\color{coscolor}\cos \alpha}$}
    (intersection of 0,0--30:1cm and 1,0--1,1) coordinate (t);
  \draw (0,0) -- (t);
  \draw[xshift=3.2cm]
    node[text width=3cm]
     {\small SeitenverhÃ¤ltnisse im rechtwinkligen Dreieck:
      \[\sin \alpha = \frac{\text{Gegenkathete}}{\text{Hypothenuse}}\]
      \[\cos \alpha = \frac{\text{Ankathete}}{\text{Hypothenuse}}\]
      \[\tan \alpha = \frac{\text{Gegenkathete}}{\text{Ankathete}}\]};
\end{tikzpicture}

\subsection{PGFPlots}

\begin{tikzpicture}
    \begin{axis}[width=190pt,axis x line=middle, axis y line=center, tick align=outside]
        \addplot+[mark=none, smooth] (\x,{1/10*\x^2});
		\addplot+[mark=none, smooth] (\x,{sin(\x r)});
    \end{axis}
\end{tikzpicture}

\end{document}


 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage {algorithmic}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}
\newtheorem{lemma}[theorem]{Lemma}
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\newtheorem{claim}{Claim}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams' lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 23} }

\begin{centering}
\section*{Single-Source Shortest Paths with Negative Edge Weights}
\end{centering}

\section{Bellman-Ford Algorithm}
In this section, we study the Bellman-Ford algorithm that solves the single source shortest paths problem on graphs with edges with potentially negative weights. Given a directed graph $G = (V, E)$ with edge weights given by $w(x, y )$ for $(x, y ) \in E$, we want to compute the shortest path distances $d(s, v )$ from source $s$ for all $v \in V$ . More specifically, the Bellman-Ford algorithm:

\begin{itemize}
  \item Detects a negative cycle if it exists and is reachable from $s$, or
  \item Computes the shortest path distances $d(s, v )$ for all $v \in V$
\end{itemize}

Note $\pi(\cdot)$ is used to store the shortest paths found and $\pi(v )$ represents the predecessor of $v$ on the shortest path from $s$ to $v$.

\textbf{NOTE}: This version of Bellman-Ford is a bit different than the one we presented in class! As mentioned in class, we changed it up slightly to be more in line Dynamic Programming. However, the analysis is basically the same. We'll analyze the above version here.

\begin{algorithm}
\caption{Bellman-Ford Algorithm}
\label{alg:1}
\begin{algorithmic}
\STATE $d[v] \gets \infty, \forall v \in V$ \texttt{// set initial distance estimates}
\STATE \texttt{to maintain paths: set } $\pi(v) \gets $nill for all $v, \pi(v)$ represents the predecessor of $v$
\STATE $d[s] \gets 0$ \texttt{// set distance to start node trivially as 0}
\FOR{$i$ from $1 \to n - 1$}
  \FOR{$(u,v) \in E$}
    \STATE $d[v] \gets \min\{d[v], d[u] + w(u,v) \}$ \texttt{// update the distance estimates for v}
    \STATE \texttt{to maintain paths, if } $d[v]$ \texttt{changes, then } $\pi(v) \gets u$
  \ENDFOR
\ENDFOR
\STATE \texttt{// Negative Cycle Step}
\FOR{$(u,v) \in E$}
  \IF{$d[v] > d[u] + w(u,v)$}
    \RETURN "Negative Cycle" \texttt{// negative cycle detected}
  \ENDIF
\ENDFOR
\RETURN $d[v] \forall v \in V$
\end{algorithmic}
\end{algorithm}

For an example run of the Bellman-Ford algorithm, please refer to the lecture slides or CLRS.

The total runtime of the Bellman-Ford algorithm is $O(mn)$. In the first for loop, we repeatedly update the distance estimates $n - 1$ times on all $m$ edges in time $O(mn)$. In the second for loop, we go through all $m$ edges to check for negative cycles in time of $O(m)$.

We prove the correctness of the Bellman-Ford algorithm in two steps:

\begin{claim}[If there is a negative cycle reachable from $s$, then the Bellman-Ford algorithm detects and reports ``Negative Cycles'']
\begin{proof}

For the sake of contradiction, suppose there exists a negative cycle $C$ reachable from the source $s$ and the Bellman-Ford algorithm does not report ``Negative Cycles''. Assume $C$ contains nodes $v_1, v_2, \cdots , v_k$ with edges $(vi , vi+1)$ for $i = 1, \cdots , k$ such that $\sum^{k}_{i=1} w(vi , vi+1) < 0$, where $v_k+1 = v_1$. See Figure \ref{fig:cycle}. Let $d[\cdot]$ be the distance estimates determined in the first for loop of the algorithm. Since $C$ is reachable from $s$, there is a path from $s$ to $v_1$ and to all nodes on $C$. In particular, there exist simple paths, i.e., paths without cycles, of at most $n - 1$ edges to the nodes of $C$. In the first for loop, the edges on each such simply path get relaxed in order and consequently, $d[vi ]$ will be some finite number less than $\infty$ for $i = 1, \cdots , k$. Since the Bellman-Ford algorithm does not report ``Negative Cycles'' in the second for loop, it must be that $d[vi+1] \leq d[vi ] + w(vi , vi+1)$ for $i = 1, \cdots , k$. Adding the inequalities, we obtain
$$
\sum_{i=1}^k d[v_{i+1}] \leq \sum_{i=1}^k d[v_i] + w(v_i, v_{i+1})
$$

\begin{figure}
\centering
\includegraphics[scale=0.5]{Cycle.png}
\caption{A negative cycle reachable from source $s$}
\label{fig:cycle}
\end{figure}

As we are summing over the cycle $C$, the terms $\sum_{i=1}^k d[v_{i+1}]$ and $\sum_{i=1}^k d[v_i ]$ are equal and can be cancelled. It follows that $0 \leq \sum_{i=1}^k w(vi , vi+1)$. This contradicts that $C$ is a negative cycle.

\end{proof}
\end{claim}

In the next claim, we show that if the graph has no negative cycles reachable from the source, then the Bellman-Ford algorithm returns the correct shortest path distances.

\begin{claim}[If G has no negative cycles reachable from $s$, then $d[v\text{]} = d(s, v ), \forall v \in V$]

\begin{proof}

Let $d_k (v )$ be the value of $d[v ]$ after $k$ iterations of the first for loop. We prove by induction the statement that $d_k (v )$ is at most the minimum distance of a path from $s$ to $v$ with at most $k$ edges. Then, we will have $d_{n-1}(v ) = d[v ]$ for all node $v$ at termination. We'll argue below that if there is any path from $s$ to $v$ , then there is some shortest path with at most $n - 1$ edges, so this means
$$
\text{dist}(s, v ) \leq d[v ] = d_{n-1}(v ) \leq \text{ minimum cost of a path with at most } n - 1 \text{ edges } = \text{dist}(s, v )
$$

Thus, everything in the above inequality chain is equal, and in particular $d[v ]$ is equal to the distance from $s$ to $v$ . Above, we used the fact that $d[v ]$ is an over-estimate on dist$(s, v )$, which follows from our analysis of Dijkstra's algorithm.

We now argue that if there is a path from $s$ to $v$ , then there exists a shortest path from $s$ to $v$ has at most $n - 1$ edges. If a shortest path has a cycle, the cycle cannot be negative and we can remove it and improve its total distance. If the cycle has a positive weight, removing the cycle will strictly improve the shortest path's distance. If the cycle has zero weight, we can ignore the cycle. Hence, we can assume that shortest paths are simple, that is, do not have cycles.

\textit{Base Case}: When $k = 0$, the distance estimates have been just initialized. So, $d_0(v ) = \infty$ if $v \neq s$. Furthermore, $d_0(s) = 0 = d(s, s)$, which is the minimum distance of length-$0$ paths from $s$ to $s$. The statement is satisfied for the base case.

\textit{Inductive Step}: Assume that $d_{k-1}(v )$ is at most the minimum distance of a $s \to v$ path on at most $k - 1$ edges for all $v$.

Consider $v \neq s$. Let $P$ be a shortest simple $s \to v$ path on at most $k$ edges. Let $u$ be the node just before $v$ on $P$, and let $Q$ be the sub-path of $P$ from $s$ to $u$. The path $Q$ would have at most $ k - 1 $ edges and is a shortest path from $s$ to $u$ with at most $k - 1$ edges, since sub-paths of shortest paths are also shortest paths. By the inductive hypothesis, $Q$ has cost at most $d_{k-1}(u)$. In the $k$-th iteration, we update $d_k (v )$ such that $d_k (v ) \leq d_{k-1}(u) + w(u, v ) \leq w(Q) + w(u, v ) = w(P)$. The induction is complete, and the claim is proved.

\end{proof}
\end{claim}

\section{Amortized Time}
Let's return to the Fibonacci heaps that we only very briefly mentioned above.

Note the runtimes listed for the operations of Fibonacci heaps are not worst-case runtimes. Instead, they are, what we call amortized runtimes. We say an operation on a data structure takes amortized $t(n)$ time if starting from an empty data structure, performing the operation
$L$ times takes $O(L \times t(n))$ time in total. This means the runtime of the operation is $O(t(n))$ when averaged over the sequence of $L$ instances of the operation. Each individual operation call may take much more than $t(n)$ time, but this is compensated by many cheap operation calls (that take much less than $t(n)$ time).

We analyze the amortized cost of incrementing a binary counter by one when the count is represented in binary. Consider a b-bit counter which starts at $0$ (i.e. b $0$'s). In each increment operation, we update the counter's bits correspondingly by flipping some bits from $0$ to $1$, or vice versa.

Some of the increment operations may take $\Omega(b)$ time. For example, an increment operation can require carrying $b$ bits:

\begin{align*}
1111111& \\
+ 1& \\
= 10000000&
\end{align*}

Other increments may take $O(1)$ time.
\begin{align*}
10000000& \\
+ 1& \\
=10000001& \\
\end{align*}

All this said, we can show the amortized cost of the increment operation on a binary counter is $O(1)$. Even though some increments take time linear in the number of bits, if we do $n$ increment operations to the counter starting from the all $0$s, each operation takes $O(1)$ time on average.

\begin{claim}{The total time to increment a binary counter $n$ times is $O(n)$}

We use what is known as the \textit{accounting} method to prove this claim. Each nonzero bit in the binary counter will get a ``credit'' obtained from earlier increment operations that will then be used to pay for later expensive operations. More specifically, we will maintain the invariant that every $1$ in the binary representation has a ``credit'', which we represent as $\bigoplus$, associated with it. Let $x$ be the binary counter. If we start with an ``empty'' integer – that is $0$ – then clearly all $1$'s have a ``credit'' as there are no $1$'s. Assume that all the 1's of $x$ have a ``credit'' at the start of an increment operation. In each increment operation, we know the first addition will require constant work for which the addition operation will be charged with. We actually ``charge'' the addition operation two ``credit''s, represented as $\bigoplus\bigoplus$, to the new $1$ to be added:

\begin{align*}
x = 1^{\bigoplus}1^{\bigoplus}0& \\
+& \\
1^{\bigoplus \bigoplus}
\end{align*}

Now we start adding. We will maintain the invariant that any ``carry'' bit will have two $\bigoplus$ credits. For completeness, we'll call the original $1$ to be added to $x$ a ``carry'' as well.

Now, at each point we are adding a carry bit to a bit in $x$. If the carry bit is $0$, we do nothing and stop. If the carry bit to be added to the $i$-th bit is $1$ and the $i$-th bit of $x$ is $0$ (Note $i$ starts at $0$), then one of the $\bigoplus$ credits of the carry bit is used to store $1$ in $x[i]$ and the other remains on this new 1 as $\bigoplus$:

\begin{align*}
x = 1^{\bigoplus}1^{\bigoplus}0& \\
+ 1^{\bigoplus \bigoplus} \\
= 1^{\bigoplus}1^{\bigoplus}1^{\bigoplus} &
\end{align*}
At this point, the carry for the $i + 1$-st slot is $0$ and we can stop the addition.

When the carry bit to be added to the $i$-th bit is $1$ and $x[i]$ is $1$, however, we will get a non-zero carry bit for the $i + 1$-st position. In this case, we will use one $\bigoplus$from the $1$ stored in $x[i]$ to pay for storing a $0$ in $x[i]$ (doing the carry addition), and we'll move the two $\bigoplus$ of the carry bit to the new carry bit for the $i + 1$-st position. This maintains the invariant thatall $1$s in $x$ have a credit and all carries have two credits.

For example, consider an increment operation on the binary counter $x = 0111$:

\begin{align*}
01^{\bigoplus}1^{\bigoplus}1^{\bigoplus}& \\
+1^{\bigoplus\bigoplus}&
\end{align*}

A new carry bit is formed:
\begin{align*}
&1^{\bigoplus\bigoplus}\\
=01^{\bigoplus}&1^{\bigoplus}0
\end{align*}

A new carry bit is formed:
\begin{align*}
&1^{\bigoplus\bigoplus}\\
=0&1^{\bigoplus}00
\end{align*}

A new carry bit is formed:
\begin{align*}
&1^{\bigoplus\bigoplus}\\
=&0000 \\
=&1^{\bigoplus}000
\end{align*}
\end{claim}

All carry propagations of additions are for free because they are paid for by the credits accumulated in previous additions of $0$'s and $1$'s. There are $O(n)$ credits overall, two for each increment operation. Thus, the total runtime is $O(n)$. The credit system allows you to pay for later long operations by depositing credits from previous short operations. Some operations are long, but over all $n$ increment operations, the total work is $O(n)$. It follows that the increment operation takes amortized $O(1)$ time.

\end{document}


\newpage\section{Complete Quadrilateral + Spiral Similarity}

	\lem{}{Three lines, $ l_a, l_b, l_c $, origin at point $ P $. Two circles $ \w_1, \w_2 $ passing through $ P $ meet the lines at $ A_1, B_1, C_1;\ A_2, B_2, C_2 $ resp. Let $ A_3 $ be the reflection of $ A_2 $ on $ A_1 $. Define $ B_3, C_3 $ similarly. Then $ PA_3B_3C_3 $ are concyclic.}

		\fig{1}{SpiralSimiLemma1}{Spiral Similarity Lemma 1: the Blue points have been reflected wrt to the Red points to get the Green points}

	\prob{https://artofproblemsolving.com/community/c6h355791p1932936}{ISL 2009 G4}{E}{Given a cyclic quadrilateral $ABCD$, let the diagonals $AC$ and $BD$ meet at $E$ and the lines $AD$ and $BC$ meet at $F$. The midpoints of $AB$ and $CD$ are $G$ and $H$, respectively. Show that $EF$ is tangent at $E$ to the circle through the points $E$, $G$ and $H$.}

	\solu{This problem generalizes to \hrf{lemma:power_of_point_on_a_midline_of_a_self-polar_triangle}{this}}

	\prob{https://artofproblemsolving.com/community/c6h587990p3480801}{All Russian 2014 Grade 10 Day 1 P4}{E}{Given a triangle $ABC$ with $AB>BC$, let $ \Omega $ be the circumcircle. Let $M$, $N$ lie on the sides $AB$, $BC$ respectively, such that $AM=CN$. Let $K$ be the intersection of $MN$ and $AC$. Let $P$ be the incenter of the triangle $AMK$ and $Q$ be the $K$-excenter of the triangle $CNK$. If $R$ is midpoint of the arc $ABC$ of $ \Omega $ then prove that $RP=RQ$.}

	\lem{}{Let $E$ and $F$ be the intersections of opposite sides of a convex quadrilateral $ABCD$. The two diagonals meet at $P$. Let $M$ be the foot of the perpendicular from $P$ to $EF$. Show that $\angle BMC=\angle AMD$. And $ PM $ is the bisector of angles $ \angle AMC, \angle BMD $.}

		\fig{1}{Complete_quad_anglebisector}{}

	\theo{https://en.wikipedia.org/wiki/Newton_line}{Newton-Gauss Line}{Among the points $A, B, C, D$ no three are collinear. The lines $AB$ and $CD$ intersect at $E,$ and $BC$ and $DA$ intersect at $F.$ Prove that either the circles with diameters $AC, BD, EF$ pass through two common points, or no two of them have any common point.

		The previous can be stated differently: The midpoints of $AC, BD, EF$ are collinear and this line is called ``Newton-Gauss Line''.}

	\solu{Either by E.R.I.Q. Lemma, Length Chase, or configurations like \href{https://www.cut-the-knot.org/Curriculum/Geometry/Varignon.shtml}{Varignon Parallelogram}}

	\prob{}{}{M}{Let $ABCD$ incribed $(O)$ and a point so-called $M$ .Call $X,\ Y,\ Z,\ T,\ U,\ V$are the projection of $M$ onto $AB,\ BC,\ CD,\ DA,\ CA,\ BD$ respectively. Call $I,\ J,\ H$ the midpoints of $XZ,\ UV,\ YT$ respectively. Prove that $N,\ P,\ Q$ are collinear.}\label{eriq_lemma_1}

	\solu{Divide the problem in cases, and prove the easiest case first.}

	\lem{}{In a cyclic quadrilateral $ ABCD $, $ AC\cap BD=P,\ AD\cap BC=Q,\ AB\cap CD=R $. $ S, T $ are the midpoints of $ PQ, PR $. And a point $ X $ is on $ ST $. Prove that the power of $ X $ wrt $ ABCD $ is $ XP^2 $.}\label{lemma:power_of_point_on_a_midline_of_a_self-polar_triangle}

		\solu{Using polar argument wrt $ P $}

		\fig{.7}{power_of_point_on_a_midline_of_a_self-polar_triangle}{}

	\prob{https://artofproblemsolving.com/community/c6h326960p1752047}{USA TST 2000 P2}{E-M}{Let $ ABCD$ be a cyclic quadrilateral and let $ E$ and $ F$ be the feet of perpendiculars from the intersection of diagonals $ AC$ and $ BD$ to $ AB$ and $ CD$, respectively. Prove that $ EF$ is perpendicular to the line through the midpoints of $ AD$ and $ BC$.}

		\fig{1}{USATST2000P2}{USA TST 2000 P2}

	\solu{First solution is using some properties of the complete quad and angle bash the angle $ \measuredangle (MN, EF) $}

	\solu{Second solution is to notice the two brow triangles and proving them congruent.}

	\prob{}{}{E}{Let $2$ equal circle $(O_1), (O_2)$ meet each other at $P, Q$. $O$ be the midpoint of $PQ$. $2$ line through $P$ meet the circles at $A,\ B,\ C,\ D,\ (A, C \in (O_1);\ B, D\in (O_2))$. $M, N$ be midpoint of $AD, BC$. Prove that $M, N, O$ are collinear.}\label{eriq_lemma_2}

	\prob{https://artofproblemsolving.com/community/c6h1502855p8912529}{AoPS}{}{In $\triangle ADE$  a circle with center $ O $, passes through $A, D$ meets $AE, ED$ respectively at $B, C$, $BD \cap AC = G$, line $OG$ meets $\odot ADE$ at $P$. Prove that $\triangle PBD, \triangle PAC$ has the same incenter (preferably without using inversion).}

	\prob{}{Archer - EChen M1P2}{E}{Let a circle $ \omega $ centered at $ A $ meet $ BC $ at $ D, E $, such that $ B, D, E, C $ all lie on $ BC $ in that order. Let $ \omega $ meet $ \odot ABC $ at $ F, G $ such that $ A, F, B, C, G $ lie on the circle in that order. Let $ \odot BFD \cap AB = K,\ \odot CGE \cap AC = L $. Prove that $ FK, GL, AO $ are concurrent.}

		\figdf{.7}{Archer_EChen_M1P2}{}

	\prob{https://artofproblemsolving.com/community/c6h477205p2672053}{Sharygin 2012 P22}{E}{A circle $\omega$ with center $I$ is inscribed into a segment of the disk, formed by an arc and a chord $AB$. Point $M$ is the midpoint of this arc $AB$, and point $N$ is the midpoint of the complementary arc. The tangents from $N$ touch $\omega$ in points $C$ and $D$. The opposite sidelines $AC$ and $BD$ of quadrilateral $ABCD$ meet in point $X$, and the diagonals of $ABCD$ meet in point $Y$. Prove that points $X, Y, I$ and $M$ are collinear.}

		\fig{1}{sharygin/2012_22}{}
		\solu{La Hire}

	\prob{https://artofproblemsolving.com/community/c6h477204p2672051}{Sharygin 2012 P21}{E}{Two perpendicular lines pass through the orthocenter of an acute-angled triangle. The sidelines of the triangle cut on each of these lines two segments: one lying inside the triangle and another one lying outside it. Prove that the product of two internal segments is equal to the product of two external segments.}

		\fig{.7}{sharygin/2012_21}{}
		\solu{Spiral Similarity}

	\prob{https://artofproblemsolving.com/community/c6h250108p1370577}{Iran TST 2004 P4}{E}{Let $ M,M'$ be two conjugates point in triangle $ ABC$ (in the sense that $ \angle MAB=\angle M'AC,\dots$). Let $ P,Q,R,P',Q',R'$ be foots of perpendiculars from $ M$ and $ M'$ to $ BC,CA,AB$. Let $ E=QR\cap Q'R'$, $ F=RP\cap R'P'$ and $ G=PQ\cap P'Q'$. Prove that the lines $ AG, BF, CE$ are parallel.}

		\figdf{.7}{irantst2004P4}{The points are collinear, by Zhao Lemmas}

	\prob{https://artofproblemsolving.com/community/c6h1629942p10229160}{Iran TST 2018 D2P6}{EM}{Consider quadrilateral $ABCD $ inscribed in circle $\omega $. $P\equiv AC\cap BD$. $E$, $F$ lie on sides $AB$, $CD$ respectively such that $\hat {APE}=\hat {DPF} $. Circles $\omega_1$, $\omega_2$ are tangent to $\omega$ at $X $, $Y $ respectively and also both tangent to the circumcircle of $\triangle PEF $ at $P $. Prove that: \[\frac {EX}{EY}=\frac {FX}{FY}\]}


\documentclass[12pt,a4paper,oneside,openany]{report}

\usepackage{verbatim}
\usepackage{epsfig}

\setlength\paperwidth{210mm}
\setlength\paperheight{297mm}
\setlength\parindent{0mm}

\setlength{\voffset}{-25.4mm}
\setlength{\hoffset}{-25.4mm}

\setlength{\topmargin}{20mm}
\setlength{\oddsidemargin}{20mm}
\setlength{\evensidemargin}{20mm}
\setlength{\textwidth}{160mm}
\setlength{\textheight}{247mm}

\setlength\footskip{10mm}
\setlength\headheight{0cm}
\setlength\headsep{0cm}
\pagestyle{plain}
\pagenumbering{arabic}
\parskip 5mm

\newcommand{\fcode}[1]{\par \small file = \textbf{#1} \verbatiminput{example/#1} \normalsize}
\newcommand{\code}[1]{\texttt{#1}}
\newcommand{\SC}{\textbf{SC} }
\newcommand{\makefile}{\code{Makefile}}

\begin{document}

\section*{More MPI examples}
There are many different ways MPI can be used to parallelise a problem.  The trivially parallel involve almost no communication, except for at the start and end to distribute work and then collect the results.  No communication is required during the running of the program.  Such parallel programs often use collective communication to push the initial information out and then to pull the final result in.

The value of $\pi = 3.14159265359$ can be computed using the following integral
\begin{equation}
\pi = \int_{0}^{1} \frac{4}{1+x^{2}}\;dx
\end{equation}
with a serial version of the code being

\fcode{src/serialpi.f90}

Notice the use of $1.0\_dp$ to make sure $1$ is interpreted correctly and the abreviated \code{if} statement.

Compiling and running the serial code
\begin{verbatim}
sc0> f90 -fast serialpi.f90 -o serialpi
sc0> ./serialpi
Enter the number of intervals: (0 quits)
100000
 pi is approximately  3.1415926535981 Error is  0.0000000000083
Enter the number of intervals: (0 quits)
10000000
 pi is approximately  3.1415926535898 Error is  0.0000000000000
Enter the number of intervals: (0 quits)
0
sc0>
\end{verbatim}

\newpage

The parallel version of this code is very simple and includes only the use of collective communications.

\fcode{src/mpipi.f90}

Every process in \code{MPI\_COMM\_WORLD} must take part in collective communications.  Every process will sit and wait until this has occured.

Compiling and running the parallel code
\begin{verbatim}
sc0> f90 -fast mpipi.f90 -o mpipi -lmpi
sc0> prun -n 4 ./mpipi
Enter the number of intervals: (0 quits)
100000
 pi is approximately  3.1415926535981 Error is  0.0000000000083
Enter the number of intervals: (0 quits)
10000000
 pi is approximately  3.1415926535898 Error is  0.0000000000000
Enter the number of intervals: (0 quits)
0
sc0>
\end{verbatim}

What do you think the compiler flat \code{-fast} does?

\newpage

The use of extensive communication during the computation process distinguishes a real parallel job from the trivially parallel job.  Such computer codes make use of high performance super computers such as the \SC which have special networks and hardware to make such programs run well.  As an example, consider the laplace equation
\begin{equation}
\nabla^{2} u = 0
\end{equation}
with a serial version of the code being

\fcode{src/seriallaplace.f90}

Notice how the arrays $u$ and $unew$ are defined so as to have different ranges and also how they are initialised.

Compiling and running the serial code
\begin{verbatim}
sc0> f90 -fast seriallaplace.f90 -o seriallaplace
sc0> ./seriallaplace
 Iteration          100 , error =    53.4471619335965
 Iteration          200 , error =    36.8855595092387
 Iteration          300 , error =    29.5396365360539
 Iteration          400 , error =    25.1585720389280
 Iteration          500 , error =    22.1680459957814
 Iteration          600 , error =    19.9601882365086
 Iteration          700 , error =    18.2440590012490
 Iteration          800 , error =    16.8606192739112
 Iteration          900 , error =    15.7146278113500
 Iteration         1000 , error =    14.7451077372352
sc0>
\end{verbatim}

\newpage

To distribute these arrays across $n$ processors requires a ``slab'' decomposition.

\fcode{src/mpilaplace.f90}

\newpage

Compiling and running the parallel code
\begin{verbatim}
sc0> f90 -fast mpilaplace.f90 -o mpilaplace -lmpi
sc0> prun -n 4 ./mpilaplace
 Iteration          100 , error =    53.4471619335964
 Iteration          200 , error =    36.8855594609273
 Iteration          300 , error =    29.5396339687446
 Iteration          400 , error =    25.1585551008231
 Iteration          500 , error =    22.1679960513792
 Iteration          600 , error =    19.9600887453752
 Iteration          700 , error =    18.2438997665307
 Iteration          800 , error =    16.8603963090278
 Iteration          900 , error =    15.7143416500720
 Iteration         1000 , error =    14.7447617007717
sc0>
\end{verbatim}

\newpage

\section*{N-body code}

\fcode{src/nbody.f90}

Compiling and running the N-body code
\begin{verbatim}
sc0> f90 -fast nbody.f90 -o nbody
sc0> /usr/bin/time ./nbody
  2.300817560736747E-018
  2.300816535849909E-018
  2.300816357189022E-018
  2.300816282865615E-018
  2.300816255406225E-018
  2.300816242766784E-018
  2.300816238288725E-018
  2.300816236761038E-018
  2.300816235520899E-018
  2.300816234545460E-018

real   5.6
user   5.5
sys    0.0
sc0>
\end{verbatim}

\newpage

\section*{N-body abstraction}
A version of the N-body code which utilised heavy data abstraction

\fcode{src/nbody.abstract.f90}

Compiling and running the N-body code
\begin{verbatim}
sc0> f90 -fast nbody.abstract.f90 -o nbody.abstract
sc0> /usr/bin/time ./nbody.abstract
  2.300817560736748E-018
  2.300816529207251E-018
  2.300816359939625E-018
  2.300816286977051E-018
  2.300816251131971E-018
  2.300816240927980E-018
  2.300816236951865E-018
  2.300816235131360E-018
  2.300816233604450E-018
  2.300816233354569E-018

real   13.8
user   13.7
sys    0.0
sc0>
\end{verbatim}

What's the difference?  Well, look at the times to run.

\end{document}


\subsection{Local minima, maxima and inflection points}