\global\long\def\bias{\mathrm{bias}}
\global\long\def\var{\mathrm{var}}
\global\long\def\mse{\mathrm{MSE}}
\global\long\def\mise{\mathrm{MISE}}
\global\long\def\lognormalkernel{\mathrm{LN}}
\global\long\def\gammakernel{\mathrm{G}}
\global\long\def\bskernel{\mathrm{BS}}
\global\long\def\igkernel{\mathrm{IG}}
\global\long\def\rigkernel{\mathrm{RIG}}
\global\long\def\euler{\mathrm{e}}

\section{Introduction}

Kernel density estimation with fixed bandwidth has become the most common method to estimate unknown densities of continuous random variables. The standard kernel density estimator (KDE) is defined as
\begin{equation}
\hat{f}\left(x\right)=\frac{1}{nh}\sum_{i=1}^{n}K\left(\frac{X_{i}-x}{h}\right),\label{eq:standard-kde}
\end{equation}
where $\left\{ X_{i}:i=1,\ldots,n\right\} $ is a set of $n$ i.i.d. samples of a univariate random variable $X$ drawn from a generating distribution $f$, $K$ is a non-negative function that integrates to unity, and $h$ is the bandwidth which controls the smoothness of the estimate \citep{Silverman1986}. We assume that $f$ is supported on the real line. Unless otherwise stated $\int$ will refer to an integral over the interval $\left(-\infty,\infty\right)$.

Before outlining the remainder of this paper, we briefly review three topics for which we will provide a unified treatment: variable-bandwidth KDEs, shifted KDEs and KDEs with asymmetric kernels.

\emph{Variable-bandwidth KDEs: }Choosing the fixed bandwidth $h$ entails a trade-off. If the bandwidth is small, high-density regions are smoothed appropriately and we can recover small-scale features. Yet low-density regions will be undersmoothed. If the bandwidth is large, low-density regions are smoothed appropriately. Unfortunately, high-density regions will be oversmoothed and small-scale features will be masked. \citet{Terrell1992} considered two variable-bandwidth KDEs to alleviate this problem. \emph{Balloon estimators} have a bandwidth that depends on the \emph{evaluation point}, i.e. the point, $x$, at which $f\left(x\right)$ needs to be estimated,
\begin{equation}
\hat{f}\left(x\right)=\frac{1}{nh\left(x\right)}\sum_{i=1}^{n}K\left(\frac{X_{i}-x}{h\left(x\right)}\right).\label{eq:balloon-estimator}
\end{equation}
By contrast, the bandwidth of \emph{sample-smoothing estimators} depends on the sample associated with each kernel such that
\begin{equation}
\hat{f}\left(x\right)=\frac{1}{n}\sum_{i=1}^{n}\frac{1}{h\left(X_{i}\right)}K\left(\frac{X_{i}-x}{h\left(X_{i}\right)}\right).\label{eq:smoothing-estimator}
\end{equation}
The latter yield properly normalised density estimates, i.e. $\int dx\,\hat{f}\left(x\right)=1$, because each term of the sum in Eq.~\eqref{eq:smoothing-estimator} integrates to unity. In general, balloon estimators are not proper densities themselves.

\emph{Shifted KDEs: }Standard KDEs tend to overestimate densities in regions with few samples and underestimate densities in regions with many samples. \citet{Samuiddin1990} proposed to shift samples slightly from low-density regions to high-density regions to achieve lower bias. \citet{Hall2002a} extended this approach to decrease the bias further using data sharpening estimators. KDEs with variable bandwidth and shifted samples were considered by \citet{Jones1994} in the context of sample-smoothing estimators.

\emph{Asymmetric KDEs:} A literature on using asymmetric kernels for density estimation has developed in parallel: gamma \citep{Chen2000}, log-normal and Birnbaum-Saunders \citep{Jin2003}, generalized Birnbaum-Saunders \citep{Marchant2013}, as well as inverse Gaussian and reciprocal inverse Gaussian kernels \citep{Scaillet2004} eliminate bias on or near the boundary of a (semi-)bounded interval. Recent improvements include estimators that yield properly-normalised densities \citep{Kuruwita2010,Jeon2014} and further reduce bias near the boundary \citep{Malec2014}. Whenever a new kernel is being considered, its asymptotic properties have to be carefully derived. The mean, median, and mode of asymmetric kernels are not identical in general. If the kernel is parametrised such that its mode or median match the corresponding sample, the mean may differ by a small amount, which is similar to shifted KDEs. Furthermore, asymmetric KDEs share properties with variable-bandwidth estimators because their effective smoothing scale changes across the domain on which the distribution is supported.

The three topics are intimately connected and can be unified by considering variable-bandwidth KDEs whose kernels have been shifted by a small amount. We provide a general treatment of shifted sample-smoothing estimators, introduce shifted balloon estimators, and develop a generic framework to obtain the asymptotic properties of a range of asymmetric kernel density estimators. After discussing mathematical preliminaries in Section~\ref{sec:preliminaries}, we define shifted balloon and sample-smoothing estimators in Sections~\ref{sec:balloon} and \ref{sec:smoothing}, respectively. We derive the asymptotic bias, variance and mean (integrated) squared error for each class of estimators. In Section~\ref{sec:application}, we apply our results to density estimators with gamma, log-normal, Birnbaum-Saunders, inverse Gaussian and reciprocal inverse Gaussian kernels, derive plugin expressions for bandwidth selection, and propose two new density estimators for density estimation on the positive real line. We conclude in Section~\ref{sec:discussion}.

\section{Preliminaries\label{sec:preliminaries}}

The estimators in Eqs.~\eqref{eq:standard-kde} to \eqref{eq:smoothing-estimator} can all be expressed as general weight function estimators defined by
\begin{align}
\hat{f}\left(x\right) & =\frac{1}{n}\sum_{i=1}^{\infty}W\left(X_{i},x\right),\label{eq:generalised-estimator}
\end{align}
where $W\left(y,x\right)$ is a non-negative function that peaks near $x=y$. Because the samples are i.i.d., the first moment of the estimator is
\begin{align}
\left\langle \hat{f}\left(x\right)\right\rangle  & =\left\langle W\left(X,x\right)\right\rangle ,\label{eq:generalised-mean}
\end{align}
where the expectation is with respect to the samples $X$. The second moment is
\begin{align*}
\left\langle \hat{f}^{2}\left(x\right)\right\rangle  & =\frac{1}{n^{2}}\left\langle \sum_{i,j=1}^{n}W\left(X_{i},x\right)W\left(X_{j},x\right)\right\rangle \\
 & =\frac{1}{n}\left\langle W^{2}\left(X,x\right)\right\rangle +\frac{n-1}{n}\left\langle W\left(X,x\right)\right\rangle ^{2}
\end{align*}
such that the variance becomes
\begin{align}
\var\hat{f}\left(x\right) & =\left\langle \hat{f}^{2}(x)\right\rangle -\left\langle \hat{f}(x)\right\rangle ^{2}\nonumber \\
 & =\frac{1}{n}\left(\left\langle W^{2}\left(X,x\right)\right\rangle -\left\langle W\left(X,x\right)\right\rangle ^{2}\right).\label{eq:generalised-variance}
\end{align}

The canonical criterion for assessing the local quality of density estimates is the mean squared error (MSE) and is defined as

\[
\mse\hat{f}\left(x\right)=\left\langle \left(f(x)-\hat{f}(x)\right)^{2}\right\rangle
\]
for a given evaluation point $x$. We rewrite the MSE in terms of squared bias and variance such that
\begin{align}
\mse\hat{f}(x) & =\bias^{2}\hat{f}(x)+\var\hat{f}(x),\nonumber \\
\text{where}\quad\bias\hat{f}(x) & =\left\langle \hat{f}(x)\right\rangle -f(x)\label{eq:generalised-bias}\\
 & =\left\langle W\left(X,x\right)\right\rangle -f\left(x\right).\nonumber
\end{align}
The mean integrated squared error (MISE) is defined as
\[
\mise\hat{f}=\int dx\,\mse\hat{f}\left(x\right)
\]
and will be our criterion for the global quality of density estimates.

The balloon estimator in Eq.~\eqref{eq:balloon-estimator}, the sample-smoothing estimator in Eq.~\eqref{eq:smoothing-estimator}, and the estimators we consider in the following two sections can be expressed in terms of a kernel. Following \citet{Terrell1992}, we formalize the notion of a kernel in the following definition.
\begin{defn}
\label{def:order-p-kernel}An order-$p$ kernel is a univariate, non-negative function $K$ such that
\begin{equation}
\begin{aligned}m_{k} & \equiv\int dz\,K(z)z^{k} & \kappa & \equiv\int dz\,K^{2}(z)\\
m_{k} & =\begin{cases}
1 & \text{if }k=0,\,p\\
0 & \text{if }0<k<p
\end{cases} & 0 & <\kappa<\infty,
\end{aligned}
\label{eq:kernel-properties}
\end{equation}
where $m_{k}$ is the $k^{\mathrm{th}}$ moment of the kernel.
\end{defn}

\section{Shifted balloon estimator\label{sec:balloon}}

We now provide our modified version of the balloon estimator.
\begin{defn}
A shifted balloon estimator is a kernel density estimator whose bandwidth depends on the point $x$ at which it is evaluated, i.e.
\begin{equation}
\hat{f}(x)=\frac{1}{n}\sum_{i=1}^{n}\frac{1}{h(x)}K\left(\frac{X_{i}-x-h^{p}\left(x\right)\delta(x)}{h(x)}\right),\label{eq:balloon-definition}
\end{equation}
where $p$ is the order of the kernel $K$, and $\delta(x)$ is a shift which remains bounded as $n\rightarrow\infty$, i.e. $\lim_{n\rightarrow\infty}\left|\delta\left(x\right)\right|<\infty$.
\end{defn}
The kernel is thus shifted with respect to the sample by a small amount $h^{p}\left(x\right)\delta\left(x\right)$ which depends on the evaluation point and vanishes as the bandwidth decreases. Shifted balloon estimators do not yield properly-normalised density estimates in general, similar to their unshifted counterparts.

We need to consider the bias and variance to evaluate the MSE in the limit of small bandwidth.
\begin{thm}
\label{thm:balloon-moments} The mean and variance of a shifted balloon estimator with order-$p$ kernel are
\begin{align}
\left\langle \hat{f}\left(x\right)\right\rangle  & =A_{0}\left(x\right)+A_{p}\left(x\right)+o\left(h^{p}\left(x\right)\right)\label{eq:balloon-mean}\\
\text{and}\quad\var\hat{f}_{h}\left(x\right) & =\frac{f\left(x\right)\kappa}{nh\left(x\right)}+o\left(n^{-1}h^{-1}\left(x\right)\right),\label{eq:balloon-var}\\
\text{where}\quad A_{k}\left(x\right) & =h^{k}\left(x\right)\sum_{j=k}^{\infty}\frac{f^{\left(j\right)}\left(x\right)}{k!\left(j-k\right)!}\left[h^{p}\left(x\right)\delta\left(x\right)\right]^{j-k}\label{eq:balloon-coefficient}
\end{align}
and $f^{(j)}(x)$ denotes the $j^{\text{th}}$ derivative of $f$ evaluated at $x$. The factorial is defined such that $0!\equiv1$.
\end{thm}
A proof is given in Appendix \ref{app:proofs}. Substituting Eqs.~\eqref{eq:balloon-coefficient} and \eqref{eq:balloon-mean} into Eq.~\eqref{eq:generalised-bias} and dropping higher-order terms in $h\left(x\right)$, the bias is
\begin{equation}
\bias\hat{f}\left(x\right)=h^{p}\left(x\right)\delta\left(x\right)f'\left(x\right)+\frac{h^{p}\left(x\right)}{p!}f^{\left(p\right)}\left(x\right)+o\left(h^{p}\left(x\right)\right),\label{eq:balloon-bias}
\end{equation}
which is the same as the expression derived by \citet{Terrell1992} if we let $\delta\left(x\right)=0$. We thus see from Eq.~\eqref{eq:balloon-bias} that any dependence of the bias on the first derivative of the generating distribution is due to the shift between the kernel mean and the associated sample as \citet{Chen2000} noted in the context of density estimators using gamma kernels. Adding Eq.~\eqref{eq:balloon-var} and the square of Eq.~\eqref{eq:balloon-bias} yields the MSE
\begin{multline*}
\mse\hat{f}\left(x\right)=\left[h^{p}\left(x\right)\delta\left(x\right)f'\left(x\right)+\frac{h^{p}\left(x\right)}{p!}f^{\left(p\right)}\left(x\right)\right]^{2}\\
+\frac{f\left(x\right)\kappa}{nh\left(x\right)}+o\left(n^{-1}h^{-1}\left(x\right)+h^{2p}\left(x\right)\right).
\end{multline*}

\section{Shifted sample-smoothing estimators\label{sec:smoothing}}

In the following, we discuss our modified version of the sample-smoothing estimator.
\begin{defn}
A shifted sample-smoothing estimator is a kernel density estimator whose bandwidth depends on the samples $X_{i}$, i.e.
\begin{equation}
\hat{f}\left(x\right)=\frac{1}{n}\sum_{i=1}^{n}\frac{1}{h\left(X_{i}\right)}K\left(\frac{X_{i}-x+h^{p}\left(X_{i}\right)\delta\left(X_{i}\right)}{h\left(X_{i}\right)}\right),\label{eq:smoothing-definition}
\end{equation}
where $p$ is the order of the kernel $K$, and $\delta(X_{i})$ is a shift which remains bounded as $n\rightarrow\infty$, i.e. $\lim_{n\rightarrow\infty}\left|\delta\left(X_{i}\right)\right|<\infty$.
\end{defn}
Again, the kernel is shifted with respect to the sample by a small amount but the magnitude of the shift depends on the sample and not the evaluation point. Shifted sample-smoothing estimators are guaranteed to yield properly-normalised density estimates.

As in the previous section, we consider the mean and the variance assuming the bandwidth is small. See Appendix \ref{app:proofs} for a proof of the following theorem.
\begin{thm}
\label{thm:smoothing-moments} If
\begin{equation}
z=\frac{y-x+h^{p}\left(y\right)\delta\left(y\right)}{h\left(y\right)}\label{eq:change-of-variables-condition}
\end{equation}
is a monotonic function of $y$ for all $x$, the mean and variance of a shifted sample-smoothing estimator with order-$p$ kernel are
\begin{align}
\left\langle \hat{f}\left(x\right)\right\rangle  & =B_{0}\left(x\right)+B_{p}\left(x\right)+o\left(h^{p}\left(x\right)\right)\nonumber \\
\text{and}\quad\var\hat{f}\left(x\right) & =\frac{f(x)\kappa}{nh(x)}+o\left(n^{-1}h^{-1}\left(x\right)\right),\label{eq:smoothing-var}\\
\text{where}\quad B_{k} & =\sum_{j=k}^{\infty}\frac{1}{k!\left(j-k\right)!}\frac{d^{j}}{dx^{j}}\left(f\left(x\right)h^{k+p\left(j-k\right)}\left(x\right)\left(-\delta\left(x\right)\right)^{j-k}\right).\nonumber
\end{align}

\end{thm}
The condition in Eq.~\eqref{eq:change-of-variables-condition} is satisfied by most kernels although exceptions exist as we will see in the next section. Using the same argument as in the previous section and dropping higher-order terms in $h\left(x\right)$, we have

\begin{align}
\bias\hat{f}\left(x\right) & =-\frac{d}{dx}\left(f\left(x\right)h^{p}\left(x\right)\delta\left(x\right)\right)+\frac{1}{p!}\frac{d^{p}}{dx^{p}}\left(h^{p}\left(x\right)f\left(x\right)\right)+o\left(h^{p}\left(x\right)\right).\label{eq:smoothing-bias}
\end{align}
The expressions agree with \citet{Terrell1992} if we let $\delta\left(x\right)=0$. \citet{Jones1994} provide an alternative derivation for second-order kernels, i.e. $p=2$. The MSE is
\begin{multline*}
\mse\hat{f}(x)=\left[-\frac{d}{dx}\left(f\left(x\right)h^{p}\left(x\right)\delta\left(x\right)\right)+\frac{1}{p!}\frac{d^{p}}{dx^{p}}\left(h^{p}\left(x\right)f\left(x\right)\right)\right]^{2}+\frac{\kappa f\left(x\right)}{nh\left(x\right)}\\
+o\left(n^{-1}h^{-1}\left(x\right)+h^{2p}\left(x\right)\right).
\end{multline*}

\section{Application\label{sec:application}}

Having derived the asymptotic properties of shifted balloon and shifted sample-smoothing estimators allows us to obtain the asymptotic properties of a range of kernel density estimators easily. Before applying our method, we need to make a distinction that is immaterial for symmetric kernels. Recall the definition of the general weight function estimator which we repeat here for ease of reference:
\begin{equation}
\hat{f}\left(x\right)=\frac{1}{n}\sum_{i=1}^{n}W\left(X_{i},x\right).\label{eq:generalised-estimator-2}
\end{equation}
There are two fundamentally different choices for the weight function.
\begin{defn}
A weight function $W\left(y,x\right)$ is called \emph{proper} if and only if
\[
\int dy\,W\left(y,x\right)=1.
\]
It is called \emph{improper} otherwise.
\end{defn}
Estimators with proper weight functions are guaranteed to be densities themselves because each term in Eq.~\eqref{eq:generalised-estimator-2} integrates to unity. Sample-smoothing estimators belong to this class. Estimators with improper weight functions, such as balloon estimators, do not in general yield normalised densities. If the weight function is symmetric with respect to exchange of its argument, the distinction becomes immaterial. Standard kernels with fixed bandwidth as defined in Eq.~\eqref{eq:standard-kde} are examples of symmetric weight functions if the kernel is an even function of its argument.

\subsection{Density estimation using improper weight functions}

Improper weight functions have recently become popular for estimating densities with (semi-)bounded support to avoid boundary bias. See Section 2.11 in \citet{Wand1995} for a detailed discussion of boundary bias. In particular, \citet{Brown1999} used beta-distribution kernels for non-parametric regression on a finite interval. \citet{Chen2000} proposed the use of gamma-distribution kernels to estimate densities of positive random variables, which spurred further research into positive density estimation using Birnbaum-Saunders and log-normal kernels \citep{Jin2003} as well as inverse Gaussian and reciprocal inverse Gaussian kernels \citep{Scaillet2004}.

Intuitively, the contribution from any weight function becomes ever more sharply peaked as the bandwidth of the estimator decreases. Because the global properties become less important, we should be able to approximate general weight functions by simpler kernels that peak where the sample and evaluation point are close. We will see that the improper estimators in the literature are asymptotically equivalent to shifted balloon estimators as defined in Eq.~\eqref{eq:balloon-definition}. This observation enables us to derive their asymptotic properties using the general results from Section \ref{sec:balloon} instead of having to derive them independently. We consider the gamma-kernel density estimator proposed by \citet{Chen2000} in detail and discuss other kernels in Section~\ref{subsec:other-kernels}.

After reparametrisation, Chen's improper estimator is given by
\[
\hat{f}\left(x\right)=\sum_{i=1}^{n}\mathrm{G}\left(X_{i};1+\frac{x}{\sigma^{2}},\sigma^{2}\right),
\]
where $\sigma$ is a bandwidth parameter, and $\mathrm{G}\left(t;k,\theta\right)=\frac{\theta^{-k}}{\Gamma\left(k\right)}x^{k-1}\exp\left(-\frac{t}{\theta}\right)$ denotes the probability density function of a gamma random variable $t$ with shape parameter $k$ and scale parameter $\theta$. If the bandwidth $\sigma$ is small, the shape parameter of the gamma distribution is large and it can be approximated by a Gaussian with the same mean \citep{Johnson1994}, i.e.
\[
\mathrm{G}\left(X_{i};1+\frac{x}{\sigma^{2}},\sigma^{2}\right)\rightarrow\mathcal{N}\left(X_{i};x+\sigma^{2},\sigma^{2}\left(x+\sigma^{2}\right)\right).
\]
The approximation takes the form of a shifted balloon estimator with Gaussian kernel and
\begin{align*}
h\left(x\right) & =\sigma\sqrt{x+\sigma^{2}}\\
h^{2}\left(x\right)\delta\left(x\right) & =\sigma^{2}.
\end{align*}

Having represented Chen's improper estimator as a shifted balloon estimator, we can find the bias and variance using Eqs.~\eqref{eq:balloon-bias} and \eqref{eq:balloon-var}:
\begin{align*}
\bias\hat{f}\left(x\right) & =\sigma^{2}f'\left(x\right)+\frac{\sigma^{2}\left(x+\sigma^{2}\right)}{2}f''\left(x\right)+o\left(\sigma^{2}\right)\\
 & =\sigma^{2}\left(f'\left(x\right)+\frac{x}{2}f''\left(x\right)\right)+o\left(\sigma^{2}\right)\\
\text{and}\quad\var\hat{f}\left(x\right) & =\frac{f\left(x\right)}{2n\sigma\sqrt{\pi\left(x+\sigma^{2}\right)}}+o\left(n^{-1}\sigma^{-1}\right)\\
 & =\frac{f\left(x\right)}{2n\sigma\sqrt{\pi x}}+o\left(n^{-1}\sigma^{-1}\right),
\end{align*}
where we have used that Gaussians are second-order kernels with $\kappa=\frac{1}{2\sqrt{\pi}}$. The expressions agree with the ones derived by \citet{Chen2000}. The MSE is
\[
\mse\hat{f}\left(x\right)=\sigma^{4}\left(f'\left(x\right)+\frac{x}{2}f''\left(x\right)\right)^{2}+\frac{f\left(x\right)}{2n\sigma\sqrt{\pi x}}+o\left(\sigma^{4}+n^{-1}\sigma^{-1}\right).
\]

\begin{figure}
\begin{centering}
\includegraphics{improper-gamma}
\par\end{centering}

\caption{\label{fig:improper-gamma}Density estimate of $n=300$ samples drawn from a log-normal distribution with logarithmic mean $\mu=1$ and logarithmic variance $\Sigma^{2}=1$ using an improper gamma estimator (solid black line) and a Gaussian approximation (dashed black line). The generating distribution is shown as a dotted black line and individual samples are shown as markers on the horizontal axis. The bandwidth for both estimators was obtained using the plugin expression in Eq.~\eqref{eq:chen-plugin} which minimizes the MISE in Eq.~\eqref{eq:chen-lognormal-mise}.}
\end{figure}

Extending the work by \citet{Chen2000}, we approximate $f\left(x\right)$ by a reference distribution with known functional form, which enables us to obtain a plugin expression for selecting the bandwidth. The parametric form of the reference distribution is largely a free choice; we assume that $f(x)$ is approximately log-normal with logarithmic mean $\mu$ and variance $\Sigma^{2}$ for convenience. Integrating yields
\begin{equation}
\mise\hat{f}=\sigma^{4}\frac{\exp\left(\frac{9\Sigma^{2}}{4}-3\mu\right)\left(12+20\Sigma^{2}+9\Sigma^{4}\right)}{128\sqrt{\pi}\Sigma^{5}}+\frac{\exp\left(\frac{\Sigma^{2}}{8}-\frac{\mu}{2}\right)}{2n\sigma\sqrt{\pi}}\label{eq:chen-lognormal-mise}
\end{equation}
and setting the derivative of the MISE with respect to $\sigma$ to zero gives the optimal bandwidth parameter
\begin{equation}
\sigma^{*}=\frac{2^{4/5}\Sigma\exp\left(\frac{\mu}{2}-\frac{17\Sigma^{2}}{40}\right)}{\left(12+20\Sigma^{2}+9\Sigma^{4}\right)^{1/5}}n^{-1/5}.\label{eq:chen-plugin}
\end{equation}
The expression allows us to obtain a bandwidth by estimating $\mu$ and $\Sigma$ from the samples without the need for computationally-expensive cross-validation. We provide a comparison of cross-validation and the plugin method in the following section. Plugin bandwidth estimators are particularly useful for preliminary data exploration. An example density estimate is shown in Figure~\ref{fig:improper-gamma}. All source code used to generate the figures in this publication is publicly available \citep{SourceCode}.

\subsection{Density estimation using proper weight functions}

Improper weight functions have the unappealing property that they do not yield properly normalised density estimates in general. \citet{Jeon2014} proposed a density estimator using a proper gamma kernel to alleviate this problem. After reparametrisation, their estimator is given by
\begin{equation}
\hat{f}\left(x\right)=\frac{1}{n}\sum_{i=1}^{\infty}\mathrm{G}\left(x;1+\frac{X_{i}}{\sigma^{2}},\sigma^{2}\right),\label{eq:jeon-kim-estimator}
\end{equation}
i.e. the role of the evaluation point and the samples have been swapped. The kernel can be approximated by a normal distribution if the bandwidth is small and the estimator takes the form of a shifted sample-smoothing estimator
\[
\mathrm{G}\left(x;1+\frac{X_{i}}{\sigma^{2}},\sigma^{2}\right)\rightarrow\mathcal{N}\left(x;X_{i}+\sigma^{2},\sigma^{2}\left(X_{i}+\sigma^{2}\right)\right).
\]
Using Eqs.~\eqref{eq:smoothing-bias} and \eqref{eq:smoothing-var}, the bias and variance are
\begin{align*}
\bias\hat{f}\left(x\right) & =-\sigma^{2}f'\left(x\right)+\frac{1}{2}\frac{d^{2}}{dx^{2}}\left(\sigma^{2}\left(x+\sigma^{2}\right)f\left(x\right)\right)+o\left(\sigma^{2}\right)\\
 & =\frac{\sigma^{2}x}{2}f''\left(x\right)+o\left(\sigma^{2}\right)\\
\text{and}\quad\var\hat{f}\left(x\right) & =\frac{f\left(x\right)}{2n\sigma\sqrt{\pi x}}+o\left(n^{-1}\sigma^{-1}\right).
\end{align*}
Jeon and Kim obtained an upper bound for the MSE of the estimator defined in Eq.~\eqref{eq:jeon-kim-estimator} but we can provide an explicit form:
\[
\mse\hat{f}\left(x\right)=\frac{\sigma^{4}}{4}\left[xf''(x)\right]^{2}+\frac{f(x)}{2n\sigma\sqrt{\pi x}}+o\left(\sigma^{4}+n^{-1}\sigma^{-1}\right).
\]
Employing a log-normal distribution with logarithmic mean and variance $\mu$ and $\Sigma^{2}$ as a reference distribution, the MISE is
\begin{equation}
\mise\hat{f}=\frac{\exp\left(\frac{9\Sigma^{2}}{4}-3\mu\right)\left(12+4\Sigma^{2}+\Sigma^{4}\right)}{128\sqrt{\pi}\Sigma^{5}}+\frac{\exp\left(\frac{\Sigma^{2}}{8}-\frac{\mu}{2}\right)}{2n\sqrt{\pi}\sigma}.\label{eq:jeon-kim-lognormal-mise}
\end{equation}
Minimizing the MISE with respect to $\sigma$ yields the plugin expression for the optimal bandwidth
\[
\sigma^{*}=\frac{2^{4/5}\Sigma\exp\left(\frac{\mu}{2}-\frac{17\Sigma^{2}}{40}\right)}{\left(12+4\Sigma^{2}+\Sigma^{4}\right)^{1/5}}n^{-1/5}.
\]

\begin{figure}
\begin{centering}
\includegraphics{bandwidth-comparison}
\par\end{centering}

\caption{\label{fig:bandwidth-comparison}MISE profiles for the estimator in Eq.~\eqref{eq:jeon-kim-estimator} as a function of bandwidth $\sigma$. The solid line shows the asymptotic MISE in Eq.~\eqref{eq:jeon-kim-lognormal-mise} obtained using a Gaussian approximation of the kernel. The dashed line shows the MISE obtained using leave-one-out (LOO) cross-validation. The shaded bands correspond to the 90\% confidence interval from one thousand independent samples of size $n=300$ drawn from a log-normal distribution with $\mu=\Sigma=1$. The offset between the two curves is arbitrary and chosen to enhance readability.}
\end{figure}
As discussed in Appendix~\ref{app:cross-validation}, cross-validation is straightforward for Jeon and Kim's estimator. A comparison between the asymptotic MISE and leave-one-out cross-validation is shown in Figure~\ref{fig:bandwidth-comparison}. The results are encouraging even for a modest sample size of $n=300$ because the two methods select similar bandwidths. The MISE profile obtained from cross-validation is more variable between realizations because it depends on each individual sample whereas the asymptotic MISE profile only depends on the logarithmic mean and variance of the whole sample.

\subsection{Application to estimators with other kernels\label{subsec:other-kernels}}

\begin{sidewaystable}
\textwidth=2.7\textheight%
\begin{tabular*}{1\textheight}{@{\extracolsep{\fill}}llllll}
Distribution & $\left\langle t\right\rangle $ & Variance & Parametrisation & $h^{2}\left(x\right)\delta\left(x\right)$ & $h\left(x\right)$\tabularnewline
\hline
$\gammakernel\left(t;k,\theta\right)=\frac{\theta^{-k}t^{k-1}}{\Gamma(k)}\exp\left(-\frac{t}{\theta}\right)$ & $k\theta$ & $k\theta^{2}$ & $\gammakernel\left(y;\frac{x}{\sigma^{2}}+1,\sigma^{2}\right)$ & $\sigma^{2}$ & $\sigma\sqrt{x+\sigma^{2}}$\tabularnewline
$\lognormalkernel\left(t;\mu,\sigma^{2}\right)=\frac{1}{\sqrt{2\pi}\sigma x}\exp\left(-\frac{\left(\log t-\mu\right)^{2}}{2\sigma^{2}}\right)$ & $\euler^{\mu+\frac{\sigma^{2}}{2}}$ & $\left\langle t\right\rangle ^{2}\left(\euler^{\sigma^{2}}-1\right)$ & $\lognormalkernel\left(y;\log x,\sigma^{2}\right)$ & $x\left(\euler^{\frac{\sigma^{2}}{2}}-1\right)$ & $x\euler^{\frac{\sigma^{2}}{2}}\sqrt{\euler^{\sigma^{2}}-1}$\tabularnewline
$\bskernel\left(t;\sigma,\lambda\right)=\frac{(1+t\lambda)}{2\sigma\sqrt{2\pi\lambda t^{3}}}\exp\left(-\frac{(t\lambda-1)^{2}}{2t\lambda\sigma^{2}}\right)$ & $\frac{2+\sigma^{2}}{2\lambda}$ & $\frac{\sigma^{2}\left(4+5\sigma^{2}\right)}{4\lambda^{2}}$ & $\bskernel\left(y;\sigma,x^{-1}\right)$ & $\frac{x\sigma^{2}}{2}$ & $x\sigma\sqrt{1+5\sigma^{2}/4}$\tabularnewline
$\igkernel\left(t;\mu,\lambda\right)=\sqrt{\frac{\lambda}{2\pi t^{3}}}\exp\left(-\frac{\lambda(t-\mu)^{2}}{2t\mu^{2}}\right)$ & $\mu$ & $\frac{\mu^{3}}{\lambda}$ & $\igkernel\left(y;x,\sigma^{-2}\right)$ & 0 & $\sigma x^{3/2}$\tabularnewline
$\rigkernel\left(t;\mu,\lambda\right)=\sqrt{\frac{\lambda}{2\pi t}}\exp\left(-\frac{\lambda(\mu t-1)^{2}}{2t\mu^{2}}\right)$ & $\frac{1}{\mu}+\frac{1}{\lambda}$ & $\frac{1}{\lambda\mu}+\frac{2}{\lambda^{2}}$ & $\rigkernel\left(y;\frac{1}{x-\sigma^{2}},\sigma^{-2}\right)$ & 0 & $\sigma\sqrt{x+\sigma^{2}}$\tabularnewline
\end{tabular*}

\vspace{2em}
\caption{\label{tbl:kernels}List of probability distributions with their mean and variance. The fourth column shows the parametrisation used for improper weight functions, where $x$ is the evaluation point and $y$ is the associated sample. The parametrisation for proper weight functions can be obtained by swapping $x$ and $y$. The fifth column lists the shift between the mean of the kernel and the sample. The effective bandwidth is shown in the last column. }
\vspace{2em}

\begin{tabular*}{1\textheight}{@{\extracolsep{\fill}}llllll}
 &  & \multicolumn{2}{l}{Improper weight function} & \multicolumn{2}{l}{Proper weight function}\tabularnewline
Kernel with ref. & $n\times\var_{\sigma}$  & $\bias_{\sigma}$  & $n^{1/5}\times\sigma*$ & $\bias_{\sigma}$  & $n^{1/5}\times\sigma*$\tabularnewline
\hline
G \citep{Chen2000,Jeon2014} & $\frac{f(x)}{2\sigma\sqrt{\pi x}}$  & $\sigma^{2}\left(f'(x)+\frac{xf''(x)}{2}\right)$ & $\frac{2^{4/5}\Sigma\exp\left(\frac{\mu}{2}-\frac{17\Sigma^{2}}{40}\right)}{\left(12+20\Sigma^{2}+9\Sigma^{4}\right)^{1/5}}$ & $\frac{\sigma^{2}}{2}xf''\left(x\right)$ & $\frac{2^{4/5}\Sigma\exp\left(\frac{\mu}{2}-\frac{17\Sigma^{2}}{40}\right)}{\left(12+4\Sigma^{2}+\Sigma^{4}\right)^{1/5}}$\tabularnewline
LN \citep{Jin2003} & $\frac{f(x)}{2\sqrt{\pi}\sigma x}$  & $\frac{\sigma^{2}x}{2}\left(f'\left(x)+xf''\left(x\right)\right)\right)$ & $\frac{2^{4/5}\Sigma\exp\frac{\Sigma^{2}}{20}}{\left(12+4\Sigma^{2}+\Sigma^{4}\right)^{1/5}}$ & $\frac{\sigma^{2}}{2}\frac{d}{dx}\left(x\frac{d}{dx}\left(xf\left(x\right)\right)\right)$ & $\frac{2^{4/5}\Sigma\exp\frac{\Sigma^{2}}{20}}{\left(12+4\Sigma^{2}+\Sigma^{4}\right)^{1/5}}$\tabularnewline
BS \citep{Jin2003} & \multicolumn{5}{l}{same as log-normal}\tabularnewline
IG \citep{Scaillet2004} & $\frac{f(x)}{2\sqrt{\pi}\sigma x^{3/2}}$  & $\frac{\sigma^{2}}{2}x^{3}f''\left(x\right)$ & $\frac{2^{4/5}\Sigma\exp\left(\frac{7\Sigma^{2}}{40}-\frac{\mu}{2}\right)}{\left(12+68\Sigma^{2}+225\Sigma^{4}\right)^{1/5}}$ & \multicolumn{2}{c}{method not applicable}\tabularnewline
RIG \citep{Scaillet2004} & $\frac{f(x)}{2\sigma\sqrt{\pi x}}$  & $\frac{\sigma^{2}}{2}xf''\left(x\right)$ & $\frac{2^{4/5}\Sigma\exp\left(\frac{\mu}{2}-\frac{17\Sigma^{2}}{40}\right)}{\left(12+4\Sigma^{2}+\Sigma^{4}\right)^{1/5}}$ & $\sigma^{2}\left(f'(x)+\frac{xf''(x)}{2}\right)$ & $\frac{2^{4/5}\Sigma\exp\left(\frac{\mu}{2}-\frac{17\Sigma^{2}}{40}\right)}{\left(12+20\Sigma^{2}+9\Sigma^{4}\right)^{1/5}}$\tabularnewline
\end{tabular*}

\vspace{2em}
\caption{\label{tbl:asymptotic-properties}List of kernels with their asymptotic variance. The bias and optimal plugin bandwidth $\sigma^{*}$ are shown separately for proper and improper estimators. }
\end{sidewaystable}
We use the methods developed above to derive the asymptotic properties of estimators with proper and improper weight functions. The gamma, log-normal (LN), Birnbaum-Saunders (BS), inverse Gaussian (IG) and reciprocal inverse Gaussian (RIG) distributions are listed in Table~\ref{tbl:kernels}. To serve as a kernel, we parametrise the distributions in terms of the evaluation point $x$, associated sample $y$, and bandwidth parameter $\sigma$. In the limit $\sigma\rightarrow0$, all kernels converge to a normal distribution with mean $x+h^{2}\left(x\right)\delta\left(x\right)$ and variance $h^{2}\left(x\right)$ such that our method is applicable \citep{Johnson1994}. The parametrisation for proper estimators can be obtained by exchanging the role of the evaluation point and the associated sample.

The asymptotic bias and variance for proper and improper estimators are listed in Table~\ref{tbl:asymptotic-properties}. The expressions for the asymptotics of improper estimators agree with the expressions in the original papers after reparametrisation. Because improper BS and LN estimators have a higher-order dependence on the evaluation point than the improper gamma estimator, they have smaller bias near the boundary which has been confirmed by simulations \citep{Jin2003}. The same applies to the improper IG estimator because its bias is cubic in the evaluation point.

The proper estimators with LN, BS, and RIG kernels are new contributions although the LN kernel is equivalent to a log-transform of the samples followed by standard kernel density estimation using Gaussian kernels (see Section~2.11 in \citet{Silverman1986}), which is widely used. The method we have developed is not applicable to proper estimators with RIG kernels because the assumption that $z\left(y,x\right)=\frac{y-x}{\sigma y^{3/2}}$ is monotonic in $y$ is violated.  All estimators achieve the same $n^{-4/5}$ convergence rate as Gaussian KDEs because their MISE is of the form $\sigma^{4}E+\left(\sigma n\right)^{-1}F$, where $E$ and $F$ are numbers that do not depend on $n$ or $\sigma$.

\section{Discussion\label{sec:discussion}}

We have extended balloon and sample-smoothing estimators by a shift parameter and derived their asymptotic properties. Our approach facilitates the unified study of a wide range of density estimators which are subsumed under these two general classes of kernel density estimators. We have demonstrated our method by obtaining the asymptotics of estimators with improper gamma, log-normal, Birnbaum-Saunders, inverse Gaussian and reciprocal inverse Gaussian. Recently, an estimator with proper gamma kernel was proposed \citep{Jeon2014}; we have obtained the first explicit expression for its bias. We have proposed estimators with proper Birnbaum-Saunders and reciprocal inverse Gaussian kernels which have the distinct advantage of yielding normalised density estimates. Plugin expressions for bandwidth estimation are provided for both proper and improper estimators using the log-normal distribution as a reference distribution.

Given the plethora of density estimators, which ones should be used? If the generating density is expected to have a large gradients, the proper gamma estimator and the improper inverse Gaussian (IG) or reciprocal inverse Gaussian (RIG) estimators are suitable because their bias is independent of the first derivative. The improper log-normal, Birnbaum-Saunders, IG and RIG estimators as well as the proper gamma estimator have relatively low bias near the origin and are appropriate for samples that are clustered near zero. Estimators whose effective bandwidth has a higher-order dependence on the evaluation point should be used for heavily-skewed data to avoid spurious wiggles in the tails of the distribution. We believe that proper estimators are preferable because they are guaranteed to yield normalised densities.

In future work, we would like to apply our approach to further kernels and conduct a full simulation study of the finite-sample properties of the estimators.

\appendix

\section{Proofs}

\label{app:proofs}
\begin{lem}
\label{lem:integral-balloon}The integral

\[
L=\int\frac{dy}{h\left(x\right)}\phi\left(\frac{y-x-h^{p}\left(x\right)\delta\left(x\right)}{h\left(x\right)}\right)\theta\left(y\right)
\]
is given by
\begin{align*}
L & =\sum_{k=0}^{\infty}A_{k}\left(x\right)\int dz\,\phi\left(z\right)z^{k},\\
\text{where}\quad A_{k}\left(x\right) & =h^{k}\left(x\right)\sum_{j=k}^{\infty}\frac{\theta^{\left(j\right)}\left(x\right)}{k!\left(j-k\right)!}\left[h^{p}\left(x\right)\delta\left(x\right)\right]^{j-k}.
\end{align*}
\end{lem}
\begin{proof}
We make the change of variables
\[
z=\frac{y-x-h^{p}\left(x\right)\delta\left(x\right)}{h\left(x\right)}
\]
such that, holding $x$ constant,
\begin{align*}
dz & =\frac{dy}{h\left(x\right)}\\
\text{and}\quad L & =\int dz\,\phi\left(z\right)\theta\left(x+h^{p}\left(x\right)\delta\left(x\right)+zh\left(x\right)\right).
\end{align*}
Expanding $\theta$ as a Taylor series about $x$ yields
\[
L=\int dz\,\phi\left(z\right)\sum_{j=0}^{\infty}\frac{\theta^{\left(j\right)}\left(x\right)}{j!}\left(h^{p}\left(x\right)\delta\left(x\right)+zh\left(x\right)\right)^{j}.
\]
Using the binomial theorem and exchanging the order of summation gives
\begin{align*}
L & =\int dz\,\phi\left(z\right)\sum_{j=0}^{\infty}\frac{\theta^{\left(j\right)}\left(x\right)}{j!}\sum_{k=0}^{j}\binom{j}{k}z^{k}h^{k+p\left(j-k\right)}\left(x\right)\delta^{j-k}\left(x\right)\\
 & =\sum_{k=0}^{\infty}A_{k}\left(x\right)\int dz\,\phi\left(z\right)z^{k},\\
\text{where}\quad A_{k}\left(x\right) & =h^{k}\left(x\right)\sum_{j=k}^{\infty}\frac{\theta^{\left(j\right)}\left(x\right)}{k!\left(j-k\right)!}\left[h^{p}\left(x\right)\delta\left(x\right)\right]^{j-k}.
\end{align*}

\end{proof}

\begin{proof}[Proof of Theorem~\ref{thm:balloon-moments}]
From Eqs.~\eqref{eq:generalised-mean} and \eqref{eq:balloon-definition}, the first moment of a shifted balloon estimator is
\[
\left\langle \hat{f}\left(x\right)\right\rangle =\int\frac{dy}{h\left(x\right)}\,K\left(\frac{y-x-h^{p}\left(x\right)\delta\left(x\right)}{h\left(x\right)}\right)f\left(y\right).
\]
Let $\phi\left(z\right)=K\left(z\right)$ and $\theta\left(y\right)=f\left(y\right)$. Then by Lemma~\ref{lem:integral-balloon}
\begin{align}
\left\langle \hat{f}\left(x\right)\right\rangle  & =\sum_{k=0}^{\infty}A_{k}\left(x\right)\int dz\,K\left(z\right)z^{k},\label{eq:balloon-mean-proof}\\
\text{where}\quad A_{k}\left(x\right) & =h^{k}\left(x\right)\sum_{j=k}^{\infty}\frac{f^{\left(j\right)}\left(x\right)}{k!\left(j-k\right)!}\left[h^{p}\left(x\right)\delta\left(x\right)\right]^{j-k}.\nonumber
\end{align}
Recall that by Definition~\ref{def:order-p-kernel}, the integral in Eq.~\eqref{eq:balloon-mean-proof} vanishes for $0<k<p$ and is equal to unity for $k=0,\,p$ such that
\[
\left\langle \hat{f}\left(x\right)\right\rangle =A_{0}\left(x\right)+A_{p}\left(x\right)+o\left(h^{p}\left(x\right)\right).
\]

According to Eqs.~\eqref{eq:generalised-variance} and \eqref{eq:balloon-definition}, the variance of a shifted balloon estimator is
\begin{equation}
\var\hat{f}\left(x\right)=\frac{1}{n}\int\frac{dy}{h^{2}\left(x\right)}\,K^{2}\left(\frac{y-x-h^{p}\left(x\right)\delta\left(x\right)}{h\left(x\right)}\right)f\left(y\right)-\frac{\left\langle \hat{f}\left(x\right)\right\rangle ^{2}}{n}.\label{eq:balloon-var-proof}
\end{equation}
We use Lemma~\ref{lem:integral-balloon} with $\phi\left(z\right)=K^{2}\left(z\right)$and $\theta\left(y\right)=f\left(y\right)/h\left(x\right)$ such that the variance becomes
\[
\var\hat{f}\left(x\right)=\frac{f\left(x\right)\kappa}{nh\left(x\right)}+o\left(n^{-1}h^{-1}\left(x\right)\right).
\]
The second term in Eq.~\eqref{eq:balloon-var-proof} is absorbed by $o\left(n^{-1}h^{-1}\left(x\right)\right)$ because of its zero-order dependence on $h$$\left(x\right)$.\end{proof}
\begin{lem}
\label{lem:integral-smoothing}The integral
\[
L=\int\frac{dy}{h\left(y\right)}\phi\left(\frac{y-x+h^{p}\left(y\right)\delta\left(y\right)}{h\left(y\right)}\right)\theta\left(y\right)
\]
is given by
\begin{align*}
L & =\sum_{k=0}^{\infty}B_{k}\left(x\right)\int dz\,\phi\left(z\right)z^{k},\\
\text{where}\quad B_{k}\left(x\right) & =\sum_{j=k}^{\infty}\frac{1}{k!\left(j-k\right)!}\frac{d^{j}}{dx^{j}}\left(\theta\left(x\right)h^{k+p\left(j-k\right)}\left(x\right)\left(-\delta\left(x\right)\right)^{j-k}\right)
\end{align*}
if
\begin{equation}
z\left(y,x\right)=\frac{y-x+h^{p}\left(y\right)\delta\left(y\right)}{h\left(y\right)}\label{eq:smoothing-change-of-variables}
\end{equation}
is a monotonic function of $y$ for all $x$ such that the inverse $y\left(z,x\right)$ exists.\end{lem}
\begin{proof}
We make the change of variables in Eq.~\eqref{eq:smoothing-change-of-variables} such that, holding $x$ constant,
\begin{align}
dz & =\left(\frac{\frac{d}{dy}\left(y-x+h^{p}\left(y\right)\delta\left(y\right)\right)}{h\left(y\right)}-\frac{y-x+h^{p}\left(y\right)\delta\left(y\right)}{h\left(y\right)}\frac{h'\left(y\right)}{h\left(y\right)}\right)dy\nonumber \\
 & =\frac{dy}{h\left(y\right)}\frac{d}{dy}\left(y-x+h^{p}\left(y\right)\delta\left(y\right)-zh\left(y\right)\right)\nonumber \\
\text{and}\quad L & =\int dz\,\phi\left(z\right)\times\frac{\theta\left(y\left(z,x\right)\right)}{\left[\frac{d}{dy}\left(y-x+h^{p}\left(y\right)\delta\left(y\right)-zh\left(y\right)\right)\right]_{y=y\left(z,x\right)}}.\label{eq:smoothing-transformed}
\end{align}

For fixed $z$, we express the fraction in the integrand of Eq.~\eqref{eq:smoothing-transformed} as the contour integral
\begin{equation}
\ell=\frac{1}{2\pi i}\oint_{\gamma}d\tau\,\frac{\theta\left(\tau\right)}{\tau-x+h^{p}\left(\tau\right)\delta\left(\tau\right)-zh\left(\tau\right)},\label{eq:contour-integral}
\end{equation}
where $\tau$ is an integration variable. The contour $\gamma$ encloses the points defined by $\tau=y\left(z,x\right)$ and $\tau=x$. The simple pole at $\tau=y\left(z,x\right)$ has residue (see Theorem 8.15 in \citet{Howie2008})
\[
\frac{\theta\left(y\left(z,x\right)\right)}{\left[\frac{d}{dy}\left(y-x+h^{p}\left(y\right)\delta\left(y\right)-zh\left(y\right)\right)\right]_{y=y\left(z,x\right)}}
\]
such that the contour integral in Eq.~\eqref{eq:contour-integral} is indeed equal to the second term in Eq.~\eqref{eq:smoothing-transformed} by Cauchy's residue theorem.

Expanding $\ell$ as a power series about $\tau=x$ yields
\begin{align*}
\ell & =\frac{1}{2\pi i}\oint_{\gamma}d\tau\,\frac{\theta\left(y\right)}{\tau-x}\sum_{j=0}^{\infty}\left(\frac{zh\left(\tau\right)-h^{p}\left(x\right)\delta\left(\tau\right)}{\tau-x}\right)^{j}\\
 & =\frac{1}{2\pi i}\oint_{\gamma}d\tau\,\theta\left(y\right)\sum_{j=0}^{\infty}\sum_{k=0}^{j}\binom{j}{k}\frac{z^{k}h^{k+p\left(j-k\right)}\left(\tau\right)\left(-\delta\left(\tau\right)\right)^{j-k}}{\left(\tau-x\right)^{j+1}}.
\end{align*}
The second equality follows by the binomial theorem. The $j^{\text{th}}$ term gives rise to a pole of order $j+1$ at $\tau=x$ with residue (see Theorem 8.17 in \citet{Howie2008})
\[
R_{j}=\sum_{k=0}^{j}\frac{z^{k}}{k!\left(j-k\right)!}\frac{d^{j}}{dx^{j}}\left(\theta\left(x\right)h^{k+p\left(j-k\right)}\left(x\right)\left(-\delta\left(x\right)\right)^{j-k}\right)
\]
such that $\ell=\sum_{j=0}^{\infty}R_{j}$. Substituting back into Eq.~\eqref{eq:smoothing-transformed} and exchanging the order of summation yields
\begin{align*}
L & =\int dz\,\phi\left(z\right)\sum_{j=0}^{\infty}\sum_{k=0}^{j}\frac{z^{k}}{k!\left(j-k\right)!}\frac{d^{j}}{dx^{j}}\left(\theta\left(x\right)h^{k+p\left(j-k\right)}\left(x\right)\left(-\delta\left(x\right)\right)^{j-k}\right)\\
 & =\sum_{k=0}^{\infty}B_{k}\left(x\right)\int dz\,\phi\left(z\right)z^{k}\\
\text{where}\quad B_{k}\left(x\right) & =\sum_{j=k}^{\infty}\frac{1}{k!\left(j-k\right)!}\frac{d^{j}}{dx^{j}}\left(\theta\left(x\right)h^{k+p\left(j-k\right)}\left(x\right)\left(-\delta\left(x\right)\right)^{j-k}\right).
\end{align*}

\end{proof}

\begin{proof}[Proof of Theorem~\ref{thm:smoothing-moments}]
From Eqs.~\eqref{eq:generalised-mean} and \eqref{eq:smoothing-definition}, the first moment of a shifted sample-smoothing estimator is
\[
\left\langle \hat{f}\left(x\right)\right\rangle =\int\frac{dy}{h\left(y\right)}\,K\left(\frac{y-x+h^{p}\left(y\right)\delta\left(y\right)}{h\left(y\right)}\right)f\left(y\right).
\]
Let $\phi\left(z\right)=K\left(z\right)$ and $\theta\left(y\right)=f\left(y\right)$. Then by Lemma~\ref{lem:integral-smoothing}
\begin{align}
\left\langle \hat{f}\left(x\right)\right\rangle  & =\sum_{k=0}^{\infty}B_{k}\left(x\right)\int dz\,K\left(z\right)z^{k},\label{eq:smoothing-mean-proof}\\
\text{where}\quad B_{k}\left(x\right) & =\sum_{j=k}^{\infty}\frac{1}{k!\left(j-k\right)!}\frac{d^{j}}{dx^{j}}\left(f\left(x\right)h^{k+p\left(j-k\right)}\left(x\right)\left(-\delta\left(x\right)\right)^{j-k}\right).\nonumber
\end{align}
Recall that by Definition~\ref{def:order-p-kernel}, the integral in Eq.~\eqref{eq:smoothing-mean-proof} vanishes for $0<k<p$ and is equal to unity for $k=0,\,p$ such that
\[
\left\langle \hat{f}\left(x\right)\right\rangle =B_{0}\left(x\right)+B_{p}\left(x\right)+o\left(h^{p}\left(x\right)\right).
\]

According to Eqs.~\eqref{eq:generalised-variance} and \eqref{eq:smoothing-definition}, the variance of a shifted sample-smoothing estimator is
\begin{equation}
\var\hat{f}\left(x\right)=\frac{1}{n}\int\frac{dy}{h^{2}\left(y\right)}\,K^{2}\left(\frac{y-x+h^{p}\left(y\right)\delta\left(y\right)}{h\left(y\right)}\right)f\left(y\right)-\frac{\left\langle \hat{f}\left(x\right)\right\rangle ^{2}}{n}.\label{eq:smoothing-var-proof}
\end{equation}
We use Lemma~\ref{lem:integral-balloon} with $\phi\left(z\right)=K^{2}\left(z\right)$and $\theta\left(y\right)=f\left(y\right)/h\left(y\right)$ such that the variance becomes
\[
\var\hat{f}\left(x\right)=\frac{f\left(x\right)\kappa}{nh\left(x\right)}+o\left(n^{-1}h^{-1}\left(x\right)\right).
\]
The second term in Eq.~\eqref{eq:smoothing-var-proof} is absorbed by $o\left(n^{-1}h^{-1}\left(x\right)\right)$ because of its zero-order dependence on $h$$\left(x\right)$.
\end{proof}

\section{Cross-validation}

\label{app:cross-validation}The quantity
\begin{equation}
M=\int dx\,\hat{f}^{2}\left(x\right)-\frac{2}{n}\sum_{i=1}^{n}\hat{f}_{i}\left(X_{i}\right)\label{eq:mise-estimator}
\end{equation}
is an unbiased estimator of the MISE of the density estimator $\hat{f}\left(x\right)$ up to an additive constant, where $\hat{f_{i}}\left(X_{i}\right)$ is the density estimated from all samples except $X_{i}$ evaluated at $X_{i}$. Details can be found in Section~3.4.3 of \citet{Silverman1986}. The second term in Eq.~\eqref{eq:mise-estimator} is straightforward to evaluate whereas the first requires integration of the density estimate over the whole domain. Fortunately, the integral can be evaluated analytically for the proper gamma estimator in Eq.~\eqref{eq:jeon-kim-estimator} and is given by
\begin{align*}
\int dx\,\hat{f}^{2}\left(x\right) & =\frac{1}{n^{2}}\sum_{i,j=1}^{n}\int dx\,\mathrm{G}\left(x;1+\frac{X_{i}}{\sigma^{2}},\sigma^{2}\right)\mathrm{G}\left(x;1+\frac{X_{j}}{\sigma^{2}},\sigma^{2}\right)\\
 & =\frac{1}{n^{2}}\sum_{i,j=1}^{n}\frac{\Gamma\left(1+\frac{X_{i}+X_{j}}{\sigma^{2}}\right)2^{-1-\frac{X_{i}+X_{j}}{\sigma^{2}}}}{\Gamma\left(1+X_{i}/\sigma^{2}\right)\Gamma\left(1+X_{j}/\sigma^{2}\right)}.
\end{align*}
The MISE estimate can thus be evaluated easily without having to resort to numerical integration.

\bibliographystyle{agsm}
\bibliography{paper}


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\renewcommand{\footnoterule}{}
\markright{scale}
\section*{\hspace*{-1.6cm} scale}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Scale a signal using the Mellin transform.
\end{minipage}
\vspace*{.2cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
S = scale(x,a,fmin,fmax,N)
\end{verbatim}
\end{minipage}
\vspace*{.3cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty scale} computes the {\ty a}-scaled version of signal {\ty x} :
        $x_a(t)\ =\ a^{-\frac{1}{2}}\ x(\frac{t}{a})$ using the Mellin
        transform.\\

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty x} & signal in time to be scaled ({\ty Nx=length(x)})\\
        {\ty a} & scale factor. {\ty a} $<$ 1 corresponds to a compression
		in the time domain and {\ty a} $>$ 1 to a dilation. {\ty a}
		can be a vector. 		& {\ty 2}\\
        {\ty fmin, fmax} & respectively lower and upper frequency bounds of
           the analyzed signal. These parameters fix the equivalent
           frequency bandwidth (expressed in Hz). When unspecified, you
           have to enter them at the command line from the plot of the
           spectrum. {\ty fmin} and {\ty fmax} must be $>${\ty 0} and
	   $\leq${\ty 0.5}\\
        {\ty N} & number of analyzed voices & auto\footnote{This value,
	determined from {\ty fmin} and {\ty fmax}, is the
	next-power-of-two of the minimum value checking the non-overlapping
	condition in the fast Mellin transform.}\\
\hline  {\ty S} & the {\ty a}-scaled version of signal {\ty x}. Length of
		{\ty S} can be larger than length of {\ty x} if {\ty a} $>$
		1. If {\ty a} is a vector of length {\ty L}, {\ty S} is
           a matrix with {\ty L} columns. {\ty S} has the same energy as {\ty x}.\\
\hline
\end{tabular*}
\end{minipage}
\vspace*{.3cm}

{\bf \large \sf Example}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Dilate a Klauder-wavelet by a factor of 2 :
\begin{verbatim}
         sig=klauder(100); S=scale(sig,2,.05,.45,100);
         subplot(211); plot(sig);
         subplot(212); plot(real(S(51:150)));
\end{verbatim}
\end{minipage}
\vspace*{.3cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
fmt.
\end{verbatim}
\end{minipage}


\section{Value Size}
\label{sec:value-size}

In Shelley, the protocol parameter $\var{minUTxOValue}$ is used to
disincentivize attacking nodes by putting many small outputs on the
UTxO, permanently blocking memory. Because members of the $\Value$
type can be arbitrary large in principle, the multi-asset ledger must
accomodate by scaling the minimum amount of Ada in an output by the
size of the output, which means that we need a function to determine
the size of a member of $\Value$.

How much memory is used exactly is an implementation detail, but some facts are:
\begin{itemize}
  \item Every output contains Ada, so Ada can be regarded as part of the constant amount of space.
  \item The size will scale linearly with the number of $\PolicyID$s and tokens.
  \item The number of $\PolicyID$s is less than or equal to the number of tokens.
\end{itemize}

Because of the first two items in that list, we could use the following definition for $\fun{valueSize}$:

\begin{figure*}[h]
  \begin{align*}
    & \fun{valueSize} \in \Value \to \N \\
    & \fun{valueSize} ~\var{v} = k_0 + k_1 * |\{ (\var{pid}, \var{aid}) : \var{v}~\var{pid}~\var{aid} \neq 0
            \land (\var{pid}, \var{aid}) \neq (\mathsf{adaID}, \mathsf{adaToken}) \}| \\
    & \phantom{\fun{valueSize} ~\var{v} = k_0} + k_2 * |\{ \var{pid} : \exists aid : v~pid~aid \neq 0 \}|
  \end{align*}
  \caption{Value Size}
  \label{fig:test}
\end{figure*}

Here, $k_0$ reflects the constant part, i.e. Ada, but also everything
in the output that is not included in the $\Value$ (i.e. the address)
(maybe this means the function should be renamed). $k_1$ reflects the
size of non-ada tokens, and $k_2$ the size of $\PolicyID$s.

There are some trade-offs here. Because of the third item in the above
list, $k_2$ could be set to zero without changing the asymptotic
behaviour, somewhat simplifying this function. This is important,
because this function influences how much users have to send each
other for transactions to be valid, so having a function that a user
can easily evaluate by themselves is important.


\section{Sort and Grade}
\label{sortandgrade}

A simple sort-up/down algorithm for Booleans counts the number
of {\apl 1}s in the argument, generates a vector of that
length of {\apl 1}s, then pads it at the front/back with the appropriate
number of {\apl 0}s, to give a result whose shape matches the argument:

\medskip
{\apl SortAscending\qlarrow\qlbrace\qlpar\qbar\qrho\qomega\qrpar\quarrow\qlpar\qplus\qslash\qomega\qrpar\qrho\01\qrbrace}\\
~\\
{\apl SortDescending\qlarrow\qlbrace\qlpar\qrho\qomega\qrpar\quarrow\qlpar\qplus\qslash\qomega\qrpar\qrho\01\qrbrace}\\
\medskip

\noindent Upgrade and downgrade on Boolean vectors can be written as:

\medskip
{\apl ug\qlarrow\qlbrace\qlpar\qlpar\qtilde\qomega\qrpar\qslash\qiota\qrho\qomega\qrpar\qcomma\qomega\qslash\qiota\qrho\qomega\qrbrace}\\
~\\
{\apl dg\qlarrow\qlbrace\qlpar\qlpar\qomega\qslash\qiota\qrho\qomega\qrpar\qcomma\qlpar\qtilde\qomega\qrpar\qslash\qiota\qrho\qomega\qrbrace}\\

\fixme{ Get Roger Hui grade example. Perhaps in J?}

Although all of these algorithms perform quite well, compared
to normal upgrade, but they are not as data-parallel as we would like.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS630: Database Management Systems
% Copyright 2014 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 1}

A university database contains information about professors (identified by social security number \textit{SSN}) and courses (identified by \textit{courseid}).
Professors also have a name, an address and a phone number.
Courses have a name and a number of credits.
Professors teach courses.
For each of the following situations, draw an ER diagram that describes it (assuming no further constraints hold).

\begin{enumerate}[label=(\alph*)]
\item Every professor must teach some course.

\textbf{Solution:}

\begin{figure}[H]\centering
\input{\texDirectory/hw03/hw03q01f01}
\caption{ER Diagram for Question 1$\left(a\right)$ with Chen's Notation} \label{fig:ER1}
\end{figure}

\item Every professor teaches exactly one course (no more, no less).

\textbf{Solution:}

\begin{figure}[H]\centering
\input{\texDirectory/hw03/hw03q01f02}
\caption{ER Diagram for Question 1$\left(b\right)$ with Chen's Notation} \label{fig:ER2}
\end{figure}

\item Every professor teaches exactly one course (no more, no less), and every course must be taught by some professor.

\textbf{Solution:}

\begin{figure}[H]\centering
\input{\texDirectory/hw03/hw03q01f03}
\caption{ER Diagram for Question 1$\left(c\right)$ with Chen's Notation} \label{fig:ER3}
\end{figure}

\item Modify the diagram from (a) such that a professor can have a set of addresses (which are street, city and state triples) and a set of phones.
Recall that in the ER model there can be only primitive data types (no sets).

\textbf{Solution:}

\begin{figure}[H]\centering
\input{\texDirectory/hw03/hw03q01f04}
\caption{ER Diagram for Question 1$\left(d\right)$ with Chen's Notation} \label{fig:ER4}
\end{figure}

It is assumed multiple professors may live at a single address.
There might be addresses at which no professor lives.
There might be phone numbers that do not belong to any professor.
There might be professors who have not declared their addresses or phone numbers.
As each entity must have a key attribute, \texttt{Address ID} and \texttt{Phone ID} of \texttt{Address} and \texttt{Phone} entities are also indicated.

\item Modify the diagram from (d) such that professors can have a set of addresses, and at each address there is a set of phones.

\textbf{Solution:}

It is assumed multiple professors may live at a single address.
There might be addresses at which no professor lives.
There might be phone numbers that do not belong to any professor.
There might be professors who have not declared their addresses.
There might be addresses with no phone.
As each entity must have a key attribute, \texttt{Address ID} and \texttt{Phone ID} of \texttt{Address} and \texttt{Phone} entities are also indicated.

\begin{figure}[H]\centering
\input{\texDirectory/hw03/hw03q01f05}
\caption{ER Diagram for Question 1$\left(e\right)$ with Chen's Notation} \label{fig:ER5}
\end{figure}

\end{enumerate}


\subsection{Intertemporal decision theory}

\(U_T=\sum_[t=T]d_tU(x_t)\)


\addtocontents{toc}{\setcounter{tocdepth}{2}}
\chapter{Integration}
Integration is the opposite to differentiation. While differentiation is finding the tangent to a line, integration is finding the area under this line (usually called ``area under the graph").

\section{What Is Integration?}
Suppose a car is decelerating and the velocity-time graph of this car is conveniently the equation $v=-\left(t^2\right)+4$ as shown.

\begin{figure}[h!]
	\centering
	\begin{tikzpicture}
		\begin{axis}
		[
			restrict y to domain=0:6,
			restrict x to domain=0:3,
			xlabel=$t\ (s)$,
			ylabel=$v\ (m\ s^{-1})$,
			xmin=0,
			xmax=3,
			xtick={0,1,2},
			ymin=0,
			ymax=5,
			ytick={0,1,...,4},
			axis lines=center,
			smooth,
			samples=150
		]
			\addplot [color=black,mark=none] {-x^2+4};
		\end{axis}
	\end{tikzpicture}
	\caption{The curve of a car decelerating.}
	\label{fig:integrationBasicEx1}
\end{figure}

How can the distance (referred to as ``displacement", symbol $s$) that the car has travelled during this time be found? From a velocity-time graph, the displacement travelled is the area under the graph.

Note how finding the derivative is finding the ``higher" level; for a travelling car it the derivative of the distance over time is the speed, and the derivative of the velocity over time is the acceleration. When integrating, the ``lower" level is found; so the integral of the acceleration becomes the velocity, and the integral of the velocity is the displacement.

One way to find the area under the graph (and therefore the displacement travelled) is to approximate it using a rectangle (blue in figure \ref{fig:integrationBasicEx2}). This will give an okay estimate, but not a very accurate one. The area can be found as shown.

\begin{figure}[h!]
	\centering
	\begin{tikzpicture}
	\begin{axis}
	[
		restrict y to domain=0:6,
		restrict x to domain=0:3,
		xlabel=$t\ (s)$,
		ylabel=$v\ (m\ s^{-1})$,
		xmin=0,
		xmax=3,
		xtick={0,1,2},
		extra x ticks={0},
		ymin=0,
		ymax=5,
		ytick={0,1,...,4},
		extra y ticks={0},
		axis lines=center,
		smooth,
		samples=150
	]
		\addplot [color=black,mark=none] {-x^2+4};
		\addplot [color=blue,mark=none,domain=0:2] {4};
		\addplot [color=blue,mark=none,domain=0:2] {0};
		\addplot [color=blue,mark=none] coordinates {(2,0) (2,4)};
		\addplot [color=blue,mark=none] coordinates {(0,0) (0,4)};
	\end{axis}
	\end{tikzpicture}
	\caption{In blue is an approximation for the area.}
	\label{fig:integrationBasicEx2}
\end{figure}

\begin{align*}
	s&=\text{area under graph}\\
	s&=\text{length} \cdot \text{height}\\
	s&=2 \cdot 4\\
	s&=8 \text{ m}
\end{align*}

How could the area under the graph be found in an even more accurate way? More rectangles can be drawn as shown in figure \ref{fig:integrationBasicEx3}. Instead of using a time interval of $2$, a smaller interval could be used. For example, suppose that the differences in time are $0.5$ and not $2$ like before. $t=0.5$ is drawn until it touches the graph. The same is done for $t=1.0$, $t=1.5$, and $t=2.0$ to give four rectangles as shown. This is quite a bit more accurate than using just one rectangle.

\begin{figure}[h!]
	\centering
	\begin{tikzpicture}
	\begin{axis}
	[
		restrict y to domain=0:6,
		restrict x to domain=0:3,
		xlabel=$t\ (s)$,
		ylabel=$v\ (m\ s^{-1})$,
		xmin=0,
		xmax=3,
		xtick={0,0.5,...,2},
		extra x ticks={0},
		ymin=0,
		ymax=5,
		ytick={0,0.25,...,4},
		yticklabels={0,,0.5,,1,,1.5,,2,,2.5,,3,,3.5,,4},
		extra y ticks={0},
		axis lines=center,
		smooth,
		samples=150
	]
		\addplot [color=black,mark=none] {-x^2+4};
		\addplot [color=blue,mark=none,domain=0.0:0.5] {4};
		\addplot [color=blue,mark=none,domain=0.5:1.0] {3.75};
		\addplot [color=blue,mark=none,domain=1.0:1.5] {3};
		\addplot [color=blue,mark=none,domain=1.5:2.0] {1.75};
		\addplot [color=blue,mark=none,domain=0.0:2.0] {0};
		\addplot [color=blue,mark=none] coordinates {(2,0) (2,1.75)};
		\addplot [color=blue,mark=none] coordinates {(1.5,0) (1.5,3)};
		\addplot [color=blue,mark=none] coordinates {(1,0) (1,3.75)};
		\addplot [color=blue,mark=none] coordinates {(0.5,0) (0.5,4)};
		\addplot [color=blue,mark=none] coordinates {(0,0) (0,4)};
	\end{axis}
	\end{tikzpicture}
	\caption{In blue is a more accurate approximation for the area.}
	\label{fig:integrationBasicEx3}
\end{figure}

To figure out the area of figure \ref{fig:integrationBasicEx3}, the area of the four rectangles would be found as shown below.

\begin{align*}
	s&=\text{area under graph}\\
	s&=0.5 \cdot 4 + 0.5 \cdot 3.75 + 0.5 \cdot 3 + 0.5 \cdot 1.75\\
	s&=6.25 \text{ m}
\end{align*}

By using smaller differences in time, a more accurate value can be gotten. When this difference in time gets smaller, more rectangles need to be added together. It's almost like a bunch of sums. Eventually, the width of those rectangles will be, eventually the width will be so small that it's negligible. So it can be assumed that the area is just its height, as when $A = \text{length} \cdot \text{height}$ and length becomes negligible, area becomes $A = \text{height}$.

In other words, to work out the area under the graph, the heights of all the rectangles can be multiplied by whatever difference in time we chose (the ``width") and added together. However, the difference in time becomes so small that it becomes negligible.

This is written formally as
\begin{equation*}
	\int_{0}^{2} -t^2+4 \, dt
\end{equation*}
where:
\begin{itemize}
	\item the weird S sign is the integral symbol (it looks like an S for sum, since a lot of rectangles are summed together),
	\item the little $0$ and $2$ show where the start and end-points of the function are (see how in our above examples time only went from $0$ to $2$ seconds), this is called the lower-bound and upper-bound respectively,
	\item the $-x^2+4$ is whatever formula we chose to have,
	\item and the $dt$ stands for the difference in time that we eventually let approach to 0. Note leaving out that this and the constant of integration (covered later) are the two most often made mistakes, so definitely check that you've included them at the end of an exam!
\end{itemize}

More on areas under graphs is explained in a later chapter, focussing on further examples on finding areas under graphs and showing how you can actually use integrals to find the exact area.

\section{Indefinite Integrals}
Integrals without any lower- or upper-bounds are called indefinite integrals. This is also called doing ``antidifferentiation", as it's essentially the opposite of differentiating.

To integrate, raise the variable's power by $1$ and then divide the variable with its coefficient by the new raised power. In other words,
\begin{align}
	&\int x^n \, dx\\
	&= \frac{x^{(n+1)}}{n+1}\text{.}
\end{align}

\subsection{The Constant of Integration}
When differentiating an expression such as $f(x) = 2x^2+5$, the $5$ gets lost and all that's left is $f'(x) = 4x$. So when integrating $4x$ all that can be gotten is $\int 4x \, dx = 2x^2$. Notice that $5$ cannot be gotten back out. Therefore any indefinite integral must include the constant of integration, a $+C$.

Why is this not needed for a definite integral? Well, since two different integrals are subtracted, the two $+C$s will cancel, so it can just be left out.

\addtocontents{toc}{\setcounter{tocdepth}{1}}
\subsection{Examples}
\begin{enumerate}
	\item
	\begin{align*}
		&\int x^2 \, dx\\
		&= \frac{x^3}{3} + C
	\end{align*}

	\item
	\begin{align*}
		&\int 4x^3 \, dx\\
		&= \frac{4x^4}{4} + C\\
		&= x^4+C
	\end{align*}

	\item
	\begin{align*}
		&\int 9 \, dx\\
		&= 9x + C
	\end{align*}

	\item
	\begin{align*}
		&\int \left(x^2+x^{-7}\right) \, dx\\
		&= \frac{x^3}{3} + \frac{x^{-6}}{-6} + C\\
		&= \frac{x^3}{3} + \frac{1}{-6x^6} + C\\
		&= \frac{x^3}{3} - \frac{1}{6x^6} + C
	\end{align*}
\end{enumerate}

\section{Definite Integrals (Finding Areas Under (or Over) Graphs)}
When finding the area under a graph between two points, the integral of the graph between the two points can be found. The integral of figure \ref{fig:integralDefiniteEx1} is as shown.
\begin{figure}[h!]
	\centering
	\begin{tikzpicture}
	\begin{axis}
	[
		restrict y to domain=0:25,
		restrict x to domain=0:5,
		xlabel=$x$,
		ylabel=$y$,
		xmin=0,
		xmax=5.5,
		xticklabels={,,,$a$,$b$,},
		extra x ticks={0},
		ymin=0,
		ymax=16,
		yticklabels={,,,,},
		extra y ticks={0},
		axis lines=center,
		smooth,
		samples=150
	]
		\addplot [color=black,mark=none,name path=y] {(x-1)^2};
		\path [name path = xaxis] (axis cs:2,0) -- (axis cs:3,0);
		\addplot [pattern=north west lines,pattern color=blue] fill between[of=y and xaxis, soft clip={domain=2:3}];
		\addplot [color=blue,mark=none,dashed] coordinates {(2,0) (2,1)};
		\addplot [color=blue,mark=none,dashed] coordinates {(3,0) (3,4)};
	\end{axis};
	\end{tikzpicture}
	\caption{The hatched area is what is to be found.}
	\label{fig:integralDefiniteEx1}
\end{figure}
\begin{equation}
	\int_{a}^{b} f(x) dx
\end{equation}

If the antiderivative of $f(x)$ is given the variable $F(x)$, then the fundamental theorem of calculus can be described as shown.

\begin{equation}
	\int_{a}^{b} f(x) \, dx = F(b) - F(a)
\end{equation}

In other words, once the antiderivative is found, substitute $x$ for $a$ and subtract it by the antiderivative with $x$ substituted for $b$.

The notation for subtracting two antiderivatives is shown in the examples below; seeing it is often easier to understand than reading it.

\subsection{Examples}
\begin{enumerate}
	\item
	The graph of the function $y=x^2+2x$ is sketched in figure \ref{fig:integralDefiniteEx2}. Find the area of the shaded part of the graph.
	\begin{figure}[h!]
		\centering
		\begin{tikzpicture}[cmark/.style={label={[anchor=center]:\pgfuseplotmark{#1}}}]
			\begin{axis}
			[
				restrict y to domain=-5:25,
				restrict x to domain=-1:4,
				xlabel=$x$,
				ylabel=$y$,
				xmin=-1,
				xmax=4.5,
				xtick={-1,0,...,4},
				xticklabels={,,1,,3},
				ymin=-5,
				ymax=26,
				ytick={-5,0,...,25},
				yticklabels={,,,,,,,},
				axis lines=center,
				smooth,
				samples=150
			]
				\addplot [color=black,mark=none,name path=y] {x^2+2*x};
				\path [name path = xaxis] (axis cs:1,0) -- (axis cs:3,0);
				\addplot [pattern=north west lines,pattern color=blue] fill between[of=y and xaxis, soft clip={domain=1:3}];
				\addplot [color=blue,mark=none,dashed] coordinates {(1,0) (1,3)};
				\addplot [color=blue,mark=none,dashed] coordinates {(3,0) (3,15)};
				\coordinate [label=below right:{$0$},black,cmark=*] (o) at (0,0);
			\end{axis};
		\end{tikzpicture}
		\caption{Graph of $y=x^2+2x$, not to scale.}
		\label{fig:integralDefiniteEx2}
	\end{figure}

	\item
	Find the positive value of $Z$ for which $\int_{1}^{Z} (1+2x) \, dx = 4$
\end{enumerate}

\begin{enumerate}
	\item
	\begin{align*}
		&\int_{1}^{3} (x^2 + 2x) \, dx\\
		&=\left[ \frac{x^3}{3} + \frac{2x^2}{2} \right]_1^3\\
		&=\left[ \frac{x^3}{3} + x^2 \right]_1^3\\
		&=\left[ \frac{3^3}{3} + 3^2 \right] - \left[ \frac{1^3}{3} + {1^2} \right]\\
		&= \left[ 18 \right] - \left[ \frac{4}{3} \right]\\
		&= \frac{50}{3} \text{ units}^2
	\end{align*}
	Note that the units is squared, as an area is found.

	\item
	\begin{align*}
		\int_{1}^{Z} (1+2x) \, dx &= 4\\
		\left[ x+x^2 \right]_1^Z &= 4\\
		\left[ Z+Z^2 \right] - \left[ 1+1^2 \right] &= 4\\
		Z+Z^2 - 2 &= 4\\
		Z^2+Z - 6 &= 0\\
		\left(Z+3\right)\left(Z-2\right) &= 0\\
	\end{align*}
	\begin{align*}
		&Z=-3 & Z&=2\\
		&\text{Disregard, as $Z$ must be positive.} & &
	\end{align*}
\end{enumerate}

\section{Working With Negative Area}
Integrating a graph which goes below the $x$-axis will give a negative area. Is this useful?

Using the car example, the graph goes below the $x$-axis when it goes backwards. If the total displacement is to be found then yes, it's very helpful. But the total distance that the car drove would always be positive.

Generally in maths, when you have to find an area under a graph that goes into the negative then you must change a negative area to a positive one.

\subsection{Examples}
\begin{enumerate}
	\item
	Find the area under the graph of $f(x) = 2x+4$ between $x=-2$ and $x=2$.

	\item
	Figure \ref{fig:integralNegativeAreaEx1} shows the graph of $y=8-2x-x^2$. Find the area of the graph between $x=-5$ and $x=2$.
\end{enumerate}

\begin{figure}[h!]
	\centering
	\begin{tikzpicture}[cmark/.style={label={[anchor=center]:\pgfuseplotmark{#1}}}]
		\begin{axis}
		[
			restrict y to domain=-7.75:9.5,
			restrict x to domain=-5.5:9,
			xlabel=$x$,
			ylabel=$y$,
			xmin=-5.5,
			xmax=2.75,
			xtick={-5,-4,...,2},
			xticklabels={,,},
			ymin=-7.75,
			ymax=9.75,
			ytick={-7,-6,...,9},
			yticklabels={,,},
			axis lines=center,
			smooth,
			samples=75
		]
			\addplot [color=black,mark=none,name path=y] {8-2*x-x^2};
			\path [name path = xaxis] (axis cs:-5,0) -- (axis cs:2,0);
			\addplot [pattern=north west lines,pattern color=blue] fill between[of=y and xaxis, soft clip={domain=-5:2}];
			\addplot [color=blue,mark=none,dashed] coordinates {(-5,0) (-5,-7)};
			\coordinate [label=below right:{$0$},black,cmark=*] (o) at (0,0);
			\coordinate [label=above:{$-5$},black,cmark=*] (a) at (-5,0);
			\coordinate [label=below right:{$-4$},black,cmark=*] (b) at (-4,0);
			\coordinate [label=below left:{$2$},black,cmark=*] (c) at (2,0);
		\end{axis};
	\end{tikzpicture}
	\caption{Graph of $y=8-2x-x^2$, not to scale.}
	\label{fig:integralNegativeAreaEx1}
\end{figure}

\begin{enumerate}
	\item
	\begin{align*}
		&\int_{-2}^{2} 2x-4 \, dx\\
		&=\left[ x^2-4x \right]_{-2}^2\\
		&=\left[ 2^2-4\left(2\right) \right] - \left[ \left(-2\right)^2-4\left(-2\right) \right]\\
		&=-4-12\\
		&=-16 \text{ units}^2
	\end{align*}
	As the area must be positive, the area is $16$ units$^2$.

	\item
	$A_{total} = A_{below} + A_{above}$
	\begin{align*}
		A_{below} &= \int_{-5}^{-4} (8-2x-x^2) dx\\
		&= \left[ 8x-x^2-\frac{x^3}{3} \right]_{-5}^{-4}\\
		&= \left[ 8\left(-4\right) - \left(-4\right)^2 - \frac{(-4)^3}{3} \right] - \left[ 8\left(-5\right) - \left(-5\right)^2 - \frac{(-5)^3}{3} \right]\\
		&= \left[ -32-16 + \frac{64}{3} \right] - \left[ -40-25 + \frac{125}{3} \right]\\
		&= \left[ -\frac{80}{3} \right] - \left[ -\frac{80}{70}\right]\\
		&= -\frac{10}{3} \text{ units}^2
	\end{align*}
	However, area is positive, so $A_{below} = \frac{10}{3}$ units$^2$.

	\begin{align*}
		A_{above} &= \int_{-4}^{2} (8-2x-x^2) dx\\
		&= \left[ 8x-x^2-\frac{x^3}{3} \right]_{-4}^{2}\\
		&= \left[ 8\left(2\right) - \left(2\right)^2 - \frac{(2)^3}{3} \right] - \left[ 8\left(-4\right) - \left(-4\right)^2 - \frac{(-4)^3}{3} \right]\\
		&= \left[ 16-4 - \frac{8}{3} \right] - \left[ -32-16 + \frac{64}{3} \right]\\
		&= \left[ \frac{28}{3} \right] - \left[ -\frac{80}{3}\right]\\
		&= \frac{28}{3} + \frac{80}{3}\\
		&= 36 \text{ units}^2
	\end{align*}

	\begin{align*}
		A_{total} &= A_{below} + A_{above}\\
		&= \frac{10}{3} + 36\\
		&= \frac{10+108}{3}\\
		&= \frac{118}{3}\\
		&= 39\frac{1}{3} \text{ units}^2
	\end{align*}
\end{enumerate}

\section{Area Between Two Graphs}
To find the area between two graphs, the are of the graph that is higher than the other along the $y$-axis needs to be subtracted by the area of the graph that is lower.

\begin{figure}[h!]
	\centering
	\begin{tikzpicture}[cmark/.style={label={[anchor=center]:\pgfuseplotmark{#1}}}]
		\begin{axis}
		[
			restrict y to domain=0:16,
			restrict x to domain=0:5,
			xlabel=$x$,
			ylabel=$y$,
			xmin=0,
			xmax=5,
			xtick={0,0.5,...,4},
			xticklabels={,,$a$,,,,,$b$,},
			ymin=0,
			ymax=4.5,
			ytick={0,1,...,4},
			yticklabels={,,},
			axis lines=center,
			smooth,
			samples=125
		]
			\addplot [color=black,mark=none,name path=Lower] {(x-2)^2+0.5};
			\addplot [color=black,mark=none,name path=Upper] {0.5*x+1};
			\addplot [pattern=north west lines,pattern color=blue] fill between[of=Lower and Upper, soft clip={domain=1:3.5}];
			\coordinate [label=below left:{$y=f(x)$},black] (a) at (4,4.5);
			\coordinate [label=below:{$y=g(x)$},black] (b) at (4.25,3.125);
		\end{axis}
	\end{tikzpicture}
	\caption{The area between the two graphs is shaded here, they intersect at points $a$ and $b$.}
	\label{fig:integrationAreaBetweenExplanation}
\end{figure}

In other words,
\begin{enumerate}
	\item make a sketch, then judge which graph will be the upper and which will be the lower one,
	\item find the points of intersection,
	\item integrate and evaluate between the two points.
\end{enumerate}

For the example in figure \ref{fig:integrationAreaBetweenExplanation}, the area could be found by calculating,
\begin{equation}
	A=\int_{a}^{b}(f(x)) - (g(x)) \, dx\text{.}
\end{equation}

\subsection{Example}
Find the area enclosed by $g(x) = x+3$ and $f(x) = 6-x-x^2$.

The lines meet when $f(x) = g(x)$.
\begin{align*}
	x+3&=6-x-x^2\\
	x^2+2x-3&=0\\
	(x+3)(x-1)&=0\\
\end{align*}
\begin{align*}
	x&=-3 & x&=1
\end{align*}
The lines meet at $x=-3$ and $x=1$.
\begin{align*}
	A&=\int_{-3}^{1} (f(x)) - (g(x)) \, dx\\
	&= \int_{-3}^{1} (6-x-x^2) - (x+3) \, dx\\
	&= \int_{-3}^{1} 6-x-x^2 -x -3 \, dx\\
	&= \int_{-3}^{1} 3-2x-x^2 \, dx\\
	&= \left[ 3x-x^2-\frac{x^3}{3} \right]_{-3}^{1}\\
	&= \left[ 3(1)-1^2-\frac{1^3}{3} \right] - \left[ 3(-3)-(-3)^2-\frac{(-3)^3}{3} \right]\\
	&= \left[ \frac{5}{3} \right] - \left[ -9 \right]\\
	&= 10 \frac{2}{3} \text{ units}^2
\end{align*}


\section{Images as layers of fluorescence}
\label{imaging:model}

The goal of quantitative biological imaging, in overly general terms, is to perform
a set of meaningful mathematical operations on fluorescent images.
We therefore require a mathematical model of fluorescence images
to use as a reference when discussing image correction and analysis.
Note that the model I construct below is designed
to be intuitive with respect to cell culture microscopy,
and so it differs slightly from the
more general models that it is founded upon
\cite{Schultz1974,Madiset1998,L2000,Model2001}.

A fluorescence microscopy image $I$ can be thought of as
a series of fluorescing layers, each with its own distinct properties,
that add together to create the final image \arp{fig:imaging:layers}.
In this simple model, a fluorescence image is composed of multiple
\b{foreground} and \b{background} layers that come
from distinct components of the sample.
(As described below, I use specific
definitions of these terms ``foreground'' and ``background''
that may differ from one's intuition.)
It is important to note that additivity is a special property
of fluorescence microscopy, as bright-field and other non-fluorescence signals do
not necessarily behave in this way \cite{L2000}.
Further, there are cases where the layers
of fluorescence will become non-additive,
as the fluorophores and the camera
will display non-linear behaviors in certain ranges \cite{Hiraoka1987}.
The model I describe
here, and all analysis in this chapter, assumes that
all contributing components are behaving within their linear ranges.

  \begin{figure}[!bt]
  \centering
  \includegraphics[width=3in]{FIGS/imaging/layers.pdf}
  {\singlespacing
  \caption[ Images as layers of fluorscence.]
            { A fluorescence image $I$ of a cell is the sum of
			distinct fluorescence layers. Here,
			$F_1$ is the foreground signal of interest, $F_2$
			is non-specific staining within the cell, and $B$
			is background fluorescence from the imaging surface.}
  \label{fig:imaging:layers}}
  \end{figure}

\subsubsection{Experimental image layers $F$ and $B$}

In this dissertation, foreground ($F$) refers
to any image layer that emits fluorescence in a spatially non-uniform manner
within an image; the layer is not ``flat.''
The most useful foreground layer is due to the specific binding of a
fluorescent probe to its target, as this layer is
likely the one that is under study.
Where my definition of ``foreground'' may diverge from others
is that I include spatially non-uniform fluorescence artifacts
as foreground layers. Such artifacts might include
non-specific cellular staining, cellular autofluorescence
(as for layer $F_2$ in \ar{fig:imaging:layers}), and staining artifacts
such as halos, bubbles, or bright puncta.
Many biologists refer to these artifacts
as ``background'', but I have a specific definition for this term as well.

Also specific to this dissertation, ``background'' ($B$) refers to any image
layer that emits fluorescence in a spatially uniform manner within an image
(e.g. layer $B$ in \ar{fig:imaging:layers}).
In other words, the layer is ``flat,'' aside from variation between
pixels due to measurement error.
Background layers may include reflections from the imaging
surface, autofluorescence from unbound fluorophore in the solvent, or fluorophore
that has adhered to the imaging surface.

Finally, all fluorescent image layers scale linearly
with excitation light intensity and
exposure time (again, so long as we are in the
linear range for all image components).
For convenience
we can fold excitation intensity and exposure time into a single term $t$,
which I refer to simply as ``exposure.''
Taken together we get the (yet-incomplete)
image model in \ar{eq:imaging:simpleModel} that contains $n$ foreground
and $m$ background layers.
%
\begin{equation} \label{eq:imaging:simpleModel}
I = t \left( \sum_{i=1}^n F_i + \sum_{j=1}^m B_j \right)
\end{equation}

\subsubsection{Image modification by the microsope}

The model in \ar{eq:imaging:simpleModel} describes the fluorescing
layers of an image, but there are also non-fluorescent properties
of images.
In digital fluorescence microscopy, the camera typically has some
non-zero baseline value that I refer to as the ``detector value'' $D$.
This value is a constant and does not change with
exposure \cite{Goldman2005}.

  \begin{figure}[!bt]
  \centering
  \includegraphics[width=5in]{FIGS/imaging/components.pdf}
  {\singlespacing
  \caption[ Components of a fluorescence image.]
            { A fluorescence image can be modeled by
            $I=D+S(F+B)$, where $D$ is the baseline detector value,
            $S$ is shading, $F$ is the foreground, and $B$ is the
            background. Here, synthetic images of two foreground
            objects demonstrate each of these
            components. Plots indicate pixel values along the white line
            drawn across the image as i \b{a}. \b{a}, Two foreground objects that
            vary 2-fold in intensity are shown. \b{b}, Addition of
            background reduces the apparent fold-difference between the
            two foreground objects, as does addition of the detector
            (\b{c}). \b{d} Shading distorts the foreground and background,
            but leaves the detector contribution unchanged.}
  \label{fig:imaging:components}}
  \end{figure}

Uneven illumination of the image by the light source,
which I refer to as ``shading'' ($S$),
is an optical problem inherent to microscopy
\cite{Hiraoka1987,Inoue1997,Murray2007,Fiolka2008}. Shading
is a consequence of how light moves through lenses and so it is
not just a property of wide-field microscopy,
though this is a common misconception: shading is also present in confocal
and total internal fluorescence (TIRF) microscopy \cite{Fiolka2008,Herbert2012}.

Shading can be interpreted as variation in relative
exposure as a function of position within the image.
In other words, $S$ modifies
$t$ in a pixel-coordinate dependent manner. Because fluorescence units
are typically arbitrary, in that their absolute values carry
no meaning, I simplify the model further by setting $t=1$
and dropping it from the equation
(note that analyses making use of varied exposure
times could also collapse $S$ and $t$ into a single term).
By including the $S$ and $D$ components,
and a noise term $\epsilon$ to absorb
experimental error, we get the complete model in
\ar{eq:imaging:fullModel}.

I simplify the model further by collapsing all
foreground and background layers into
single terms. The resulting image model in \ar{eq:imaging:simpleFullModel}
is used throughout this text.
This simplification is useful because the basic image
correction and analysis methods presented in this chapter do not
distinguish between foreground layers or between background layers.
It is useful to keep the multi-layer model in mind, however, as
it will help when trying to untangle the overall fluorescence behavior
of experimental images.
See \ar{fig:imaging:components} for a visual
demonstration of the simple model.

\begin{gather}
I = D + S \left( \sum_{i=1}^n F_i + \sum_{j=1}^m B_j \right)\epsilon  \label{eq:imaging:fullModel} \\
I = D + S (F+B)\epsilon  \label{eq:imaging:simpleFullModel}
\end{gather}

\subsection{Properties of the image components}

With a model in place, it is useful to obtain some intuition for how to
think about images in this framework. First, note that the image itself,
and each layer that makes it up, is a matrix of intensity values
(\ar{fig:imaging:stack}). Many
analytical operations can be performed per pixel coordinate, in effect
ignoring the presence of neighboring pixels.
Other operations take into account
those neighbors (this is especially true of segmentation,
discussed later). Finally, we can think of a ``stack'' of images,
in the same way
one would stack a deck of cards. We can perform operations ``down the
stack'' at particular pixel coordinate.
Such operations include ``per-pixel'' means and medians.

  \begin{figure}[!bt]
  \centering
  \includegraphics[width=4in]{FIGS/imaging/stack.pdf}
  {\singlespacing
  \caption[ Pixel coordinate system.]
            { An image $I_{r,c}$ is a matrix of intensity values,
            with pixel coordinates
			given by row $r$ and column $c$. Analyses can
			be performed on single pixels, the entire image, image
			neighborhoods, or image stacks as shown. In the case of an image stack,
			$z$ refers to the image position within the stack. Note that $z$ need not
			refer to a height, as in the common z-stacks of confocal microscopy,
			but can also indicate different color
			channels or even entirely unrelated images. In effect, $z$
            is a ``height'' within the image stack that may not
            correspond to the physical height of an imaged within a sample.}
  \label{fig:imaging:stack}}
  \end{figure}

Each component of the image model in \ar{eq:imaging:simpleFullModel}
is a matrix of the same dimensions,
and each has distinct properties. The foreground components
of $F$ have completely idiosyncratic
values, both within and between images, as these
values depend on where cells are, what is
being stained, and what kinds of randomly-positioned
artifacts are present. Indeed, the
unknown behaviors of the foreground layers are
what we typically aim to understand
via image analysis.

The background layers of $B$ are defined to
be more predictable, in that they are
unchanging by position within an image. The background should
be constant between images as well,
when identical experimental conditions
are used to obtain those images.
However, the total background
may change as a consequence of some experimental
perturbation. For example, some small
molecules used as drugs may fluoresce
and so add an additional background layer.
Finally, note that due to shading these
``flat'' background layers will appear
distorted in uncorrected images.
To clarify, then, I define background
layers as those that are constant across an image
in the absence of shading.

The detector layer $D$ has the simplest behavior, as it can be considered
constant regardless of the imaging and experimental conditions. ``Constant'' in this
case means that the value at any given pixel position $D_{r,c}$ does not change
over time or between images.
The values within an image, on the other hand, may vary
(see \ar{fig:imaging:properties}a).

The shading value $S$ is problematic for analysis in that it distorts the
$B$ and $F$ layers in non-trivial ways.
Like $D$, this layer also shows variation within an image.
The shading pattern caused by the objective lens
tends to show brighter fluorescence at image centers
and weaker fluorescence at the edges. However,
the presence of other components
in the light path, such as filters, can modify this shading pattern
(\ar{fig:imaging:properties}b) \cite{VandenDoel1998}.
Unlike $D$, the shading pattern can vary between
images as well, though in \ar{imaging:correction} I show that this variation
can be both predictable and correctable. $S$ distorts the $F$ and $B$ layers
multiplicatively, and can cause as
large as 1.5- to 2-fold intensity differences across an image
\cite{Bray2013}.

  \begin{figure}[!bt]
  \centering
  \includegraphics[width=4in]{FIGS/imaging/properties.pdf}
  {\singlespacing
  \caption[ Example detector $D$ and shading $S$ patterns.]
            { Example within-image variation of the detector $D$
			and shading $S$ image components. \b{a}, Per-pixel
            averages of 10 detector images from
			two cameras are shown. Camera 1, Andor Zyla sCMOS 11-bit.
			Camera 2, Roper Scientific CoolSnap HQ2 CCD 14-bit.
            Intensities and
			images are scaled independently.
			\b{b}, Shading patterns from two optical channels. Uniformly-fluorescent
			background images were made with dissolved Hoechst or wheat
			germ agglutinin (WGA)-TRITC in the DAPI and TRITC optical channels. Note
			that each channel has a dramatically different shading pattern and overall
			degree of shading. Histograms show the all-pixel distributions
			of shading values, which are defined to have
			a median of one (a robust variant of $\E[S]\equiv 1$, defined in
			\ar{imaging:correction}).
			}
  \label{fig:imaging:properties}}
  \end{figure}

Finally, we are left with the noise term $\epsilon$. I use this
term specifically to capture measurement noise, not true biological
variability. The largest source of measurement
noise for modern fluorescence imaging
is probably due to the combination of two factors: the error in converting
photon counts to electrons, and the error introduced during transmission
of those electrons from the camera. Combined, this measurement error
introduces an intensity-dependent uncertainty in the total measured
intensity of the pixel $I_{r,c}$. In general, the relative error
decreases as a function of the square root of intensity \cite{Goldman2005}.


\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{indentfirst}
\usepackage{tikz-cd}
\usepackage{mathtools}

\theoremstyle{definition}
\newtheorem{defn}{Definition}[section]
\newtheorem{ex}{Exercise}[section]

\newcommand{\mo}[1]{\lvert #1 \rvert}
\newcommand{\mos}[1]{\lvert #1 \rvert^2}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\T}{\text{true}}
\newcommand{\F}{\text{false}}

\title{Applied Category Theory}
\author{Linxuan Ma}

\begin{document}
	\maketitle

	\abstract{Category theory is the study of formalization of mathematical structures and concepts as an abstract directed graph. This note mainly consists of topics explored in \textit{An Invitation to Applied Category Theory}, with some bonus information from theoretical category theory and general abstract algebra to aid the understanding of certain concepts illustrated in the book.}

	\section{Generative Effects}

	A central focus of category theory is the abstraction of mathematical structures and their relationships. In particular, a mapping is a crucial structure in category theory that represents an abstracted relationship between two objects. A mapping $f: X \to Y$ denotes a relationship of $X$ with $Y$.

	Notice the notational resemblance of mappings to functions, which uses $f: X \to Y$ to denote a transformation from members of $X$ to members of $Y$ (i.e. a member of the dependent product formed over the constant dependent type $\prod_{(\_:A)}B$). However, in the scope of category theory, abstraction is imperative for the sake of generalizability, and therefore we do not limit mappings to represent concrete structures of transformation between objects. Such pattern of lacking concreteness is common among structures related to category theory and abstract algebra (oMg it literally has the word "abstract" in its name what were you expecting you dum dum).

	\begin{defn}
		A \emph{mapping} $f: X \to Y$ is an observation of $X$ via the relationship it has with $Y$, conforming to:
		\begin{itemize}
			\item \textbf{Composition}: given $f: A \to B$ and $g: B \to C$, there exists an associative composition $g \circ f: A \to C$
			\item \textbf{Identity}: for every mapping $f: A \to B$ there exists a unit mapping $\textbf{1}$ such that: $$f \circ \textbf{1}_A = f = \textbf{1}_B \circ f$$
		\end{itemize}
	\end{defn}

	In other words, mappings form a monoid over the composition operation.

	One noteworthy implication of mappings is that a mapping preserves some form of structure or information. To illustrate, a monotonous function $f$ preserves order: $x \leq y \iff f(x) \leq f(y)$, a metric-preserving function preserves the distance defined in the given system: $\mo{x - y} = \mo{f(x) - f(y)}$, etc. The specific information which the mapping preserve is determined by the category it belong to.

	\subsection{Introduction to Generative Effects}

	As mentioned, the notions of generative effect and structure preservation do not exist without a bounded context. We shall first construct a simple system to illustrate such ideas.

	Consider the set $U = \{A, B, C\}$ where a subset of $U$ denotes a transitive closure of elements in $U$. We now defined $\Phi(p)$ as the observation that if elements $A$ and $C$ are connected (potentially through a connection through other elements) in the closure described by subset $p$. Note that in this particular system, there is an isomorphism between two element's connectivity in the closure described by $p$ and their mutual existence in $p$; however, viewing the system in a way that is dependent on such behavior is discouraged, as the subset $p$ and the observation $\Phi$ are  meant to be methods of abstracting the idea of "connections".

	We then define the joining of two closures. Two points $x$ and $y$ are connected in the closure $\alpha \lor \beta$ is true if and only if there exists a sequence of point $\{z_1, z_2, \dots, z_n\} \in \alpha \cup \beta$ such that:
	\begin{itemize}
		\item $x$ is connected to $z_1$
		\item $z_i$ is connected to $z_{i + 1}$
		\item $z_n$ is connected to $y$
	\end{itemize}

	The above definition of join in our system may sound unnecessary for a system with only $3$ objects, yet it is useful if our system extends with more objects in the future.

	Now consider $p = \{A, B\}$ and $q = \{B, C\}$. Observe that the observation is false in both closure represented by $p$ and $q$:
	\begin{align*}
		\Phi(p) &= \text{false} \\
		\Phi(q) &= \text{false}
	\end{align*}

	However, $p \lor q$ connects $A$ and $C$ via the sequence of objects $\{B\} \in p \cup q$, resulting in: $$\phi(p \lor q) = \text{true}$$

	We observe that the observation $\Phi$ is not preserved over the join operation, i.e. $\Phi$ is inherently lossy with respect to join. This is a simple illustration of a generative effect: a property of a composite system that cannot be observed or determined given only its constituent subsystems.

	\subsection{Ordering}

	We now introduce more intricate structures based on the system of $U$ which we devised in the last section. An ordering system is a system whose objects can be compared with an object to yield a boolean result.

	To adapt to more sophisticated observations regarding $U$, we shall expand our representation of connected closures to a partition of $U$, where each element denotes a closure of elements in $U$. To perform $P \lor Q$, for each pair of connected elements $a$ and $b$ in $Q$, we add the union of the elements which $a$ and $b$ are located in $P$ to $P$, and remove the respective constituent subset.
	\begin{align*}
		P &= \{\{A, B\}, \{C\}\} \\
		Q &= \{\{A\}, \{B, C\}\} \\
		P \lor Q &= \{\{A, B, C\}\}
	\end{align*}

	Consider an order on $U$ where $P \preceq Q$ if for every connected pair $x$ and $y$ in $p$, $x$ and $y$ are connected in $Q$. For instance: $$\{\{A\}, \{B\}, \{C\}\} \preceq \{\{A, B\}, \{C\}\}$$

	The notion of $X \preceq Y$ can be viewed as a mapping: a relationship of $X$ with $Y$. Note that the relationship $\preceq$ does not occur on any two systems of $U$. This is equivalent to the fact that not all elements are comparable, as $\preceq$ is merely a mapping denoting a relationship between two elements; there is no guarantee of its existence (unless in the case of composition, as mentioned in its definition).

	By listing all possible systems of $U$ and representing $preceq$ with arrows, we obtain a \emph{Hasse diagram}:
	\begin{equation*}\begin{tikzcd}
		{} & \{\{A, B, C\}\} \\
		\{\{A, C\}, \{B\}\} \ar[ur] & \{\{A, B\}, \{C\}\} \ar[u] & \{\{A\}, \{B, C\}\} \ar[ul] \\
		{} & \{\{A\}, \{B\}, \{C\}\} \ar[ul] \ar[u] \ar[ur]
	\end{tikzcd}\end{equation*}

	Note that in $P \lor Q$, $P \preceq (P \lor Q)$ and $Q \preceq (P \lor Q)$. Interestingly, for any system $R$, $P \preceq R$ and $Q \preceq R$ implies that $(P \lor Q) \preceq R$. From the above diagram, we can conclude the formal definition of join:
	\begin{defn}
		A joined system $P \lor Q$ is the smallest system that is bigger than both $P$ and $Q$, and thereby satisfies:
		\begin{itemize}
			\item $P \preceq (P \lor Q)$
			\item $Q \preceq (P \lor Q)$
			\item $P \preceq R,\ Q \preceq R \iff (P \lor Q) \preceq R$
		\end{itemize}
	\end{defn}

	There is also an order on the boolean values $\mathbb{B} = \{\T, \F\}$: $\F \preceq \T$. In this context, the relationship $\preceq$ can be viewed as the term "implies". The joins of $\mathbb{B}$ are:
	\begin{align*}
		\T \lor \T &= \T \\
		\T \lor \F &= \T \\
		\F \lor \T &= \T \\
		\F \lor \F &= \F
	\end{align*}

	Note the similarity with the logical operator "or".
	The relationship $P \preceq Q$ is preserved over the aforementioned observation $\Phi$: $\Phi(P) \lor \Phi(Q) \preceq \Phi(P \lor Q)$.

	\subsubsection{Partitions and Equivalence}

	The partition of a set is useful in denoting a relationship between elements of the set, such as a closure under certain conditions. Formally, the definition of a partition is:
	\begin{defn}
		A \emph{partition} of $A$ is a set $P$, where each element $A_p$ is a nonempty subset of $A$, satisfying:
		\begin{itemize}
			\item $A = \bigcup_{p \in P} A_p$
			\item $p \neq q \Rightarrow A_p \cap A_q = \varnothing$
		\end{itemize}
	\end{defn}
	\begin{defn}
		An \emph{equivalence relation} on set $A$ is a binary relation $\sim$ that satisfies the following:
		\begin{itemize}
			\item \textbf{Reflexivity}: $\forall a \in A.\ a \sim a$
			\item \textbf{Symmetry}: $a \sim b \iff b \sim a$
			\item \textbf{Transitivity}: $a \sim b,\ b \sim c \Rightarrow a \sim c$
		\end{itemize}
	\end{defn}

	There exists a bijection between all partitions of a set $A$ and the equivalence relations on $A$.

	Observe that a equivalence relation $a \sim b$ can be defined as $a \in A_p$ and $b \in A_p$ given a partition $P$. In other words, $a$ is equivalent to $b$ if they are elements of the same part of a given partition. This satisfies the reflexivity ($a$ is always in the part as itself), symmetry and transitivity axioms.

	Dually, given an equivalence relation $\sim$ on set $A$, the set of transitive closures of $A$ in terms of $\sim$ forms a partition.

	Similarly, a partition of set $A$ can be expressed via a surjective function $f: A \to P$, where $P(a)$ denotes the part of the partition containing $a$ for $a \in A$. The pre-images $f^{-1}(p)$ for each $p \in P$ form a partition of $A$.

	\subsubsection{Preorders}

	As mentioned in the previous section, an ordering system is a system equipped with an order relation $\preceq$. The formal definition of an order relation is:
	\begin{defn}
		A \emph{preorder relation} on a set $A$ is a binary relation (denoted with $\preceq$) on the elements of $A$ that satisfies the following constraints:
		\begin{itemize}
			\item \textbf{Reflexivity}: $a \preceq a$ for all $a \in A$
			\item \textbf{Transitivity}: $a \preceq b,\ b \preceq c \Rightarrow a \preceq c$ for all $a, b, c \in A$
			\item \textbf{Equivalence}: $a \preceq b,\ b \preceq a \iff a \cong b$
		\end{itemize}
	\end{defn}

	The aforementioned set of booleans $\mathbb{B}$ forms a preorder with the relation $\F \preceq \T$:
	\begin{equation*}\begin{tikzcd}
		\T \ar[loop right] \\ \F \ar[u] \ar[loop left]
	\end{tikzcd}\end{equation*}

	Observe that a preorder $(A, =)$ can be defined on any set $A$. By defining the preorder relation as $=$, we enforce that a preorder relation $a \preceq b$ only occur when $a = b$. If $a \neq b$ then neither $a \preceq b$ nor $b \preceq a$ holds. A preorder satisfying the above is a \emph{discrete preorder}.

	The dual of discrete preorder is \emph{codiscrete preorder}, where for any $a, b \in A$ there exist relations $a \preceq b$ and $b \preceq a$. The formal definition of such preorder is a preorder with the total binary relation: $$A \times A \subseteq A \times A$$.

	\subsubsection{Partial Orders}

	\emph{Skeletality} refers to the requirement that $a \cong b$ implies $a = b$. A skeletal preorder is a \emph{partially ordered set}:
	\begin{defn}
		A \emph{partially ordered set} (poset) is a preorder $A$ with the additional constraint that $\forall a, b \in A.\ a \cong b \iff a = b$.
	\end{defn}

	Any discrete order already satisfies the constraint for a partially ordered set as there is no relation mapping between elements.

	Given the above definitions for orders and their representation in the form of Hasse diagrams, we can formalize the definition of a \emph{graph}:
	\begin{defn}
		A \emph{graph} $G = \{V, A, s, t\}$ is an abstract mathematical structure that encapsulates relations between elements. A graph consists of a set of vertices $V$ (often viewed as the elements of a set), a set of arrows $A$ (the relations/mappings between elements) and functions $s: A \to V$ (source function) and $t: A \to V$ (target function).
	\end{defn}

	A \emph{path} in a graph $G$ from $v_a$ to $v_b$ ($v_a, v_b \in V_G$) is any (potentially empty) sequence of composing arrows $\{a_1, a_2, \dots, a_n\}$ such that $s_G(a_1) = v_a$ and $t_G(a_n) = v_b$. Note the similarity in arrow composition and mapping composition.

	Any graph $G$ already establish a preorder relation between its vertices, as the relation can represent the existence of a path between two vertices in the graph, i.e. $a \to_G b \iff a \preceq b$ for $a, b \in V_G$. This satisfies the transitivity law due to the composition of arrows, and satisfies the reflexivity law due to the existence of a $0$-length path between a vertex and itself.

	A partial order does not enforce the existence of relation between any two objects; a partial order with a relation between any two elements is a \emph{total order}:
	\begin{defn}
		A \emph{total order} $A$ is a partial order that satisfies:
		\begin{itemize}
			\item For all $a, b \in A$, either $a \preceq b$ or $b \preceq a$. In order words, any two elements in $A$ is comparable.
		\end{itemize}
	\end{defn}

	\subsubsection{Monotone Maps}

	One significant relationship between preorders is \emph{monotone map}. A monotone map preserves ordering in a way.

	\begin{defn}
		A \emph{monotone map} $f: A \to B$ between preorders $(A, \preceq_A)$ and $(B, \preceq_B)$ satisfies:
		\begin{gather*}
			\forall x, y \in A.\ x \preceq_A y \Rightarrow f(x) \preceq_B f(y)
		\end{gather*}
	\end{defn}

	As an example, a map $f: \mathbb{N} \to \mathbb{B}$ that maps the upper set of $x \in \mathbb{N}$ to $\T$ and other elements to $\F$ is a monotone map.

	\begin{ex}
		The set $\uparrow p \coloneqq \{p'\in P \mid p \preceq p'\}$ is an upper set for any $p \in P$. This can be proven via considering
	\end{ex}

\end{document}