%!TEX root = /home/renaud/Documents/EPL/tfe/latex/tfe.tex
\section{Preliminaries: the theory of Stochastic Differential Equations}
We introduce here the notions of stochastic differential equations (SDE's) and stochastic integrals. Those are the fundamental tools at the basis of the Lagrangian numerical methods. The discussion that comes next is based on several references, including \cite{spivakovskaya2007lagrangian} and \cite{gardiner1985stochastic,heemink2011slides,keunings,shah2017tracing}. Other references have been used for local parts of the work; they are then cited when they come in handy. Some results are stated without proofs. In such cases, unless otherwise stated, the reader may refer to \cite{gardiner1985stochastic} for formal proofs.

The idea behind the Lagrangian methods developed in this chapter is to estimate the concentration obeying an advection-diffusion-reaction equation by simulating the trajectories of a large number of particles in the flow. In this work, we restrict ourselves to advection diffusion equations of the form~\eqref{eq:C_PDE_vec}, which we recall here for the sake of readability:
\begin{equation} \label{eq:ADE}
	\frac{\partial C}{\partial t} = \nabla \cdot (-\b u C + \b K \nabla C)
\end{equation}
In the next, equation~\eqref{eq:ADE} will be referred to as the \textit{transport model}. In order to implement a numerical method tracking the fates of individual particles, an equation describing the fate of such a particle must be derived, and that equation must be consistent with the transport model. Formally, the transport model can be interpreted as a Fokker-Planck equation, namely the partial differential equation governing the time evolution of the probability density function $p(\b x,t)$ of the position of a particle. The correspondence is made by interpreting the concentration as the probability density function: $p=C$.

At the microscopic scale, Brownian diffusion is modeled by a stochastic force acting on the particles. This force is interpreted as the resultant of atomic bombardment on the particle. Intuitively, the direction of the force due to atomic bombardment is constantly changing, and at different times the particle is hit more on one side than another, leading to the seemingly random nature of the force, and hence of the motion. Therefore, the differential equation governing the position $\b x(t)$ of a particle is stochastic. For example, Langevin proposed in 1908 an equation governing the position of a Brownian particle, which in one dimension can be written in the form :
\begin{equation} \label{eq:Langevin}
	\frac{dx}{dt} = a(x,t) + b(x,t)\xi(t),
\end{equation}
where $x$ is the position of the particle, $a(x,t)$ and $b(x,t)$ are known functions and $\xi(t)$ is the rapidly fluctuating random term. The simplest model is obtained by considering that $\xi(t)$ is a white noise, i.e.
\begin{subnumcases}{}
		\esp{\xi(t)} = 0,\\
		\esp{\xi(t)\xi(t')} = \delta(t-t'),
\end{subnumcases}
where $\esp{\cdot}$ denotes expectation. The fact that $\xi$ has zero mean is because any nonzero mean can be absorbed in the term $a(x,t)$. The second condition states that $\xi(t)$ is uncorrelated, namely that the random force acting on a particle at a time is independent of the random forces acting on that particle at any other time. This simple form of the noise is justified by the fact that the correlation time $t_c$ of the bombardment forces (of the order of $10^{-13}$ s for Brownian motion) is much smaller than the viscous time scale ($t_v \sim 10^{-9}$ s for Brownian motion) which is itself much smaller than the Brownian motion time scale ($t_B \sim 10^{-3}$ s). One must keep in mind that is an idealization of the atomic bombardment force. It is possible to show that
\begin{equation} \label{eq:whitenoise_integral}
	\int_0^t \xi(t') \rm dt' = W(t),
\end{equation}
where $W(t)$ is the \textit{Wiener process}, a \textit{continuous} stochastic process defined by the following characteristics:
\begin{subnumcases}{\label{eq:WienerProcess}}
	W(0) = 0,\\
	W(t_2) - W(t_1)  \sim \mathcal{N}(0,t_2-t_1),\\
	\esp{[W(t_4)-W(t_3)][W(t_2)-W(t_1)]} = 0, \label{eq:independent_inc}
\end{subnumcases}
where $t_1 < t_2 < t_3 < t_4$. In other words, $W(t)$ is a zero mean gaussian process of variance $t$ which has the property of independent increments. Suppose now that $a(x,t) = a$ and $b(x,t) = b$ are constant. The above relations imply that the solution to $\eqref{eq:Langevin}$ is
\begin{equation}
	x(t) = at + bW(t),
\end{equation}
where we implicitly assumed that $x(0) = 0$. However, one can show that the Wiener process is not differentiable with probability 1. We are thus faced with a paradox here since this implies that $x(t)$ is itself non-differentiable, and hence that the Langevin equation as stated in~\eqref{eq:Langevin} \textit{does not exist mathematically}. In fact, $\xi(t)$ is the derivative of $W(t)$ in the \textit{distributive sense}. From~\eqref{eq:whitenoise_integral}, it follows directly that
\begin{equation}\label{eq:dW}
	\rm dW(t) \equiv W(t+\rm dt) - W(t) = \xi(t) \rm dt,
\end{equation}
but it is incorrect (or at least very misleading) to write $\frac{dW(t)}{dt} = \xi(t)$, since the Wiener process is nowhere differentiable with probability 1, as already stated above.

Hopefully this introductory example shows the need for some preliminary steps in order to rigorously define a Stochastic Differential Equation (SDE), and to formalize the link between SDE's and Fokker-Planck equations. This is precisely the goal of the next pages.

\subsection{Formal definition of a SDE}
In this section, we restrict ourselves to a one-dimensional problem. This allows to make the notations less cumbersome while still introducing all the tools and concepts that are needed in order to understand the two-dimensional Lagrangian models which are at the basis of our numerical resolution of two-dimensional passive tracer's transport problems. Indeed, all the results presented here can almost straightforwardly be extended to several dimensions. Consider again the Langevin equation~\eqref{eq:Langevin}. We have shown in the introduction that the equation does not really make sense under that form. What we are now going to show is that the corresponding \textit{integral equation}
\begin{equation} \label{eq:SDEint_xsi}
	x(t) = x(t_0) + \int_{t_0}^t a(x(s),s) \rm ds + \int_{t_0}^t b(x(s),s) \xi(s) \rm ds
\end{equation}
can be interpreted consistently. The first integral is a standard Lebesgue integral of a function $a$ of the stochastic process $x(t)$. The second integral has to be defined carefully, because of the presence of the white noise. By~\eqref{eq:dW}, we rewrite it as
\begin{equation} \label{eq:stoch_int}
	\int_0^t b(x(s),s) \rm dW(s),
\end{equation}
which is a kind of stochastic Stieltjes integral (see appendix~\ref{app:stieltjes}). Consider the following partition of the interval $[t_0,\,t]$:
\begin{equation}
	t_0 \le t_1 \le t_2 \le \dots \le t_{n-1} \le t_n = t,
\end{equation}
and define intermediate points $\tau_i \in [t_{i-1},\, t_i]$. Such a partition will often be used thereafter without introducing it explicitly every time it appears. The stochastic integral of $b$ with respect to the Wiener process $\int_{t_0}^t b(x(s),s) \rm dW(s)$ is defined as a mean-square limit (see appendix~\ref{app:mslim}) of the partial sum
\begin{equation}
	S_n = \sum_{i=1}^n b(x(\tau_i),\tau_i) [W(t_i)-W(t_{i-1})].
\end{equation}
Such a limit is not unique: it depends on the particular choice of $\tau_i$. This sensibility to the choice of location $x(\tau_i)$ at which the function is evaluated is a consequence of the unbounded variation of the Wiener process. Popular choices are $\tau_i = t_{i-1}$, $\tau_i = \frac{1}{2}(t_{i-1}+t_i)$ or $\tau_i = t_i$, corresponding to the \textit{Itô}, \textit{Stratonovich} and \textit{backward Itô} stochastic integrals, respectively. Hence, for a stochastic integral such as~\eqref{eq:stoch_int} to be well-defined, its \textit{interpretation} must be stated explicitly. Namely, one must specify if the integral is taken in the \textit{Itô}, \textit{Stratonovich} or \textit{backward Itô} sense, or any other interpretation corresponding to a given choice of $\tau_i$. Only the Itô and backward Itô interpretations will be considered in this work. To avoid any confusion, we will denote the Itô integral by $\I\int$ and the backward Itô integral by $\bI\int$. Hence
\begin{subequations}
\begin{align}
        \I\int_{t_0}^t b(x(s),s) \rm dW(s) &= \mslim_{n\rightarrow \infty} \sum_{i=1}^n b(x(t_{i-1}),t_{i-1}) [W(t_i)-W(t_{i-1})], \label{def:I}\\
        \bI\int_{t_0}^t b(x(s),s) \rm dW(s) &= \mslim_{n\rightarrow \infty} \sum_{i=1}^n b(x(t_{i}),t_{i}) [W(t_i)-W(t_{i-1})]. \label{def:bI}
\end{align}
\end{subequations}
\paragraph{Remark}\label{remark:backwardIto} In the backward Itô definition of the stochastic integral, only $x$ needs to be evaluated at $t_i$: if $b(x,t)$ is differentiable in $t$ (which is assumed for all the functions considered in this work), the integral is independent of the particular choice of value for $t$ in the range $[t_{i-1},\, t_i]$. Hence, we could replace $b(x(t_i),t_i)$ by $b(x(t_i),\tau_i)$ with $\tau_i \in [t_{i-1},\, t_i]$ in the definition~\eqref{def:bI}. Note that this remark is not restricted to backward Itô integration: if $b$ is differentiable in $t$, only the location $x(\tau_i)$ at which $b$ is evaluated in the sum affects the limit.

Conventionally, a stochastic differential equation such as $\eqref{eq:Langevin}$ is written in the form
\begin{equation} \label{eq:generalSDE}
	\begin{cases}
		\rm dx(t) = a(x(t),t) \rm dt + b(x(t),t) \rm dW(t),\\
		x(t_0) = x_0.
	\end{cases}
\end{equation}
Equation~\eqref{eq:generalSDE} has to be interpreted as the implicit integral equation
\begin{equation}
	x(t) = x_0 + \int_{t_0}^t a(x(s),s) \rm ds + \int_{t_0}^t b(x(s),s) \rm dW(s),
\end{equation}
and the interpretation of the stochastic integral must thus be stated. For example, one will talk about an Itô SDE or a backward Itô SDE, and the stochastic process $x(t)$ which is the solution to the SDE~\eqref{eq:generalSDE} is different whether the stochastic integral is computed in the Itô or backward Itô sense.

It can be shown that an Itô stochastic integral $\I\int_{t_0}^t G(s) \rm dW(s)$ exists whenever the function $G$ is \textit{continuous} and \textit{nonanticipating} on the closed interval $[t_0,\,t]$. $G$ is a \textit{nonanticipating} function of $t$ is for all $t$ and $s$ such that $t<s$, $G(t)$ is statistically independent of $W(s)-W(t)$. This condition seems obviously satisfied for any deterministic function $b(x(t),t)$ of the stochastic process $x(t)$ obeying the SDE~\eqref{eq:generalSDE}, since $x(t)$ only depends on anterior values of the Wiener process. For example, in the context of the position of a brownian particle, it is intuitively obvious that the unknown future collisions cannot affect the present position of the particle. From now on, we assume thus that we are dealing with \textit{nonanticipating} functions.

\subsection{Properties of the Itô integral}
When working with nonanticipating functions, it is possible to take advantage of the fact that $G(t_{i-1})$ is independent of $W(t_i) - W(t_{i-1})$ to derive particularly useful properties of the Itô integral. Such properties are generally not true for Stratonovich or backward Itô integrals. However, there is a formula called Ito's formula that allows to build rules to transform any SDE into an Itô SDE. Hence, when working with a Stratonovich or backward Itô SDE, it is often useful to transform that SDE into an equivalent Itô SDE so that the previously mentioned properties can be applied. In this section, we first derive those properties, and then we show Itô's formula for the change of variables. Those results are fundamental for connecting a SDE to the Fokker-Plank equation and for building consistent numerical schemes, as we shall see later.
\subsubsection{Rules for the differentials}
One preliminary result of first-order importance in the context of stochastic integration is that, for any nonanticipating function $G$
\begin{equation} \label{eq:dW=dt_int}
	\I\int_{t_0}^t G(s) \rm [dW(s)]^2 = \int_{t_0}^t G(s) \rm ds,
\end{equation}
i.e. that
\begin{equation} \label{eq:dw1}
	\lim_{n\rightarrow\infty} \esp{\left(\sum_{i=1}^n G(t_{i-1})[W(t_i)-W(t_{i-1})]^2-\int_{t_0}^t G(s) \rm ds\right)^2} = 0.
\end{equation}
Let $\Delta W_i := W(t_i) - W(t_{i-1})$ and $\Delta t_i = t_i - t_{i-1}$. By the properties of the Wiener process, $\Delta W_i \sim \mathcal{N}(0,\Delta t_i)$. Hence, $\Delta W_i^2/\Delta t_i$ follows a $\chi$-squared distribution with one degree of freedom. It has thus mean $1$ and variance $2$ and thus
\begin{subequations} \label{eq:dwprop}
\begin{align}
        &\esp{\Delta W_i^2} = \Delta t_i, \quad \mbox{and} \label{eq:dwesp}\\
        &\esp{(\Delta W_i^2 - \Delta t_i)^2} = 2 \Delta t_i^2. \label{eq:dwvar}
\end{align}
\end{subequations}
Using Riemann sums, the left hand side of equation~\eqref{eq:dw1} can be rewritten as
\begin{equation}
	\lim_{n\rightarrow\infty} \esp{\left(\sum_{i=1}^n G(t_{i-1})(\Delta W_i^2-\Delta t_i)\right)^2},
\end{equation}
which, by developing the square and using the linearity of the expectation operator is equal to
\begin{equation} \label{eq:dw2}
	\lim_{n\rightarrow\infty} \sum_{i=1}^n \bigg[\esp{G^2(t_{i-1})(\Delta W_i^2-\Delta t_i)^2} + \sum_{j=1}^{i-1} \esp{2G(t_{i-1})G(t_{j-1})(\Delta W_j^2 - \Delta t_j)(\Delta W_i^2 - \Delta t_i)}\bigg].
\end{equation}
Since $G$ is nonanticipating, we can use independence between terms to rewrite~\eqref{eq:dw2} as
\begin{multline} \label{eq:dw3}
	\lim_{n\rightarrow\infty} \sum_{i=1}^n \bigg[\esp{G^2(t_{i-1})}\underbrace{\esp{(\Delta W_i^2-\Delta t_i)^2}}_{=2\Delta t_i^2} \\ + \sum_{j=1}^{i-1} \esp{2G(t_{i-1})G(t_{j-1})(\Delta W_j^2 - \Delta t_j)}\underbrace{\esp{(\Delta W_i^2 - \Delta t_i)}}_{=0}\bigg].
\end{multline}
Notice that this step is only possible for Itô integration since otherwise we would not have independence. Equation~\eqref{eq:dw3} simplifies to
\begin{equation}
	2 \lim_{n\rightarrow\infty} \sum_{i=1}^n \esp{G^2(t_{i-1})}\Delta t_i^2.
\end{equation}
If $G$ is bounded on $[t_0,\,t]$, the latter goes to zero, which concludes the proof. Since $[\rm dW(t)]^2$ only appears in the context of stochastic integration, property~\eqref{eq:dW=dt_int} is often written
\begin{equation} \label{eq:dw2=dt}
	\left[\rm dW(t)\right]^2 = \rm dt,
\end{equation}
but one must not forget that the underlying meaning of this expression is relation~\eqref{eq:dW=dt_int}, which is only valid in the context of Itô integration.

By a similar method, one can show that for any $N \ge 3$
\begin{equation}
	\left[\rm dW(t)\right]^{N} = 0,
\end{equation}
and that for any $N_1 \ge 1$, $N_2 \ge 1$
\begin{equation} \label{eq:dwdt=0}
	\left[\rm dt\right]^{N_1}\left[\rm dW(t)\right]^{N_2} = 0.
\end{equation}

Those results are often summarized by saying that $\rm dW(t)$ is an infinitesimal of order $\frac{1}{2}$ in $\rm dt$ and that infinitesimals of order higher than $1$ are discarded when it comes to compute differentials. Intuitively, $\rm dW(t)$ is a gaussian of variance $\rm dt$; a characteristic magnitude for $\rm dW(t)$ is its standard deviation, $\sqrt{\rm dt}$ which is indeed of order $\frac{1}{2}$.

\subsubsection{The Itô formula}
Consider an arbitrary function $\phi(x(t),t)$ with $x(t)$ obeying the Itô SDE~\eqref{eq:generalSDE}. We are interested in the SDE governing $\phi$. By definition,
\begin{equation}
	\rm d\phi(x(t),t) = \phi(x(t)+\rm dx(t),t+ \rm dt) - \phi(x(t),t).
\end{equation}
Expanding $\phi(x(t)+\rm dx(t),t+ \rm dt)$ in Taylor series and keeping the terms up to order 1 in $\rm dt$ yields
\begin{align}
\rm d\phi(x,t) &= \frac{\partial \phi}{\partial t}\rm dt + \frac{\partial \phi}{\partial x}\rm dx(t) + \frac{1}{2} \frac{\partial^2 \phi}{\partial x^2} [\rm dx(t)]^2 + \dots \nonumber\\
&= \frac{\partial \phi}{\partial t}\rm dt + \frac{\partial \phi}{\partial x}\left(a \rm dt + b \rm dW(t)\right) + \frac{1}{2} \frac{\partial^2 \phi}{\partial x^2} \left(b^2 [\rm dW(t)]^2 + \dots \right) + \dots, \nonumber
\end{align}
where $a$, $b$ and the derivatives are evaluated at $(x(t),t)$. Now, using relation~\eqref{eq:dw2=dt}, we get the Itô formula:
\begin{equation} \label{eq:ItoFormula}
	\rm d\phi(x(t),t) = \left[\frac{\partial \phi}{\partial t} + \frac{\partial \phi}{\partial x} a(x(t),t) + \frac{1}{2}\frac{\partial^2 \phi}{\partial x^2} b^2(x(t),t)\right] \rm dt + \frac{\partial \phi}{\partial x}b(x(t),t) \rm dW(t).
\end{equation}

\subsection{Link between Itô and backward Itô SDE's}
A same stochastic process $x(t)$ can be described both by a Itô and by a backward Itô SDE. Suppose that $x(t)$ obeys the Itô SDE
\begin{equation} \label{eq:I}
	dx(t) = a_I(x(t),t) \rm dt + b_I(x(t),t) \rm dW(t),
\end{equation}
and the equivalent backward Itô SDE
\begin{equation} \label{eq:bI}
	dx(t) = a_{bI}(x(t),t) \rm dt + b_{bI}(x(t),t) \rm dW(t).
\end{equation}
The goal is to compute the relations between the functions $a_I$, $b_I$ and $a_{bI}$, $b_{bI}$. By~\eqref{eq:bI},
\begin{equation}
	x(t) = x(t_0) + \int_{t_0}^t a_{bI}(x(s),s) \rm ds + \bI \int_{t_0}^t b_{bI}(x(t),t) \rm dW(t).
\end{equation}
Let us rewrite the backward Itô stochastic integral term
\begin{equation} \label{eq:bI-int}
	\bI \int_{t_0}^t b_{bI}(x(t),t) \rm dW(t) \triangleq \mslim_{n\rightarrow \infty} \sum_{i=1}^n b_{bI}(x(t_{i}),t_{i}) [W(t_i)-W(t_{i-1})]
\end{equation}
as an Itô integral. Since $x(t)$ satisfies the Itô SDE~\eqref{eq:I}, we can apply Itô formula. This yields
\begin{align} \label{eq:equiv1}
	b_{bI}(x(t_i),t_{i}) =~& b_{bI}(x(t_{i-1}),t_{i-1}) \nonumber\\
		&+ \left[\frac{\partial b_{bI}}{\partial t} + \frac{\partial b_{bI}}{\partial x} a_I(x(t_{i-1}),t_{i-1}) + \frac{1}{2}\frac{\partial^2 b_{bI}}{\partial x^2} b_I^2(x(t_{i-1}),t_{i-1})\right] (t_i - t_{i-1}) \nonumber\\
		&+ \frac{\partial b_{bI}}{\partial x}b_I(x(t_{i-1}),t_{i-1}) (W(t_i) - W(t_{i-1})),
\end{align}
where the derivatives are evaluated at $(x(t_{i-1}),t_{i-1})$. Introducing~\eqref{eq:equiv1} in~\eqref{eq:bI-int} and setting $[\rm dW(t)]^2 = \rm dt$ yields, after dropping the terms in $\rm dt \rm dW(t)$ and $\rm dt^2$:
\begin{multline}
	\bI \int_{t_0}^t b_{bI}(x(t),t) \rm dW(t) = \mslim_{n\rightarrow \infty} \sum_{i=1}^n \bigg( b_{bI}(x(t_{i-1}),t_{i-1}) [W(t_i)-W(t_{i-1})] \\
	+ b_I(x(t_{i-1}),t_{i-1}) \frac{\partial b_{bI}}{\partial x}[t_i - t_{i-1}]\bigg),
\end{multline}
and finally
\begin{equation}
	\bI \int_{t_0}^t b_{bI}(x(t),t) \rm dW(t) = \I\int_{t_0}^t b_{bI}(x(t),t) \rm dW(t) + \int_{t_0}^t b_I(x(s),s) \frac{\partial b_{bI}}{\partial x}(x(s),s) \rm ds.
\end{equation}
Therefore, we have the equivalences
\begin{equation}
	\begin{array}{ccc}
	\mbox{\underline{Itô SDE}} & & \mbox{\underline{backward Itô SDE}}\\[.2cm]
	\rm dx(t) = a_I \rm dt + b_I \rm dW(t) & \Leftrightarrow & \rm dx(t) = \left[a_I-b_I\dfrac{\partial b_I}{\partial x}\right]\rm dt + b_I \rm dW(t)
	\end{array}
\end{equation}
and conversely
\begin{equation} \label{eq:bI-I}
	\begin{array}{ccc}
	\mbox{\underline{backward Itô SDE}} & & \mbox{\underline{Itô SDE}}\\[.2cm]
	\rm dx(t) = a_{bI} \rm dt + b_{bI} \rm dW(t) & \Leftrightarrow & \rm dx(t) = \left[a_{bI}+b_{bI}\dfrac{\partial b_{bI}}{\partial x}\right]\rm dt + b_{bI} \rm dW(t).
	\end{array}
\end{equation}
Here, the dependence of $a_I$, $b_I$, $a_{bI}$ and $b_{bI}$ on $x(t)$ and $t$ have been made implicit to simplify the notations.

\subsection{Connection between Itô and backward Itô SDE's and the Fokker Planck equation}
Consider a particle in one dimension whose position $x(t)$ obeys the Itô SDE
\begin{equation} \label{eq:itoSDE}
	\I\
	\begin{cases}
		\rm dx(t) = a(x(t),t) \rm dt + b(x(t),t) \rm dW(t),\\
		x(t_0) = x_0.
	\end{cases}
\end{equation}
Let $p(x,t;y,s)$ be the probability density function of the position $x$ of the particle at time $t$ given that the particle was in position $y$ at time $s$, with $s < t$. For an infinitesimal $\rm dx$ and $\bar{x} \in \Omega$, the probability that the random variable $x(t)$ describing the position of the particle at time $t$ has value between $\bar{x}$ and $\bar{x}+\rm dx$ is given by:
\begin{equation}
	\Pr\big(\bar{x} < x(t) < \bar{x} + \rm dx \,\big|\, x(s) = y\big) = p(\bar{x},t;y,s)\rm dx.
\end{equation}
 From~\eqref{eq:itoSDE} we are going the derive the partial differential equation governing the evolution of $p(x,t;x_0,t_0)$. Let $K(x)$ be an arbitrary function of $\mathcal{C}^2$ with compact support. In the next, all the derivative terms of functions $a$ and $b$ are evaluated at $(x(t),t)$ and the derivatives of $K$ are evaluated at $x(t)$, unless otherwise stated. By Itô's formula:
\begin{equation}
	\rm dK(x(t)) = \left[\frac{\partial K}{\partial x} a(x(t),t) + \frac{1}{2}b^2(x(t),t) \frac{\partial^2 K}{\partial x^2} \right] \rm dt + \frac{\partial K}{\partial x}b(x(t),t) \rm dW(t).
\end{equation}
Taking the expectation yields
\begin{equation}
	\esp{\rm dK(x(t))} = \esp{\frac{\partial K}{\partial x} a(x(t),t) + \frac{1}{2}b^2(x(t),t) \frac{\partial^2 K}{\partial x^2}} \rm dt,
\end{equation}
and thus
\begin{equation} \label{eq:dEKdt}
	\frac{d\esp{K(x(t))}}{dt} = \esp{\frac{\partial K}{\partial x} a(x(t),t) + \frac{1}{2}b^2(x(t),t) \frac{\partial^2 K}{\partial x^2}}.
\end{equation}
Using the probability density $p(x,t;x_0,t_0)$, we can rewrite~\eqref{eq:dEKdt} as
\begin{multline} \label{eq:dEKdt_int}
	\frac{d}{dt} \int_{-\infty}^{\infty} K(x) p(x,t;x_0,t_0) \rm dx = \underbrace{\int_{-\infty}^{\infty} \frac{\partial K}{\partial x} a(x,t) p(x,t;x_0,t_0) \rm dx}_{:= I_1}\\ + \frac{1}{2} \underbrace{\int_{-\infty}^{\infty} \frac{\partial^2 K}{\partial x^2} b^2(x,t) p(x,t;x_0,t_0) \rm dx}_{:= I_2}.
\end{multline}
Integrating $I_1$ by parts yields:
\begin{align}
	I_1 &= \left[ a(x,t)p(x,t;x_0,t_0)K(x)\right]_{-\infty}^{\infty} - \int_{-\infty}^{\infty} K(x) \frac{\partial{(ap)}}{\partial x} \rm dx \nonumber\\
	&= - \int_{-\infty}^{\infty} K(x) \frac{\partial{(ap)}}{\partial x} \rm dx, \label{eq:I1}
\end{align}
where the second equality is obtained because $K$ has a compact support. Integrating $I_2$ by parts yields successively:
\begin{align}
	I_2 &= 	\left[ b^2(x,t) p(x,t;x_0,t_0) \frac{\partial K}{\partial x} \right]_{-\infty}^{\infty} - \int_{-\infty}^{\infty} \frac{\partial K}{\partial x} \frac{\partial (b^2 p)}{\partial x} \nonumber\\
	&= \left[ b^2(x,t) p(x,t;x_0,t_0) \frac{\partial K}{\partial x} \right]_{-\infty}^{\infty} - \left[ \frac{\partial (b^2 p)}{\partial x} K(x) \right]_{-\infty}^{\infty} +  \int_{-\infty}^{\infty} K(x) \frac{\partial^2 (b^2 p)}{\partial x^2} \rm dx\nonumber\\
	&= \int_{-\infty}^{\infty} K(x) \frac{\partial^2 (b^2 p)}{\partial x^2} \rm dx. \label{eq:I2}
\end{align}
Again, the surface terms vanish because of the compact support of $K$. Inserting~\eqref{eq:I1} and~\eqref{eq:I2} in~\eqref{eq:dEKdt_int} and rearranging the terms yields
\begin{equation}
	\int_{-\infty}^{\infty} K(x) \left( \frac{\partial p}{\partial t} + \frac{\partial (ap)}{\partial x} - \frac{1}{2}\frac{\partial^2 (b^2 p)}{\partial x^2}\right) \rm dx= 0.
\end{equation}
Since $K$ is arbitrary we must have that
\begin{equation} \label{eq:FPito}
	\frac{\partial p}{\partial t} = - \frac{\partial (ap)}{\partial x} + \frac{1}{2}\frac{\partial^2 (b^2 p)}{\partial x^2},
\end{equation}
the Fokker-Planck equation (or Kolmogorov forward equation) corresponding to the Itô SDE~\eqref{eq:itoSDE}.

Now suppose that the particle's position $x(t)$ obeys the backward Itô SDE
\begin{equation} \label{eq:bitoSDE}
	\bI\
	\begin{cases}
		\rm dx(t) = a(x(t),t) \rm dt + b(x(t),t) \rm dW(t),\\
		x(t_0) = x_0.
	\end{cases}
\end{equation}
By~\eqref{eq:bI-I}, $x(t)$ is equivalently governed by the Itô SDE
\begin{equation} \label{eq:biSDE-i}
	\I\
	\begin{cases}
		\rm dx(t) = \left(a(x(t),t)+b(x(t),t)\dfrac{\partial b}{\partial x}\right)\rm dt + b(x(t),t) \rm dW(t)\\
		x(t_0) = x_0.
	\end{cases}
\end{equation}
By the above results, the probability density of $x(t)$ is governed by the partial differential equation
\begin{equation}
	\frac{\partial p}{\partial t} = - \frac{\partial}{\partial x}\left[\left(a+b\dfrac{\partial b}{\partial x}\right)p\right] + \frac{1}{2}\frac{\partial^2 (b^2 p)}{\partial x^2}.
\end{equation}
The latter can be simplified to
\begin{equation}
	\frac{\partial p}{\partial t} = - \frac{\partial(ap)}{\partial x} + \frac{1}{2}\frac{\partial}{\partial x}\left(b^2 \frac{\partial p}{\partial x} \right),
\end{equation}
the Fokker-Planck equation corresponding to the backward Itô SDE~\eqref{eq:bitoSDE}.

\subsection{Generalization to multiple dimensions}
Here we generalize the results of previous sections to the cases with $n$ variables and $m$ independent noise components. In that case, $\b x(t)$ and $\b a(\b x,t)$ are $n$-dimensional vectors, $\b B(\b x, t)$ is a $n \times m$ matrix and $\b W(t)$ is a $m$-dimensional multivariate Wiener process of mean $\b 0$. Hence, $\b W(t) = [W_1(t),W_2(t),\dots,W_m(t)]^{\t}$ is a vector of $m$ \textit{independent} Wiener processes such as defined in~\eqref{eq:WienerProcess}. The general form of a multidimensional SDE is then
\begin{equation} \label{eq:SDE_ndim}
	\begin{cases}
		\rm d \b{x}(t) =  \b a(\b x(t),t) \rm dt + \b B(\b x(t),t) \rm d \b W(t),\\
		\b x(t_0) = \b x_0.
	\end{cases}
\end{equation}
The Itô differential rules are similar to the one-dimensional case. For $N \ge 3$, $N_1,\, N_2 \ge 1$ and $i,j \in \{1,2,\dots,m\}$, we have (in the context of Itô integration):
\begin{subnumcases}{}
	\rm dW_i(t)\rm dW_j(t) = \delta_{ij}\rm dt,\\[.1 cm]
	[\rm dW_i(t)]^{N} = 0,\\[.1 cm]
	[\rm dt]^{N_1}[\rm dW_i(t)]^{N_2} = 0.
\end{subnumcases}
Itô's formula for a function $\phi$ of the $n$-dimensional vector $\b x(t)$ satisfying the Itô SDE~\eqref{eq:SDE_ndim} is
\begin{align}
	\rm df(\b x) =~&\left[ \sum_{i=1}^n a_i(\b x(t),t) \frac{\partial f}{\partial x_i} + \frac{1}{2}\sum_{i=1}^n \sum_{j=1}^n [\b B(\b x(t),t)\b B^{\t}(\b x(t),t)]_{ij} \frac{\partial^2 f}{\partial x_i \partial x_j} \right] \rm dt \nonumber\\
	&+ \sum_{i=1}^{n}\sum_{j=1}^{m} [\b B(\b x(t),t)]_{ij}\frac{\partial f}{\partial x_i} \rm dW_j(t),
\end{align}
where the derivatives of $f$ are evaluated at $\b x(t)$.

Let $\b D = \b B\b B^{\t}$. If~\eqref{eq:SDE_ndim} is interpreted as an Itô SDE, the corresponding Fokker-Planck equation is
\begin{equation} \label{eq:FP-I-ndim}
	\frac{\partial p}{\partial t} = -\sum_{i=1}^n \frac{\partial(a_ip)}{\partial x_i} + \frac{1}{2} \sum_{i=1}^n \sum_{j=1}^n \frac{\partial^2 \left([\b D]_{ij}p\right)}{\partial x_i \partial x_j}.
\end{equation}

If~\eqref{eq:SDE_ndim} is interpreted as a backward Itô SDE, the corresponding Fokker-Planck equation is
\begin{equation} \label{eq:FP-bI-ndim}
	\frac{\partial p}{\partial t} = -\sum_{i=1}^n \frac{\partial(a_ip)}{\partial x_i} + \frac{1}{2} \sum_{i=1}^n \sum_{j=1}^n \frac{\partial}{\partial x_i}\left( [\b D]_{ij} \frac{\partial p}{\partial x_j} \right).
\end{equation}
In equations~\eqref{eq:FP-I-ndim} and~\eqref{eq:FP-bI-ndim}, all the derivatives are evaluated at $(\b x(t),t)$.

Generally, if we have a Fokker-Planck equation and want to compute a corresponding SDE, we have to compute $\b B$ from $\b D = \b B \b B^{\t}$. The solution to that equation is not unique, so that SDE's with different $\b B$ could be consistent with the same Fokker-Planck equation. If $\b D$ is positive-semidefinite, the Cholesky decomposition \textit{exists}, namely there is a \textit{lower triangular} matrix $\b B$ which is the solution to $\b D = \b B \b B^{\t}$. Besides, if $\b D$ is positive-definite, the Cholesky decomposition is \textit{unique} if we require that the diagonal elements of $\b B$ are strictly positive. This provides a way to compute $\b B$ in the case of a positive-semidefinite matrix $\b D$.

\paragraph{Remark} The Itô and backward Itô SDE's considered in this section have exactly the same form, given by equation~\eqref{eq:SDE_ndim}. However, if $\b B$ is not constant, they correspond to different Fokker-Planck equations and thus to different \textit{transport processes}. An important implication that does not arise in the context of deterministic integration is that the numerical method has to be chosen consistently with the type (Itô, backward Itô, Stratonovich, etc.) of the SDE. The derivation of simple numerical schemes consistent with either an Itô or a backward Itô SDE is the topic of the next section.

\subsection{Numerical methods} \label{sec:numericalmethods}
We present here the \textit{Euler} and \textit{backward Euler} methods, which are the simplest numerical schemes for the simulation of an Itô and a backward Itô stochastic process, respectively. The goal of this section is really to provide some intuition about why the \textit{Euler method} is relevant to simulate an Itô process whereas the \textit{backward Euler method} is relevant in the context of a backward Itô process. We do not aim at providing fully rigorous proofs of convergence and consistency. Such a formalism can be found in the well-known book by Kloeden and Platen about stochastic numerical methods \cite{kloedenplaten1995numerical}. A more recent handbook about stochastic numerical methods is \cite{colet2014stochastic}, and a very nice introductory article is found in \cite{higham2001algorithmic}. The backward Euler method has been introduced more recently by LaBolle \cite{labolle2000diffusion}.

Consider the one-dimensional SDE
\begin{equation} \label{eq:numSDE}
	\begin{cases}
		\rm dx(t) = a(x(t),t) \rm dt + b(x(t),t) \rm dW(t),\\
		x(t_0) = x_0.
	\end{cases}
\end{equation}
\begin{sloppypar}
Through a numerical approximation of~\eqref{eq:numSDE}, we can only compute $x$ at discrete times $t_0 < t_1  < \dots < t_{n-1} < t_n = T$, where $T$ is the final integration time. We consider a constant time step $\Delta t$ such that $t_{i+1} = t_i + \Delta t$ for any $i \in \{0,1,\dots,n-1\}$. Let $X_i$ denote the numerical approximation of $x(t_i)$, and let $\Delta W_{i+1} := W(t_{i+1}) - W(t_i) \sim \mathcal{N}(0,\Delta t)$. $\Delta W_i$ is thus a gaussian noise of mean $0$ and variance $\Delta t$, and $\Delta W_i$ is independent of $\Delta W_j$ for any $i \neq j$. In the next, we shall only verify that the schemes are \textit{consistent}, namely that they tend to the proper SDE when $t_i \rightarrow t$, $\Delta t \rightarrow \rm dt$ and $\Delta W_i \rightarrow \rm dW(t)$.
\end{sloppypar}

Let us first consider the case where~\eqref{eq:numSDE} is a Itô SDE. Suppose that $x(t_i)$ is known, and we want to compute $x(t_{i+1})$. Now, consider a partition $t_i = t_{i_0} < t_{i_1} < \dots < t_{i_{m-1}} < t_{i_m} = t_{i+1}$ of $[t_i,\, t_{i+1}]$. By~\eqref{eq:numSDE} and the definition of the Itô stochastic integral:
\begin{multline}
	x(t_{i+1}) = x(t_{i}) + \lim_{m\rightarrow\infty} \sum_{k=0}^{m-1} a\left(x(t_{i_k}),t_{i_k}\right)(t_{i_{k+1}} - t_{i_{k}})\\ + \mslim_{m\rightarrow\infty} \sum_{k=0}^{m-1} b\left(x(t_{i_{k}}),t_{i_k}\right)[W(t_{i_{k+1}}) - W(t_{i_{k}})].
\end{multline}
The simplest approximation to that expression is to take $m=1$. This gives precisely the \textit{Euler method}, also called the \textit{Euler-Maruyama method}:
\begin{equation}
	X_{i+1} = X_i + a(X_i,t_i) \Delta t + b(X_i,t_i) \Delta W_{i+1}
\end{equation}
Note that if $R_0,R_1,\dots,R_{n-1}$ are independent standard gaussian random variables, then we can replace $\Delta W_{i+1}$ by $\sqrt{\Delta t}R_i$ which could be more practical to implement.

Now consider the case where~\eqref{eq:numSDE} is a backward Itô SDE. A similar reasoning as set out above for the Itô case yields :
\begin{multline}
	x(t_{i+1}) = x(t_{i}) + \lim_{m\rightarrow\infty} \sum_{k=0}^{m-1} a\left(x(t_{i_k}),t_{i_k}\right)(t_{i_{k+1}} - t_{i_{k}})\\ + \mslim_{m\rightarrow\infty} \sum_{k=0}^{m-1} b\left(x(t_{i_{k+1}}),t_{i_k}\right)[W(t_{i_{k+1}}) - W(t_{i_{k}})].
\end{multline}
Notice that since the first limit corresponds to a deterministic integral, we can choose to evaluate $a$ at any time in $[t_{i_k},\, t_{i_{k+1}}]$.\footnote{The choice $a(x(t_{i_k}),t_{i_k})$ corresponds to the Darboux integration, which can be seen as a particular case of the Riemann integration. Remember that a function is Darboux-integrable if and only if it is Riemann-integrable, and the values of the two integrals, if they exist, are equal.} The fact that $b$ is evaluated at time $t_{i_k}$ follows from the implicit assumption that $b$ is differentiable in $t$, cfr. the remark at page \pageref{remark:backwardIto}. Taking $m=1$ yields the approximation
\begin{equation} \label{eq:biImplicit}
	X_{i+1} = X_i + a(X_i,t_i) \Delta t + b(X_{i+1},t_i) \Delta W_{i+1},
\end{equation}
which is an \textit{implicit} scheme since $b$ has to be evaluated at $X_{i+1}$. In the general case $b$ is nonlinear in $x$ and solving~\eqref{eq:biImplicit} is nontrivial. In particular, it is not always possible to invert $b$ and hence to find an explicit formula for $X_{i+1}$. The idea is thus to rely on a predictor-corrector method: we first compute an estimate $X^{*}_{i+1}$ of $X_{i+1}$ using an explicit formula, and then we compute $X_{i+1}$ as
\begin{equation} \label{eq:biCorrector}
	X_{i+1} = X_i + a(X_i,t_i) \Delta t + b(X^*_{i+1},t_i) \Delta W_{i+1}.
\end{equation}
Now the question is: how to compute $X^*_{i+1}$ ? One might be tempted to use the Euler method, which is explicit. However, we will see shortly that we do not need to include the advective transport term in the estimation $X^*_{i+1}$ of $X_{i+1}$. The predictor-corrector scheme is thus
\begin{subnumcases}{\label{eq:backwardEuler}}
		\Delta Y_{i+1} = b(X_i,t_i)\Delta W_{i+1}, \label{eq:backwardEuler-dY}\\
		X_{i+1} = X_i + a(X_i,t_i) \Delta t + b(X_{i}+\Delta Y_{i+1},t_i) \Delta W_{i+1}. \label{eq:backwardEuler-dX}
\end{subnumcases}
In order to see that the scheme~\eqref{eq:backwardEuler} consistently approximate the backward Itô SDE~\eqref{eq:numSDE}, we can consider a stochastic process $y(t)$ and interpret $\Delta Y_{i+1}$ as the difference between two successive iterates: $\Delta Y_{i+1} = Y_{i+1} - Y_{i}$. But then, the scheme~\eqref{eq:backwardEuler} is precisely the \textit{Euler method} applied on the Itô SDE with two variables
\begin{subnumcases}{\I\ \label{eq:ito2}}
	\rm dy(t) = b(x(t),t) \rm dW(t),\\
	\rm dx(t) = a(x(t),t) \rm dt + b(x(t)+\rm dy(t),t) \rm dW(t). \label{eq:ito2.2}
\end{subnumcases}
Therefore, by the equivalence formula between Itô and backward Itô SDE's~\eqref{eq:bI-I}, showing that the scheme~\eqref{eq:backwardEuler} is consistent with the backward Itô SDE~\eqref{eq:numSDE} amounts to show that~\eqref{eq:ito2} is equivalent to the Itô SDE
\begin{equation} \label{eq:bE-ito}
	\rm dx(t) = \left(a(x(t),t)+b(x(t),t)\dfrac{\partial b}{\partial x}\right)\rm dt + b(x(t),t) \rm dW(t),
\end{equation}
where $\partial_x b$ is evaluated at $(x(t),t)$. Using the same convention for all the derivatives of $b$, a stochastic Itô-Taylor expansion on $b(x(t)+\rm dy(t),t)$ yields
\begin{subequations}
	\begin{align}
		b(x+\rm dy(t),t) &= b(x,t) + \frac{\partial b}{\partial x} \rm dy(t) + \frac{1}{2}\frac{\partial^2 b}{\partial x^2} [\rm dy(t)]^2 + \dots\\
		&= b(x,t) + \frac{\partial b}{\partial x} \rm b(x,t) \rm dW(t) + \frac{1}{2}\frac{\partial^2 b}{\partial x^2} b^2(x,t) [\rm dW(t)]^2 + \dots\\
		&= b(x,t) + \frac{\partial b}{\partial x} \rm b(x,t) \rm dW(t) + \frac{1}{2}\frac{\partial^2 b}{\partial x^2} b^2(x,t) \rm dt + \dots \label{eq:bE-inter}
	\end{align}
\end{subequations}
where we have noted $x$ instead of $x(t)$ in order to save space.
Finally, introducing~\eqref{eq:bE-inter} in~\eqref{eq:ito2.2} yields
\begin{subequations}
	\begin{align}
		\rm dx(t) &= a(x,t) \rm dt + b(x,t)\rm dW(t) + \frac{\partial b}{\partial x} \rm b(x,t) [\rm dW(t)]^2 + \frac{1}{2}\frac{\partial^2 b}{\partial x^2} b^2(x,t) \rm dt \rm dW(t), \label{eq:bE-ito1}\\
		&= \left(a(x,t)+b(x,t)\dfrac{\partial b}{\partial x}\right)\rm dt + b(x,t) \rm dW(t), \label{eq:bE-ito2}
	\end{align}
\end{subequations}
where we have used the properties~\eqref{eq:dw2=dt} and~\eqref{eq:dwdt=0} of Itô integration. Since~\eqref{eq:bE-ito2} is exactly~\eqref{eq:bE-ito}, we have shown the consistency of the backward Euler method~\eqref{eq:backwardEuler} for the numerical integration of the backward Itô SDE~\eqref{eq:numSDE}. Notice that including the advective transport in the computation of $\Delta Y_{i+1}$ in equation~\eqref{eq:backwardEuler-dY} leads to a term of order $\rm dt \rm dW$ in equation~\eqref{eq:bE-ito1}. By~\eqref{eq:dwdt=0}, $\rm dt \rm dW = 0$ and this term does not appear in~\eqref{eq:bE-ito2}.

\paragraph{Remark} We have implicitly assumed through all the above developments that $b$ is \textit{smooth enough}, and the equivalence between the Itô and backward Itô formulation has only be proven in that case. In the case of discontinuous $b$, the consistency of the backward Itô formulation with the Fokker-Planck equation is shown in \cite{labolle2000diffusion} for the one-dimensional case and demonstrated in the two-dimensional case. There is however no proof for the multi-dimensional case. The efficacy of the backward Euler method in the case of discontinuous diffusivities is assessed in \cite{spivakovskaya2007backward} on two one-dimensional test cases. The second test case is an advection-diffusion equation with constant velocity and a piecewise constant diffusivity. It shows a significantly better performance of the backward Euler method with respect to Itô and Stratonovich methods in estimating the residence time of a tracer. %This result is particularly interesting in the context of the overturner model, where the vertical diffusivity $K_v$ is discontinuous.
\subsubsection{Strong and Weak convergence of the schemes}
Let us now present briefly the notion of strong and weak convergence for the numerical approximation of a SDE. The difference between those two definitions of the convergence resides in the criterion used to measure the difference between the exact solution $X(t_i)$ and its numerical approximation $X_i$. Using the expected value $\esp{|X(t_i) - X_i|}$ leads to the concept of \textit{strong convergence}. Formally, a method has a strong order of convergence equal to $\gamma$ if there exists a constant $\beta$ such that
\begin{equation}
	\esp{|X(t) - X_i|} \le \beta \Delta t^{\gamma}
\end{equation}
at any fixed $t = i\Delta t$ and $\Delta t$ sufficiently small. Hence, the strong order of convergence measure the rate at which the average error decreases when $\Delta t$ goes to zero.

An alternative, less demanding definition of convergence is obtained if we measure the "error of the means" instead of the "mean of the error". This leads to the concept of \textit{weak convergence}. Formally, a method has a weak order of convergence equal to $\gamma$ if there exists a constant $\beta$ such that
\begin{equation}
	|\esp{f(X(t))} - \esp{f(X_i)}| \le \beta \Delta t^{\gamma}
\end{equation}
at any fixed $t = i\Delta t$ and $\Delta t$ sufficiently small, and where the function $f$ is an arbitrary function with polynomial growth.

Under appropriate conditions on the functions $a$ and $b$, it can be shown that the Euler-Maruyama and the backward Euler scheme both have strong order of convergence $\frac{1}{2}$ and weak order of convergence $1$.

\subsection{Dealing with no-through boundary conditions}
Only no-through boundary conditions are encountered in the problems that we will consider. Such conditions are enforced by bouncing particles off boundaries, as suggested in \cite{tompson1992particle}. Under such conditions, no particle can leave the domain. Bouncing is assumed to be perfect, and we only consider straight lines boundaries. Hence, bouncing amounts to remap particles leaving the domain into the domain by an orthogonal symmetry with the boundary.

\subsection{Computation of the concentration} \label{boxcounting_kernel}
The above discussions shows how to simulate individual particles trajectories. However, what we ultimately want is to properly estimate the solution to~\eqref{eq:ADE}, namely to compute an approximation of the concentration from the particles locations. Several methods exists, amongst which the \textit{box counting method} and the \textit{kernel density estimation}. Both methods are discussed in \cite{spivakovskaya2007lagrangian} and \cite{dehaan1999densitykernel}.

The \textit{box counting} method has been extensively used to estimate the concentration in studies using random walk modeling, see e.g. \cite{riddle1998specification}. The method goes as follows: suppose a grid discretization of the domain, the \textit{box counting methods} consists in counting the number of particles in a grid cell\footnote{Such grid cells are also called \textit{boxes} in the literature. To avoid confusion, we prefer to reserve the term "box" to design compartments in this work.}; the estimation of the concentration is then obtained by multiplying the number of particles with their mass, and dividing this total mass by the volume of the grid cell. This method amounts to compute histograms: the concentration profile computed this way is constant in each grid cell and discontinuous at the boundaries between the grid cells. Such an estimation of the concentration depends thus on the size and center of the grid cells, but there is unfortunately no physical argument that would allow to choose those parameters. The choice of the averaging volumes is thus often made empirically as the result of a trade-off between regularity and a satisfying resolution of the solution. On one hand, choosing large grid cells could lead to the loss of important details since the estimated concentration cannot be described in a grid cell more precisely than a constant. Hence the solution tends to be oversmoothed when the number of grid cells is too small. On the other hand, opting for many small grid cells lead to a concentration profile that tends to become very irregular or noisy. Another disadvantage of the method is that the number of particles needed to make the uncertainties becomes negligible might be excessively large. However, this method is the most efficient for problem where the goal is precisely to compute volume averages over grid cells; but if one aims at computing point concentration, then the kernel density estimation turns out to be more efficient.

The \textit{kernel density estimation} method allows to reduce drastically the number of particles, and does not suffer from the resolution limit inherent to the box counting method. However, the kernel estimation method introduces some difficulties. A first difficulty is that this method depends on a smoothing parameter called the \textit{bandwidth} for which finding a relevant value is not trivial. Second, the method as such does not perform well at the boundaries, and a specific treatment of the boundaries must be introduced. Classical references on the density kernel estimation are \cite{silverman1986density} and \cite{wand1995kernel}.

For the purposes of this work, we will exclusively be interested in volume averages of the concentration over grid cells so that only the box-counting method will be used. Indeed, grid cells are precisely the nodes of the underlying network, and the transition probabilities between the nodes are estimated from the concentration in the grid cells.


\begin{frame}{Algoritmo - AD*}
\begin{algorithm}[H]
\caption{key(s)}
\begin{algorithmic}[1]
\If {$g(s) > rhs(s)$}
\State \textbf{return} [$rhs(s) + \epsilon \cdot h(s_{start}, s); rhs(s)$];
\Else
\State \textbf{return} [$g(s) + h(s_{start}, s); g(s)$];
\EndIf
\end{algorithmic}
\end{algorithm}
\begin{algorithm}[H]
\caption{UpdateState(s)}
\begin{algorithmic}[1]
\setalglineno{5}
\If {$s$ was not visited before}
    \State $g(s) = \infty$;
\EndIf
\If {($s \neq s_{goal}$)} $rhs(s)=min_{s' \in Succ(s)}(c(s,s') + g(s'));$
\EndIf
\If {($s \in OPEN$)} remove $s$ from OPEN;
\EndIf
\If {($g(s) \neq rhs(s)$)}
  \If {$s \notin CLOSED$}
    \State insert $s$ into OPEN with $key(s)$;
    \Else
    \State insert $s$ into INCONS;
    \EndIf
\EndIf
\end{algorithmic}
\end{algorithm}
\end{frame}

\begin{frame}{Algoritmo - AD* função ComputeOrImprovePath}
\begin{algorithm}[H]
\caption{ComputeOrImprovePath()}
\begin{algorithmic}[1]
\setalglineno{5}
\While {($min_{s \in OPEN}(key(s)) < key(s_{start}) \:\Or\: rhs(s_{start}) \neq  g(s_{start}) $)}
\State remove state $s$ with the minimum key from OPEN;
\If {($g(s) > rhs(s)$)}
\State $g(s) = rhs(s)$;
\State CLOSED = CLOSED $\cup \{s\}$;
\For {\textbf{all} $s' \in Pred(s)$} $UpdateState(s')$;
\EndFor
\Else
\State $g(s) = \infty$;
\For {\textbf{all} $s' \in Pred(s) \cup \{s\}$} $UpdateState(s')$;
\EndFor
\EndIf
\EndWhile
\end{algorithmic}
\end{algorithm}
\end{frame}

\section{Resultados}
\begin{frame}{A*}
\input{tikzfiles/astar}
\end{frame}


\setcounter{table}{0}
\renewcommand{\thetable}{S\arabic{table}}%
\setcounter{figure}{0}
\renewcommand{\thefigure}{S\arabic{figure}}%
\renewcommand{\thesection}{SI~\Alph{section}}%

\section{An external, spatially uniform potential $\V$ shifts the chemical potential $\mu$ of the gas} \label{sec:V_shifts_chem_pot}
We show that imposing an external, spatially uniform potential $\V$ to a gas
has the effect of shifting the chemical potential $\mu$ of the gas, recovering
eqn.~\ref{eq:mof-density}. Consider a control volume $\Omega$ with volume
$V=|\Omega|$ (large enough to neglect boundary effects) that is endowed with
the external, spatially uniform potential $\V$. Impose the grand-canonical
ensemble, where this control volume can exchange energy and particles with a
bath of gas at temperature $T$ and chemical potential $\mu$.

To denote a microstate of this system, let $N$ be the number of gas particles
in the control volume and $\mathbf{r}_1,...,\mathbf{r}_N$ be their positions.
Then, the potential energy $E$ of a microstate of the control volume is:

\begin{equation} E(\rvec_1,...,\rvec_N) = N\V +
E_{gg}(\rvec_1,...,\rvec_N),
\end{equation}
where $E_{gg}$ is the (unknown and complicated) interatomic potential for
gas-gas interactions that governs the (real) gas properties. The first term
arises from each gas molecule experiencing the external potential $\V$, where
$\V < 0$ corresponds to attraction. To account for molecular rotational and
vibrational degrees of freedom, we could treat the positions $\rvec_i$ as the
locations of atoms rather than molecules. In this case the gas-gas interactions
will include both \emph{intra-molecular} interactions and
\emph{inter-molecular} interactions.

The grand canonical partition function of the control volume is:

\begin{multline}
    \Xi(\mu, V, T)= \\ \displaystyle \sum_{N=0}^\infty \frac{1}{\Lambda^{3N}N!} \int_{\Omega} \cdots \int_{\Omega} e^{-\beta E_{gg}(\rvec_1, ..., \rvec_N)} e^{\beta (\mu - \V) N} d\rvec_1 \cdots d\rvec_N.
    \label{eq:gcpf}
\end{multline}
We recognize this as the grand canonical partition function of the bulk gas in
the control volume without the external potential, but shifted in chemical
potential:

\begin{equation}
    \Xi(\mu, V, T)=\Xi_0(\mu - \V, V, T)
    \label{eq:xi_vs_xi0}
\end{equation}
where $\Xi_0(\mu, V, T)$ is the grand partition function of the bulk
gas in the control volume in the absence of an external potential. Importantly,
this equivalency depends on the potential $\V$ being spatially uniform.
Therefore, the thermodynamic properties of the gas atoms in the spatially
uniform external potential $\V$ (adsorbed in our idealized substrate)
are equivalent to the properties of the bulk gas at chemical potential $\mu-\V$
(where $T$ is held fixed). As $\V$ becomes more negative, corresponding to a
more attractive adsorbent, the thermodynamic properties of the adsorbed gas in
our ideal substrate are equivalent to the gas at a higher chemical potential.

\section{Proof of extremum}\label{sec:proof-extremum}
To show that a uniform potential gives the highest deliverable capacity, we
consider an interaction potential between gas and substrate $\V(\rvec)$ that
varies in space.  In this proof, we make use of the Fourier transform of
this potential:

\begin{align}
    \Vk \equiv \iiint \V(\rvec) e^{-i\kvec\cdot \rvec} d\rvec
\end{align}
The Fourier transform of a uniform potential is a Dirac delta function $\tilde{\V}(\kvec)\propto\delta(\kvec)$. In order for a uniform potential
to extremize the deliverable capacity, we must show that the functional
derivative of the deliverable capacity with respect to $\Vk$ is zero for
\emph{nonzero} values of $\kvec$, i.e.

\begin{align}
    \frac{\delta D}{\delta \Vk} &= \mathbf{0}, \text{ if } \kvec\ne \mathbf{0}.
\end{align}
We note that this functional derivative may be non-zero for $\kvec=\mathbf{0}$
because we separately maximize with respect to the particular uniform potential
$\V$. This means that

\begin{align}
    \frac{\delta N_\text{full}}{\delta \Vk} &= \frac{\delta N_\text{empty}}{\delta \Vk}
\end{align}
where $N_\text{full}$ and $N_\text{empty}$ are the number of particles at the full and empty pressure, $\pfull$ and $\pempty$, respectively.

Because the chemical potential $\mu$ varies monotonically with $N$ at fixed
temperature, we can consider how the chemical potential varies as we change
$\Vk$ with the number of molecules held fixed. We demonstrate this using the
cyclic chain rule, which shows us that

\begin{align}
    \left(\frac{\delta N}{\delta \Vk}\right)_{\mu} &=
    -\left(\frac{\delta \mu}{\delta \Vk}\right)_{N}
    \left(\frac{\partial N}{\partial \mu}\right)_{\Vk}.
\end{align}
Since changing the chemical potential changes the number of molecules in the general case, if we can show that $\left(\frac{\delta \mu}{\delta \Vk}\right)_{N}=0$ then we will have shown that $\left(\frac{\delta N}{\delta \Vk}\right)_{\mu}=0$.  Thus we consider

\begin{align}
    \left(\frac{\delta \mu}{\delta \Vk}\right)_N
    &= \left(\frac{\delta \left(\frac{\partial F}{\partial N}\right)_{\volume}}{\delta \Vk}\right)_N
    \\
    &= \left(\frac{\partial \left(\frac{\delta F}{\delta \Vk}\right)_{\volume}}{\partial N}\right)_{\volume}
    \label{eq:dmudpot}
\end{align}
where we have made use of the derivative relationship between $\mu$ and the Helmholtz free energy $F$, and have then reordered the functional and partial derivatives.
Let us consider the interior derivative first.  The derivative of the Helmholtz free energy with respect to the external potential $\Vk$ yields the number density:

\begin{align}
    \frac{\delta F}{\delta \Vk} &= \rho(\kvec)
\end{align}
The number density is itself spatially uniform for any stable system in a fluid state at this density (i.e. does not spontaneously crystallize), and thus has a Fourier transform that is proportional to a Dirac $\delta$-function.  Thus, the functional derivative $\frac{\delta F}{\delta \V(\rvec)}$ is actually a uniform function.
We can insert this expression into Eq.~\ref{eq:dmudpot} to find that

\begin{align}
    \left(\frac{\delta \mu}{\delta \Vk}\right)_N &\propto \delta(\kvec) \\
    \left(\frac{\delta N}{\delta \Vk}\right)_\mu &\propto \delta(\kvec)
\end{align}
Thus, the functional derivative of both $\mu$ and $N$ with regard to $\V(\rvec)$ are themselves spatially uniform.  Since we already maximize $D$ with respect to the spatially uniform component of the potential (i.e. $\kvec=0$), the derivative of $D$ with respect to any change of potential is zero.

This demonstrates that a spatially uniform potential leads to an extremum value
of the deliverable capacity. This proof is insufficient, however, to show that
it must be a true maximum.

\section{The real-substrate analog of $\V$ is $\gst$}
\label{sec:phi-is-delta-g}
\begin{figure}
    \centering
    \includegraphics[width=\columnwidth]{g_st_simple.pdf}
    \caption{Cartoon illustrating the definition of $\gst$. For a given adsorbent and gas, $\gst$=$\gst (\rho, T)$ is defined as the difference in chemical potential between the adsorbed gas system and bulk gas system when both exhibit the same density $\rho$ and temperature $T$.}
    \label{fig:delta-G-cartoon}
\end{figure}

The parameter describing our idealized substrate is $\V$, the spatially uniform
potential felt by a gas molecule adsorbed in the idealized substrate. A natural
question is how this potential relates to the properties of real substrates.
The effect of $\V$ in our model is to shift the chemical potential $\mu$ of the
gas (see Sec.~\ref{sec:V_shifts_chem_pot}). Because our ideal substrate shifts
the chemical potential of the gas molecules by providing a spatially uniform
potential energy field, the entropy of the gas in the ideal substrate is equal
to the entropy of the gas in its bulk state at the same density and
temperature. In contrast, a real substrate provides a non-spatially uniform
potential. Consequently, the entropy of the gas inside a real substrate is
\emph{not} equal to the entropy of the bulk gas at the same density and
temperature. Therefore, the parameter analogous to $\V$ in a real substrate
will involve both energy and entropy. The real-substrate analog of $\V$ is the
shift of molar Gibbs free energy provided by the substrate, specifically an
\emph{isosteric} (or constant-density) shift of the Gibbs free energy:

\begin{equation}
   \gst(\rho, T) \equiv
   %\frac{G_{\text{ads}}(\rho, T) - G_{\text{gas}}(\rho, T)}{N}\\
    \mu_{\text{ads}}(\rho, T) - \mu_{\text{gas}}(\rho, T).
  % \gst(\rho, T) &\equiv g_{\text{gas}}(\rho, T) - g_{\text{ads}}(\rho, T)
  \label{eq:g_st}
\end{equation}
The isosteric Gibbs free energy difference $\gst$ is the difference in molar
Gibbs free energy (equivalent to chemical potential) between the adsorbed gas system and
the bulk gas \emph{with the same density of gas molecules}. The quantity $\gst$
does \emph{not} correspond to a change in the molar Gibbs free energy as a
molecule is adsorbed, which is zero under conditions of coexistence. The
quantity $\gst$ in a real substrate is a direct analog to $\V$ in our ideal
substrate because it is the chemical potential shift needed to impose on the
bulk gas in coexistence with the real substrate to achieve the same density as
in the substrate (compare with eqn.~\ref{eq:xi_vs_xi0}).
Figure~\ref{fig:delta-G-cartoon} illustrates a hypothetical experiment to
measure $\gst$ via a piston with a removable partition that separates a volume
of free space from the same volume of substrate. Note that $\gst$ is a property
of both the substrate and the identity of the gas. Because real substrates
offer a \emph{non}-spatially uniform potential, $\gst(\rho, T)$ is a function
of $\rho$ and $T$, unlike our ideal, homogenous substrate where $\gst(\rho,
T)=\V$. Consequently, throughout this article, we show $\gst(\rho, T)$ for real
substrates at both conditions relevant to gas storage and delivery, $\pfull$
and $\pempty$.

In practice, we can readily compute $\gst(\rho, T)$ of a real gas/substrate
system from (i) the (experimental or simulated) equilibrium adsorption isotherm
of the gas in the substrate and (ii) the (experimental or simulated) chemical
potential of the bulk gas. Consider the real substrate in thermodynamic
equilibrium with a bulk gas at fixed temperature $T$ and pressure $p$, and let
$\rho=\rho(p, T)$ be the density of gas in the substrate. At coexistence, the
chemical potential of the bulk gas is equal to the chemical potential of the
adsorbed gas in the substrate. Thus, we can use the experimentally known molar
Gibbs free energy of the pure, bulk gas system at temperature $T$ and pressure
$p$ to determine the molar Gibbs free energy of the adsorbed system:
$\mu_{\text{ads}}(\rho, T)=\mu_{\text{gas}}(p, T)$. We can then also look up
the known chemical potential of the bulk gas at the same density and
temperature as in the substrate, $\mu_{\text{gas}}(\rho, T)$. Via
eqn.~\ref{eq:g_st}, $\gst$ follows from subtracting the two quantities.

An interesting question is how $\gst$ relates to the commonly measured and
reported isosteric heat of adsorption $q_{st}$, which is roughly the energy
change when a gas molecule is adsorbed~\cite{sircar1999isosteric,
tian2017differential}. Figure~\ref{fig:qst-vs-delta-G} shows how $q_{st}$
compares to $\gst$ for several prominent adsorbents. In every case,
$|q_{st}|>|\gst|$ because the gas in the adsorbent always has less entropy than
the gas in the bulk at the same density and temperature. That is, while
adsorption is energetically favored, it is entropically disfavored due to the
restrictions imposed on the configuration of the gas molecules via steric
interactions with the substrate itself; this counters the energetic attraction.

\begin{figure}
    \centering
    \includegraphics[width=0.95\columnwidth]{qst-vs-delta-G}
    \caption{Relationship between $\gst$ and the isosteric heat
      $q_{st}$ for several prominent adsorbents at room
      temperature (data from Ref.~\cite{mason2014evaluating, garcia2018benchmark}). The dots represent the properties of methane
      adsorption at 5.8~bar and 65~bar. The $+$'s represent the
      properties of hydrogen adsorption at 5~bar and
      100~bar.}
    \label{fig:qst-vs-delta-G}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=0.95\columnwidth]{methane-298-gst}
    \caption{The density-dependence of $\gst$ of methane in several
      adsorbents (298\ K). Note that $\gst$ is monotonic in
      $\rho$. Data from
      Refs.~\cite{mason2014evaluating, furukawa2009storage}.
    }
    \label{fig:methane-gst}
\end{figure}

\begin{figure}
\centering
\includegraphics[width=0.95\columnwidth]{four-cases-pro}
\caption{A bulk gas reservoir is connected to a volume filled with adsorbent material (green/blue: material atoms, red: gas particles). The adsorbent is also connected, but with a closed valve, to an equal volume containing the ideal substrate (shaded blue background represents the uniform potential energy field $\V$) with an equal density of gas as in the adsorbent. The top (bottom) shows a realization of this connected system where the pressure of the bulk gas is $\pfull$ ($\pempty$), and thus the density of gas in the adsorbent is $\rhofull$ ($\rhoempty$) by definition. The deliverable capacity of the adsorbent is $\rhofull - \rhoempty$. We can determine if the deliverable capacity of gas in the ideal substrate is lower or higher than in the adsorbent by considering what happens when we open the valves connecting the two pairs of volumes containing adsorbent and the ideal substrate.}
\label{fig:delta-gst-maximum}
\end{figure}

\section{An upper bound when $\gst(\rho)$ is monotonic}\label{sec:monotonic}
Every adsorbent has a $\Delta g_\text{st}$ at $\pfull$ and $\pempty$ corresponding
to a full and empty density, $\rhofull$ and $\rhoempty$, respectively. The deliverable capacity of the adsorbent is equal to
the difference between the full and empty density. Examining
Fig.~\ref{fig:methane-gst}, we see that a significant variety of known
experimental $\Delta g_\text{st}$ curves are monotonic. Furthermore, our
qualitative argument in Section~\ref{sec:upper-bound} suggests that this
function \emph{should} monotonically increase for rigid substrates, as
increasing the density of gas causes some of the gas to reside in higher-energy
sites. If this is always the case, the deliverable capacity of a real material
(with a non-spatially uniform potential) with a given $\gst$ at $\pfull$ or
$\pempty$ is bounded below by the deliverable capacity in our idealized
substrate with $\V=\gst$, even for a value of $\V$ that is not optimal.

To show this, we perform a thought experiment illustrated in
Fig.~\ref{fig:delta-gst-maximum}. We begin with two pairs of volumes.  Two of
these volumes contain a real adsorbent material.  These adsorbent volumes are
connected to bulk gas reservoirs at $\pfull$ and $\pempty$ respectively, and
by definition contain gas at density $\rhofull$ and $\rhoempty$.  The other two volumes
volumes contain gas at a density equal to the two adsorbent volumes ($\rhofull$ and $\rhoempty$), but contain our
idealized substrate with uniform potential energy $\V$.

We now consider what happens if we open a diffusive connection between a
volume of adsorbent and a volume with an idealized substrate that initially contain
the same density of gas, for instance by connecting them with a tube. If the
chemical potential in the idealized substrate is higher than in the porous material,
then gas will flow out of the idealized substrate, lowering its density.  Conversely,
if the chemical potential is lower in the idealized substrate than in the
volume of porous material, then gas will flow into the idealized substrate, increasing
its density.
The difference in chemical potential between those two volumes is
\begin{align}
   \gst(\rho) - \V &= \left(\mu_{\text{ads}}(\rho) - \mu_{\text{gas}}(\rho)\right)
   - \left(\mu_{\V}(\rho) - \mu_{\text{gas}}(\rho)\right)
   \\
   &= \mu_{\text{ads}}(\rho) - \mu_{\V}(\rho).
\end{align}
where $\mu_{\text{ads}}(\rho)$ and $\mu_{\V}(\rho)$ are the chemical potentials
of gas in the porous material and ideal substrate with potential $\V$, respectively.
Thus if $\gst(\rhoempty) =
\V$, the low-density containers will remain at their initial density after
they are connected. Thus, the deliverable capacity of the adsorbent will be greater
than the deliverable capacity of the ideal substrate with uniform potential $\V$ if and
only if $\gst(\rhofull)<\V$, i.e. if $\gst(\rho)$ does \emph{not} monotonically
increase, since then the gas in the high-density system will flow from the ideal
substrate to the adsorbent. By the same token, if we consider the case where
$\gst(\rhofull) = \V$, then for the adsorbent to achieve a greater deliverable
capacity than the idealized substrate, the adsorbent must have less residual gas,
which means that gas must spontaneously flow from the low-density adsorbent to the
volume with potential $\V$, which means that $\gst(\rhoempty)>\V$. Once again,
exceeding our upper bound requires a material with a $\gst(\rho)$ that does not
increase monotonically.

Taken together, this indicates that not only is our absolute upper bound an
upper bound for rigid adsorbents, but the green curve labeled $\rho_D(\gst)$ in
Figs.~\ref{fig:methane-298-D}, \ref{fig:hydrogen-298-D}, and
\ref{fig:hydrogen-77-D} is an upper bound for materials with a non-optimal
$\gst$ at either $\pfull$ or $\pempty$.

\section{Cryogenic hydrogen storage}\label{sec:cryo-hydrogen}
One approach to increase the deliverable capacity is to reduce the storage
temperature. This is illustrated in Fig.~\ref{fig:hydrogen-77-D}, which shows
the upper bound to the deliverable capacity of hydrogen at 77\ K, the boiling
point of nitrogen. The DOE ULTIMATE target in this case looks far more
achievable, and with a much lower $|\gst|$. In fact, an empty tank at this
temperature can satisfy the DOE 2020 target. The DOE ULTIMATE target is 14\%
below the upper bound. Actual adsorbents fall far short of the theoretical maximum.

\begin{figure}
    \centering
    \includegraphics[width=0.95\columnwidth]{hydrogen-77-n-vs-G}
    \caption{Deliverable capacity of hydrogen at 77\ K as a function of the attractive Gibbs free energy $\gst$. Experimental deliverable capacities for several porous materials (data from Ref.~\cite{garcia2018benchmark}) are shown along with the experimental values for $\gst$ at the empty and full pressures shown as $+$'s connected by a line.}
    \label{fig:hydrogen-77-D}
\end{figure}


\hypertarget{cheatsheet-for-tactics}{%
\section{Cheatsheet for tactics}\label{cheatsheet-for-tactics}}

\hypertarget{intro-h}{%
\subsection{\texorpdfstring{\texttt{intro\ h}}{intro h}}\label{intro-h}}

\begin{itemize}
\tightlist
\item
  `Assume h is a proof of P'
\item
  `Assume P is true'
\item
  'Let a be an arbitrary element of X\texttt{in\ a}∀ a ∈ X` term.
\end{itemize}

If the goal is \texttt{P\ →\ \ Q}, creates a hypothesis h : P and
changes the goal to \texttt{Q.}

If the goal is \texttt{∀\ a∈\ X,\ p\ x} then \texttt{intro\ a} creates a
term \texttt{a} of type \texttt{X} and changes the goal to
\texttt{p\ x}.

\hypertarget{intros-h1-h2}{%
\subsection{\texorpdfstring{\texttt{intros\ h1\ h2}}{intros h1 h2}}\label{intros-h1-h2}}

Does several \texttt{intro} commands at once.

\hypertarget{exact-h}{%
\subsection{\texorpdfstring{\texttt{exact\ h}}{exact h}}\label{exact-h}}

\begin{itemize}
\tightlist
\item
  Our hypothesis \texttt{h} establishes the result
\end{itemize}

Applies the hypothesis \texttt{h} to prove the theorem. \texttt{h} must
be exactly the desired goal. .

\hypertarget{assumption}{%
\subsection{\texorpdfstring{\texttt{assumption}}{assumption}}\label{assumption}}

\begin{itemize}
\tightlist
\item
  Now we see that our hypotheses yield the result.
\end{itemize}

If one of the hypotheses exactly proves the theorem, apply that
hypothesis.

\hypertarget{apply}{%
\subsection{\texorpdfstring{\texttt{apply}}{apply}}\label{apply}}

\begin{itemize}
\tightlist
\item
  Given \texttt{h}, to show \texttt{Q} it suffices to show \texttt{P}.
\end{itemize}

If a hypothesis says \texttt{h\ :\ P\ →\ Q} and our goal is \texttt{Q}
then \texttt{apply\ h} replaces \texttt{Q} by \texttt{P}.

\hypertarget{rw-or-rewrite}{%
\subsection{\texorpdfstring{\texttt{rw} or
\texttt{rewrite}}{rw or rewrite}}\label{rw-or-rewrite}}

\begin{itemize}
\item
  Given \texttt{h\ :\ a=b} we can replace \texttt{a} by \texttt{b} (or
  in the goal or in a hypothesis.
\item
  By hypothesis \texttt{h}, we can replace \emph{left side of
  \texttt{h}} with \emph{right side of \texttt{h}}. Or, with the
  \texttt{←} version, we can replace the \emph{right side of \texttt{h}}
  with the \emph{left side of \texttt{h}} in the goal.
\end{itemize}

Given a hypothesis that asserts the equality of two things (\texttt{=}
or \texttt{⟺}), replace one thing by the other.

\hypertarget{to-replace-in-the-goal}{%
\subsubsection{\texorpdfstring{To replace in the
\emph{goal}:}{To replace in the goal:}}\label{to-replace-in-the-goal}}

By default, given \texttt{h\ :\ a=b}, the command \texttt{rw\ h}
replaces \texttt{a} by \texttt{b} The command \texttt{rw\ ←\ h} replaces
\texttt{b} by \texttt{a}.

\hypertarget{to-replace-in-a-hypothesis-use-at.}{%
\subsubsection{\texorpdfstring{To replace in a \emph{hypothesis}, use
\texttt{at}.}{To replace in a hypothesis, use at.}}\label{to-replace-in-a-hypothesis-use-at.}}

Given hypotheses \texttt{hab:\ a=b} and \texttt{hbc:\ b=c} then
\texttt{rw\ hbc\ at\ hab} changes \texttt{hab} to \texttt{a=c}. Note
that you are using \texttt{hbc} to rewrite \texttt{hab}.

\hypertarget{by_contra}{%
\subsection{\texorpdfstring{\texttt{by\_contra}}{by\_contra}}\label{by_contra}}

\begin{itemize}
\tightlist
\item
  We will prove the contrapositive, so assume the conclusion is false.
\end{itemize}

The tactic \texttt{by\_contra} makes the negation of the goal a
hypothesis and changes the goal to \texttt{false}.

\hypertarget{cases}{%
\subsection{\texorpdfstring{\texttt{cases}}{cases}}\label{cases}}

\begin{itemize}
\tightlist
\item
  Given \texttt{P\ ∨\ Q}, we consider separately the cases when
  \texttt{P} is true and when \texttt{Q} is true. (\texttt{P\ ∨\ Q} is a
  hypothesis)
\item
  To prove \texttt{P\ ∧\ Q} we prove \texttt{P} and \texttt{Q}
  separately.
\item
  If we know there exists an \texttt{x} of type \texttt{T} satisfying a
  property \texttt{p\ x}, we can assume separately that \texttt{x} is of
  type \texttt{T} and that \texttt{p\ x} is a hypothesis. In other
  words, decompose a \texttt{∃\ x,\ p\ x} hypothesis into \texttt{∃\ x}
  hypothesis and a \texttt{p\ x} hypothesis.
\end{itemize}

If \texttt{x} is a \emph{hypothesis} about an inductive type then
\texttt{cases} breaks up that hypothesis into its component parts. For
example, if \texttt{h} is a proof of an \texttt{and} term then
\texttt{cases\ h} is are the separate proofs of the terms. If `h' is a
proof about a product, then \texttt{cases\ h} are the separate proofs of
the terms.

\hypertarget{left-and-right}{%
\subsection{\texorpdfstring{\texttt{left} and
\texttt{right}}{left and right}}\label{left-and-right}}

\begin{itemize}
\tightlist
\item
  It suffices to prove the proposition on the left/right.
\end{itemize}

When a goal is made up of two parts, either of which suffices,
\texttt{left} and \texttt{right} replace the goal with the corresponding
part. For example, if the goal is \texttt{P\ ∨\ Q} then \texttt{left} is
like \texttt{apply} for the implication \texttt{P\ →\ P\ ∨\ Q} and right
is \texttt{apply} for \texttt{Q\ →\ P\ ∨\ Q}.

\hypertarget{use}{%
\subsection{\texorpdfstring{\texttt{use}}{use}}\label{use}}

\begin{itemize}
\tightlist
\item
  Then \texttt{x} satisfies the desired conditions.
\end{itemize}

\texttt{use\ x} says to instantiate the \texttt{∃\ y,\ p\ y} clause of a
goal with x, turning the goal into \texttt{p\ x}.

\hypertarget{split}{%
\subsection{\texorpdfstring{\texttt{split}}{split}}\label{split}}

\begin{itemize}
\tightlist
\item
  We consider the component propositions to our conclusion in turn.
\end{itemize}

\texttt{split} breaks up a compound goal (like \texttt{P\ ∧\ Q} or
\texttt{P\ ⟺\ Q}) into subgoals.

\hypertarget{refl}{%
\subsection{\texorpdfstring{\texttt{refl}}{refl}}\label{refl}}

\begin{itemize}
\tightlist
\item
  True by definition.
\end{itemize}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{tikz}
\setlength{\parindent}{0pt}

\newtheorem*{theorem}{Theorem}
\newtheorem*{definition}{Definition}
\newtheorem*{lemma}{Lemma}
\newtheorem*{corollary}{Corollary}
\newtheorem{example}{Example}
\newtheorem*{trick}{Trick}
\newtheorem*{question}{Question}

\title{Lecture 2: Determinants, Cross Product}
\author{}
\date{}

\begin{document}

\maketitle

\section{Determinants}

\subsection{Determinants in 2D plane}

With two vectors, how do we calculate the area of the triangle enclosed by
these two vectors?

\subsubsection{Calculate the area of the triangle formed by two vectors with an acute angle}

\begin{tikzpicture}
  [help line/.style={dashed}]
  \draw[->] (0, 0) -- (3, 0) node[above] {$\vec{A}$};
  \draw[->] (0, 0) -- (2, 2) node[above] {$\vec{B}$};
  \draw[help line] (2, 2) -- (2, 0);
  \draw[->, help line] (0, 0) -- (0, 3) node[left] {$\vec{A'}$};
  \draw (2, 0.2) -- (1.8, 0.2) -- (1.8, 0);
  \draw (0.2, 0) arc [start angle=0, end angle=45, radius=0.2] node[right]
    {$\theta$};
  \draw (0, 0.2) arc [start angle=90, end angle=45, radius=0.2] node[above]
    {$\theta'$};
\end{tikzpicture}

Apparently
\[
  Area = \frac{1}{2}|\vec{A}||\vec{B}|\sin\theta
\]

But it is hard to calculate $\sin\theta$, whereas it is easy to calculate
$\cos\theta$ with dot products. Therefore, we can rotate $\vec{A}$ towards
$\vec{B}$ by $90^{\circ}$ to get $\vec{A'}$ and hence
\begin{gather*}
  \sin\theta = \cos(90^{\circ} - \theta) = \cos(\theta') \\
  \begin{split}
  Area  &= \frac{1}{2} |\vec{A}| |\vec{B}| \sin\theta \\
        &= \frac{1}{2} |\vec{A'}| |\vec{B}| \cos\theta' \\
        &= \frac{1}{2} \vec{A'} \cdot \vec{B} \\
  \end{split} \\
\end{gather*}

Suppose that $\vec{A} = <a_1, a_2>$, and $\vec{B} = <b_1, b_2>$, then
$\vec{A'} = <-a_2, a_1>$. Therefore,

\[
  \begin{split}
  \vec{A'} \cdot \vec{B}  &= <-a_2, a_1> \cdot <b_1, b_2> \\
                          &= a_1 b_2 - a_2 b_1 \\
                          &= det(\vec{A}, \vec{B}) \\
                          &=  \begin{vmatrix}
                                a_1 & a_2 \\
                                b_1 & b_2
                              \end{vmatrix}
  \end{split}
\]

Notice that it is also possible that $\vec{A'} = <a_2, -a_1>$, depending on
the relative positions of $\vec{A}$ and $\vec{B}$, so the result could be the
negative value of the area.

\subsubsection{Calculate the area of the triangle formed by two vectors with an obtuse angle}

\begin{tikzpicture}
  [help lines/.style={dashed}]
  \draw[->] (0, 0, 0) -- (3, 0, 0) node[anchor=south] {$\vec{A}$};
  \draw[->] (0, 0, 0) -- (-2, 2, 0) node[anchor=south] {$\vec{B}$};
  \draw[help lines] (0, 0, 0) -- (-3, 0, 0);
  \draw[help lines] (-2, 2, 0) -- (-2, 0, 0);
  \draw (-2, 0.2, 0) -- (-1.8, 0.2, 0) -- (-1.8, 0, 0);
  \draw (0.2, 0) arc [start angle=0, end angle=135, radius=0.2]
    node[anchor=south west] {$\theta$};
  \draw[->, help lines] (0, 0, 0) -- (0, 3, 0) node[anchor=east] {$\vec{A'}$};
  \draw (0, 0.4) arc [start angle=90, end angle=135, radius=0.4]
    node[anchor=south] {$\theta'$};
\end{tikzpicture}

\[
  \begin{split}
  Area  &= \frac{1}{2} |\vec{A}| |\vec{B}| \sin(\pi - \theta) \\
        &= \frac{1}{2} |\vec{A}| |\vec{B}| \sin\theta \\
        &= \frac{1}{2} |\vec{A}| |\vec{B}| \sin(\pi + \theta') \\
        &= \frac{1}{2} |\vec{A'}| |\vec{B}| \cos\theta' \\
        &= \frac{1}{2} \vec{A'} \cdot \vec{B}
  \end{split}
\]

Similarly, suppose that $\vec{A} = <a_1, a_2>$, and $\vec{B} = <b_1, b_2>$, then
$\vec{A'} = <-a_2, a_1>$. Therefore,

\[
  \begin{split}
  \vec{A'} \cdot \vec{B}  &= <-a_2, a_1> \cdot <b_1, b_2> \\
                          &= a_1 b_2 - a_2 b_1 \\
                          &= det(\vec{A}, \vec{B}) \\
                          &=  \begin{vmatrix}
                                a_1 & a_2 \\
                                b_1 & b_2
                              \end{vmatrix}
  \end{split}
\]

\subsubsection{Geometric interpretation of determinants in 2D plane}

$det(\vec{A}, \vec{B})$ measures the area of the parallelogram formed by
$\vec{A}$ and $\vec{B}$, but the value can be positive or negative of it.

\subsection{Determinants in 3D space}

Suppose that $\vec{A} = <a_1, a_2, a_3>$, $\vec{B} = <b_1, b_2, b_3>$, and
$\vec{C} = <c_1, c_2, c_3>$, then
\[
  \begin{split}
  det(\vec{A}, \vec{B}, \vec{C}) &= \begin{vmatrix}
                                    a_1 & a_2 & a_3 \\
                                    b_1 & b_2 & b_3 \\
                                    c_1 & c_2 & c_3
                                    \end{vmatrix} \\
                                 &= a_1 \cdot \begin{vmatrix}
                                              b_2 & b_3 \\
                                              c_2 & c_3
                                              \end{vmatrix} -
                                    a_2 \cdot \begin{vmatrix}
                                              b_1 & b_3 \\
                                              c_1 & c_3
                                              \end{vmatrix} +
                                    a_3 \cdot \begin{vmatrix}
                                              b_1 & b_2 \\
                                              c_1 & c_2
                                              \end{vmatrix}
  \end{split}
\]

We can expand the $3 \times 3$ determinants to expressions of $2 \times 2$
determinants according to the first row to calculate their values. The rules
are as follows:

\subsubsection{Geometric interpretation of determinants in 3D space}

\begin{theorem}
Geometrically, $det(\vec{A}, \vec{B}, \vec{C}) = \pm$ volume of the
parallelepiped formed by $\vec{A}$, $\vec{B}$, and $\vec{C}$.
\end{theorem}

\section{Cross Product}

\begin{definition}
  The cross product of two vectors $\vec{A} = <a_1, a_2, a_3>$ and
  $\vec{B} = <b_1, b_2, b_3>$ is a vector defined as:
  \[
    \begin{split}
    \vec{A} \times \vec{B} &= \begin{vmatrix}
                                \vec{i} & \vec{j} & \vec{k} \\
                                a_1 & a_2 & a_3 \\
                                b_1 & b_2 & b_3
                              \end{vmatrix} \\
                           &= \begin{vmatrix}
                                a_2 & a_3 \\
                                b_2 & b_3
                              \end{vmatrix} \vec{i} -
                              \begin{vmatrix}
                                a_1 & a_3 \\
                                b_1 & b_3
                              \end{vmatrix} \vec{j} +
                              \begin{vmatrix}
                                a_1 & a_2 \\
                                b_1 & b_2
                              \end{vmatrix} \vec{k}
    \end{split}
  \]
\end{definition}

Notice that the $3 \times 3$ determinants in the above definition is not a
legit determinant expression. It is just a symbolic notation helping memorize
the formula.

\begin{theorem}
$|\vec{A} \times \vec{B}| = $ the area of the parallelogram formed by $\vec{A}$
and $\vec{B}$ in the 3D space.
\end{theorem}

\begin{theorem}
The direction of $\vec{A} \times \vec{B}$, or $dir(\vec{A} \times \vec{B})$
is perpendicular to the parallelogram formed by $\vec{A}$ and $\vec{B}$ in
the 3D space.
\end{theorem}

There are two directions which is perpendicular to the parallelogram formed by
$\vec{A}$ and $\vec{B}$, which one is the direction of $\vec{A} \times \vec{B}$?

Right-hand rule: Use your right hand to rotate from $\vec{A}$ to $\vec{B}$ then
your right thumb is pointing to the direction of $\vec{A} \times \vec{B}$.

\begin{example}

\end{example}

\subsection{Properties of cross product}

\subsubsection{Commutative property}
Cross product does not satisfy the commutative property.

\begin{gather*}
  \vec{A} \times \vec{B} \neq \vec{B} \times \vec{A} \\
  \vec{A} \times \vec{B} = - \vec{B} \times \vec{A} \\
\end{gather*}

In particular:
\[
  \vec{A} \times \vec{A} = \vec{0}
\]

\subsubsection{Associative property}
Cross product does not satisfy the associative property.

Counter example: Suppose $\vec{A} = <1, 0, 1>$, $\vec{B} = <1, 1, 0>$,
$\vec{C} = <0, 1, 1>$, then we can get
\begin{gather*}
  (\vec{A} \times \vec{B}) \times \vec{C} = <0, 1, -1> \\
  \vec{A} \times (\vec{B} \times \vec{C}) = <1, 0, -1> \\
\end{gather*}

\subsection{Application of cross product}

\begin{example}
Given the coordinates of three points that are not in the same line in 3D space,
find the equation of the plane determined by these three points.

Solution:

Finding the equation of a plane is equivalent to finding a method of
determining whether a random point is in the plane or not.

Suppose the three known points are $P_1$, $P_2$, and $P_3$, and a random point
is $P$.

Method 1: $P$ is in the plane if and only if the parallelepiped formed by
$\vec{P_1P_2}$, $\vec{P_1P_3}$, and $\vec{P_1P}$ has a volume $0$. Therefore,
according to the geometric interpretation of determinants in 3D space, we can
derive that $P$ is in the plane if and only if
\[
  det(\vec{P_1P_2}, \vec{P_1P_3}, \vec{P_1P}) = 0
\]

Method 2: Using the normal vector. The normal vector of a plane refers to the
vector that are perpendicular to the plane. A plane can have infinite normal
vectors. We usually use $\vec{N}$ to represent a normal vector.

The result of a cross product of two vectos is a normal vector of the
parallelogram formed by the two vectos. With that being said, $P$ is in the
plane if and only if $\vec{P_1P} \perp \vec{N}$, which is equivalent to
\begin{gather*}
  \vec{P_1P} \cdot \vec{N} = 0 \\
  \vec{P_1P} \cdot (\vec{P_1P_2} \times \vec{P_1P_3}) = 0 \\
\end{gather*}

Therefore, the two equations are actually equivalent:
\[
  det(\vec{P_1P_2}, \vec{P_1P_3}, \vec{P_1P}) = \vec{P_1P} \cdot (\vec{P_1P_2} \times \vec{P_1P_3}) = 0
\]

\end{example}

\end{document}


\clearpage %or \cleardoublepage
\phantomsection
\chapter*{Notation}
\markright{}
\addcontentsline{toc}{chapter}{Notation}

\begin{longtable}[l]{ l p{0.7\textwidth} }
  $a$                                          & [m] ellipsoid equatorial radius \\
  $b$                                          & [m] ellipsoid polar radius \\
  $\hat c$                                     & [m] mean aerodynamic chord \\
  $c$                                          & [N/(m/s)] damping coefficient \\
  $c_S$                                        & [m/s] speed of sound \\
  $C_D$                                        & [-] drag coefficient \\
  $C_l$                                        & [-] rolling moment coefficient \\
  $C_L$                                        & [-] lift coefficient \\
  $C_m$                                        & [-] pitching moment coefficient \\
  $C_n$                                        & [-] yawing moment coefficient \\
  $C_P=P / \left( \rho n^3 D^5 \right)$        & [-] power coefficient \\
  $C_T=T / \left( \rho n^2 D^4 \right)$        & [-] thrust coefficient \\
  $C_Y$                                        & [-] side force coefficient \\
  $C_{\Delta P}$                               & [-] engine power losses coefficient \\
  $D$                                          & [N] drag \\
  $D$                                          & [m] propeller diameter \\
  $g$                                          & [m/s\textsuperscript{2}] gravity acceleration \\
  $h$                                          & [m] altitude \\
  $\vec H=\left[ H_X, H_Y, H_Z \right]^T$      & [kg$\cdot$m\textsuperscript{2}/s] angular momentum vector \\
  $i$                                          & [rad] incidence angle \\
  $\boldsymbol I$                              & [kg$\cdot$m\textsuperscript{2}] inertia matrix \\
  $J$                                          & [-] propeller advance ratio \\
  $k$                                          & [N/m] spring constant \\
  $L$                                          & [N] lift \\
  $m$                                          & [kg] mass \\
  $\boldsymbol M$                              & [kg],[kg$\cdot$m],[kg$\cdot$m\textsuperscript{2}] generalized inertia matrix \\
  $n$                                          & [rpm] engine revolutions per minute \\
  $n$                                          & [rev./s] propeller revolution speed \\
  $p$                                          & [Pa] pressure \\
  $P$                                          & [W] power \\
  $\vec P=\left[ P_X, P_Y, P_Z \right]^T$      & [kg$\cdot$m/s] momentum vector \\
  $Q$                                          & [N$\cdot$m] torque \\
  $\vec Q=\left[ L, M, N \right]^T$            & [N$\cdot$m] moment of force vector \\
  $\vec r=\left[ x, y, z \right]^T$            & [m] coordinates vector \\
  $R$                                          & [J/(kmol$\cdot$K)] universal gas constant \\
  $\vec R=\left[ X, Y, Z \right]^T$            & [N] force vector \\
  $\boldsymbol s=\left[ u,v,w,p,q,r \right]^T$ & [m/s],[rad/s] aircraft state vector \\
  $S$                                          & [m\textsuperscript{2}] wing area \\
  $S$                                          & [K] Sutherland constant \\
  $\vec S=\left[ S_X, S_Y, S_Z \right]^T$      & [kg$\cdot$m] first moment of mass \\
  $T$                                          & [K] temperature \\
  $T$                                          & [N] thrust \\
  $V$                                          & [m/s] velocity \\
  $\vec V=\left[ u, v, w \right]^T$            & [m/s] velocity vector \\
  \multicolumn{2}{l}{$\boldsymbol x=\left[ x,y,z,e_0,e_x,e_y,e_z \right]^T$} \\
                                               & [m],[-] aircraft coordinates vector \\
  $\alpha$                                     & [rad] angle of attack \\
  $\beta$                                      & [rad] angle of sideslip \\
  $\hat \delta$                                & [-] normalized controls position \\
  $\theta$                                     & [rad] pitch angle \\
  $\lambda$                                    & [rad] geographic longitude \\
  $\mu$                                        & [-] friction coefficient \\
  $\mu$                                        & [Pa$\cdot$s] dynamic viscosity \\
  $\nu$                                        & [m\textsuperscript{2}/s] kinematic viscosity \\
  $\rho$                                       & [kg/m\textsuperscript{3}] air density \\
  $\varphi$                                    & [rad] geographic latitude \\
  $\phi$                                       & [rad] roll angle \\
  $\psi$                                       & [rad] yaw angle \\
  $\vec \omega=\left[ p, q, r \right]^T$       & [rad/s] angular velocity vector \\
  $\cfrac{\partial \epsilon}{\partial \alpha}$ & [-] horizontal stabilizer downwash angle derivative with respect to the aircraft angle of attack \\
  & \\
  \multicolumn{2}{l}{\textbf{Indices:}} \\
  $a$   & Aerodynamic Axis System \\
  $A$   & aerodynamics \\
  $AC$  & aerodynamic center \\
  $b$   & Body Axis System \\
  $CM$  & Center of Mass \\
  $e$   & Earth-fixed Axis System \\
  $g$   & Gravity (North-East-Down) Axis System \\
  $h$   & horizontal stabilizer \\
  $LG$  & landing gear \\
  $M$   & mass \\
  $MAP$ & manifold absolute pressure \\
  $N$   & normal \\
  $O$   & coordinate system’s origin \\
  $P$   & propulsion \\
  $s$   & Stability Axis System \\
  $T$   & tangent \\
  $v$   & vertical stabilizer \\
  $X$   & x-axis component \\
  $Y$   & y-axis component \\
  $Z$   & z-axis component \\
\end{longtable}


\chapter{Single-compartment model}

\section{Equivalent circuits}

The essential characteristic of a neuron is its electrical properties. The individual components of a cell and its surroundings, such as the membrane, the intracellular plasma or channel proteins behave like electronic circuit components such as resistors, capacitors or batteries. This means we can describe neurons in terms of \textbf{equivalent circuits} and use electrical laws to predict its behavior.

For example, let us consider a cylindrical axon segment. The resistance along the axon is calculated by:
\begin{align*}
R_L &= \frac{r_LL}{\pi a^2} \\
R_L &\qquad \text{longitudinal resistance, measured in Ohm (\si{\ohm})} \\
r_L &\qquad \text{resistivity, measured in resistance times length (e.g.\ \si{\ohm\meter})}\\
L &\qquad \text{axon length}\\
a &\qquad \text{axon radius}
\end{align*}
The resistivity is an intrinsic property of a conducting material. In the case of cell plasma it generally depends on ionic concentrations. With $L=100\si{\micro\meter}, a=2\si{\micro\meter}$ and $r_L=1\si{\kilo\ohm\milli\meter}$ the formula above gives us $R_L\approx 8 \si{\mega\ohm}$. Then, with Ohm's law, we can calculate the voltage drop necessary to drive a 1\si{\nano\ampere} current through this axon segment. Note that causation in Ohm's law works in both directions. Voltage differences cause currents and currents cause voltage differences.
\begin{align*}
R=\frac{V}{I} &\Leftrightarrow V=R\cdot I \qquad \text{Ohm's law}\\
V_L &= 8\si{\mega\ohm}\cdot 1\si{\nano\ampere} = 8\si{mV}\\
R &\qquad \text{resistance in Ohm (\si{\ohm})} \\
V &\qquad \text{voltage difference in Volt (\si{\volt})}\\
I &\qquad \text{current in Ampere (\si{\ampere})}
\end{align*}

Next we look at Ohm's law with conductance instead of resistance. Conductance is the inverse of resistance, uses the symbol $G$ and is measured in Siemens (\si{\siemens}). We consider a channel protein with a cylindrical pore of length $L=6\si{nm}$ and cross sectional area $A=0.15\si{\square\nano\meter}$
\begin{align*}

\end{align*}

\section{Hydraulic analogue}

\section{Single-compartment model}

\section{Solving for voltage}

\section{Solving for currents}

\section{Ion-specific conductances}


\input{../header.tex}
\title{\vspace{-2cm}INF3490/INF4490 Exercises - Evolutionary Computing}
\author{Eivind Samuelsen\input{../author_footnote.tex}}
\date{}

% Removing paragraph indents is sometimes useful:
\setlength\parindent{0pt}
% ==============================================================================

% ================================= DOCUMENT ===================================
\begin{document}
    \renewcommand\marginsymbol[1][0pt]{%
  \tabto*{0cm}\makebox[-1cm][c]{$\mathbb{P}$}\tabto*{\TabPrevPos}}

\maketitle
\input{../intro.tex}

\section{Evolution strategy(ES)} % 1
\subsection{} % a
A common variant of evolution strategies used for (local) search is the \((1 + 4)\) ES.
How would this differ from the \((1 + 1)\) ES in how the search space is explored?
How does this, and \((1 + \lambda)\) in general, compared to hill climbing and greedy search?
\subsection{} % b
What effect does an adaptive search strategy have on optimization performance?
\subsection{} % c
How would it affect the search if the strategy parameters were mutated after the solution parameters instead of before?
\section{ES Implementation} % 2
\subsection{\marginsymbol} % a
\label{subsec:w4e3a}
Ignoring mutation, and starting with the population \(\{1, 2, 3, 4\}\),
implement and run 3 generations of a \((4 + 8)\) ES maximizing \(g(x) = x\),
and observe what the end population looks like (use intermediary recombination).
\subsection{} % b
If a \((4, 8)\) ES had been used in Problem \ref{subsec:w4e3a},
what would the probability of the optimal solution \((x = 4)\) surviving the first generation have been?
\subsection{\marginsymbol} % c
Repeat Problem \ref{subsec:w4e3a} with an EP with \(q = 2\).
How do the two algorithms compare?
\section{Knapsack problem}
In a 0-1 \href{https://en.wikipedia.org/wiki/Knapsack_problem}{\textbf{knapsack problem}},
how could you implement a repair mutation to transform infeasible solutions into feasible ones
(i.e. make the sum of costs of the selected items go below the budget)?

\input{../contact.tex}
\end{document}
% ==============================================================================


\documentclass{article}
\usepackage{amsmath}

\begin{document}

\section{Modified Encoder for Pointer Network}
Since the original pointer network architecture is based on a LSTM encoder-decoder model, it is likely to be suspectible to the same problems of being unable to leverage long range information. This is especially critical as the architecture is presented for combinatoric problems where the ordering of the input sequence is irrelevant. We propose a modified encoder network based on dilated CNNs with gated activation units and skipped connections. Specifically, let $x_k$ be the output of layer $k$, then,
\begin{align}
x_{k} = x_{k-1} + [\tanh(W_{f,k}* x_{k-1})\odot\sigma(W_{g,k}* x_{k-1})]
\end{align}
where $*$ denotes a dilated convolution operator, $\odot$ denotes an element-wise multiplication, $\sigma(\cdot)$ is the sigmoid function, and $W_{(f,g),k}$ are learnable filters. For the first layer, $x_{-1}$ are the embedded inputs to the network and we did not include the skip connection. \\

In order to initialize the decoder LSTM hidden state and utilize the attention mechanism, we concatenated the outputs of each layer to form the collection of encoder outputs. For the initial decoder hidden state, we applied average pooling to this encoder outputs collection.

\end{document}