\documentclass[12point]{article}

\begin{document}

\section{Diffusion}
We model external virus as a concentration that diffuses and can be uptake or exported. We use the standard PhysiCell equations:

\begin{eqnarray}
\frac{partial \rho }{\partial t} & = &
D \nabla^2 \rho - \lambda \rho +
\sum_{\textrm{cells }i}
\delta( \vec{x} - \vec{x}_i ) \left[
V_i U_i \rho + E_i
\right]
\end{eqnarray}
where $D$ is the diffusion coefficient, $\lambda$ is the decay rate, $U$ is the cell's uptake rate, and $E$ is the net export rate (see more below).

We use a PDE like this for viral particles $V$, uncoated virus $U$ (which might be released by dead cells), viral RNA $R$ (might be released by dead cells), viral protein $P$ (might be released by dead cells), and also
release assembled virus (used as a technical assist on viral internalization and export dynamics--all virions in this last equation are immediately transferred to the viral concentration. See below).

\section{Internal virus model}
Let's define the following variables:
\begin{enumerate}
\item
$V$ are viruses that are bound to the cell surface and in the process of endocytosis / adsorption.

\item
$U$ are viruses that have been fully endocytoses and coated.

\item
$R$ is RNA that has been delivered from uncoated virus $U$

\item
$P$ are (full sets) of viral proteins that have been synthesized from RNA $R$

\item
$A$ are assembled virions that are ready for export.

\end{enumerate}
Let's model just the internal processes, noting that PhysiCell automatically takes care of virus intake (source for $V$) and export (sink for $A$).

Our first model of this will be a system of ODEs:
\begin{eqnarray}
\frac{dV}{dt} & = & -r_U V \\
\frac{dU}{dt} & = & r_U V - r_P U \\
\frac{dR}{dt} & = & r_P R - \lambda_R R \\
\frac{dP}{dt} & = & r_S R - \lambda_P P - r_A P \\
\frac{dA}{dt} & = & r_A P
\end{eqnarray}

Next, we need to figure out net virus export. We use PhysiCell's standard form. If $\rho_A$ is the external concentration of exported (and assembled) viral particles, then:
\begin{eqnarray}
\frac{d\rho_A}{dt} = ... \sum_{\textrm{cells } i} \delta( \vec{x}-\vec{x}_i ) E_{A,i}
\end{eqnarray}
where $\delta$ is the Dirac delta function centered at the cell's position
$\vec{x}_i$, and $E_{A,i}$ is the net export rate (in assembled virions per time).

We use the following constitutive relation:
\begin{eqnarray}
E_A & = & r_E A.
\end{eqnarray}

As one final note: we want exported $A$ diffuse and be able to infect other cells. We do this with a ``cleanup'' step at each PhysiCell diffuison time step: in each voxel, we move all of $\rho_A$ to $\rho_V$ (the concentration
of viral particles).

\section{Cell apoptotic response (PD)}
We use a fairly standard Hill function: let $e$ be the effect, which we model as increasing with assembled viral particles:
\begin{eqnarray}
e & - & \frac{ A^n  }{ A^n + A_{\frac{1}{2}}^n  }
\end{eqnarray}
where $A_{\frac{1}{2}}$ is the half max, and $n$ is the Hill coefficient.

Then, the cell's apoptosis rate is
\begin{eqnarray}
r_{\textrm{apoptosis}} = \overline{r}_\textrm{max} e.
\end{eqnarray}

At death, some fraction $f$ of the cell's $V$, $U$, $R$, $P$, and $A$ are released into the microenvironment. These can potentially be individually specified.

\section{Later}
Use a molecular-scale model of ACE2 to model how the uptake rate changes.

Use a receptor trafficking model to also modulate virion uptake.

Better parameter estimates of viral replication.

Add immune response with more cell types. Cell death can release immunostimulatory factors that bring in macrophages. Those cause more damage and feedback. Connect that to ARDS.

\end{document}


\section{Systems and Higher Order Equations}
There is a connection between a system of linear first order equations and a single higher order equation. We'll see how to convert between the two, which will help explain why methods like the auxiliary equation work.

% System to higher order
\input{./linearSystems/systemsAndHigherOrder/systemToHigherOrder.tex}
% Higher order to system
\input{./linearSystems/systemsAndHigherOrder/higherOrderToSystem.tex}


\documentclass[a4paper, 12pt]{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{dsfont}
\usepackage[left=2cm, right=2cm, bottom=3cm, top=2cm]{geometry}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{natbib}
\newcommand{\lbc}{\ell}

\title{lbrycred}
\author{Brendon J. Brewer}
\date{}

\begin{document}
\maketitle

%\abstract{\noindent Abstract}

% Need this after the abstract
\setlength{\parindent}{0pt}
\setlength{\parskip}{8pt}

Let $\boldsymbol{y} = \{y_1, y_2, ..., y_n\}$ be the vector of credibility
scores (where $y_i \geq 0$) and $\boldsymbol{w} = \left\{w_{ij}\right\}$
a matrix of signed supports, where
$w_{ij}$ is the support from channel $i$ to channel $j$. Let the diagonal
elements $w_{ii}$ include the bid, self supports, and anonymous supports.
In practice the $\boldsymbol{w}$ will be sparse and concentrated along the
diagonal. The total staked amount on channel $j$ is the column sum
\begin{align}
\ell_j = \sum_{i=1}^n w_{ij}.\label{eqn:total_lbc}
\end{align}
It would be nice if $\boldsymbol{y}$ were in units of LBC and if
it was normalised to the same total as the staked LBC:
\begin{align}
\sum_{i=1}^n \ell_i = \sum_{i=1}^n y_i,
\end{align}
and what I propose here and demonstrate in {\tt simulate.py} satisfies this
criterion.

We need to compute the $\boldsymbol{y}$ values from the $\boldsymbol{w}$,
and Jeremy (or someone else at LBRY Inc) proposed an iterative procedure which
reminds me of the
\href{https://en.wikipedia.org/wiki/Jacobi_method}{Jacobi Method} or
(more accurately) Gauss-Seidel. However,
I tried it out and it diverged --- presumably the matrix $\boldsymbol{w}$ had
eigenvalues above 1.

{\bf Aside mostly for my own thinking}:
One can think of these iterative methods as optimisation techniques; basically
Gibbs sampling at zero temperature in order to optimise some objective
function. Since I'm better at optimisation than linear algebra, I will try to
work in these terms. You can also think about these as numerical PDE solvers
like in undergrad COSC.

\section{Taking Supporter Credibility Into Account}
Now let's try replacing $\ell_j$ with an alternative that accounts for the
credibilities of the supporting channels. We'll need to treat the diagonal
value $w_{jj}$ differently because self and anonymous supports don't get
modified due to credibility.

To do this, I will introduce another quantity for each channel,
$\boldsymbol{m} = \{m_1, m_2, ..., m_n\}$. The {\em m} stands for
{\em multiplier}, as the effect of these is going to be multiplication of LBC
(so if a channel has $m=2$, their staked LBC is twice as valuable as someone
with $m=1$). All $m$ values are greater than or equal to 1.
Before continuing, let's define a nonlinear softening function which takes
an LBC amount as input and returns a multiplier value. Similar to trending,
I will use a power, with a modification so that the initial value
is 1:
\begin{align}
\Phi(x) &= (x + 1)^\alpha.
\end{align}
The inverse of this is
\begin{align}
\Phi^{-1}(x) &= x^{\frac{1}{\alpha}} - 1,
\end{align}
and in the demo I have set $\alpha = 1/3$.

If the $y$ values are correct, the $m$-values are:
\begin{align}
m_j &= \Phi(y_j). \label{eqn:grade}
\end{align}

The iterative procedure to compute the $m$s and thus the $y$s is as follows,
starting from an initial estimate of the $y$s which should probably be set
equal to the $\ell$s (the total staked LBC).
\begin{enumerate}
\item Compute the $m$s from the $y$s using Equation~\ref{eqn:grade}.
\item Update the $y$s using $y_j = w_{jj} + \sum_{i \neq j} m_i w_{ij}$
($\equiv$~Eqn~\ref{eqn:total_lbc} if $m$s are 1)
\item Re-normalise the $y$s by dividing by their sum and multiplying by total
staked LBC
\end{enumerate}
This process is repeated until the change in the $y$s is neligible. Typically
not many iterations are needed.

\bibliographystyle{plainnat}
\bibliography{references}

\end{document}


\section{How common are split votes?}
\label{leaderelection:split:rate}

The previous section analyzed the performance of normal elections when
no split vote occurs. In practice, two or more candidates may time out
at similar times, leading to split votes.  Split votes cause additional
election timeout delays, and if they occur too frequently, they can
impact election performance dramatically. This section first analyzes
split votes under a simplifying assumption that network latencies are
constant, then subsequently relaxes this assumption.

\subsubsection{Split vote rate with fixed latency}

\begin{figure}
\centering
\includegraphics[scale=.5]{leaderelection/splitvotediagram}
\vcaption[split vote example with fixed latency]{
These examples show two similar elections in a five-server cluster when
one server has failed and network messages have a fixed latency. Each
server's random timeout value is shown on the timelines, where $t_{(1)}$
is the smallest value chosen, $t_{(2)}$ is the second-smallest, and so
on. In the top election, the first server is able to collect votes from
itself, the third server, and the fourth server. However, in the bottom
election, its RequestVote RPC cannot reach the third server in time
before that server times out; thus, the election ends in a split vote.
}
\label{fig:leaderelection:theory:splitvotediagram}
\end{figure}

Split votes can be calculated more simply if network latencies are
fixed. Let the constant $l$ be the one-way latency between any two
servers in the cluster, measured as a fraction of the election timeout
range.
Because of the fixed network latency,
the first server to time out is guaranteed to get the
votes of all servers that don't time out within $l$ of it, and it will
receive none of the votes of the other servers, who will each vote for
themselves.
The probability of a split vote is thus the probability that
too many candidates time out within $l$ of each other. For example,
consider a five-server cluster in which only four servers are available.
As illustrated in Figure~\ref{fig:leaderelection:theory:splitvotediagram},
if only two servers time out within $l$ of each other, the earliest server
will be able to collect votes from itself and the other two servers and
become leader. However, if three servers time out within $l$, then the
earliest server will only be able to reach one other server in time to
receive its vote, so the vote is split.

To derive a general formula for when split votes
occur, let $c$ denote the number of servers that time out within $l$ of
each other and let $n$ be the size of the full cluster.
The first server will get its own vote plus votes from the $s-c$ servers
that time out at least $l$ time after the first. Thus, a split vote
occurs when the following condition holds:
\begin{align*}
\text{votes needed}              &> \text{votes available to earliest server} \\
\Big\lfloor \dfrac{n}{2} \Big\rfloor + 1 &> 1 + (s - c) \\
c &> s - \Big\lfloor \frac{n}{2} \Big\rfloor
\end{align*}

How often split votes occur thus depends on how often at least $c$ servers
timeout within $l$ of each other. Let $D_{c,s} = T_{(c)} - T_{(1)}$,
where $T_{(i)}$ is a random variable representing the timeout of the
$i$-th of $s$ servers in sorted order; $D_{c,s}$ is the time after the
first server times out that the $c^\text{th}$ server times out. The
probability of split votes is then $\Pr(D_{c,s} < l)$, where $c$ is
determined by the formula given above ($s - \Big\lfloor \dfrac{n}{2}
\Big\rfloor + 1$).

We now derive the CDF for $D_{c,s}$, denoted $\Pr(D_{c,s} \leq l)$.
Suppose the first server times out at $t$.
%
First, if $t < 1-l$,
each of the following servers times out within $l$ time after $t$ with probability
$\dfrac{l}{1-t}$.
The probability that the second through $c^\text{th}$ servers time out
within $l$ time after $t$, and the remaining $s-c$ servers do not, is given by:
\begin{align*}
& \dbinom{s-1}{c-1}
  \left(\frac{l}{1-t}\right)^{c-1}
  \left(\frac{1-t-l}{1-t}\right)^{s-c}
\end{align*}
\noindent
%
Instead, if $t \geq 1-l$, then any server that times out after the first
must time out within $l$ time of $t$. Thus, all $s$ servers will
time out within $l$ of the first with probability $1$, and the
probability that any server does not timeout within $l$ of the first is
$0$. Putting this together, we can now derive the CDF:
%
\begin{align*}
\Pr(D_{c,s} \leq l)
%
 &= \mathlarger{\sum_{k=c}^s} \left(
\begin{aligned}
     &\int_0^{1-l}
     \Pr(\text{exactly $k$ servers time out in $t$ to $(t+l)$ range} \, | \, M_s=t)
     f_{M_s}(t) \, dt \, +
     \\
     &\int_{1-l}^1
     \Pr(\text{exactly $k$ servers time out in $t$ to $(t+l)$ range} \, | \, M_s=t)
     f_{M_s}(t) \, dt
\end{aligned}
\right) \displaybreak[0]\\
%
 &= \mathlarger{\sum_{k=c}^s}
     \left(
     \int_0^{1-l}
     \Pr(\text{exactly $k$ servers time out in $t$ to $(t+l)$ range} \, | \, M_s=t)
     f_{M_s}(t) \, dt
    \right) \, + \\
 &\quad \int_{1-l}^1 f_{M_s}(t) \, dt
  \displaybreak[0]\\
%
 &= \mathlarger{\sum_{k=c}^s}
     \left(
     \int_0^{1-l}
     \dbinom{s-1}{k-1}
     \left(\frac{l}{1-t}\right)^{k-1}
     \left(\frac{1-t-l}{1-t}\right)^{s-k}
     f_{M_s}(t) \, dt
    \right) \, + \\
 &\quad \int_{1-l}^1 f_{M_s}(t) \, dt
  \displaybreak[0]\\
%
 &= \mathlarger{\sum_{k=c}^s}
     \dbinom{s-1}{k-1}
     \left(
     \int_0^{1-l}
     \left(\frac{l}{1-t}\right)^{k-1}
     \left(\frac{1-t-l}{1-t}\right)^{s-k}
     s
     (1-t)^{s-1} \, dt
    \right) \, + \\
 &\quad \int_{1-l}^1 s (1-t)^{s-1} \, dt
  \displaybreak[0]\\
%
 &= \mathlarger{\sum_{k=c}^s}
     \dbinom{s-1}{k-1}
     \left. \left(
     -\frac{s}{s-k+1}
     l^{k-1}
     \left(1-t-l\right)^{s-k+1}
    \right) \right|_{t=0}^{1-l}
     \, + \\
 &\quad \mathlarger{\left. \left( -(1-t)^s \right)
\right|_{t=1-l}^1}
  \displaybreak[0]\\
%
 &= \left( \mathlarger{\sum_{k=c}^s}
     \dbinom{s-1}{k-1}
     \frac{s}{s-k+1}
     l^{k-1}
     (1-l)^{s-k+1}
    \right)
     \, + l^s
 \displaybreak[0] \\
%
 &= \left( \mathlarger{\sum_{k=c}^s}
     \frac{(s-1)! (s)}{(k-1)!(s-k)! (s-k+1)}
     l^{k-1}
     (1-l)^{s-k+1}
     \right)
     \, + l^s
    \displaybreak[0] \\
%
 &= \left(\mathlarger{\sum_{k=c}^s}
     \frac{s!}{(k-1)!(s-k+1)!}
     l^{k-1}
     (1-l)^{s-k+1}
     \right)
     \, + l^s
    \displaybreak[0] \\
%
 &= \left(\mathlarger{\sum_{k=c}^s}
     \dbinom{s}{k-1}
     l^{k-1}
     (1-l)^{s-k+1}
     \right)
     \, + l^s
    \displaybreak[0] \\
%
 &= \left(\mathlarger{\sum_{k=c-1}^{s-1}}
     \dbinom{s}{k}
     l^k
     (1-l)^{s-k}
     \right)
     \, + l^s
    \displaybreak[0] \\
%
 &= \mathlarger{\sum_{k=c-1}^s}
     \dbinom{s}{k}
     l^k
     (1-l)^{s-k}
    \displaybreak[0]
\end{align*}
%
(The CDF somewhat resembles a binomial distribution, which hints that
there may exist an easier derivation.)

For example, consider a five-server
cluster with four available servers ($s=4$). A split vote will occur if the
earliest three servers time out within $l$ of each other ($c=3$). If the
election timeout range is \SI{100}{\milli\second},
the earliest three servers will time
out within $l$ of each other, resulting in a split vote:
\begin{compactitem}
\item In about 0.06\% of elections, if the one-way network latency is
\SI{1}{\milli\second},
$\Pr(D_{3,4} \leq .01)$;
\item In about 5.2\% of elections, if the one-way network latency is
\SI{10}{\milli\second}, $\Pr(D_{3,4} \leq .1)$; and
\item In about 18.1\% of elections, if the one-way network latency is
\SI{20}{\milli\second}, $\Pr(D_{3,4} \leq .2)$.
\end{compactitem}

\begin{figure}
\centering
\includegraphics{leaderelection/distance}
\vspace{-2ex}
\vcaption[split vote probability with fixed network latency]{
The graphs show the likelihood of split votes for various cluster
sizes and numbers of server failures, given a fixed network latency $l$.
Each graph shows a different full cluster size, and the curves on each
graph show different numbers of failed servers in that cluster.
Each value represents the likelihood that a split vote will occur
because the first $c$ of the $s$ servers timed out within $l$ of each
other, where $c$ is determined by $s - \Big\lfloor \dfrac{n}{2}
\Big\rfloor + 1$.
For example, a five-server
cluster with two failures and $l=0.2$ will see about half of
elections end in split votes.
}
\label{fig:leaderelection:theory:distance}
\end{figure}

Figure~\ref{fig:leaderelection:theory:distance} graphs the CDF formula for
a range of cluster sizes. The first thing to observe is that failures
have a very large effect on split vote rates, especially if the cluster
is down to a bare majority of its original members. For example, a
five-server cluster
with $l=0.2$ will encounter less than 20\% split votes after one
failure;
if the same cluster encounters a second failure, about half of election
terms will encounter split votes. To prepare for worst-case scenarios,
the election timeout range should be set to produce tolerable values
when a bare majority of the cluster is available.

Second,
larger clusters experience fewer split votes with the same number of
failures, but they experience an even larger worst-case split vote rate
as a result of being able to tolerate more failures. For example, a
nine-server cluster with $l=0.2$ will experience only about a
15\% rate of
split votes after two failures (compare with 50\% for a five-server
cluster). However, when it is down to its bare majority with four
failures, the nine-server cluster will experience a nearly 70\% split vote rate.

Third, keeping the number of available servers constant, larger full
cluster sizes will have more split votes. For example, with $l=0.2$, a
nine-server cluster with six available servers will experience about a
35\% rate split votes; a seven-server cluster with six available servers will
experience only about a 10\% rate split votes. This is because larger full
clusters require more votes to win an election; fewer candidates need to
time out within $l$ of each other in order to produce a split vote.

Finally, the graphs suggest that choosing an election timeout range of
10--20 times the one-way network latency (so $l=0.1$) will result in low split
vote rates in all clusters, assuming network latencies are nearly
constant. With this setting, a nine-server cluster that has experienced
four failures will encounter 40\% split votes, and most typical clusters
will encounter much fewer. Smaller election timeouts (larger $l$ values)
may also work in many deployments, but they should be tested more
carefully to make sure.

\subsubsection{Split vote rate with variable latency}

When network latency is variable, calculating split vote rates is more
complicated. The problem is that a RequestVote message sent by one
server can overtake a RequestVote message sent earlier by a different
server. Thus, the first server to time out is no longer guaranteed to
collect all of the votes of servers that do not vote for themselves.
The first server is still the most likely candidate to win, by virtue of
sending requests for votes first, but its advantage depends on how much
earlier it timed out. Thus, with variable latency, we intuitively expect
somewhat higher rates of split votes (there is more competition).

\begin{figure}
\centering
\includegraphics{leaderelection/splits.pdf}
\vspace{-3ex}
\vcaption[split vote probability with variable network latency]{
The contour graphs show split vote rates for various cluster sizes and
numbers of failed servers. Each narrow contour line denotes a 1\%
increase in the probability of split votes; each medium contour line
denotes a 10\% increase; the thick contour line visible in some graphs
denotes the 50\% barrier. Split votes are always
0\% at the origin,
where messages are instantaneous. The points on the $x$ axis, where the
latency range is zero, correspond to the split vote rates with fixed
latencies in Figure~\ref{fig:leaderelection:theory:distance}.
\\
For example, the probability of a split vote for a five-server cluster
after one failure can be found in the graph in the second column and
second row. When latencies are chosen randomly and uniformly between 0.1
and 0.2, the point with a
minimum latency of 0.1 and a latency range of 0.1 reveals, by counting
contour lines, that the split vote rate is about 16\%. With two
failures, the probability of a split vote in the same cluster is nearly
40\%.
}
\label{fig:leaderelection:theory:splitvotes}
\end{figure}

Rather than model this mathematically, we used a small simulation. Each
run followed the following steps (before optimization):
%
\begin{enumerate}
%
\item Assign random timeouts to each of $s$ servers.
%
\item If a server has not voted by the time it times out, it votes for
itself and schedules RequestVote messages to be delivered to other
servers after random latencies.
%
\item If any server collects a majority of votes, the election term is
considered successful; otherwise, it is considered a split vote.
%
\end{enumerate}
%
After \num{10000} runs, the fraction of split votes was calculated.

Figure~\ref{fig:leaderelection:theory:splitvotes} shows the split vote
rates for messages with uniform random latencies in the range
$[l_\text{min},l_\text{max}]$, as calculated by the simulation.
(Uniform random latencies may not be a realistic distribution, but they
are the simplest case and can help with estimating more complex
distributions.)
The overall conclusion from these graphs is the same as
with fixed latency: more failures result in significantly higher split
vote rates.

In clusters with only a bare majority of servers available (the bottom
graph in each column), the contour lines are very linear with a slope of
about $-2$: they have about the same split vote rate when keeping the
average network latency constant. This indicates that the split vote
rates for bare-majority clusters
can be accurately approximated with our fixed latency model using
the average of the variable latency range. For example in a nine-server
cluster with four failures, a variable latency chosen randomly from the
range $[0.1, 0.2]$ results in a similar split vote rate as a fixed
latency of $0.15$.

In clusters with fewer failures, the contour lines aren't always linear,
and they typically have less slope in general (they are flatter). For
example, in a five-server cluster with no failures, a variable latency
between 0.1 and 0.2 has a similar split vote rate as a fixed latency of
about 0.2 (the slope of the contour lines is only about $-1$).
Typically, split vote rates can be bounded with our fixed latency model
using the maximum of the variable latency range. This is true for about
78\% of the data points shown in the figure; however, this approximation
works least well for large clusters with few failures, as these contour
lines are most curved.


\subsection{Benchmark: Lorenz 1963}

We first verified that our choice of implementation for a conventional \gls{esn}
produced similar results to those found in the literature
\cite{pathak_using_2017}. Table \ref{tab:lorenzparam} contains the hyper-parameters
that minimized the RMSE of the model. Our optimized values differ
somewhat from the literature, but our \gls{esn} implementation
succesfully replicated the climate of the Lorenz Attractor similar to Pathak
et. al 2017 \cite{pathak_using_2017}. Figure \ref{fig:lorenz63} shows the
\gls{esn} forecast ten seconds into the future for the Lorenz 1963 model.

\begin{table}[H]
  \centering
  \caption{Hyper-parameters for the \gls{esn} that predicted the Lorenz 1963
  model. The random seed was generated by the open source package
  \texttt{numpy}.}
  \label{tab:lorenzparam}
  \begin{tabular}{l r r}
    \hline
    Parameter & This paper & Literature \cite{pathak_using_2017}\\
    \hline
    $N$ & 2000& 300\\
    $\rho$& 0.9&1.2\\
    \texttt{sparsity}& 0.1& 0.1\\
    \texttt{noise}& 0.001& 0\\
    Training Length & 3200& Not Specified\\
    Random Seed & 85 & Not Specified\\
    \hline
  \end{tabular}
\end{table}
\begin{figure}[H]
  \centering
  % \includegraphics[width=\columnwidth]{lorenz63_prediction.png}
  \resizebox{\columnwidth}{!}{\input{./images/lorenz63_prediction.pgf}}
  \caption{Using an \gls{esn} to replicate the climate of the Lorenz Attractor.}
  \label{fig:lorenz63}
\end{figure}


\documentclass{article}
\usepackage{fullpage}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{tikz}
\usepackage{bm}
\usepackage[colorlinks,linkcolor=blue,citecolor=blue,urlcolor=blue]{hyperref}
\usepackage{siunitx}
\usepackage{eufrak}
\usepackage{filecontents}
\usepackage{natbib}
\usepackage{doi}

\begin{filecontents}{matrix-comparison.bib}
@article{devendran2017,
	title={A fourth-order {Cartesian} grid embedded boundary method for {Poisson's} equation},
	author={Devendran, Dharshi and Graves, Daniel and Johansen, Hans and Ligocki, Terry},
	journal={Comm.\ App.\ Math.\ Comp.\ Sci.},
	volume={12},
	number={1},
	pages={51--79},
	year={2017},
	publisher={Mathematical Sciences Publishers},
	doi={10.2140/camcos.2017.12.51}
}

@article{shaw2017,
	title={Multidimensional method-of-lines transport for atmospheric flows over steep terrain using arbitrary meshes},
	author={Shaw, James and Weller, Hilary and Methven, John and Davies, Terry},
	journal={J.\ Comp.\ Phys.},
	volume={344},
	pages={86--107},
	year={2017},
	publisher={Elsevier},
	doi={10.1016/j.jcp.2017.04.061}
}
\end{filecontents}

\newcommand{\vect}{\mathbf}
\newcommand{\moment}{\mathfrak{m}}

\title{Comparison of one-dimensional cubicFit and highOrderFit}
\author{James Shaw}

\begin{document}
\nocite{*}
\maketitle

\begin{figure}
	\centering
	\begin{tikzpicture}[
	  scale=0.53,
	  cpnt/.style={fill=gray},
	]

	\draw [->] (-3,0) -- (5.5,0) node [right] {$x$};
	\draw (-1,-0.5) -- (-1,0.5);
	\draw (1,-0.5) -- (1,0.5);
	\draw (3,-0.5) -- (3,0.5) node [above] {$f$};

	\path [cpnt] (-2,0) circle [radius=0.1] node [below] {$\phi_{uuu}$};
	\path [cpnt] (0,0) circle [radius=0.1] node [below] {$\phi_{uu}$};
	\path [cpnt] (2,0) circle [radius=0.1] node [below] {$\phi_u$};
	\path [cpnt] (4,0) circle [radius=0.1] node [below] {$\phi_d$};
	\draw [thick] (3,0) circle [radius=0.15];
	\end{tikzpicture}
%
	\caption{One-dimensional four-point upwind-biased stencil used to approximate the flux at face $f$.}
	\label{fig:stencil}
\end{figure}

Here we make a comparison between two transport schemes: cubicFit \citep{shaw2017} and highOrderFit, which is based on the high-order formulation by \citet{devendran2017}.  Both schemes form a matrix equation that is solved to find coefficients used to calculate the flux.  We define a one-dimensional, four-point, upwind-biased stencil (figure~\ref{fig:stencil}) with equispaced cell centres.
For cubicFit we approximate a field $\phi$ using a cubic polynomial
\begin{align}
\phi = a_1 + a_2 x + a_3 x^2 + a_4 x^3
\end{align}
that interpolates the four stencil points.  A matrix equation is formed in order to calculate the unknown coefficients $a_1 \ldots a_4$,
\begin{align}
	\begin{bmatrix}
		1 & x_{uuu} & x_{uuu}^2 & x_{uuu}^3 \\
		1 & x_{uu}  & x_{uu}^2  & x_{uu}^3 \\
		1 & x_u     & x_u^2     & x_u^3 \\
		1 & x_d     & x_d^2     & x_d^3
	\end{bmatrix}
	\begin{bmatrix}
		a_1 \\
		a_2 \\
		a_3 \\
		a_4
	\end{bmatrix}
	=
	\begin{bmatrix}
		\phi_{uuu} \\
		\phi_{uu} \\
		\phi_u \\
		\phi_d
	\end{bmatrix}
	\text{.}
\end{align}
If the equispaced cell centres are positioned at $x_{uuu} = -2.5$, $x_{uu} = -1.5$, $x_u = -0.5$, $x_d = 0.5$ then
\begin{align}
	\begin{bmatrix}
		1 & -2.5 & 6.25 & -15.625 \\
		1 & -1.5 & 2.25 & -3.375 \\
		1 & -0.5 & 0.25 & -0.125 \\
		1 &  0.5 & 0.25 &  0.125
	\end{bmatrix}
	\begin{bmatrix}
		a_1 \\
		a_2 \\
		a_3 \\
		a_4
	\end{bmatrix}
	=
	\begin{bmatrix}
		\phi_{uuu} \\
		\phi_{uu} \\
		\phi_u \\
		\phi_d
	\end{bmatrix}
	\text{.}
	\label{eq:cubicFitMatrix}
\end{align}
For highOrderFit, we solve the matrix equation
\begin{align}
	\begin{bmatrix}
		\moment_{uuu}^0/\moment_{uuu}^0 & \moment_{uuu}^1/\moment_{uuu}^0 & \moment_{uuu}^2/\moment_{uuu}^0 & \moment_{uuu}^3/\moment_{uuu}^0 \\
		\moment_{uu}^0/\moment_{uu}^0 & \moment_{uu}^1/\moment_{uu}^0 & \moment_{uu}^2/\moment_{uu}^0 & \moment_{uu}^3/\moment_{uu}^0 \\
		\moment_u^0/\moment_u^0 & \moment_u^1/\moment_u^0 & \moment_u^2/\moment_u^0 & \moment_u^3/\moment_u^0 \\
		\moment_d^0/\moment_d^0 & \moment_d^1/\moment_d^0 & \moment_d^2/\moment_d^0 & \moment_d^3/\moment_d^0
	\end{bmatrix}
	\begin{bmatrix}
		a_1 \\
		a_2 \\
		a_3 \\
		a_4
	\end{bmatrix}
	=
	\begin{bmatrix}
		\phi_{uuu} \\
		\phi_{uu} \\
		\phi_u \\
		\phi_d
	\end{bmatrix}
\end{align}
where $\moment_V^p = \int_V x^p dV$ is the $p$th moment of volume $V$, and the zeroth moment $\moment_V^0$ is equal to the volume.
If the equispaced cells each have $\moment^0 = 1$ with the cell centres positioned as before, then
\begin{align}
	\begin{bmatrix}
		1 & -2.5 & 6.\dot{3} & -16.25 \\
		1 & -1.5 & 2.\dot{3} & -3.75 \\
		1 & -0.5 & 0.\dot{3} & -0.25 \\
		1 &  0.5 & 0.\dot{3} & 0.25
	\end{bmatrix}
	\begin{bmatrix}
		a_1 \\
		a_2 \\
		a_3 \\
		a_4
	\end{bmatrix}
	=
	\begin{bmatrix}
		\phi_{uuu} \\
		\phi_{uu} \\
		\phi_u \\
		\phi_d
	\end{bmatrix}
	\text{.}
	\label{eq:highOrderMatrix}
\end{align}
Notice how the matrix in equation~\eqref{eq:highOrderMatrix} is similar to, but not equal to, the matrix in equation~\eqref{eq:cubicFitMatrix}.

\section*{Taylor series analysis}

\begin{figure}
	\centering
	\begin{tikzpicture}[
	  scale=0.53,
	  cpnt/.style={fill=gray},
	]
	\draw [->] (-5,0) -- (5.5,0) node [right] {$x$};
	\draw (-3,-0.5) -- (-3,0.5);
	\draw (-1,-0.5) -- (-1,0.5);
	\draw (1,-0.5) -- (1,0.5) node [above] {$\phi_L$};
	\draw (3,-0.5) -- (3,0.5) node [above] {$\phi_R$};

	\path [cpnt] (-4,0) circle [radius=0.1] node [below] {$\phi_{j-2}$};
	\path [cpnt] (-2,0) circle [radius=0.1] node [below] {$\phi_{j-1}$};
	\path [cpnt] (0,0) circle [radius=0.1] node [below] {$\phi_j$};
	\path [cpnt] (2,0) circle [radius=0.1] node [below] {$\phi_{j+1}$};
	\path [cpnt] (4,0) circle [radius=0.1] node [below] {$\phi_{j+2}$};
	\draw [thick] (1,0) circle [radius=0.15];
	\draw [thick] (3,0) circle [radius=0.15];
	\end{tikzpicture}
	\caption{One-dimensional fluxes through a cell $\phi_{j+1}$ using four-point upwind-biased stencils to approximate fluxes $\phi_L$ and $\phi_R$.}
	\label{fig:flux}
\end{figure}

To find a polynomial that interpolates the 4-point upwind-biased stencil we construct Taylor series approximations centred at $\phi_L$ for the stencil points $\phi_{j-2} \ldots \phi_{j+1}$
\begin{align}
	\begin{bmatrix}
		1 & -\frac{5}{2} & \frac{1}{2!} \left(-\frac{5}{2}\right)^2 & \frac{1}{3!} \left(-\frac{5}{2}\right)^3 \\[4pt]
		1 & -\frac{3}{2} & \frac{1}{2!} \left(-\frac{3}{2}\right)^2 & \frac{1}{3!} \left(-\frac{3}{2}\right)^3 \\[4pt]
		1 & -\frac{1}{2} & \frac{1}{2!} \left(-\frac{1}{2}\right)^2 & \frac{1}{3!} \left(-\frac{1}{2}\right)^3 \\[4pt]
		1 & \frac{1}{2} & \frac{1}{2!} \left(\frac{1}{2}\right)^2 & \frac{1}{3!} \left(\frac{1}{2}\right)^3
	\end{bmatrix}
	\begin{bmatrix}
		\phi_L \\
		\phi'_L \\
		\phi''_L \\
		\phi'''_L
	\end{bmatrix}
	&=
	\begin{bmatrix}
		\phi_{j-2} \\
		\phi_{j-1} \\
		\phi_j \\
		\phi_{j+1}
	\end{bmatrix}
	\text{.}
\end{align}
Solving this system results in the fourth-order accurate approximation,
\begin{align}
	\phi_L = \frac{1}{16} \phi_{j-2} - \frac{5}{16} \phi_{j-1} + \frac{15}{16} \phi_j + \frac{5}{16} \phi_{j+1} + \mathcal{O}(\Delta x^5)
\end{align}
The approximate flux divergence (figure~\ref{fig:flux}) is then
\begin{align}
	\phi_R - \phi_L = -\frac{1}{16} \phi_{j-2} + \frac{6}{16} \phi_{j-1} - \frac{20}{16} \phi_j + \frac{20}{16} \phi_{j+1} + \frac{5}{16} \phi_{j+2} \label{eq:cubicFitCoeffs}
\end{align}
but this not a fourth-order accurate flux divergence approximation as we will see next.
A fourth-order accurate approximation to the flux divergence is derived from a Taylor series approximation of $\partial \phi / \partial x$ centred at $\phi_{j+1}$ (figure~\ref{fig:flux}),

\begin{align}
	\begin{bmatrix}
		1 & -3 & \frac{1}{2!} \left(-3\right)^2 & \frac{1}{3!} \left(-3\right)^3 & \frac{1}{4!} \left(-3\right)^4 \\[4pt]
		1 & -2 & \frac{1}{2!} \left(-2\right)^2 & \frac{1}{3!} \left(-2\right)^3 & \frac{1}{4!} \left(-2\right)^4 \\[4pt]
		1 & -1 & \frac{1}{2!} \left(-1\right)^2 & \frac{1}{3!} \left(-1\right)^3 & \frac{1}{4!} \left(-1\right)^4 \\[4pt]
		1 & 0 & \frac{1}{2!} 0^2 & \frac{1}{3!} 0^3 & \frac{1}{4!} 0^4 \\[4pt]
		1 & 1 & \frac{1}{2!} 1^2 & \frac{1}{3!} 1^3 & \frac{1}{4!} 1^4
	\end{bmatrix}
	\begin{bmatrix}
		\phi_{j+1} \\[4pt]
		\phi'_{j+1} \\[4pt]
		\phi''_{j+1} \\[4pt]
		\phi'''_{j+1} \\[4pt]
		\phi''''_{j+1}
	\end{bmatrix}
	&=
	\begin{bmatrix}
		\phi_{j-2} \\
		\phi_{j-1} \\
		\phi_j \\
		\phi_{j+1} \\
		\phi_{j+2}
	\end{bmatrix} \label{eq:highOrderFluxDiv}
\intertext{which yields}
\left. \frac{\partial \phi}{\partial x} \right|_{j+1} = -\frac{1}{12} \phi_{j-2} + \frac{6}{12} \phi_{j-1} - \frac{18}{12} \phi_j + \frac{10}{12} \phi_{j+1} + \frac{3}{12} \phi_{j+2} + \mathcal{O}(\Delta x^4) \text{.} \label{eq:highOrderFluxDivCoeffs}
\end{align}
To find the fourth-order face flux we invert the matrix in equation~\eqref{eq:highOrderMatrix} and find that
\begin{align}
	\phi_L = \frac{1}{12} \phi_{j-2} - \frac{5}{12} \phi_{j-1} + \frac{13}{12} \phi_j + \frac{3}{12} \phi_{j+1} \text{.} \label{eq:highOrderFlux}
\end{align}
Using equation~\eqref{eq:highOrderFluxDiv} and equation~\eqref{eq:highOrderFlux} we can verify that $\left. \partial \phi / \partial x \right|_{j+1} = \phi_R - \phi_L + \mathcal{O}(\Delta x^4)$ as desired (figure~\ref{fig:fluxDivComparison}).

\begin{figure}
	\centering
	\includegraphics[width=5in]{interim-results/taylor-series.png}
	\caption{Flux divergence cofficients using a 5-point Taylor series approximation (equation~\ref{eq:highOrderFluxDivCoeffs}), cubicFit approximation (equation~\ref{eq:cubicFitCoeffs}) and highOrderFit approximation (equation~\ref{eq:highOrderFlux}).}
	\label{fig:fluxDivComparison}
\end{figure}

\bibliographystyle{ametsoc2014}
\bibliography{matrix-comparison}

\end{document}


\section*{Web Appendix E: Additional simulations}
In this section we provide details for two additional simulations using different priors on $\tau$. The setup for the simulations are the same as in Section 3 in the paper. The uniform prior has support on $[0, 3]$. The inverse Gamma prior has shape parameter $2$ and scale parameter $1/2$. These parameters were chosen in order to make the prior have sufficient mass close to $0$. All results are based on $N = 100$ replications.

\subsection*{Inverse Gamma prior on $\tau^2$}
\input{tables/invGamma_cl}
\input{tables/invGamma_pb}
\input{tables/invGamma_ph}

\newpage

\subsection*{Uniform prior on $\tau$}

\input{tables/uniform_cl}
\input{tables/uniform_pb}
\input{tables/uniform_ph}


\documentclass[twoside, 11pt]{article}

\usepackage[preprint]{jmlr2e}

\usepackage{natbib}

\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{mathtools}

% for bayesian network diagrams
% ref: https://github.com/jluttine/tikz-bayesnet
\usepackage{float}
\usepackage{tikz}
\usetikzlibrary{bayesnet}

% ensure sufficient marginspace for todos
\setlength {\marginparwidth }{2cm}
\usepackage[obeyFinal]{todonotes}
% \setuptodonotes{inline}

% define notation for norm and abs that scale nicely.
% ref: https://tex.stackexchange.com/a/297263
\let\oldnorm\norm
\let\norm\undefined
\DeclarePairedDelimiter\norm{\lVert}{\rVert}

\let\oldabs\abs
\let\abs\undefined
\DeclarePairedDelimiter\abs{\lvert}{\rvert}

\DeclarePairedDelimiter\card{\lvert}{\rvert}

\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\gammad}{Gamma}
\DeclareMathOperator*{\poissond}{Poisson}
\DeclareMathOperator*{\negbind}{Neg-bin}

\newcommand{\reals}[0] {\mathbb{R}}
\newcommand{\nonnegint}[0] {\mathbb{N}_{\geq 0}}

\newcommand{\E}[0] {\mathbb{E}} %expect

\newcommand{\bigO}[0] {\mathcal{O}}

\newtheorem{thm}{Theorem}

\begin{document}

% style self-loop edges.
\tikzset{every loop/.style={min distance=10mm,in=60,out=120,looseness=10}}

\author{\name Reuben Fletcher-Costin}

\title{}

\maketitle

\begin{abstract}%
*DRAFT*

We consider a hidden Markov model for event counts with an additive Poisson noise term. The model is otherwise assumed to have a finite state space. Computing the posterior distrubtion is intractable. Instead, we define an approximation scheme for the posterior distribution by approximating the conditional posterior distribution for the Poisson rate parameter $\lambda$ with one scaled Gamma distribution per discrete state in the model's state space. We derive an approximate forward algorithm to advance approximate posterior one timestep. The approximate forward update algorithm minimises the KL-divergence from the true forward-updated approximate posterior distribution (in general, a mixture of Gamma distributions) to its projection back as a single Gamma distribution in the approximation space.
\end{abstract}

\section{Summary}

Here's the rough plan of attack:
\begin{enumerate}
\item define basic hidden Markov model
\item define hidden Markov model with Poisson noise
\item compute the likelihood for hidden Markov model with Poisson noise
\item follow \citet*{russell2002artificial} derivation of the forward update equation for the posterior
\item give a brief example illustrating why we need a way to encode belief of varying rates $\lambda$ for different states in the discrete space, to motivate approximations
\item make the working assumption that we can approximate the posterior distribution with $\gammad$ distributions, one for each hidden state $s \in S$ in the discrete state space.
\item note how this family of approximations are not closed under the transition operator on $S$ -- whenever we apply the transition operator we end up with a mixture of $\gammad$ distributions. Similarly for conditioning on evidence.
\item look for a best approximation of the resulting mixture within our approximation space, in the sense of minimal $KL$-divergence
\item derive closed-form equations for our approximation to project each mixture back down to a single $\gammad$ distribution
\item comment about approximation error
\end{enumerate}

\section{probabilistic model}
\subsection{Prelude: hidden Markov model}

Consider a first-order hidden Markov model (HMM) with a hidden state $x_t$ at each time $t=0,\ldots,T$ that generates observable evidence $z_t$ for $t=1,\ldots,T$. We use $z_{1:T}$ to denote $z_1, \ldots, z_T$. Due to the Markov assumptions, the joint distribution of $Z_{1:T}$ and $x_{0:T}$ factorises as
\begin{equation}
P(z_{1:T}, x_{0:T}) = P(x_0) \prod_{t=1, \ldots, T} P(z_t \mid x_t) P(x_t \mid x_{t-1} ) \; .
\end{equation}
The conditional dependence structure of the HMM is illustrated in Figure~\ref{fig:hmm}.Each hidden state $x_t$ takes values in a finite state space $S = \{ s_1, \ldots, s_n \}$. Each generated observation $z_t$ is an event count in $\nonnegint$. The probability of transitioning from a state $x_t=s_j$ at time $t$ to a state $x_{t+1}=s_i$ at time $t+1$ is given by
\begin{equation}
P(x_{t+1}=s^{\prime} \mid x_t=s) = A_{s^{\prime},s}
\end{equation}
where $A$ is an $n \times n$ stochastic matrix indexed by the hidden states of $S$. The probability of observing $z_t=k$ events at time $t$ is given by
\begin{equation}
P(z_t=k \mid x_t=s) = B_{k, s}
\end{equation}
where for $s \in S$ the column $B_s$ is a distribution for $k$ over $\nonnegint$.

% standard hmm

\begin{figure}[H]
\tikz{
	% nodes
	\node[latent] (x) {$x_t$}; %
	\node[latent,left=of x] (x0) {$x_0$}; %
	\node[obs,below=of x] (z) {$z_t$}; %
	% plates
	\plate {timeplate} {(x) (z)} {$T$};%
	% edges
	\edge {x0} {x}
	\edge {x} {z}
	% self-loop edges
	\path[->] (x) edge [loop above] node {$t=1,\ldots,T$; $t-1 \mapsto t$} ();
}
\caption{Hidden Markov model}
\label{fig:hmm}
\end{figure}

\subsection{Hidden Markov model with additive Poisson noise}

Now consider a HMM where the event count $z_t$, generated from the hidden state $x_t$, is no longer directly observable. Instead, at each time $t$ we observe an event count $y_t \in \nonnegint$ such that
\begin{equation}
y_t = x_t + k_t
\end{equation}
where $k_t \in \nonnegint$ is generated from a Poisson distribution with a rate $\lambda > 0$ that does not vary over time. We assume a prior distribution $\gammad(\alpha, \beta)$ over $\lambda$. The joint probability distribution for this model is assumed to be
\begin{align}
& P(y_{1:T}, z_{1:T}, x_{0:T}, k_{1:t}, \lambda, \alpha, \beta) \nonumber \\
= & P(x_0) P(\alpha, \beta) P(\lambda \mid \alpha, \beta) \prod_{t=1, \ldots, T} P(y_t \mid k_t, z_t) P(k_t \mid \lambda) P(z_t \mid x_t) P(x_t \mid x_{t-1} ) \; .
\end{align}
This conditional dependence structure is shown in Figure~\ref{fig:hmmpoisson}.

The likelihood $P(y_t=y \mid x_t, \alpha, \beta)$ of observing $y_t=y$ events given the latent variables $x_t, \alpha, \beta$ reduces to a discrete convolution between the likelihood of the noise-free HMM $P(z_t=k \mid x_t)$ and the likelihood of the noise model $P(k_t=k \mid \alpha, \beta)$.
\begin{align}
& P(y_t=y \mid x_t, \alpha, \beta) \nonumber \\
= & \int_{(z, k, \lambda)} P(y_t=y \mid x_t, \alpha, \beta, z_t=z, k_t=k, \lambda=\lambda) P(z_t=z, k_t=k, \lambda=\lambda \mid x_t, \alpha, \beta) \, d(z, k, \lambda) \nonumber \\
= & \int_{(z, k, \lambda)}
P(y_t=y \mid z_t=z, k_t=k)
P(z_t=z \mid x_t)
P(k_t=k \mid \lambda)
P(\lambda \mid \alpha, \beta)
\, d(z, k, \lambda) \nonumber \\
= & \int_{(k, \lambda)}
P(z_t=y-k \mid x_t)
P(k_t=k \mid \lambda)
P(\lambda \mid \alpha, \beta)
\, d(k, \lambda) \nonumber \\
= & \sum_{k \in \nonnegint} P(z_t = y-k \mid x_t)
\int_{\lambda} P(k_t = k \mid \lambda) P(\lambda \mid \alpha, \beta) \, d \lambda \, .
\end{align}
The choice of $P(k_t = k \mid \lambda)$ as the Poisson distribution and $P(\lambda \mid \alpha, \beta)$ as its conjugate prior Gamma distribution leads to a closed-form expression for the integral:
\begin{align}
& \int_{\lambda}
P(k_t = k \mid \lambda) P(\lambda \mid \alpha, \beta)
\, d \lambda \nonumber \\
= & \int_{\lambda}
\poissond( k ; \lambda) \gammad( \lambda ; \alpha, \beta)
\, d \lambda \nonumber \\
= &
\binom{\alpha+k-1}{k} \left( \frac{\beta}{\beta+1} \right)^{\alpha} \left( \frac{1}{\beta+1} \right)^{k} \, .
\end{align}
A short proof of this identity is given in Appendix~\ref{appendix:gammaconj}. This final expression is known~\citep{gelman2013bayesian} to be the negative binomial density $k \sim \negbind\left(\alpha, \beta\right)$. Combining the above two derivations produces a terse expression for the likelihood
\begin{align}
P(y_t=y \mid x_t, \alpha, \beta)
= & \sum_{k \in \nonnegint} P(z_t = y-k \mid x_t) \negbind\left(k ; \alpha, \beta\right) \, . \label{eqn:hybridconjpriorobs}
\end{align}

% hmm with additive Poisson noise

\begin{figure}
\tikz{
	% nodes
	\node[obs] (y) {$y_t$}; %
	\node[latent,above=of y] (z) {$z_t$}; %
	\node[latent,above left=of y] (k) {$k_t$}; %
	\node[latent,above=of z] (x) {$x_t$}; %
	\node[latent,left=of x, xshift=-1cm] (x0) {$x_0$}; %

	\node[latent,left=of k] (lambda) {$\lambda$}; %
	\node[latent,left=of lambda] (alphabeta) {$\alpha, \beta$}; %
	% plates
	\plate {timeplate} {(x) (z) (k) (y)} {$T$};%
	% edges
	\edge {x0} {x}
	\edge {x} {z}
	\edge {z} {y}
	\edge {k} {y}
	\edge {lambda} {k}
	\edge {alphabeta} {lambda}
	% self-loop edges
	\path[->] (x) edge [loop above] node {$t=1,\ldots,T$; $t-1 \mapsto t$} ();
}
\caption{Hidden Markov model with Poisson noise}
\label{fig:hmmpoisson}
\end{figure}

\subsection{Inference}
\subsection{Standard HMM inference tasks}

As explained by \citet{rabiner1989tutorial}, \citet*{russell2002artificial}, given a hidden Markov model, there are a number of standard inference tasks we may wish to perform:

\begin{enumerate}
\item ``filtering'': estimating the posterior distribution at the current time $t=T$ over latent variables given evidence to date $y_{1:t}$
\item ``smoothing'': estimating the posterior distribution over latent variables given the evience to date at some historical time $t<T$
\item ``prediction'': estimating the posterior distribution over latent variables at future times $t>T$ given evidence to date
\item most likely explanation: recovering a trajectory of latent states that is best, in some particular sense (e.g. via the Viterbi algorithm)
\item estimating or training the parameters of our Markov model (E.g. via the Baum-Welch algorithm, an instance of Expectation Maximisation algorithms).
\end{enumerate}

We will focus on the first task, ``filtering''.

\subsection{Forward update}

The goal is to derive a computationally tractable way to update the posterior of the latent variables $P(x_t, \lambda \mid y_{1:t})$ given the information from one new observation of an event count $y_{t+1}$. A suitable update would be some function $F$ expressing the following relation:
\begin{equation}
P(x_{t+1}, \lambda \mid y_{1:t+1}) = F\left( P(x_t, \lambda \mid y_{1:t}), y_{t+1}\right)
\end{equation}

Although the rate parameter $\lambda$ does not vary in time, it can be notationally helpful to write $(x_t, \lambda_t)$ to denote the random variables $x_t$ and $\lambda_t=\lambda$ at time $t$. We can regard $(x_t, \lambda_t)$ as the hidden state of a HMM on the product state space $S \times \reals_{>0}$, with a transition model defined naturally component-wise -- note the transition model for $\lambda_t$ is trivial since $\lambda_t=\lambda$ for all $t$.

We structure the derivation of the forward upate for the model's posterior distribution $P(x_{t+1}, \lambda_t \mid y_{1:{t+1}})$ following \citet*{russell2002artificial}:
\begin{align}
& P(x_{t+1}, \lambda_{t+1} \mid y_{1:{t+1}}) \nonumber \\
= & P(x_{t+1}, \lambda_{t+1} \mid y_{1:t}, y_{t+1}) \nonumber \\
= & P(y_{t+1} \mid y_{1:t})^{-1} P(y_{t+1} \mid x_{t+1}, \lambda_{t+1}, y_{1:t}) P(x_{t+1}, \lambda_{t+1} \mid y_{1:t}) \nonumber \\
= & P(y_{t+1} \mid y_{1:t})^{-1} P(y_{t+1} \mid x_{t+1}, \lambda_{t+1}) P(x_{t+1}, \lambda_{t+1} \mid y_{1:t}) , \label{eqn:rawforward}
\end{align}
which follows from Bayes' theorem and the conditional dependence structure of the model.

The first factor $P(y_{t+1} \mid y_{1:t})^{-1}$ does not depend upon our latent variables and can be dropped if we are only interested in estimating relative posterior probabilities for different $(x_t, \lambda)$. We retain this factor to keep us honest - later on we will need to perform careful normalisation. The second factor, the likelihood $P(y_{t+1} \mid x_{t+1}, \lambda_{t+1})$ is a discrete convolution between the noise-free HMM's observation model $P(z_{t+1}=w \mid x_{t+1}=s^{\prime}) = B_{k,s^{\prime}}$ and our noise model $P(k_t=w \mid \lambda_t) = \poissond(w ; \lambda_t)$, that is,
\begin{equation}
P(y_{t+1}=k \mid x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime})
=
\sum_{w \in \nonnegint} B_{k-w,s^{\prime}} \poissond(w ; \lambda^{\prime}) \, \label{eqn:hybridobs} .
\end{equation}
The latter factor $P(x_{t+1}, \lambda_{t+1} \mid y_{1:t})$ needs to be rewritten as a function of $P(x_{t}, \lambda_{t} \mid y_{1:t})$, which can be obtained from the transition model after conditioning on $x_t=s, \lambda_t=\lambda$:
\begin{align}
& P(x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime} \mid y_{1:t}) \nonumber \\
= & \sum_{s \in S} \int_{\lambda \in \reals_{>0}}
P(x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime} \mid y_{1:t}, x_t=s, \lambda_t=\lambda ) P(x_t=s, \lambda_t=\lambda \mid y_{1:t} ) \,d\lambda \nonumber \\
= & \sum_{s \in S} \int_{\lambda \in \reals_{>0}}
P(x_{t+1}=s^{\prime}, \mid x_t=s)
P(\lambda_{t+1}=\lambda^{\prime} \mid \lambda_t=\lambda ) P(x_t=s, \lambda_t=\lambda \mid y_{1:t} ) \,d\lambda \nonumber \\
= & \sum_{s \in S}
P(x_{t+1}=s^{\prime}, \mid x_t=s)
P(x_t=s, \lambda_t=\lambda^{\prime} \mid y_{1:t} ) \nonumber \\
= & \sum_{s \in S}
A_{s^{\prime}, s}
P(x_t=s, \lambda_t=\lambda^{\prime} \mid y_{1:t} ) \nonumber \\
\label{eqn:advance}
\end{align}

Combining equation~\ref{eqn:rawforward} with equation~\ref{eqn:advance} and the expression for the likelihood~\ref{eqn:hybridobs} gives
\begin{align}
& P(x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime} \mid y_{1:{t+1}}) \nonumber \\
= &
\frac{1}{P(y_{t+1} \mid y_{1:t})}
\left[
\sum_{s \in S}
A_{s^{\prime}, s}
\sum_{w \in \nonnegint}
B_{k-w,s^{\prime}} \poissond(w ; \lambda^{\prime})
P(x_t=s, \lambda_t=\lambda^{\prime} \mid y_{1:t} )
\right] \label{eqn:exactfwdupdate}
\end{align}

One complication impeding exact computation of the posterior $P(x_{t+1}, \lambda_{t+1} \mid y_{1:t+1})$ is the coupling between the latent variables $\lambda_{t}$ and $x_{t}$ after conditioning on $y_{1:t}$. For a simple example of this, consider a state space $S$, consisting of two hidden states $s_1$ and $s_2$, equipped with observation models $P(z_t \mid x_t=s_1)$ and $P(z_t \mid x_t=s_2)$ such that $\E \left[ P(z_t \mid x_t=s_1) \right] = 1$ and $\E \left[ P(z_t \mid x_t=s_2) \right] = 2$. Assume transitions between $s_1$ and $s_2$ are impossible, but that we have very little prior knowledge about the magnitude of $\lambda$, the rate of the Poisson noise, or the initial state $x_0$. Suppose we observe event data $y_1, \ldots, y_T$ over a period of time, and learn that the average event count $\sum_{t=1}^T y_t \approx 3$.  Then if the system were dwelling in state $x_t=s_1$ we would infer that $\lambda \approx 2$, in order for the expected rate of ``signal'' event counts $Z_t$ and ``noise'' event counts $k_t$ to sum to give the observed event counts $y_t \approx 3$. Alternatively, if the system had occupied state $x_t=s_2$ we would infer that $\lambda \approx 1$. This means that an accurate representation of our posterior belief $P(x_{t+1}, \lambda_{t+1} \mid y_{1:t+1})$ needs to be rich enough to model coupling between $\lambda_{t+1}$ and $x_{t+1}$.

The previous example also serves to illustrate how the marginal posterior distribution for $\lambda$ can be multi-modal. This is also true for the conditional posterior distribution $P(\lambda_t \mid x_t=s, y_{1:t})$. Consider a singleton state space $S=\{s\}$ with an observation model $P(z_t \mid x_t=s)$ that emits an event count of $z_t=0$ or $z_t=100$ with uniform probability $1/2$. Suppose we have little prior knowledge about the noise rate $\lambda$. At $t=1$ we observe our first data $y_1=100$. We infer either $z_1=100$ hence $k_1=0$ or $z_1=0$ hence $k_1=100$. Therefore our conditional posterior belief for the noise rate $P(\lambda_1 \mid x_1=s, y_1)$ will be bimodal, having one mode near $\lambda=100$ and another mode near $\lambda=0$.

\subsection{Approximate forward updates of approximate posteriors}
We can regard the forward update of the posterior as $\phi_{t+1} = F \left( \phi_t, y_{t+1} \right)$ where the $\phi$ are abstract elements of some space $\Phi$ that is closed under application of the following operators:
\begin{enumerate}
\item a transition operator $T : \Phi \rightarrow \Phi$ that evolves the latent states forward one time step.
\item a condition-on-observation operator $O_y : \Phi \rightarrow \Phi$, where $y$ is a new observation.
\end{enumerate}
If we have $T$ and $\{O_y\}_{y}$ then we can define $F(\phi, y) = O_{y} \circ T (\phi)$. Instead of representing the posterior exactly, we can select some approximation space $\Phi$ such that $P(x_t, \lambda \mid y_{1:t} ) \approx \phi_t \in \Phi$.

An approximation scheme may also be able to handle the case where the approximation space $\Phi$ is \emph{not} closed under application of $T$ or $O_y$. Suppose there exists some enclosing space $\bar{\Phi} \supset \Phi$ such that $\bar{\Phi}$ is closed under application of $T$ and $O_y$. Then, provided we can define a projection operator $\Pi_{\Phi} : \bar{\Phi} \rightarrow \Phi$ that maps each element in $\bar{\Phi} \setminus \Phi$ to its ``best'' approximation in $\Phi$, we can then define an approximate forward update function $\tilde{F}(\circ, y): \Phi \rightarrow \Phi$ by
\begin{equation}
\tilde{F}(\phi, y) \coloneqq \Pi_{\Phi} \circ O_{y} \circ T (\phi) \, .
\end{equation}

A natural way to project onto an approximation space inside a larger space $\bar{\Phi}$ equipped with a distance metric $d_{\bar{\Phi}}$ is to find any point in the approximation space with minimal distance from the point $\phi \in \bar{\Phi} \setminus \Phi$ we wish to approximate:
\begin{equation}
\Pi_{\Phi}(\phi)
\coloneqq
\argmin_{\phi^{\prime} \in \Phi } d_{\bar{\Phi}}(\phi, \phi^{\prime})
\end{equation}
Our objects $\phi$ are distributions, so instead of a distance metric we will instead measure error in the sense of the the (directed) Kullback–Leibler divergence of the original distribution $\phi$ from the approximant:~\footnote{
We chose this direction as intuitively it feels more correct to evaluate the expectation of the relative entropy with respect to the orginal, not the approximation. But perhaps this is mistaken, or perhaps it doesn't matter.}
\begin{align}
\Pi^{KL}_{\Phi}(\phi)
\coloneqq &
\argmin_{\phi^{\prime} \in \Phi } D_{KL}(\phi \mid \mid \phi^{\prime}) \\
 = & \argmin_{\phi^{\prime} \in \Phi } \mathbb{E}_{\phi} \left[ \log \left( \frac{ \phi^{\prime}}{ \phi } \right) \right]
\end{align}

\subsection{Discrete approximation of rate parameter $\lambda$}

One simple approximation scheme is to fix a-priori $m$ different values of the noise rate parameter $\{ \lambda_1, \ldots, \lambda_m \} = \Lambda_m \subset \reals_{>0}$. Then, if the state space $S$ contains $n$ hidden states $s_1, \ldots, s_n$, we can reduce the model to a HMM over the $n \times m$ discrete state space $(s , \lambda_j ) \in S \times \Lambda_m$. One way to construct this is to consider $m$ trivial single-state HMMs for each value of $\lambda_j$, then combine them into a single HMM with state space $\{ \lambda_1, \ldots, \lambda_m \}$ but no possible transitions between distinct states by taking the disjoint union of their state spaces $\{ \lambda_j \}_j$, then take the product of the resulting HMM with the standard noise-free discrete HMM for our states $x_t \in S$.  A downside to this approach is that if we pick too few values of $m$ we may get a poor approximation, but as we pick more values of $m$, the cumulative computational effort to repeatedly apply the transition operator over $t = 1, \ldots, T$ while updating the approximate posterior is $ \bigO (T n^2 m ) $.

\subsection{Approximation with Gamma distributions}

The simple discrete approximation scheme described in the previous subsection required us to store $m$ non-negative values $P(x_t=s, \lambda=\lambda_j \mid y_t)$ for each $s$, $t$ in order to approximate the distribution $\lambda \sim P(x_t=s, \lambda \mid y_t)$. We may be able to approximate the distribution of $\lambda$ for fixed $x_t=s$, $t$ more compactly with an appropriately chosen parametrised family of distributions. One obvious choice to investigate is the conjugate prior distribution for $\lambda$, $\gammad(\alpha, \beta)$. This suggests the following approximation:
\begin{equation}
P\left(x_t=s, \lambda \mid y_{1:t} \right) \approx \phi(x_t=s, \lambda) \coloneqq \gammad(\lambda ; \alpha_{s,t}, \beta_{s,t}) P(x_t=s \mid y_{1:t} ) ,
\end{equation}
where we regard the Gamma distribution $\gammad(\lambda ; \alpha_{s,t}, \beta_{s,t})$ as an approximation to the posterior distribution for the rate conditioned on the hidden state: $P(\lambda | x_t=s, y_{1:t})$. With this approximation, we need only maintain three scalar values for each $s$, $t$:
\begin{enumerate}
\item $P(x_t=s \mid y_{1:t})$, the value of the marginal distribution,
\item $\alpha_{s,t}>0$, a shape parameter, and
\item $\beta_{s,t}>0$ a rate parameter.
\end{enumerate}

We define our approximation space $\Phi$ as
\begin{equation}
\Phi \coloneqq \left\{ \phi_{\alpha, \beta, \gamma} : \alpha = (\alpha_s)_s, \beta = (\beta_s)_s, \gamma = (\gamma_s)_s, \alpha_s > 0, \beta_s > 0, \gamma_s \geq 0, \sum_{s \in S} c_s = 1 \right\}
\end{equation}
where each element $\phi_{\alpha, \beta, \gamma} \in \Phi$ is a function $\phi_{\alpha, \beta, \gamma} : S \times \reals_{>0} \rightarrow \reals_{\geq 0}$ that acts on a state-rate pair $(s, \lambda)$ by
\begin{equation}
\phi_{\alpha, \beta, \gamma} (s, \lambda) \coloneqq \gamma_s \; \gammad(\lambda ; \alpha_s, \beta_s) .
\end{equation}

This approximation scheme compresses the storage requirements per time $t$ from $n m$ scalar values of the previous approximation scheme down to $3 n$ scalar values. This reduction in parameters comes at the cost of no longer being able to represent multi-modal behaviour of $\lambda \sim P(\lambda | x_t=s_i y_{1:t})$. This approximation retains the ability to represent different noise rates $\lambda$ coupled with different hidden states $s \in S$. We are not directly concerned about storage requirements, but a smaller state space hints at the possibility of reduced computational effort.

\subsection{Approximate forward update for Gamma distribution approximation}

Substituting approximations $\phi(x_t, \lambda_t)$ for the true posterior $p(x_t, \lambda_t \mid y_{1:t})$ into the forward update equation~\ref{eqn:exactfwdupdate} gives:
\begin{align}
& \phi(x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime}) \nonumber \\
\approx &
\frac{1}{P(y_{t+1} \mid y_{1:t})}
\left[
\sum_{s \in S}
A_{s^{\prime}, s}
\sum_{w \in \nonnegint}
B_{k-w,s^{\prime}} \poissond(w ; \lambda^{\prime})
\phi(x_t=s, \lambda_t=\lambda^{\prime})
\right]  \, . \label{eqn:rawapproxupdate}
\end{align}
where $y_{t+1} = k$.
Expanding $\phi(x_t=s, \lambda_t=\lambda^{\prime}) \coloneqq \gamma_{s,t} \gammad(\lambda^{\prime} ; \alpha_{s,t}, \beta_{s,t})$ allows us to use an identity between the Poisson and conjugate-prior Gamma distributions to eliminate the Poisson pmf factor:
\begin{align}
\poissond(w ; \lambda^{\prime}) \phi(x_t=s, \lambda_t=\lambda^{\prime}) & =
\gamma_{s,t} \poissond(w ; \lambda^{\prime}) \gammad(\lambda^{\prime} ; \alpha_{s,t}, \beta_{s,t}) \nonumber \\
& = \gamma_{s,t} \negbind(w ; \alpha_{s,t}, \beta_{s,t}) \gammad(\lambda^{\prime} ; \alpha_{s,t} + w, \beta_{s,t} + 1) \nonumber \\
\end{align}
The identity used above is:
\begin{equation}
\poissond(w ; \lambda) \gammad(\lambda ; \alpha, \beta) =
\negbind(w ; \alpha, \beta)
\gammad(\lambda ; \alpha + w, \beta + 1) \, . \nonumber
\end{equation}
The proof of this identity is elementary given the definitions of the three distributions, see Appendix~\ref{appendix:gammaconj}.

To simplify notation, define the non-negative real-valued tensor $c_{s^{\prime}, s, w, k, t}$ as
\begin{equation}
c_{s^{\prime}, s, w, k, t} \coloneqq A_{s^{\prime}, s} B_{k-w, s^{\prime}} \gamma_{s,t} \negbind(w ; \alpha_{s,t}, \beta_{s,t}) \, .
\end{equation}

The right hand side of the update equation~\ref{eqn:rawapproxupdate} is seen to be a finite mixture of Gamma distributions with non-negative weights:
\begin{align}
& \phi(x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime}) \nonumber \\
\approx &
\frac{1}{P(y_{t+1} \mid y_{1:t})}
\left[
\sum_{s \in S}
\sum_{w \in \nonnegint}
c_{s^{\prime}, s, w, k, t}
\gammad(\lambda^{\prime} ; \alpha_{s,t} + w, \beta_{s,t} + 1)
\right]  \, .
\end{align}
Since the right-hand side is a mixture of Gamma distributions, while the left hand side is but a single scaled Gamma distribution, the error involved in this approximation has the potential to be large.

The plan is to approximate this mixture of Gamma distributions with a single approximating Gamma distribution. Before we can proceed, we note the mixture may not be normalised. We need to factor the mixture into a normalised mixture of Gamma distributions, and a scaling coefficient. To normalise we first drop the factor $P(y_{t+1} \mid y_{1:t})^{-1}$ as it is irrelevant if we only seek to compute relative probabilities. Then, since summing the posterior distribution over $s^{\prime} \in S$ and integrating over $\lambda^{\prime} \in \reals_{>0}$ should give unity, we require a normalisation constant $Z > 0$ such that
\begin{equation}
\frac{1}{Z}
\sum_{s^{\prime} \in S}
\sum_{s \in S}
\sum_{w \in \nonnegint}
c_{s^{\prime}, s, w, k, t}
\int_{\lambda}
\gammad(\lambda^{\prime} ; \alpha_{s,t} + w, \beta_{s,t} + 1)
\,d\lambda = 1 \nonumber
\end{equation}
Each integral of a Gamma distribution evaluates to one, hence we define
\begin{equation}
Z \coloneqq
\sum_{s^{\prime} \in S}
\sum_{s \in S}
\sum_{w \in \nonnegint}
c_{s^{\prime}, s, w, k, t} \, .
\end{equation}
Again simplifying notation, define the non-negative real valued tensor $c^{\prime}_{s^{\prime}, s, w, k, t}$ as
\begin{equation}
c^{\prime}_{s^{\prime}, s, w, k, t} \coloneqq \frac{1}{Z}
c_{s^{\prime}, s, w, k, t} \, .
\end{equation}
Similarly, for each $x_t=s^{\prime}$, we want to recover an approximation for the posterior distribution for $\lambda$ conditioned on $x_{t}=s^{\prime}$, and this distribution should also sum to unity, so we seek additional normalizing constants $Z_{t,s^{\prime}} > 0$ such that
\begin{equation}
\frac{1}{Z_{t,s^{\prime}}}
\sum_{s \in S}
\sum_{w \in \nonnegint}
c^{\prime}_{s^{\prime}, s, w, k, t}
\int_{\lambda}
\gammad(\lambda^{\prime} ; \alpha_{s,t} + w, \beta_{s,t} + 1)
\,d\lambda = 1 \nonumber
\end{equation}
Hence we normalise once more by defining
\begin{equation}
Z_{t,s^{\prime}}
\coloneqq
\sum_{s \in S}
\sum_{w \in \nonnegint}
c^{\prime}_{s^{\prime}, s, w, k, t}
\end{equation}
and defining another non-negative real valued tensor $c^{\prime\prime}_{s^{\prime}, s, w, k, t}$ as
\begin{equation}
c^{\prime\prime}_{s^{\prime}, s, w, k, t} \coloneqq \frac{1}{Z_{t, s^{\prime}}}
c^{\prime}_{s^{\prime}, s, w, k, t} \, .
\end{equation}
Recall our approximation for the posterior at the next timestep $t+1$ is assumed to have the form
\begin{equation}
\phi(x_{t+1}=s^{\prime}, \lambda_{t+1}=\lambda^{\prime}) \coloneqq \gamma_{s^{\prime},t+1} \gammad(\lambda^{\prime} ; \alpha_{s^{\prime},t+1}, \beta_{s^{\prime},t+1})
\end{equation}
Therefore the scaling coefficient must be
\begin{equation}
\gamma_{s^{\prime},t+1} = Z_{t,s^{\prime}} \, .
\end{equation}
The remaining task is to derive expressions for $(\alpha_{s^{\prime},t+1}, \beta_{s^{\prime},t+1})$, which we can achieve by application of the following theorem:

\begin{thm}
Let $p(\lambda)$ be a finite mixture of Gamma distributions,
\begin{equation}
p(\lambda) = \sum_j c_j \gammad(\lambda \mid \theta_j)
\end{equation}
such that $\sum_j c_j = 1$, $c_j \geq 0$ for all $j$, $\theta_j = (\alpha_j, \beta_j)$ with $\alpha_j > 0$, $\beta_j > 0$.
Let $q(\lambda \mid \theta)$ be a Gamma distribution
\begin{equation}
q(\lambda \mid \theta ) = \gammad(\lambda \mid \theta) \, .
\end{equation}
Then the best approximation of $p(\lambda)$ by $q(\lambda \mid \theta)$ in the sense of KL-divergence from $q$ to $p$,
\begin{equation}
\theta^{*} = \argmin_{\theta} \int_{\lambda} p(\lambda) \log \left( \frac{q(\lambda \mid \theta) }{ p(\lambda) } \right) \, d\lambda ,
\end{equation}
is characterised by $\theta^* = (\alpha^*, \beta^*)$ satisfying the constraints:
\begin{alignat}{3}
\E_q \left[ \lambda \right]
& = \frac{\alpha^*}{\beta^*}
&& = \sum_j c_j \left( \frac{\alpha_j}{\beta_j} \right)
&& = \E_p \left[ \lambda \right] \\
\E_q \left[ \log(\lambda) \right]
& = \psi(\alpha^*) - \log(\beta^*)
&& = \sum_j c_j \left( \psi(\alpha_j) - \log(\beta_j) \right)
&& = \E_p \left[ \log(\lambda) \right]
\end{alignat}
where $\psi$ denotes the digamma function.

\end{thm}
\begin{proof}

Define $L(\theta) : \reals_{>0} \times \reals_{>0} \rightarrow \reals_{\geq 0}$ as
\begin{equation}
L(\theta) \coloneqq
\int_{\lambda} p(\lambda) \log \left( \frac{q(\lambda \mid \theta) }{ p(\lambda) } \right) \, d\lambda .
\end{equation}
Provided $L$ is sufficiently smooth, local extreme points of $L$ occur in the interior where the gradient vanishes, so set $\theta^*$ as
\begin{equation}
D_{\theta} L(\theta) \bigr \rvert_{\theta=\theta^*} = \bar{0}
\end{equation}
Provided the argument of the integral is sufficiently smooth, we can commute the gradient inside the integral:
\begin{align}
D_{\theta}
\int_{\lambda} p(\lambda) \log \left( \frac{q(\lambda \mid \theta) }{ p(\lambda) } \right) \, d\lambda
& =
\int_{\lambda}
D_{\theta}
\left[
p(\lambda) \log \left( \frac{q(\lambda \mid \theta) }{ p(\lambda) } \right)
\right] \, d\lambda \\
& =
\int_{\lambda}
p(\lambda)
\frac{p(\lambda)}{q(\lambda \mid \theta)}
\frac{1}{p(\lambda)}
D_{\theta} \left[ q(\lambda \mid \theta) \right]
\, d\lambda \\
& =
\int_{\lambda}
\frac{p(\lambda)}{q(\lambda \mid \theta)}
D_{\theta} \left[ q(\lambda \mid \theta) \right]
\, d\lambda
\end{align}
Hence the vanishing gradient of $L$ at $\theta^*$ can be expressed equivalently as the two constraints
\begin{align}
\int_{\lambda}
\frac{p(\lambda)}{q(\lambda \mid \theta)}
\frac{\partial}{\partial \alpha} \left[ q(\lambda \mid \theta) \right]
\, d\lambda \bigr \vert_{\theta=\theta^*} & = 0 \, , \\
\int_{\lambda}
\frac{p(\lambda)}{q(\lambda \mid \theta)}
\frac{\partial}{\partial \beta} \left[ q(\lambda \mid \theta) \right]
\, d\lambda \bigr \vert_{\theta=\theta^*} & = 0 \, .
\end{align}
The partial derivative of $q(\lambda \mid \theta)$ with respect to $\beta$ is
\begin{align}
\frac{\partial}{\partial \beta} q(\lambda \mid \theta)
= &
\frac{\partial}{\partial \beta} \left( \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta \lambda} \right) \\
= &
(\alpha) \frac{\beta^{\alpha-1}}{\Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta \lambda}
+
(-\lambda) \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta \lambda}
\\
= & \left( \frac{\alpha}{\beta} - \lambda \right) q(\lambda \mid \theta) \, .
\end{align}
Substituting back into the first constraint gives
\begin{equation}
\int_{\lambda}
\frac{p(\lambda)}{q(\lambda \mid \theta)}
\left( \frac{\alpha}{\beta} - \lambda \right) q(\lambda \mid \theta)
\, d\lambda \bigr \vert_{\theta=\theta^*} = 0 \, , \\
\end{equation}
so provided $q(\lambda \mid \theta)$ is nonzero we have
\begin{equation}
\int_{\lambda}
p(\lambda)
\left( \frac{\alpha}{\beta} - \lambda \right)
\, d\lambda \bigr \vert_{\theta=\theta^*} =
\int_{\lambda}
p(\lambda)
\left( \frac{\alpha^{*}}{\beta^{*}} - \lambda \right)
\, d\lambda =
0 \, ,
\end{equation}
which, combined with the property that expected value of any Gamma distribution $\gammad(\lambda \mid \alpha, \beta)$ is $\frac{\alpha}{\beta}$, gives the first characterising constraint upon $\theta^{*}$.

For the second characterising constraint, the partial derivative of $q(\lambda \mid \theta)$ with respect to $\alpha$ is
\begin{align}
\frac{\partial}{\partial \alpha} q(\lambda \mid \theta)
= &
\frac{\partial}{\partial \alpha} \left( \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta \lambda} \right) \\
= & \left( \frac{-\Gamma^{\prime}(\alpha)}{\Gamma(\alpha)} + \log(\lambda) + \log(\beta) \right) q(\lambda \mid \theta) \, .
\end{align}
Substituting back into the second constraint gives
\begin{equation}
\int_{\lambda}
\frac{p(\lambda)}{q(\lambda \mid \theta)}
\left( \frac{-\Gamma^{\prime}(\alpha)}{\Gamma(\alpha)} + \log(\lambda) + \log(\beta) \right) q(\lambda \mid \theta)
\, d\lambda \bigr \vert_{\theta=\theta^*} = 0 \, , \\
\end{equation}
so again, provided $q(\lambda \mid \theta)$ is nonzero,
\begin{equation}
\int_{\lambda}
p(\lambda)
\left( \frac{-\Gamma^{\prime}(\alpha)}{\Gamma(\alpha)} + \log(\lambda) + \log(\beta) \right)
\, d\lambda \bigr \vert_{\theta=\theta^*} = 0 \, , \\
\end{equation}
which rearranges to
\begin{equation}
\int_{\lambda}
p(\lambda) \log(\lambda) \, d\lambda
=
\int_{\lambda}
p(\lambda)
\left( \frac{\Gamma^{\prime}(\alpha^*)}{\Gamma(\alpha^*)} - \log(\beta^*) \right)
\, d\lambda
\end{equation}
where the fraction $\frac{\Gamma^{\prime}(\alpha^*)}{\Gamma(\alpha^*)} = \psi(\alpha^*)$ is recognised as the digamma aka 0-th order polygamma function. Finally, we use the following standard identity for the expected value of $\log(\lambda)$ for a Gamma distribution,
\begin{equation}
\int_{\lambda}
\gammad(\lambda ; \alpha, \beta ) \log(\lambda) \, d\lambda = \psi(\alpha) - \log(\beta) \; .
\end{equation}
This gives the second characterising constraint upon $\theta^*$ in the statement of the theorem.

Apart from a lot of loose ends to check edge cases and give a convincing argument for why the alleged extreme point is a local minima, and in fact the global minimum, this completes a sketch of the proof.

\end{proof}

\begin{thm}
Let $c_j \geq 0, \alpha_j > 0, \beta_j > 0$ be real numbers where $j = 1, \ldots n$ for some integer $n \geq 1$. Then there exist unique real numbers $\alpha^{*} > 0, \beta^{*} > 0$ satisfying the constraints
\begin{align}
\frac{\alpha^*}{\beta^*}
& = \sum_j c_j \left( \frac{\alpha_j}{\beta_j} \right) \\
\psi(\alpha^*) - \log(\beta^*)
& = \sum_j c_j \left( \psi(\alpha_j) - \log(\beta_j) \right)
\end{align}
where $\psi$ denotes the digamma function.
\end{thm}
\begin{proof}
Reduce the problem to a single equation for $\alpha^*$ by taking the logarithm of the first constraint and eliminating the $\log(\beta^*)$ term in the second constraint, to give:
\begin{equation}
\psi(\alpha^*) - \log(\alpha^*) + \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right) = \sum_j c_j \left( \psi(\alpha_j) - \log(\beta_j) \right) \, . \nonumber
\end{equation}
Define $f : (0, \infty) \rightarrow \real$ by $f(\alpha) = \log(\alpha) - \psi(\alpha)$. From inequality 2.2 of~\citet*{alzer1997some}, we have
\begin{equation}
\frac{1}{2\alpha} < f(\alpha) < \frac{1}{\alpha} \qquad (\alpha>0)\, . \label{eqn:fbounds}
\end{equation}
Using Theorem 1 from~\citet*{alzer1997some}, we have that $f$ is strictly completely monotonic on $(0, \infty)$. In particular $f$ is invertible on $(0, \infty)$.

In terms of $f$ the equation for $\alpha^*$ is equivalently expressed as
\begin{equation}
f(\alpha^*) =
\sum_j c_j \left( \log(\beta_j) - \psi(\alpha_j) \right)
+ \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right) \, \label{eqn:fconstraint}
\end{equation}
In order to define $\alpha^*$ using $f^{-1}$, we need to demonstrate that that the right hand side of equation~\ref{eqn:fconstraint} is strictly positive so it falls inside the domain of $f^{-1}$. As the logarithm is concave on $(0, \infty)$, by Jensen's inequality we have the bound
\begin{equation}
\log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right)
\geq
\sum_j c_j \left( \log\left(\frac{\alpha_j}{\beta_j}\right) \right) \, . \label{eqn:logjensenbound}
\end{equation}

Combining the bounds~\ref{eqn:fbounds},~\ref{eqn:logjensenbound} and the definitions of the $c_j$ and $\alpha_j$ shows that
\begin{align}
& \sum_j c_j \left( \log(\beta_j) - \psi(\alpha_j) \right)
+ \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right) \nonumber \\
= &
\sum_j c_j \left( \log\left(\frac{\beta_j}{\alpha_j}\right) + \log(\alpha_j) - \psi(\alpha_j) \right)
+ \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right) \nonumber \\
> &
\sum_j c_j \left( \log\left(\frac{\beta_j}{\alpha_j}\right) + \frac{1}{2\alpha_j} \right)
+ \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right) \nonumber \\
= &
\sum_j \frac{c_j}{2 \alpha_j} - \sum_j c_j \left( \log\left(\frac{\alpha_j}{\beta_j}\right) \right)
+ \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right) \nonumber \\
\geq &
\sum_j \frac{c_j}{2 \alpha_j}
> 0 \; . \nonumber
\end{align}
Hence there exists a unique $\alpha^* > 0$ such that
\begin{equation}
\alpha^* = f^{-1} \left(
\sum_j c_j \left( \log(\beta_j) - \psi(\alpha_j) \right)
+ \log\left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right)
\right) \, . \nonumber
\end{equation}
Finally, substituting $\alpha^*$ into the first constraint from the statement of the theorem shows that $\beta^*$ must take the unique value
\begin{equation}
\beta^{*} = \alpha^{*} \left( \sum_j c_j \left(\frac{\alpha_j}{\beta_j}\right) \right)^{-1}
\end{equation}
which, using the definitions of $\alpha^*$, $\alpha_j$, $\beta_j$ and $c_j$, we conclude $\beta^* > 0$.
\end{proof}

\subsection{Numerical Approximation}

In practice, in order to compute the parameters $(\alpha^*, \beta^*)$ of a Gamma distribution that best approximates a mixture of Gamma distributions, we need to compute the inverse of $f(\alpha) = \log(\alpha) - \psi(\alpha) = y$ for some $y > 0$.

Minka Appendix C~\citep{minka2000estimating} describes how to approximately invert the digamma function $\psi$ using a good initial guess followed by five iterations of the Newton-Raphson method. We tried a similar scheme to numerically invert $f$ using the Newton-Raphson method, but encountered some difficulties. During each iteration of Newton-Raphson we update the current approximation $x_n$ of the root $x$ of $f(x) - y = 0$ with a new approximation $x_{n+1}$ defined by
\begin{equation}
x_{n+1} = x_n - \frac{f(x_n)}{f^{\prime}(x_n)}
\end{equation}

However, when $x_n > 0$ is close to zero, due to the convexity of $f$, the Newton-Raphson iteration can sometimes produce a new value of $x_{n+1} < 0$, on the other side of the singularity of $f(x)$ at $x=0$, which causes iterations to diverge. One ad-hoc way to mitigate this problem is to check if a proposed $x_{n+1}$ is negative, and if so, replace it with a strictly positive value such as $x_n / 2$.

A more satisfactory method is to replace the use of Newton-Raphson with Halley's method, where the iterations are defined by:
\begin{equation}
x_{n+1}
=
x_n - \frac{
2 f(x_n) f^{\prime}(x_n)
}{
2 \left( f^{\prime}(x_n) \right)^2 - f(x_n) f^{\prime \prime}(x_n)} \nonumber
\end{equation}
In practice, empirically we find two iterations of Halley's method suffices for rough accuracy, using an initial guess of $x_0 \approx 0.6548 y^{-0.9256}$. When evaluating $f$ and its two derivatives $f^{\prime}$ and $f^{\prime\prime}$, the $\psi$, $\psi^{\prime}$ and $\psi^{\prime\prime}$ terms may be evaluated approximately using a series approximation for $\psi$. Algorithm AS 103~\citet{bernardo1976algorithm} can be used to approximate $\psi$. We based our approximation of $\psi$ on the code given by Johnson~\citep{JohnsonDigammaC}.

\subsection{todo}

\begin{enumerate}
\item restate exact update equation in more abstract form (finite linear combination of unnormalised phi functions)
\item comment that our approximation space may not be closed under this operation (done)
\item propose how to repair the situation by projecting back into the approximation space (done)
\item propose minimising the KL divergence from the finite mixture to the element of the approximation space (done)
\item sketch how we can minimise by taking the derivative and setting it to the zero vector (done)
\item plug in our definition of the approximation scheme from the previous subsection (done)
\item compute some partial derivatives (done)
\item do a little algebra and derive some pretty constraints that characterise a best approximation (done, roughly)
\item do a little more algebra to define explicit update equations for our approximation forward update
\item comment about computational cost
\item comment about approximation error
\end{enumerate}

\appendix

\section{Gamma distribution identities}
\label{appendix:gammaconj}

The Poisson distribution and its conjugate prior Gamma distribution produce a closed-form expression when multiplied together. This follows immediately from unpacking the definitions, rearranging some factors, and inserting suitable forms of the multiplicative identity:
\begin{align}
& P(k_t = k \mid \lambda) P(\lambda \mid \alpha, \beta)
\nonumber \\
= &
\poissond( k ; \lambda) \gammad( \lambda ; \alpha, \beta)
\nonumber \\
= &
\frac{\lambda^k e^{-\lambda}}{k!}
\frac{\beta^{\alpha}}{\Gamma(\alpha)}
\lambda^{\alpha-1} e^{-\beta \lambda}
\lambda \nonumber \\
= &
\frac{1}{k!}
\frac{\beta^{\alpha}}{\Gamma(\alpha)}
\lambda^{(\alpha+k)-1} e^{-(\beta+1) \lambda}
\nonumber \\
= &
\frac{\Gamma(\alpha+k)}{\Gamma(\alpha) k!}
\frac{\beta^{\alpha}}{(\beta+1)^{\alpha+k}}
\frac{(\beta+1)^{\alpha+k}}{\Gamma(\alpha+k)}
\lambda^{(\alpha+k)-1} e^{-(\beta+1) \lambda}
\nonumber \\
= &
\frac{\Gamma(\alpha+k)}{\Gamma(\alpha) k!}
\frac{\beta^{\alpha}}{(\beta+1)^{\alpha+k}}
\gammad(\lambda ; \alpha+k, \beta+1)
\nonumber \\
= &
\binom{\alpha+k-1}{k} \left( \frac{\beta}{\beta+1} \right)^{\alpha} \left( \frac{1}{\beta+1} \right)^{k}
\gammad(\lambda ; \alpha+k, \beta+1)
\nonumber \\
= &
\negbind(k ; \alpha, \beta)
\gammad(\lambda ; \alpha+k, \beta+1) \, .
\nonumber
\end{align}
The penultimate factor in front of the Gamma distribution is known~\citep{gelman2013bayesian} to be the negative binomial density $k \sim \negbind\left(\alpha, \beta\right)$. Note that the upper parameter of the binomial coefficient may be negative when $k=0$ and $\alpha < 1$ -- one way to interpret this is to consider the definition $\binom{n}{k} = n! (k! (n-k)!)^{-1}$ over non-negative integers $n$ and $k$, recall that the factorial is extended over complex numbers by $\Gamma$, where at integers $n$ $\Gamma(n) = (n-1)!$, then extend $\binom{n}{k}$ by redefining it as $\binom{n}{k} = \Gamma(n+1) (\Gamma(k+1) \Gamma(n-k+1))^{-1}$ .

This identity can also be used to give a closed-form expression for the integral:
\begin{align}
& \int_{\lambda}
P(k_t = k \mid \lambda) P(\lambda \mid \alpha, \beta)
\, d \lambda \nonumber \\
= & \int_{\lambda}
\poissond( k ; \lambda) \gammad( \lambda ; \alpha, \beta)
\, d \lambda \nonumber \\
= & \negbind(k ; \alpha, \beta) \int_{\lambda}
\gammad(\lambda ; \alpha+k, \beta+1)
\, d \lambda \nonumber \\
= & \negbind(k ; \alpha, \beta) \nonumber
\end{align}
where $\gammad(\lambda ; \alpha+k, \beta+1)$ is a distribution over $\lambda$ and hence integrates to unity.

\bibliography{note}

\end{document}


%===============================================================================
\section{Traditional beam models}

%===============================================================================
\subsection{Euler-Bernoulli beam theory}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{The Euler-Bernoulli beam (planar case)}

  \begin{itemize}
    \item assumption of the model
      \begin{itemize}
        \item all cross sections remain planar
        \item cross section normal and centerline tangent are of same orientation $\varphi$ \newline
          \null \quad $\rightarrow$ model only allows for pure bending (no shear)
      \end{itemize}
    \item bending moment $M(s) = E \, I \, \kappa(s)$
    \item centerline of beam described by function $y(s)$
    \item curvature of centerline given by $\kappa(s) = \frac{\od[2]{y}{s}(s)}{\biggl( 1 + \bigl( \od{y}{s}(s) \bigr)^2 \biggr)^{\frac{3}{2}}}$ \newline
      \null \quad $\rightarrow$ geometric nonlinearity $\rightarrow$ non-linear ODE
    \item linear approximation for small deflections: $\od{y}{s} = \tan(\varphi) \ll 1 \Rightarrow \od{y}{s} \approx \varphi$
    \item Euler-Bernoulli beam equation: $M(s) = E \, I \, \od[2]{y}{s}$
  \end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Example: Cantilever beam (part 1)}

  \begin{figure}
    \centering
    \includegraphics[width=16cm, keepaspectratio=true]{sections/traditional_beams/images/EulerCanitleverExample1part1}
  \end{figure}

  \begin{tabularx}{\linewidth}{XX}
    {
      boundary conditions at $s=0$:
      \begin{itemize}
        \item $y(0) = 0$
        \item $\od{y}{s}(0) = 0$
      \end{itemize}
    } & {
      boundary conditions at $s=L$:
      \begin{itemize}
        \item $v(L) = P$
        \item $M(L) = 0 \Rightarrow \od[2]{y}{s}(L) = 0$
      \end{itemize}
    }
  \end{tabularx}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Example: Cantilever beam (part 2)}

  \begin{figure}
    \centering
    \includegraphics[width=14cm, keepaspectratio=true]{sections/traditional_beams/images/EulerCanitleverExample1part2}
  \end{figure}
  \vspace{-1.4em}

  \begin{displaymath}
  \text{moment balance:} \quad  -M(s) + P(L-s) = 0 \quad \Rightarrow \quad M(s) = P \, (L-s)
  \end{displaymath}

  \begin{displaymath}
    M(x) = E \, I \, \od[2]{y}{s} \quad \Rightarrow \quad \od[2]{y}{s} = \frac{M(s)}{E \, I}  = \frac{P \, (L-s)}{E \, I}
  \end{displaymath}

  \begin{displaymath}
    \Rightarrow y(s) = \frac{P}{E \, I} \, \biggl( - \frac{s^3}{6} + \frac{L \, s^2}{2} \biggr) + \cancel{c_1} \,s + \cancel{c_2} = \frac{P \, L^3}{2 \, E \, I} \Biggl( \biggl( \frac{s}{L} \biggr)^2 - \frac{1}{3} \biggl( \frac{s}{L} \biggr)^3 \Biggr)
  \end{displaymath}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Another example of a cantilever beam}

  \begin{figure}
    \centering
    \includegraphics[width=16cm, keepaspectratio=true]{sections/traditional_beams/images/EulerCanitleverExample2}
  \end{figure}

  \begin{tabularx}{\linewidth}{XX}
    {
      boundary conditions at $s=0$:
      \begin{itemize}
        \item $y(0) = 0$
        \item $\od{y}{s}(0) = 0$
      \end{itemize}
    } & {
      boundary conditions at $s=L$:
      \begin{itemize}
        \item $v(L) = P$
        \item $\od{y}{s}(L) = 0 \Rightarrow M(L) \neq 0$ \newline
          \null \quad (rotation is restricted)
      \end{itemize}
    }
  \end{tabularx}

  \begin{displaymath}
    \Rightarrow M(s) = M(L) + P \, (L-s)
  \end{displaymath}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Boundary conditions}

  \vspace{4em}
  \begin{center}
    \begin{tabular}{lr}
      displacement & force \\
      \hline
      prescribed $\Rightarrow$ & unknown \\
      unknown & $\Leftarrow$ prescribed \\
      \hline \\
      \\
      rotation & moment \\
      \hline
      prescribed $\Rightarrow$ & unknown \\
      unknown & $\Leftarrow$ prescribed \\
      \hline
    \end{tabular}
  \end{center}
\end{frame}

%===============================================================================
\subsection{Timoshenko beam theory}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{The Timoshenko beam (planar case)}

  \vspace{-1em}
  \begin{figure}
    \centering
    \includegraphics[width=16cm, keepaspectratio=true]{sections/traditional_beams/images/TimoshenkoBeam1}
  \end{figure}

  \begin{itemize}
    \item individual cross sections still remain planar
    \item centerline tangent orientation (red) $\neq$ cross section normal (green)
      \begin{itemize}
        \item $-\Theta$ measures orientation of cross section
        \item centerline tangent $\od{y}{s}$ influenced by \textbf{bending and shearing}
      \end{itemize}
    \item shear strain $\gamma(s) = \od{y}{s}(s) - \Theta(s) = \frac{v(s)}{k \, G \, A}$
    \item curvature $\kappa(s) = \od{\Theta}{s}(s) = \frac{M(s)}{E \, I}$ (see next slide)
  \end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Connection between curvature and moment}

  \begin{multicols}{2}
    \noindent

    \begin{figure}
    \centering
      \includegraphics[width=13cm, keepaspectratio=true]{sections/traditional_beams/images/TimoshenkoBeam2}
    \end{figure}

    \begin{displaymath}
      L = R \, \Theta \quad \Rightarrow \quad \frac{1}{R} = \frac{\Theta}{L} = \od{\Theta}{s}
    \end{displaymath}

    \vspace{0.5em}
    fitting circles to the centerline locally:
    \begin{displaymath}
      \kappa(s) = \od{\Theta}{s}(s)
    \end{displaymath}

    \begin{displaymath}
      \epsilon_{xx} = \frac{\Delta L}{L} = \frac{(R-y) \, \Theta - R \, \Theta}{R \, \Theta} = \frac{-y}{R}
    \end{displaymath}

    \begin{displaymath}
      \sigma_{xx} = -E \, \frac{y}{R}
    \end{displaymath}

    \begin{displaymath}
      M = - \int_{\Omega} y \, \sigma_{xx} \, \dif A = E \, I \, \frac{1}{R} = E \, I \, \kappa
    \end{displaymath}
  \end{multicols}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Bending and axial compression together}

  \begin{multicols}{2}
    \noindent

    \begin{figure}
    \centering
      \includegraphics[width=13cm, keepaspectratio=true]{sections/traditional_beams/images/TimoshenkoBeam3}
    \end{figure}

    because of axial forces the beam got shorter!
    \begin{displaymath}
      \begin{alignedat}{1}
        \epsilon_{xx} &= \frac{(r-y) \, \Theta - R \, \Theta}{R \, \Theta} = \frac{r-R}{R} + \frac{-y}{R} \\
        &= \epsilon - \frac{y}{R}
      \end{alignedat}
    \end{displaymath}

    \vspace{0.2em}
    stress due to axial stretch + bending:
    \begin{displaymath}
      \sigma_{xx} = E \, \bigl( \epsilon + \frac{-y}{R} \bigr)
    \end{displaymath}

    \vspace{-0.7em}
    \begin{displaymath}
      \begin{alignedat}{1}
        M &= - \int_{\Omega} y \, \sigma_{xx} \, \dif A \\
          &= - E \, \epsilon \cancel{\int_{\Omega} y \, \dif A} + \frac{E}{R} \int_{\Omega} y^2 \dif A \\
          &= E \, I \, \kappa
      \end{alignedat}
    \end{displaymath}

    $\rightarrow$ bending and axial stretch are independent!
  \end{multicols}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Model linearity}

  \begin{displaymath}
    \begin{alignedat}{1}
      \od{y}{s} &= \frac{1}{k \, G \, A} \cdot v(s) + \Theta  \\ \\
      \od{\Theta}{s} &= \frac{1}{E \, I} \cdot M(s)
    \end{alignedat}
  \end{displaymath}

  \vspace{1em}
  we obtained a linear model because we assumed...
  \begin{itemize}
    \item small slopes of the centerline (linear kinematics)
      \begin{itemize}
        \item $\od{y}{s} = \tan{\Theta} \approx \Theta$
        \item approximation is not valid for larger deformations
      \end{itemize}
    \item material linearity (linear constitutive law)
      \begin{itemize}
        \item $v(s)$, $M(s)$ get multiplied by constants \newline
          or by functions that depend on $s$ but not on $v(s)$, $M(s)$
        \item material may eg. get stiffer with increasing deformation energy
        \item btw. $k$ is a correction factor, that depends on the shape of the beam's cross section
      \end{itemize}
  \end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Example: Cantilever beam (part 1)}

  \begin{figure}
    \centering
    \includegraphics[width=16cm, keepaspectratio=true]{sections/traditional_beams/images/EulerCanitleverExample1part1}
  \end{figure}

  \begin{tabularx}{\linewidth}{XX}
    {
      boundary conditions at $s=0$:
      \begin{itemize}
        \item $y(0) = 0$
        \item $\xcancel{\od{y}{s}(0) = 0}$ (because of shear!)
        \item $\Theta(0) = 0$ (cross section fixed)
      \end{itemize}
    } & {
      boundary conditions at $s=L$:
      \begin{itemize}
        \item $v(L) = P$
        \item $M(L) = 0$
      \end{itemize}
    }
  \end{tabularx}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Example: Cantilever beam (part 2)}

  As before with the Euler-Bernoulli cantilever beam we have
  \begin{displaymath}
    v(s) = P \quad \text{ and } \quad M(s) = P \, (L-s)
  \end{displaymath}

  \vspace{1em}
  We integrate the Timoshenko beam equations ...
  \vspace{0.3em}
  \begin{displaymath}
    \od{\Theta}{s}(s) = \frac{M(s)}{E \, I} = \frac{P \, (L-s)}{E \, I}
    \quad \Rightarrow \quad
    \Theta(s) = \frac{P}{E \, I} \biggl( L \, s - \frac{s^2}{2} \biggr) + \cancel{c}
  \end{displaymath}

  \vspace{1em}

  \begin{displaymath}
    \od{y}{s}(s) = \frac{v(s)}{k \, G \, A} + \Theta(s) = \frac{P}{k \, G \, A} + \frac{P}{E \, I} \biggl( L \, s - \frac{s^2}{2} \biggr)
  \end{displaymath}

  \begin{displaymath}
    \Rightarrow y(s) = \frac{P \, s}{k \, G \, A} + \frac{P}{E \, I} \biggl( L \frac{s^2}{2} - \frac{s^3}{6} \biggr) + \cancel{c} = \frac{P \, s}{k \, G \, A} + \frac{P \, L^3}{2 \, E \, I} \Biggl(
      \biggl( \frac{s}{L} \biggr)^2 - \frac{1}{3} \biggl( \frac{s}{L} \biggr)^3 \Biggr)
  \end{displaymath}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Comparison of Euler-Bernoulli and Timoshenko cantilever beams}

  \textbf{Euler-Bernoulli beam}
  \begin{displaymath}
    y^{(E)}(s) = \frac{P \, L^3}{2 \, E \, I} \Biggl( \biggl( \frac{s}{L} \biggr)^2 - \frac{1}{3} \biggl( \frac{s}{L} \biggr)^3 \Biggr)
    \Rightarrow y^{(E)}(L) = \frac{P \, L^3}{3 \, E \, I}
  \end{displaymath}

  \textbf{Timoshenko beam}
  \begin{displaymath}
    y^{(T)}(s) = \frac{P \, s}{k \, G \, A} + \frac{P \, L^3}{2 \, E \, I} \Biggl(
      \biggl( \frac{s}{L} \biggr)^2 - \frac{1}{3} \biggl( \frac{s}{L} \biggr)^3 \Biggr)
      \Rightarrow y^{(T)}(L) = \frac{P \, L}{k \, G \, A} + \frac{P \, L^3}{3 \, E \, I}
  \end{displaymath}

  \textbf{relative error} (taking Euler-Bernoulli as the reference)
  \begin{displaymath}
    \begin{alignedat}{1}
      \epsilon_r &= \frac{y^{(T)}(L) - y^{(E)}(L)}{y^{(E)}(L)} = \frac{\frac{P \, L}{k \, G \, A}}{\frac{P \, L^3}{3 \, E \, I}} = \frac{3 \, E \, I}{k \, G \, A \, L^2} \\
      &= \frac{3 \, E \, b \, h^3}{k \, G \, b \, h \, L^2 \, 12} = \frac{1}{4} \frac{E}{k \, G} \biggl( \frac{h}{L} \biggr)^2
    \end{alignedat}
  \end{displaymath}

  \begin{center}
    $\rightarrow$ Euler-Bernoulli beam model performs equally good as the Timoshenko model \newline when the beam is very slender ($h/L$ small)
  \end{center}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Another example of a cantilever beam (part 1)}

  \vspace{-0.7em}
  \begin{figure}
    \centering
    \includegraphics[width=16cm, keepaspectratio=true]{sections/traditional_beams/images/TimoshenkoCanitleverExample2part1}
  \end{figure}
  \vspace{-0.5em}
  cross section at $s=L$ can not rotate or displace horizontally
  \vspace{1em}
  \begin{tabularx}{\linewidth}{XX}
    {
      boundary conditions at $s=0$:
      \begin{itemize}
        \item $y(0) = 0$
        \item $\Theta(0) = 0$
      \end{itemize}
    } & {
      boundary conditions at $s=L$:
      \begin{itemize}
        \item $v(L) = P$
        \item $\Theta(L) = 0$
      \end{itemize}
    }
  \end{tabularx}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Another example of a cantilever beam (part 2)}

  \vspace{-0.7em}
  \begin{figure}
    \centering
    \includegraphics[width=20cm, keepaspectratio=true]{sections/traditional_beams/images/TimoshenkoCanitleverExample2part2}
  \end{figure}

  moment balance (in the deformed configuration):
  \begin{displaymath}
    -M(s) + M(L) + P \, (L-s) - N(y(L)-y(s))
  \end{displaymath}
  $\rightarrow$ the last term results in nonlinear behavior of the model \newline
  $\rightarrow$ $M(L)$ and $N(L)$ are two extra unknowns
\end{frame}


\input{../header_class}

%---------- start document ---------- %
 \section{factor.util -- utilities for factorization}\linkedzero{factor.util}
 \begin{itemize}
   \item {\bf Classes}
   \begin{itemize}
     \item \linkingone{factor.util}{FactoringInteger}
     \item \linkingone{factor.util}{FactoringMethod}
   \end{itemize}
 \end{itemize}

 This module uses following type:
 \begin{description}
   \item[factorlist]\linkedone{factor.util}{factorlist}:\\
     \param{factorlist} is a list which consists of pairs {\tt (base, index)}.
     Each pair means \(base^{index}\).
     The product of those terms expresses whole prime factorization.
 \end{description}

\C

 \subsection{FactoringInteger -- keeping track of factorization}\linkedone{factor.util}{FactoringInteger}
 \initialize
  \func{FactoringInteger}{\hiki{number}{integer}}{\out{FactoringInteger}}\\
  \spacing
  % document of basic document
  \quad This is the base class for factoring integers.\\
  \spacing
  % added document
  \quad  \param{number} is stored in the attribute \linkingtwo{factor.util}{FactoringInteger}{number}. The factors will be stored in the attribute \linkingtwo{factor.util}{FactoringInteger}{factors}, and primality of factors will be tracked in the attribute \linkingtwo{factor.util}{FactoringInteger}{primality}.\\
  \spacing
  % input/output document
  \quad The given \param{number} must be a composite number.\\
  \begin{at}
    \item[number]\linkedtwo{factor.util}{FactoringInteger}{number}:\\ The composite number.
    \item[factors]\linkedtwo{factor.util}{FactoringInteger}{factors}:\\ Factors known at the time being referred.
    \item[primality]\linkedtwo{factor.util}{FactoringInteger}{primality}:\\ A dictionary of primality information of known factors.
      {\tt True} if the factor is prime, {\tt False} composite, or {\tt None} undetermined.
  \end{at}
%   \begin{op}
%     \verb+A==B+ & Return whether M and N are equal or not.\\
%   \end{op}
% \begin{ex}
% >>> A = factor.util.FactoringInteger((1,2))
% >>> print A
% (1, 2)
% >>> A.point
% (1, 2)
% >>>
% \end{ex}%Don't indent!
  \method
  \subsubsection{getNextTarget -- next target}\linkedtwo{factor.util}{FactoringInteger}{getNextTarget}
   \func{getNextTarget}{\param{self},\ \hikiopt{cond}{function}{{\tt None}}}{\out{integer}}\\
   \spacing
   % document of basic document
   \quad Return the next target which meets \param{cond}.\\
   \spacing
   % added document
   If \param{cond} is not specified, then the next target is a composite (or undetermined) factor of \linkingtwo{factor.util}{FactoringInteger}{number}.\\
   \spacing
   % input, output document
   \quad \param{cond} should be a binary predicate whose arguments are base and index.\\
   \quad If there is no target factor, \linklibraryone{exceptions\#exceptions.LookupError}{LookupError} will be raised.\\
%
 \subsubsection{getResult -- result of factorization}\linkedtwo{factor.util}{FactoringInteger}{getResult}
   \func{getResult}{\param{self}}{\out{\linkingone{factor.util}{factors}}}\\
   \spacing
   % document of basic document
   \quad Return the currently known factorization of the \linkingtwo{factor.util}{FactoringInteger}{number}.\\
%
 \subsubsection{register -- register a new factor}\linkedtwo{factor.util}{FactoringInteger}{register}
   \func{register}{\param{self},\ \hiki{divisor}{integer},\ \hikiopt{isprime}{bool}{{\tt None}}}{}\\
   \spacing
   Register a \param{divisor} of the \linkingtwo{factor.util}{FactoringInteger}{number} if the \param{divisor} is a true divisor of the number.\\
   \spacing
   %added document
   \quad The number is divided by the \param{divisor} as many times as possible.\\
   \spacing
   % input/output document
   The optional argument \param{isprime} tells the primality of the
   \param{divisor} (default to undetermined).\\
%
 \subsubsection{sortFactors -- sort factors}\linkedtwo{factor.util}{FactoringInteger}{sortFactors}
   \func{sortFactors}{\param{self}}{}
   \spacing
   \quad Sort factors list.\\
   \spacing
    % added document
    \quad This affects the result of \linkingtwo{factor.util}{FactoringInteger}{getResult}.\\
%
\begin{ex}
>>> A = factor.util.FactoringInteger(100)
>>> A.getNextTarget()
100
>>> A.getResult()
[(100, 1)]
>>> A.register(5, True)
>>> A.getResult()
[(5, 2), (4, 1)]
>>> A.sortFactors()
>>> A.getResult()
[(4, 1), (5, 2)
>>> A.primality
{4: None, 5: True}
>>> A.getNextTarget()
4
\end{ex}%Don't indent!
\C

 \subsection{FactoringMethod -- method of factorization}\linkedone{factor.util}{FactoringMethod}
 \initialize
  \func{FactoringMethod}{}{\out{FactoringMethod}}\\
  \spacing
  % document of basic document
  \quad Base class of factoring methods.\\
  \spacing
  % added document
  \quad All methods defined in \linkingzero{factor.methods} are
  implemented as derived classes of this class. The method which users may call is  \linkingtwo{factor.util}{FactoringMethod}{factor} only.
  Other methods are explained for future implementers of a new factoring method.\\
  \method
  \subsubsection{factor -- do factorization}\linkedtwo{factor.util}{FactoringMethod}{factor}
   \func{factor}{\param{self},\ %
     \hiki{number}{integer},\ %
     \hikiopt{return\_type}{str}{'list'},\ %
     \hikiopt{need\_sort}{bool}{False}
   }{\out{\linkingone{factor.util}{factorlist}}}\\
   \spacing
   % document of basic document
   \quad Return the factorization of the given positive integer \param{number}.
   \spacing
   % input, output document
   \quad The default returned type is a \linkingone{factor.util}{factorlist}.\\
   \quad A keyword option \param{return\_type} can be as the following:
   \begin{enumerate}
   \item {\tt 'list'} for default type (\linkingone{factor.util}{factorlist}).
   \item {\tt 'tracker'} for \linkingone{factor.util}{FactoringInteger}.
   \end{enumerate}
   \quad Another keyword option \param{need\_sort} is Boolean:
   {\tt True} to sort the result.
   This should be specified with {\tt return\_type='list'}.\\
%
  \subsubsection{\negok continue\_factor -- continue factorization}\linkedtwo{factor.util}{FactoringMethod}{continue\_factor}
  \func{continue\_factor}{\param{self},\ %
    \hiki{tracker}{\linkingone{factor.util}{FactoringInteger}},\ %
    \hikiopt{return\_type}{str}{'tracker'},\ %
    \hikiopt{primeq}{func}{\linkingone{prime}{primeq}}
  }{\out{\linkingone{factor.util}{FactoringInteger}}}\\
  \spacing
  \quad Continue factoring of the given \param{tracker} and return the
  result of factorization.\\
  \spacing
  \quad The default returned type is \linkingone{factor.util}{FactoringInteger},
  but if \param{return\_type} is specified as {\tt 'list'}
  then it returns \linkingone{factor.util}{factorlist}.
  The primality is judged by a function specified in \param{primeq}
  optional keyword argument, which default is \linkingone{prime}{primeq}.\\
%
  \subsubsection{\negok find -- find a factor}\linkedtwo{factor.util}{FactoringMethod}{find}
  \func{find}{\param{self},\ %
    \hiki{target}{integer},\ %
    **\param{options}
  }{\out{integer}}\\
  \spacing
  \quad Find a factor from the \param{target} number.\\
  \spacing
  \quad This method has to be overridden, or \linkingtwo{factor.util}{FactoringMethod}{factor} method should be overridden not to call this method.\\
%
  \subsubsection{\negok generate -- generate prime factors}\linkedtwo{factor.util}{FactoringMethod}{generate}
  \func{generate}{\param{self},\ %
    \hiki{target}{integer},\ %
    **\param{options}
  }{\out{integer}}\\
  \spacing
 % basic document
  \quad Generate prime factors of the \param{target} number with their valuations.\\
 \spacing
 % added document
  \quad The method may terminate with yielding {\tt (1, 1)}
  to indicate the factorization is incomplete.\\
  This method has to be overridden, or \linkingtwo{factor.util}{FactoringMethod}{factor} method should be overridden not to call this method.\\
\C

%---------- end document ---------- %

\input{../footer}