text:
Q: When is it valid to pass a pointer out of a function? I'm wrestling with the minutiae of variable lifetimes in C (of course)!
I've learnt that I shouldn't leave variables that were defined within functions floating around, expect when the memory for that variable has been allocated with malloc. What I can't work out is how I know when a pointer is good to pass out of a function.
In the following example, there is a fundamental difference between fptr and p_uniqueId:
int initialize_db(ToyDatabase* p_tdb, const char* filePath) {

    FILE* fptr;
    if ((fptr = fopen(filePath, "wb")) == NULL) {
        printf("Error! opening file");
        return -1;
    }
    int uniqueId = 2;
    int* p_uniqueId = &uniqueId;
    p_tdb->pFile = fptr;  //Presumably not a dangling pointer, how to tell?
    p_tdb->pId = p_uniqueId; //Will become a dangling pointer, not Ok!
    return 0;
}

int cleanup(ToyDatabase* p_tdb)
{
    fclose(p_tdb->fptr);
    //Do I need to do anything else with this fptr? Maybe p_tdb->fptr = NULL;????

    return 0;
}

In the example, the address pointed to by p_tdb->pId won't be valid outside the initialize_db function. But presumably p_tdb->pFile will be valid, presumably because fopen allocated memory for fptr.
Is there something in the documentation for fopen that I'm looking for that delineates these two cases, or is it just assumed that any function returning a pointer has been mallocd?

A: First, p_uniqueId unambiguously points to uniqueId which has automatic storage class. If it were a global, malloc'd, static, etc then you'd be fine.

Is there something in the documentation for fopen that I'm looking for that delineates these two cases, or is it just assumed that any function returning a pointer has been mallocd?

The manpage doesn't say much. It just says that a FILE* is returned, which you can safely use and pass around. It might not have been necessarily malloc'd, as it could have been allocated in a static buffer or other implementation-dependent storage. You thus can't assume that it can be safely freed. At best you leave a dangling file descriptor because free doesn't know anything about file streams.
The only remarks about cleanup that I found were in the fclose manpage:

[after calling fclose]... any further access (including another call to fclose()) to the
stream results in undefined behavior.

    //Do I need to do anything else with this fptr? Maybe p_tdb->fptr = NULL;????

You don't need to do anything, per se. You can leave p_tdb->fptr dangling with the understanding that using it is UB. You could null it. You could replace it with a different file pointer. By setting it to NULL, you can detect that it doesn't point anywhere meaningful using a null-check. If you don't check it, it's just as harmful to use (UB wise) after being nulled as if you used it while it was dangling.


You can access almost every song ever recorded with streaming music services.
Welcome to the future. We used to buy LPs, then tapes, then CDs, then MP3s. Now we rent access to massive libraries of music. It does require a fundamental shift in your thinking — but you don't need to own physical music anymore. We've compared eight streaming music services suitable for a range of platforms and look at what features they offer.
There's a reason you see fewer video shops around these days. Video streaming services mean you can enter your credit card details and start watching from huge libraries.
Is it worth buying one? Or do you still need a laptop?
How to find the perfect mobile phone for your needs.


Becoming the Beach Boys, 1961-1963, published June 9, 2015, by McFarland Books. Eight years in the making, writing Becoming the Beach Boys, 1961-1963 (BBB) was a labor of love about the music that has accompanied me throughout most of my life. Born out of frustration in reading narratives about the band's early years that often were contradictory or did not fit together, this book attempts to bring understanding to their origin story and to prompt discussion among the vibrant community of Beach Boys fans and historians.
I look forward to hearing from those who have read the book and have comments and questions.
In writing BBB, publisher constraints and the need to rein in a first-time author meant sticking to the spine of the story which meant omitting many conversations, events, and facts, some of which detail the era in which the boys grew up. How many great scenes are deleted from movies that make us wonder why they didn't make the final cut. This website exists to share the stories that didn't make the cut, but deserve telling, or further expand on stories in the book. I hope they are enjoyable and further add to the narrative.
To receive updates on new posts, click on the Follow button.
Rockford, IL 4/26/63: Minor detail but, the appendix lists the venue as the YWCA Teen Canteen located at 4990 East St. Street. This is the current address of the YWCA. In 1963 The YWCA was located at 220 S. Madison, St., Rockford IL. The YWCA did not move to their current address until sometime in the 90s. The old Y was a classic Y facility with a residence hall, gym, pool and activity rooms. One of these rooms was where the show was held. For a brief period I actually worked in that room while I was in college. As I was only 13 at the time of the show we did not find out about it until around 1964 – 65. At that time there was a rumor circulating that The Beach Boys were coming back for another show in Rockford. As there was a huge snowstorm we shoveled walks and drives for 2 days to earn money for the show. Unfortunately it was just a rumor. I enjoyed the detailed research on the rare 45s referenced in the book.
I absolutely love this book, and I'm about halfway through reading it so far. One little thing – the book says that Pacific Ocean Park where The Beach Boys performed was on the "Santa Monica pier" – but that might be confused with the official "Santa Monica Pier" which currently contains a recently built amusement park, and is north of where the POP pier was. POP was actually built on a pier at Pier Avenue in the Ocean Park section of Santa Monica, which was halfway between the official Santa Monica Pier and the Venice Beach pier. When POP closed in 1967, you could see the dilapidated buildings, including the tall Roller Coaster of the closed pier from the other beaches until it was demolished in 1974. During the 1960s, the Santa Monica Pier was only for fishing, and I used to fish from the end of the pier, or rent rowboats to fish from.
Thank you so much for your support of the book. I am glad you are enjoying it! Thanks also for the clarification about the pier. I'm keeping track of "things to add or fix" for a second edition, and your feedback is extremely helpful.
Loved the book. Several questions: When did Brian Wilson fully lose his falsetto? What was the cause? Are their examples where he uses it in later years as a solo artist?
As with many singers, Brian's falsetto became a bit rougher with age, smoking, and other general health issues. While he never recaptured its 1960s beauty, there are glimpses of it throughout the band's later output (think "She's Got Rhythm" on MIU in 1978) and his solo career.
I am very glad you enjoyed the book. Thanks for checking it out!
I came across mentions of your book both in JAVMA and Veterinary Practice News (Oct 2016). I'm a vet and writer in central Maryland with a monthly blog called VetWrite which features veterinarians who explore their creative outlets, whether via writing, painting, singing, etc. I am reaching out to see if you'd be interested in an interview for my blog.
I reviewed your book for my website, The Recoup. Recently, I was saddened to learn about the murder of Betty Willis, and I wrote an entry sharing the two sides from that fine, rare single. But upon posting it, I received a few messages stating that Betty Willis was NOT the vocalist for the sessions that produced Rachel & The Revolvers. Considering her relationship with Leon Russell and the work she did at Gold Star, it certainly doesn't seem unreasonable that she crossed path with Wilson–or that she used said connections later on to get an entree with Russell and a record deal. Plus, the voice on the R&R single sounds like a dead ringer to the woman on the solo Betty Willis recordings.
Here is my piece, and in the comment section is one of the refutations that it was not her. I know your work was quite diligent, so I'd like some input into the matter, if at all possible?


We aim to offer our clients inside and abroad with high-quality products and perfect services. Let's cooperate and be the most reliable strategic partners, and co-develop to become the leading manufacturer in the global optical communications industry.
Всё время в формате GMT 5. Текущее время: 9:07 пп.


Freescale and ALPS Announce Collaboration for Auto Infotainment and Telematics Solutions Based on i.
AI Interview with Michael Dennis Knudsen, General Manager, Haldor Topsoe A/S.
AI interviews Curt Moore, General Manager, AEC-Automotive Business at Texas Instruments.
Automotive Industries spoke to Kevin Kott, president of dSPACE Inc.
Automotive Industries spoke to Bruce Emaus, President of Vector CANtech Inc.
Automotive Industries spoke to Massimo Mannori, director - strategy and operations, at INTECS S.p.A.
Automotive Industries interviews Jeff Kessen, President of ETAS North America.
Automotive Industries speaks to Anders Källdahl, CEO of Image Systems, AB.
Fiat 500 global launch further demonstrates continuing success of Microsoft and Fiat's Blue&Me.
Good news for advanced test systems in the automotive industry.
High soot removal rates, typically 2-4 times higher that with normal diesel oxidation catalysts.
Automotive Industries speaks with Fred Hume, chief executive officer, Data I/O Corporation.
The world's most efficient internal combustion engine?
Play Country, and Where's the Cash Machine?
IMS Introduces "World's First Hands-and-Eyes Free Email Solution for In-Vehicle Use"


text:
\section{Results}
\label{results}

Recall that an {\sl Artin system} is a finite set $S$, together with a
function $m$ from the 2-element subsets of $S$ to the set
$\{2,3,4,\ldots\}\cup\{\infty\}$.  It may be helpful to consider $m$
as an edge-labelling on the complete graph with vertex set $S$.  The
{\sl Artin group} $A=A(S,m)$ corresponding to an Artin system $(S,m)$
is defined by a presentation.  The generators for $A$ are the elements
of the set $S$.  There is one relation in the presentation for each
unordered pair $s,t\in S$ such that $m(t,s)=m(s,t)<\infty$, of the
form
\begin{equation}\label{eq:artin-relation}
  sts\cdots = tst\cdots,
\end{equation}
where there are $m(s,t)$ symbols on each side of
\eqref{eq:artin-relation}.  Call these relations the {\sl Artin
  relations} associated to $(S,m)$.  If $S'$ is a subset of $S$, and
we write $m$ for the restriction to $S'$ of $m$, inclusion induces a
group homomorphism from $A(S',m)$ to $A(S,m)$, and this homomorphism
is known to be injective \cite{vdl,paris}.  The image of such a
homomorphism is called an {\sl Artin subgroup} of $A(S,m)$.

An Artin system also gives rise to a {\sl Coxeter group} $W=W(S,m)$.
The generators of the Coxeter group are the elements of $S$, subject
to the Artin relations for $(S,m)$ together with the relations $s^2=1$
for all $s\in S$.  Call these relations the {\sl Coxeter relations}
for $W$.  Note that in the presence of the relations $s^2=1=t^2$, the
Artin relation between $s$ and $t$ is equivalent to the relation
$(st)^{m(s,t)}=1$.  For $S'$ a subset of $S$, the natural map
$W(S',m)\rightarrow W(S,m)$ is injective, and the image of such a
homomorphism is called a {\sl Coxeter subgroup} of $W(S,m)$
\cite{bourbaki,brown}.

The {\sl set of labels}, $L(S,m)$, for an Artin system $(S,m)$ is the
image of $m$, viewed as a subset of
$\set{2,3,4,\ldots}\cup\set{\infty}$.  An Artin system $(S,m)$ is
called right-angled if $L(S,m)\subseteq \{2,\infty\}$, and $A(S,m)$
(resp.\ $W(S,m)$) is then called a {\sl right-angled Artin group}
(resp.\ {\sl right-angled Coxeter group}).

Now suppose $\phi:S'\rightarrow S''$ is a bijection between two
subsets of $S$ that preserves labels in the sense that
\begin{equation}\label{eq:label-preserving}
  m(s,s')=m(\phi(s),\phi(s')) \qquad\qquad\text{for all $s,s'\in S'$.}
\end{equation}
The {\sl Artin HNN-extension} $G(S,m,\phi)$ (resp.\ {\sl Coxeter
  HNN-extension} $C(S,m,\phi)$) associated to $(S,m,\phi)$ is the free
product of $A(S,m)$ (resp.\ $W(S,m)$) and an infinite cyclic group
$\gen{t}$, modulo the relations $t^{-1}st=\phi(s)$ for all $s\in S'$.
Thus an Artin HNN-extension (resp.\ a Coxeter HNN-extension) is a
special type of HNN-extension in which the base group is an Artin
(resp.\ a Coxeter) group and the stable letter conjugates an Artin
(resp.\ a Coxeter) subgroup to another Artin (resp.\ Coxeter) subgroup
via the map induced by a bijection of standard generating sets.  The
{\sl set of labels} for $G(S,m,\phi)$ (resp.\ $C(S,m,\phi)$) is the
set of labels for the group $A(S,m)$ (resp.\ $W(S,m)$).

We require one more definition before stating our main result.  For
any property $\cal P$ of groups, a group $G$ {\sl virtually has
  property $\cal P$} if there exists a finite-index subgroup $H\leq G$
such that $H$ has property $\cal P$.

\begin{theorem}
  Any Artin (resp.\ Coxeter) HNN-extension virtually embeds in an
  Artin (resp.\ Coxeter) group.  If $L$ is the set of labels for the
  HNN-extension, then the Artin (resp.\ Coxeter) group in which it
  virtually embeds can be chosen to have its set of labels equal to
  $L\cup \{2,\infty\}$.  In particular right-angled HNN-extensions
  virtually embed in right-angled groups.
  \label{main}
\end{theorem}

Theorem~\ref{main} yields the following application, which was our
original motivation for considering Artin HNN-extensions.

\begin{corollary}\label{counterex}
  For any finite group $Q$ not of prime power order, there exists $n$
  and a subgroup $G$ of $SL(n,{\mathbb Z})$ such that $G$ virtually has a
  finite classifying space and $G$ has infinitely many conjugacy
  classes of subgroups isomorphic to $Q$.
\end{corollary}

\begin{proof} Note that by `$G$ virtually has a finite classifying
  space' we mean that there is a finite index subgroup $H\leq G$ which
  has a finite $K(H,1)$.  In \cite{leary}, a group $G$ is constructed
  which virtually has a finite classifying space and which contains
  infinitely many conjugacy classes of subgroup isomorphic to $Q$.  By
  \cite[Thm.\ 28]{leary}, we know that $G$ virtually embeds in a
  right-angled Artin HNN-extension.  But then, by Theorem~\ref{main},
  $G$ virtually embeds in a right-angled Artin group, and
  by~\cite{dj,hsuwise}, any right-angled Artin group embeds in
  $SL(m,{\mathbb Z})$ for some $m$, so the group $G$ embeds in $SL(n,{\mathbb Z})$ for
  some (possibly larger)~$n$.
\end{proof}

\begin{remark}
  Corollary \ref{counterex} was proved in \cite{learynucinkis} in the
  special case when $Q$ is not of the form
  ($p$-group)-by-cyclic-by-($q$-group) for primes $p$ and $q$.
\end{remark}

\section{Proofs}

To review the standard definitions from \cite[I.5]{trees}, recall that
a {\sl graph of groups} $(G_{(-)},\Gamma)$ consists of a directed
CW-graph $\Gamma$, together with a group $G_v$ for each vertex $v$ of
$\Gamma$ (the {\sl vertex groups}), a group $G_e$ for each edge $e$ of
$\Gamma$ (the {\sl edge groups}), and for each edge $e$, two injective
group homomorphisms $\iota_e:G_e\rightarrow G_{\iota(e)}$ and
$\tau_e:G_e\rightarrow G_{\tau(e)}$ (the {\sl attaching
  homomorphisms}), where $\iota(e)$ denotes the initial vertex of $e$
and $\tau(e)$ denotes the terminal vertex of $e$.  We define
$F(G_{(-)},\Gamma)$ to be the free product of the $G_v$ and infinite
cyclic groups $\gen{t_e}$, where $e$ runs over all edges of $\Gamma$,
modulo the relations
\begin{align}\label{eq:F}
  t_e^{-1}\iota_e(g) t_e &= \tau_e(g) &\text{for all $e$, $g\in G_e$;}
\end{align}
and for a basepoint $P_0$ in $\Gamma$, we define
$\pi_1(G_{(-)},\Gamma,P_0)$ to be the subgroup of all words in
$F(G_{(-)},\Gamma)$ that, after ignoring all syllables from $G_v$'s,
define paths in $\Gamma$ that begin and end at $P_0$.  Similarly, for
a spanning tree $T$ of $\Gamma$, we define $\pi_1(G_{(-)},\Gamma,T)$
to be the free product of the $G_v$ and infinite cyclic groups
$\gen{t_e}$, where $e$ runs over all edges of $\Gamma$ not in $T$,
modulo the relations
\begin{align}
  \iota_e(g) &= \tau_e(g) &\text{for all $e\in T$,
    $g\in G_e$, and} \label{eq:inT} \\
  t_e^{-1}\iota_e(g) t_e &= \tau_e(g) &\text{for all $e\notin T$,
    $g\in G_e$.} \label{eq:notinT}
\end{align}
Up to isomorphism, $\pi_1(G_{(-)},\Gamma,P_0)$ is independent of $P_0$
and $\pi_1(G_{(-)},\Gamma,T)$ is independent of $T$, and the two
groups are isomorphic, so we call their isomorphism class the {\sl
  fundamental group of $(G_{(-)},\Gamma)$}, denoted by
$\pi_1(G_{(-)},\Gamma)$.  We also call the elements $t_e$ {\sl stable
  letters}.

Let $k$ be a positive integer, let $V$, $E$, and $V_*$ be groups, and
let $f:E\rightarrow V$, $g:E\rightarrow V$, and $h_i:V\rightarrow V_*$
($i\in{\mathbb Z}/k$) be injective group homomorphisms.
\begin{enumerate}
\item\label{HNN-graph} Let $\Delta_1$ be a self-loop.  Taking vertex
  group $V$, edge group $E$, and attaching maps $f$ and $g$, we get a
  graph of groups $(\set{V,E},\Delta_1)$ whose fundamental group is an
  HNN extension with base group $V$ and associated subgroup $E$.  Let
  $t$ be the associated stable letter.
\item\label{index-k} Let $\Delta_k$ be a (directed) $k$-cycle, with
  vertex set ${\mathbb Z}/k$ and edges $(i,i+1)$ ($i\in{\mathbb Z}/k$).  Taking the
  vertex (resp.\ edge) groups $V_i$ (resp.\ $E_i$) to be copies of $V$
  (resp.\ $E$), and the attaching homomorphisms $f_i:E_i\rightarrow
  V_i$ and $g_i:E_i\rightarrow V_{i+1}$ to be copies of $f$ and $g$,
  with all indices running over all $i\in{\mathbb Z}/k$, we get a graph of
  groups $(\set{V_i,E_i},\Delta_k)$.  Let $\set{t_i}$ be the set of
  stable letters in the group $F(\set{V_i,E_i},\Delta_k)$.
\item\label{ambient-group} Let $\Gamma_k$ be a $k$-leaved rose, with
  edges labelled by ${\mathbb Z}/k$, and let $V_i$, $E_i$, $f_i$, and $g_i$ be
  defined as in \ref{index-k}.  Taking vertex group $V_*$, edge groups
  the $E_i$, and attaching homomorphisms $h_i\circ f_i$ and
  $h_{i+1}\circ g_i$, we get a graph of groups
  $(\set{V_*,E_i},\Gamma_k)$.
\end{enumerate}

\begin{lemma}\label{graphlem}
  Let $K$ be the kernel of the homomorphism $\rho:
  \pi_1(\set{V,E},\Delta_1)\rightarrow{\mathbb Z}/k$ defined by $\rho(V)=0$
  and $\rho(t)=1$.  Then $K=\pi_1(\set{V_i,E_i},\Delta_k)$, and $K$
  embeds in $\pi_1(\set{V_*,E_i},\Gamma_k)$.
\end{lemma}

\begin{proof}
  Let $\theta:
  \pi_1(\set{V_i,E_i},\Delta_k,0)\rightarrow\pi_1(\set{V,E},\Delta_1)$
  be the homomorphism defined by $\theta(v)=v\in V$ for all $v\in V_i$
  and $\theta(t_i)=t$.  Since $\theta$ sends normal forms to normal
  forms, $\theta$ is injective; and since the image of $\theta$ is
  precisely the set of all normal forms $t^{\pm 1}v_1t^{\pm
    1}v_2\dots$ with $t$-exponent sum divisible by $k$, the image of
  $\theta$ is precisely $K$.

  Next, let $\Delta_*$ be $\Delta_k$ with an additional vertex $*$ and
  $k$ additional edges $(i,*)$ ($i\in{\mathbb Z}/k$).  Taking vertex groups to
  be the $V_i$ and $V_*$ (corresponding to $*$), edge groups to be the
  $E_i$ and copies of the $V_i$ (corresponding to $(i,*)$), and
  attaching homomorphisms to be the $f_i$ and $g_i$, along with the
  identity on $V_i$ (corresponding to the initial vertex of $(i,*)$)
  and $h_i$ (corresponding to the terminal vertex of $(i,*)$), we get
  a graph of groups $(\set{V_*,V_i,E_i},\Delta_*)$.

  Since $(\set{V_i,E_i},\Delta_k)$ is a sub-(graph of groups) of
  $(\set{V_*,V_i,E_i},\Delta_*)$, by the normal form theorem, its
  fundamental group $K$ is a subgroup of the fundamental group
  $\pi_1(\set{V_*,V_i,E_i},\Delta_*)$.  Furthermore, if $T_\Delta$ is the
  spanning tree of $\Delta_*$ whose edges are the $(i,*)$ and $T_\Gamma$ is
  the (unique) spanning tree of $\Gamma_k$, by
  \eqref{eq:inT}--\eqref{eq:notinT}, the group
  $\pi_1(\set{V_*,V_i,E_i},\Delta_*,T_\Delta)$ is isomorphic to
  $\pi_1(\set{V_*,E_i},\Gamma_k,T_\Gamma)$.  The lemma follows.
\end{proof}

For the rest of this section, we let $S$, $m$, $\phi: S'\rightarrow
S''$, and $G(S,m,\phi)$ be as defined in Section~\ref{results}.  Let
$\Lambda$ be the (directed) graph with vertex set $S$ and an edge
$(s,\phi(s))$ for each $s\in S'$.  Since every vertex of $\Lambda$ has
in-degree and out-degree each at most 1, $\Lambda$ is a disjoint union
of loops and (non-closed) paths.  Choose an integer $k$ to be a
multiple of the length of each loop in $\Lambda$ and strictly greater
than twice the length of each path in $\Lambda$.

Let ${\overline{S}}$ be the quotient of the direct product ${\mathbb Z}/k\times S$ by
the equivalence relation generated by $(i,s)\sim(i+1,\phi(s))$ for all
$i\in {\mathbb Z}/k$ and $s\in S'$.  More geometrically, let ${\widehat\Lambda}$ be
the (directed) graph with vertex set ${\mathbb Z}/k\times S$ and an edge
$((i,s),(i+1,\phi(s)))$ for each $s\in S'$ and each $i\in {\mathbb Z}/k$.
Then the map $(i,s)\mapsto s$ on vertex sets extends to a $k$-fold
covering map $\pi:{\widehat\Lambda}\rightarrow\Lambda$, and each element of
${\overline{S}}$ is the vertex set of a connected component of ${\widehat\Lambda}$.

For $i\in {\mathbb Z}/k$, define $\psi_i:S\rightarrow {\overline{S}}$ by letting
$\psi_i(s)$ be the equivalence class of $(i,s)$.  We would like to
define a labelling function ${\overline{m}}$ on ${\overline{S}}$ by
\begin{equation}\label{eq:label-function}
  {\overline{m}}(x,y) =
  \begin{cases}
    m(s,t) &\text{if $x=\psi_i(s)$ and $y=\psi_i(t)$ for some
      $i\in{\mathbb Z}/k$,} \\
    \infty &\text{otherwise.}
  \end{cases}
\end{equation}

\begin{lemma}\label{labels-well-defined}
  The labelling function ${\overline{m}}$ is well-defined.
\end{lemma}

The proof of Lemma~\ref{labels-well-defined} uses a variation on an
idea from \cite[Lem.\ 5.7]{wise}.

\begin{proof}
  Let $p:(i,s)\mapsto i$ be the natural projection from ${\widehat\Lambda}$
  to the graph $\Delta_k$ defined above, and let
  $\pi:{\widehat\Lambda}\rightarrow \Lambda$ be the covering map defined
  above.  If ${\hat\ell}$ is a component of ${\widehat\Lambda}$ such that
  $\ell=\pi({\hat\ell})$ is a closed loop, then since the length of $\ell$
  divides $k$, $p$ induces a graph isomorphism from ${\hat\ell}$ to
  $\Delta_k$.  Similarly, if ${\hat\ell}$ is a component of ${\widehat\Lambda}$
  such that $\ell=\pi({\hat\ell})$ is a non-closed path in $\Lambda$, then
  $p$ induces a graph isomorphism from ${\hat\ell}$ to $p({\hat\ell})$, a path
  of length strictly less than $k/2$.  It follows that for any two
  components ${\hat\ell}$, ${\hat\ell}'$ of ${\widehat\Lambda}$, the intersection
  $p({\hat\ell})\cap p({\hat\ell}')$ is a connected subgraph of $\Delta_k$.

  So now, suppose we have $i,j\in{\mathbb Z}/k$ and $s,s',t,t'\in S$ such that
  $x=\psi_i(s)=\psi_j(s')$ and $y=\psi_i(t)=\psi_j(t')$.
  Equivalently, suppose the vertices $(i,s)$ and $(j,s')$ lie in the
  same component $x$ of ${\widehat\Lambda}$ and the vertices $(i,t)$ and
  $(j,t')$ lie in the same component $y$ of ${\widehat\Lambda}$.  Since $p$
  induces graph isomorphisms from $x$ and $y$ to their images $p(x)$
  and $p(y)$, respectively, and $p(x)\cap p(y)$ is connected, for some
  $n\ge 0$, either both $(j,s')=(i+n,\phi^n(s))$ and
  $(j,t')=(i+n,\phi^n(t))$, or both $(i,s)=(j+n,\phi^n(s'))$ and
  $(i,t)=(j+n,\phi^n(t'))$.  In either case, by
  \eqref{eq:label-preserving}, we see that $m(s,t)=m(s',t')$.
\end{proof}

\begin{proof}[of Theorem \ref{main}]
  Taking the Artin case first, let $V=A(S,m)$, $E=A(S',m)$, and
  $V_*=A({\overline{S}},{\overline{m}})$ (which is well-defined by
  Lemma~\ref{labels-well-defined}).  Also, let $f: E\rightarrow V$ be
  induced by the inclusion of $S'$ in $S$ and let $g: E\rightarrow V$
  be induced by $\phi:S'\rightarrow S''$.  Finally, since $k$ is a
  multiple of the length of each loop in $\Lambda$ and strictly longer
  than each path in $\Lambda$, every lift of a loop or path in
  $\Lambda$ intersects each image $\psi_i(S)$ at most once, which
  means that each of the maps $\psi_i: S\rightarrow{\overline{S}}$ is injective
  and each $\psi_i$ induces an injection $h_i: V\rightarrow V_*$.
  Applying Lemma~\ref{graphlem} with this choice of $V$, $E$, $V_*$,
  $f$, $g$, and $h_i$, we see that $G(S,m,\phi)$ has a subgroup of
  index $k$ that embeds in $G_*=\pi_1(\set{V_*,E_i},\Gamma_k)$.

  It remains to show that $G_*$ is an Artin group.  By
  \eqref{eq:notinT}, we see that $G_*$ has a presentation with
  generators ${\overline{S}}\cup\set{t_i}$ ($i\in{\mathbb Z}/k$), subject to the Artin
  relations for $({\overline{S}},{\overline{m}})$ together with the relations
  \begin{equation}\label{eq:HNN-is-artin}
    t_i^{-1}\psi_i(s)t_i=\psi_{i+1}(\phi(s))
  \end{equation}
  for all $i\in{\mathbb Z}/k$ and $s\in S'$.  However, since
  $(i,s)=(i+1,\phi(s))$ in ${\overline{S}}$, \eqref{eq:HNN-is-artin} becomes
  the Artin relation $\psi_i(s)t_i=t_i\psi_i(s)$.  Therefore, if
  \begin{equation}
    m^+(x,y) =
    \begin{cases}
      {\overline{m}}(x,y) &\text{if $x,y\in{\overline{S}}$,} \\
      2 &\text{if $\set{x,y}=\set{t_i,\psi_i(s)}$ for some
        $i\in{\mathbb Z}/k$,} \\
      \infty &\text{otherwise,}
    \end{cases}
  \end{equation}
  we see that $G_*=A({\overline{S}}\cup\set{t_i},m^+)$.

  Now consider the Coxeter case.  Replacing $V=A(S,m)$, $E=A(S',m)$, and
  $V_*=A({\overline{S}},{\overline{m}})$ with $V=W(S,m)$, $E=W(S',m)$, and
  $V_*=W({\overline{S}},{\overline{m}})$, the previous proof carries through, except
  that $G_*$ is not a Coxeter group.  However, consider
  $W^+=W({\overline{S}}\cup\set{u_i,u'_i},m^+)$, where
  \begin{equation}
    m^+(x,y) =
    \begin{cases}
      {\overline{m}}(x,y) &\text{if $x,y\in{\overline{S}}$,} \\
      2 &\text{if $\set{x,y}=\set{u_i,\psi_i(s)}$ for some
        $i\in{\mathbb Z}/k$,} \\
      2 &\text{if $\set{x,y}=\set{u'_i,\psi_i(s)}$ for some
        $i\in{\mathbb Z}/k$,} \\
      \infty &\text{otherwise.}
    \end{cases}
  \end{equation}
  It follows from Tits' solution to the word problem for Coxeter
  groups that the homomorphism $\theta: G_*\rightarrow W^+$ defined by
  $\theta(s)=s$ for $s\in{\overline{S}}$ and $\theta(t_i)=u_i u'_i$ is
  injective, as reduced (graph of groups) words in $G_*$ are sent to
  reduced (Coxeter) words in $W^+$.  The theorem follows.
\end{proof}

\section{An open question}

Can Theorem~\ref{main} be extended to ``Artin graphs of Artin groups''
(defined similarly to Artin HNN-extensions)?  Note that since an
``Artin amalgamated free product'' is itself an Artin group, it is
enough to consider multiple-Artin-HNN-extensions.  In that case, the
key point is to generalize Lemma~\ref{labels-well-defined}, but it is
not clear to us how (or if) that can be done.

One may of course also consider the analogous Coxeter question(s).

redpajama_set_name: RedPajamaArXiv


text:
COVENANT TRANSPORTATION GROUP INC - FORM 10-Q - EX-32.2 - SECTION 906 CERTIFICATION (RICHARD B. CRIBBS) - May 13, 2014
EX-31.2 - SECTION 302 CERTIFICATION (RICHARD B. CRIBBS) - COVENANT TRANSPORTATION GROUP INC exhibit312.htm
EXCEL - IDEA: XBRL DOCUMENT - COVENANT TRANSPORTATION GROUP INC Financial_Report.xls
EX-32.1 - SECTION 906 CERTIFICATION (DAVID R. PARKER) - COVENANT TRANSPORTATION GROUP INC exhibit321.htm
EX-10.1 - EXHIBIT 10.1 (DESCRIPTION OF 2014 BONUS PLAN) - COVENANT TRANSPORTATION GROUP INC exhibit101.htm
10-Q - FORM 10-Q (FIRST QUARTER FISCAL YEAR 2014) - COVENANT TRANSPORTATION GROUP INC form10q.htm
CERTIFICATION PURSUANT TO
18 U.S.C. SECTION 1350
AS ADOPTED PURSUANT TO
SECTION 906 OF THE SARBANES-OXLEY ACT OF 2002
In connection with the Quarterly Report of Covenant Transportation Group, Inc. (the "Company") on Form 10-Q for the period ending March 31, 2014, as filed with the Securities and Exchange Commission on the date hereof (the "Report"), I, Richard B. Cribbs, Chief Financial Officer of the Company, pursuant to 18 U.S.C. Section 1350 as adopted pursuant to Section 906 of the Sarbanes-Oxley Act of 2002, hereby certify, that to the best of my knowledge:
/s/ Richard B. Cribbs
Richard B. Cribbs
A signed original of this written statement required by Section 906 has been provided to Covenant Transportation Group, Inc. and will be retained by Covenant Transportation Group, Inc. and furnished to the Securities and Exchange Commission or its staff upon request.
Return to Form 10-Q
COVENANT TRANSPORTATION GROUP INC Page
COVENANT TRANSPORTATION GROUP INC Reports
03/30/2010 - 10-K
11/12/2009 - 10-Q
Newest Annual Reports
Leader Hill Corp
IHS Markit Ltd.
VOLT INFORMATION SCIENCES, INC.
Sundance Strategies, Inc.
Quad M Solutions, Inc.
Cannagistics Inc.
Ando Holdings Ltd.
VIRTUAL INTERACTIVE TECHNOLOGIES CORP.
Newest 8-K & 10-Q Forms
Zoompass Holdings, Inc.
Zayo Group Holdings, Inc.
WOD Retail Solutions, Inc.
WINTRUST FINANCIAL CORP
WELLS FARGO & COMPANY/MN
Wall Street Media Co, Inc.
WABASH NATIONAL Corp
Vislink Technologies, Inc.
© Copyright 2020 by Advameg. All Rights Reserved.
Based on public records. Inadvertent errors are possible. Getfilings.com does not guarantee the accuracy or timeliness of any information on this site. Use at your own risk. This website is not associated with the SEC.

redpajama_set_name: RedPajamaCommonCrawl


text:
\subsection*{Acknowledgments}
{\small The authors are grateful to John Armstrong for many stimulating and interesting discussions and for geometric intuition that helped improve the paper. In particular, the metric projection interpretation of the global approximation is based on an initial suggestion by John in related work for optimal approximation of S(P)DEs on submanifolds (\cite{armstrongBrigoproj} and \cite{armstrongbrigoicms}, building on \cite{armstrongBrigoJets}).}

\pagestyle{myheadings}
\markboth{}
{D. Brigo, G. Pistone. Optimal approximation of the Fokker-Planck-Kolmogorov eq. }

\newpage

\section{Introduction}

Problems in systems theory, especially filtering and control, may have solutions that are expressed as evolutions of probability distributions. Approximating the evolution of a probability density function with an evolution in a parametric family is an important approximation problem. If one is working in a setting where a probability density $p_t(x)$ at the point $x$ for every time $t \ge 0$ evolves in time as a curve $t \mapsto p_t$ on an infinite dimensional space, it may be important to have a sound methodology to find the best approximation of the curve $t \mapsto p_t$ in a finite dimensional parametrized family of densities, say $\{ p(\cdot,\theta), \theta \in \Theta \subset \mathbb{R}^n\}$. This way the approximated density $p(x,\theta_t)$ will be a natural approximation of the full density $p_t(x)$ and it will be much more manageable given the dimensionality reduction from infinite to a finite $n$. If one has an evolution equation for $p_t$, say a partial differential equation (PDE) or a Stochastic PDE, as happens for example in the filtering problem with the Fokker-Planck-Kolmogorov (discrete time observations) or Kushner-Stratonovich / Zakai (continuous time observations) equations, then it becomes very important to approximate such (S)PDE with an equation for $\theta_t$, which will be a (stochastic) differential equation in the finite dimensional space $\mathbb{R}^n$. Indeed, any implementation of an equation on a machine needs to be finite dimensional, so that finding optimal finite dimensional approximations $t \mapsto \theta_t$ of the original infinite-dimensional solution $t \mapsto p_t$ is of great practical importance.
For the filtering problem this has been addressed in the work initially sketched in Hanzon (1987) \cite{hanzon87} and fully developed in  Brigo et al (1998,1999)\cite{brigo98, brigo99}, where a general method for deriving finite dimensional approximations of the optimal filter stochastic PDE in a chosen exponential family has been given.
In this paper we focus on the simpler Fokker-Planck-Kolmogorov (FPK) equation. We still have a strong link with the filtering problem, since under discrete time observations the FPK equation constitutes the prediction step between observations and is still a fundamental part of the filtering algorithm, see for example \cite{jazwinski70a}.
Applications of finite dimensional approximations of the FPK equation are by no means limited to the filtering problem. Examples of possible applications include, beside signal processing, stochastic- (local-) volatility modeling in quantitative finance, the anisotropic heat equation in physics, and quantum theory evolution equations, see the related discussion and references in \cite{brigopistone}.
In order to make sense of the notion of ``best'' or ``optimal'' approximation, we need to have a measure of how close an approximation will be to the exact solution, and aim to find the closest. Mathematically, this takes the form of a metric or possibly a divergence in the space of all possible probability densities $p$. This way we will have a notion of distance of $p(\cdot,\theta_t)$ from $p_t$, a distance we may wish to minimize in some sense to find the best possible approximation. ``Optimality'' can be imposed either in a local sense, or more strongly in a global sense. Local optimality means that whenever we evolve away {\emph{locally}} from the finite dimensional family $\{ p(\cdot,\theta), \theta \in \Theta \}$ following the ``vector field'' of the original infinite dimensional (S)PDE, resulting in a ``$d {p_t}|_{p_t=p(\cdot, \theta_t)}$'' that points out of the (tangent space of the) family,  we will find the vector $d p(\cdot, \theta_t)$ staying in the (tangent space of the) family that is closest to $d p_t$, and follow $d p(\cdot, \theta_t)$ rather than the full infinite-dimensional evolution. This is only local optimality since we approximate a vector departing from our family with a vector staying in our family, but we don't approximate directly the solution, which leaves the family immediately. The main technique used for this local optimality will be, not surprisingly, the linear projection (on the tangent space), and the metric we will use in the space of densities will be mostly the $L^2$ metric on square root of densities $\sqrt{p}$, the Hellinger distance $d_H(p_1,p_2)^2 = \int (\sqrt{p_1(x)} - \sqrt{p_2(x)})^2 dx$, or alternatively the $L^2$ distance taken directly on densities $p$ themselves $d_D(p_1,p_2)^2 = \int (p_1(x) - {p_2(x)})^2 dx$, assuming these are square integrable. We will refer shortly to the projection on the tangent space as to ``tangent'' or ``linear projection".

Global optimality will clearly be stronger than local optimality. For global optimality we wish to find for every time $t$ the density $p(\cdot,\theta_t)$ in the finite dimensional family that is closest to the true solution $p_t$ in a metric or divergence defined in the infinite dimensional space where $p_t$ evolves. When the approximation $p(\cdot,\theta_t)$ evolves in an exponential family $p(x,\theta) = \exp\left(\theta_1 c_1(x) + \theta_2 c_2(x) + \ldots +\theta_n c_n(x) - \psi(\theta) \right)$
we will find the globally optimal approximation by minimizing the relative entropy or Kullback Leibler information with respect to $\theta$, $K(p_t,p(\cdot,\theta) )= \int \ln(p_t(x)/p(x,\theta)) p_t(x) dx$, rather than the Hellinger distance, because this turns out to be much easier when the family $p(\cdot, \theta)$ is an exponential family. This is a sort of metric projection of $p_t$ onto the chosen family $p(\cdot,\theta)$ in relative entropy (not exactly a metric projection because relative entropy is a divergence and not a metric). We will refer to this projection as to the ``metric projection''. This projection is nonlinear in general, although its linearization leads precisely to the tangent projection. When applied to our problem, this metric projection approach will give us a ``moment matching" characterization of the maximum likelihood exponential density, or of the globally optimal approximation. The two are the same because the maximum likelihood density results indeed from minimization of relative entropy. We will show that the globally optimal or maximum likelihood exponential family density approximating $p_t$ will be the one sharing the expectations of the chosen family sufficient statistics with $p_t$.

These expectations provide another parameterization of the exponential family, alternative to the canonical parameters $\theta$. The expectation parameters $\eta$ are readily computed given the $\theta$ via their definition, which is $\eta_i(\theta) = \int c_i(x) p(x,\theta) dx = \partial_{\theta_i} \psi$. In this paper we introduce an algebraic relation and algorithm to invert this transformation and obtain the $\theta$ given the $\eta$ in the scalar state space case with monomials sufficient statistics $p(x,\theta) = \exp\left(\theta_1 x + \theta_2 x^2 + \ldots \theta_n x^n - \psi(\theta) \right)$, summarizing earlier results in Brigo et al (1996, 1998) \cite{BrigoPhD,brigoime}.

A localized version of the globally optimal projection, based on the assumed density approximation and on the expectation parameters $\eta$, will turn out to be identical to the local ``vector field'' $d_H$ optimality above, and this is explained by the fact that relative entropy and the Hellinger distance coincide at the lowest order of approximation.

For the assumed density approximation in filtering we refer to \cite{kushner}, \cite{maybeck} for the approximation with Gaussian densities, and \cite{brigo99} for the more general approximation with exponential families. The equivalence between the Hellinger projection and the assumed density approximation we present here for the FPK  equation and exponential families is a special case of the equivalence for the filtering stochastic partial differential equation presented in \cite{brigo99}. In the earlier reference \cite{hut} this equivalence had been established for the Gaussian family.

Summarizing the above results in a nutshell, given an exponential family,

\medskip

\begin{center}
globally optimal approximation via metric projection in relative entropy \\=\\ maximum likelihood estimation \\=\\ moment matching for sufficient statistics\\ \hspace{1cm} \\
and\\ \hspace{1cm} \\
localized version of globally optimal approx via assumed density approximation
\\=\\ locally optimal approximation \\=\\ tangent (vector field) projection in Hellinger/Fisher-Rao metric.
\end{center}

\medskip

Still, even with these equivalence results proven, finding the fully optimal global approximation is hard. Solving the optimal problem is in general impossible without knowing the true density $p_t$. However, we will state a theorem where for special exponential families $p(\cdot,\theta)$ the local and globally optimal approximations coincide and can be computed without knowing the true infinite-dimensional solution $p_t$. This special case is the main result of the paper and will be based on an exponential family built on the eigenfunctions of the operator associated with the original infinite dimensional equation for the full curve $t \mapsto p_t$. In a nutshell:

\medskip

\begin{center}
sufficient statistics chosen among eigenfunctions of FPK equation operator
\\ $\Downarrow$\\
 the locally optimal projection is also globally optimal and provides maximum likelihood estimator.
\end{center}

\medskip

A second new series of results of the paper is that we will also show analogous statements to hold for simple mixture families: in that case, to find a globally optimal approximation, our metric projection will be based on minimizing the $L^2$ direct distance $d_D$ between $p_t$ and $p(\cdot,\theta)$. We will show that this also corresponds to an adjusted moment matching condition, and that a localized version of it based on the assumed density approximation is also equivalent to the tangent ``vector-field'' projection of the locally optimal approximation in $d_D$ direct metric. We also recover equivalence of the two with Galerkin type methods. Equivalent to the Galerkin method was shown more generally for the filtering SPDE in \cite{armstrongbrigomcss}.

This can be briefly summarized as follows: given a simple mixture family,

\medskip

\begin{center}
globally optimal approximation via metric projection in $L^2$ direct distance $d_D$\\=\\  moment matching for mixture components \\ \hspace{1cm} \\
and\\ \hspace{1cm} \\
localized version of globally optimal approx via assumed density approximation
\\=\\ locally optimal approximation \\=\\ tangent (vector field) projection in $L^2$ direct metric \\=\\ Galerkin method.
\end{center}

\medskip

Finally, also for the mixture/$d_D$ case we will show that in the special case where the mixture components are chosen among the FPK operator eigenfunctions, we obtain that the locally optimal tangent projection coincides with the globally optimal metric projection and that local and global optimality coincide.  Again in a nutshell:

\medskip

\begin{center}
mixture components chosen among eigenfunctions of FPK equation operator
\\ $\Downarrow$\\
 the locally optimal projection is also globally optimal and provides the metric projection.
\end{center}

\medskip

For further references and a detailed literature review see the proceedings paper \cite{brigopistone}, where the maximum likelihood eigenfunctions result for the exponential families case is presented under the different statistical manifolds geometry of Pistone and Sempi \cite{pistonesempi}, based on Orlicz spaces and charts, rather than on the minimal  $L^2$ structure we use here. For general approaches that combine the $L^2$ geometry used here and the Orlicz-based geometry with applications to filtering see for example \cite{Newton1,Newton2,Newton3}.  Notice however that our earlier proceedings paper \cite{brigopistone} does not provide conditions for the existence of the solution of the original equation in the given function space, contrary to our existence result for the $L^2$ structure here, but uses the $L^2$ case itself to proceed, so that the present paper presents the only fully rigorous and consistent analysis on the eigenfunctions maximum-likelihood theorem, see the $L^2$ existence discussion in Section~\ref{sec:vectorfieldprojhellinger} in particular. This is based again on the Hellinger distance. Moreover, this paper deals with the direct distance case as well, which was absent in the proceedings paper. More generally, measures in the space of probability distributions have been used effectively also in sampling high dimensional and strongly correlated systems, see for example the work on Hamiltonian and Langevin Monte Carlo sampling in \cite{girolami}.
We conclude this introduction by mentioning that part of this paper appeared previously as a preprint  in \cite{brigopistone2}. Finally, Figure \ref{fig:flow} summarizes the relationship we'll show in the paper between different approximation methods.

\bigskip

\begin{figure}
\begin{tikzpicture}[node distance=2cm]
\tikzstyle{startstop} = [rectangle, rounded corners, minimum width=3cm, minimum height=1cm,text centered, draw=black]
\tikzstyle{arrow} = [thick,->,>=stealth]
\tikzstyle{arrow2} = [thick,<->,>=stealth]
\node (start)[startstop] {Local optimality - tangent projection};
\node (hellinger) [startstop, below of=start,xshift=-2cm] {Hellinger, $d_H$};
\node (direct) [startstop, right of=hellinger,xshift=2cm] {direct, $d_D$};
\node (ef) [left of=hellinger,xshift=-2.3cm] {Exp. Families (EF$(c)$)} ;
\node (sm) [right of=direct,xshift=2.3cm] {Simple Mix. (SM$(\hat{q})$)} ;
\node (efadf) [startstop, below of=hellinger] {EF-ADA};
\node (smadf) [startstop, below of=direct] {SM-ADA};
\node (expmatchef) [startstop, below of=efadf] {$\mathbb{E}[c]$ matching};
\node (relentropy) [left of=expmatchef,xshift=-2cm]{Relative entropy};
\node (smmatchsm) [startstop, below of=smadf] {$\mathbb{E}[q]$ matching};
\node (directmetric) [right of=smmatchsm,xshift=2cm]{Direct $d_D$};
\node (global) [startstop, below of=smmatchsm,xshift=-2cm]{Global optimality - metric projection};
\draw [arrow] (start) -- (hellinger);
\draw [arrow] (start) -- (direct);
\draw [arrow] (ef) -- (hellinger);
\draw [arrow] (sm) -- (direct);
\draw [arrow2] (hellinger) -- (efadf);
\draw [arrow2] (direct) -- (smadf);
\draw [arrow] (expmatchef) -- node {localization} (efadf);
\draw [arrow] (smmatchsm) -- node {localization} (smadf);
\draw [arrow] (global) -- (expmatchef);
\draw [arrow] (global) -- (smmatchsm);
\draw [arrow] (relentropy) -- (expmatchef);
\draw [arrow] (global) -- (smmatchsm);
\draw [arrow] (directmetric) -- (smmatchsm);
\end{tikzpicture}
\caption{Summary of the different approximations we will introduce in this paper, their optimality and their relationship. If the functions $c$ and $\hat{q}$ are chosen among the eigenfunctions of the given FPK equation, then the locally optimal approximations are also globally optimal.}\label{fig:flow}
\end{figure}

\section{Statistical manifolds} \label{StatMan}
For a full summary see \cite{brigo99}. We consider parametric families of probability densities, $\{p(\cdot,\theta), \ \theta \in \Theta\}$ with $\Theta$ convex open set in $\mathbb{R}^n$. The set of square roots of such densities is a subset of $L^2$ that we may view as a finite dimensional manifold. In general the $L^2$ distance between square roots of densities leads to the Hellinger distance. A curve in such a manifold is given by $t \mapsto \sqrt{p(\cdot,\theta(t))}$. Differentiating with respect to $t$ we obtain that all tangent vectors at $\theta$ are in the space
$\span\left\{ \frac{\partial{\sqrt{p(\cdot,\theta)}}}{\partial \theta_i}, i=1,\ldots,n\right\}$.
We can use the $L^2$ inner product $\langle \cdot,\cdot \rangle$ to introduce an inner product on the tangent space and a metric. Recall that for $f,h \in L^2$ we have
$\langle f,h \rangle = \int f(x) h(x) dx$ and recall the related $L^2$ norm
$\| f \|_2 = \left(\int f(x)^2 dx\right)^{1/2}$.
Define $ g_{i,j}(\theta)/4 = \left\langle \frac{\partial{\sqrt{p(\cdot,\theta)}}}{\partial \theta_i}, \frac{\partial{\sqrt{p(\cdot,\theta)}}}{\partial \theta_j}\right \rangle$. This is, up to the factor $4$, the familiar Fisher-Rao information matrix.
If we have a $L^2$ vector $v$, we can project it via the $d_H$ tangent or linear orthogonal projection
\begin{equation} \label{proL2}
\Pi^g_\theta[v] = \displaystyle \sum_{i=1}^n [ \sum_{j=1}^n
   4 g^{ij}(\theta)\; \langle v,\frac{1}{2 \sqrt{p(\cdot,\theta)}}\,
   \frac{\partial p(\cdot,\theta)}{\partial \theta_j} \rangle ]\;
   \frac{1}{2 \sqrt{p(\cdot,\theta)}}\,
   \frac{\partial p(\cdot,\theta)}{\partial \theta_i}\
\end{equation}
where upper indices denote the inverse matrix.
A different possibility is a geometry that does not use the square root. This would be done most generally using a duality argument involving $L^1$ and $L^\infty$, but here we will assume that all densities are square integrable, so that all $p$'s we deal with are in $L^2$. Then we can mimic the above structure but without square roots.
The curve is $t \mapsto p(\cdot,\theta(t))$, the tangent space is
$\span\left\{ \frac{\partial{{p(\cdot,\theta)}}}{\partial \theta_i}, i=1,\ldots,n\right\}$.
Define $\gamma_{i,j}(\theta) = \left\langle \frac{\partial{{p(\cdot,\theta)}}}{\partial \theta_i}, \frac{{\partial {p(\cdot,\theta)}}}{\partial \theta_j}\right \rangle$. This leads to what we called $d_D$ tangent or linear projection in \cite{armstrongbrigomcss},
$\Pi^\gamma_\theta[v] = \displaystyle \sum_{i=1}^n [ \sum_{j=1}^n
   \gamma^{ij}(\theta)\; \langle v,
   \frac{\partial p(\cdot,\theta)}{\partial \theta_j} \rangle ]\;
   \frac{\partial p(\cdot,\theta)}{\partial \theta_i}$.
There is another way of measuring how close two densities are.
Consider the Kullback--Leibler information or relative entropy between two densities $p$
and $q$: $K(p,q) := \int \log \frac{p(x)}{q(x)}\; p(x)\, d x$.
This is not a metric, since it is not symmetric and it does
not satisfy the triangular inequality.
It is a classic result that the Fisher metric and the Kullback--Leibler
information coincide infinitesimally.
Indeed, by Taylor expansion it is easy to show that
\begin{eqnarray}\label{eq:klig}
 K(p(\cdot,\theta),p(\cdot,\theta+d\theta))
   = \sum_{i,j=1}^n g_{ij}(\theta)\, d\theta_i\, d\theta_j
   + O(\vert d\theta \vert^3)\ .
\end{eqnarray}

\section{Locally optimal approximations to FPK}
In this section we study locally optimal approximations to the FPK equation solution on a family $\{p(\cdot,\theta), \ \ \theta \in \Theta\}$. Our main tool will be projection on the tangent space of the family.

\subsection{General tangent linear (vector-field) projection of FPK}
We first summarize the key results for tangent projection in Hellinger distance $g$ on exponential families and for tangent projection in direct metric $\gamma$ on mixture families. For a full account see \cite{brigo99} for the Hellinger case and \cite{armstrongbrigomcss} for the direct metric case.
Consider a stochastic differential equation on a probability space $(\Omega, {\cal F}, ({\cal F}_t)_t, \mathbb{P})$ taking values in $\mathbb{R}^N$,
\[ dX_t = f(X_t,t) dt + \sigma(X_t,t) dW_t , \ \  X_0, \ \ a(x,t) = \sigma(x,t)\sigma'(x,t)  \]
where the prime index denotes transposition. Let us start from the FPK equation for the probability density $p_t(x)=p(x,t)$ of the solution $X_t$ of our SDE. Examples of possible applications of approximating the FPK equation and its stochastic PDE extensions are given in \cite{brigopistone}, and include signal processing, stochastic- (local-) volatility modeling in quantitative finance, the anisotropic heat equation in physics, and quantum theory evolution equations.

For our SDE the FPK equation reads
\begin{displaymath}
   \frac{\partial p(x,t)}{\partial t} = {\cal L}_t^\ast\, p(x,t), \ \  {\cal L}_t^\ast p = -\sum_{i=1}^N
   \frac{\partial}{\partial x_i}\, [f_i(\cdot,t)\, p]
   + \inverse{2} \sum_{i,j=1}^N
   \frac{\partial^2}{\partial x_i \partial x_j}\,
   [a_{ij}(\cdot,t)\, p].
\end{displaymath}

Even if the state space of the underlying diffusion $X$ is finite dimensional, in many cases the FPK equation for the density of $X$ is infinite dimensional (see for example \cite{flandoli} and references therein for FPK equations on infinite dimensional state space). We may need finite dimensional approximations. We can project this parabolic PDE  according to either the $L^2$ direct metric (inducing the metric $\gamma(\theta)$) or, by deriving the analogous equation for $\sqrt{p_t}$,
 $\frac{\partial \sqrt{p}}{\partial t} = \frac{1}{\sqrt{p}} {\cal L}_t^\ast\, p$,
according to the Hellinger metric (inducing $g(\theta)$).
The respective tangent linear projections
\begin{equation}\label{eq:projgamg} \frac{d}{dt} p(\cdot,\theta(t)) = \Pi^\gamma_{\theta(t)}\left[  {\cal L}_t^\ast\, p(\cdot,\theta)\right], \ \
\frac{d}{dt} \sqrt{p(\cdot,\theta(t))} = \Pi^g_{\theta(t)} \left[\frac{1}{\sqrt{p(\cdot,\theta)}} {\cal L}_t^\ast\, p(\cdot,\theta)\right]
\end{equation}
 transform the PDE into a finite dimensional ODE for $\theta$ via the chain rule:
\[ \frac{d}{dt} {\rho(p(\cdot,\theta_t))} =
\sum_{j=1}^n
   \frac{\partial {\rho(p(\cdot,\theta))}}{\partial \theta_j} \dot{\theta}_j(t), \ \ \rho(p) = p \ \ \mbox{or} \ \ \rho(p) = \sqrt{p}. \]
The basic idea is illustrated in Figure \ref{fig:projections}.

For brevity we write, for suitable functions $\varphi$, $E_\theta[\varphi] := \int \varphi(x) p(x,\theta) dx$.
The tangent projections in direct and Hellinger metric respectively yield, after an integration by parts using the fact that ${\cal L}^\ast$ is the formal adjoint  of $\cal L$:
\begin{eqnarray}\label{eq:projboth}
   \dot{ \theta}_t^i
   = \displaystyle
   \sum_{j=1}^n \gamma^{ij}(\theta_t)\; E_{\theta_t}
   \left[
   {\cal L}\left(\frac{\partial p(\cdot,\theta_t)}{\partial \theta_j}\right) \right], \     \dot{\theta}_t^i
   =  \sum_{j=1}^n g^{ij}(\theta_t)\;
E_{\theta_t}   \left[
   {\cal L}\left(\frac{\partial \log p(\cdot,\theta_t)}{\partial \theta_j}\right) \right]
\end{eqnarray}
both equations starting at $\theta^i_0$.
\begin{remark}[Square roots and deformed logarithms]
Given the important role played by square roots in defining the Hellinger distance, one might have expected square roots to show up in the projected equation, namely in the second equation in \eqref{eq:projboth}. We emphasized the logarithm because this is particularly natural in view of our application to exponential families, but the square roots are still there. Indeed, an alternative and equivalent representation of the second equation in \eqref{eq:projboth} would be
\[  \dot{\theta}_t^i
   =  \sum_{j=1}^n 4 \ g^{ij}(\theta_t)\; \int \left( \frac{{\cal L}_t^* p(\cdot,\theta_t)}{2 \sqrt{p(x,\theta_t)}} \frac{\partial \sqrt{p(x,\theta)}}{\partial \theta_j}\right) dx \]
Expressing the partial derivatives of square roots via the chain rule and integrating by parts gives immediately the second equation in \eqref{eq:projboth}. Finally, while here we use the two maps $p \mapsto \sqrt{p}$ and $p \mapsto p$, one might use different maps in the spirit of the theory of deformed logarithms, see \cite{naudts}, see also \cite{Newton1,Newton2,Newton3}.
\end{remark}

\begin{figure}[htp]
\begin{centering}
\includegraphics[scale=0.5]{projfig.png}
\caption{Vector field (tangent linear) projection of the FPK Equation on the manifold SM$(\hat{q})$ in direct metric. We start from an initial condition $p_0$ in the manifold. $p_0,p_1,p_2$ describes the exact solution in $L^2$, whereas $p_0,p(\cdot,\theta^1), p(\cdot,\theta^2)$ describes the evolution resulting from the projected vector field. }
\label{fig:projections}
\end{centering}
\end{figure}

\subsection{Exponential families and expectation-to-canonical formula}
We now choose specific families to carry out the tangent projection. In particular, we project on the exponential family using the Hellinger distance and on the mixture family using the direct distance.
The exponential families we consider are EF$(c)$, whose generic density is defined as $p(x,\theta):= \exp[\theta' c(x) - \psi(\theta)]$. The functions $c$ are the sufficient statistics of the family, the parameters $\theta \in \Theta \subset \mathbb{R}^n$ are the canonical parameters of the family. The quantity $\psi(\theta)$ is a normalizing constant needed for the density to integrate to one.
Exponential families work well with the Hellinger/Fisher Rao choice because the tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure.
A number of further results can be found or immediately derived  from Amari~\cite{amari85a} (Chapter 4) or
Barndorff-Nielsen~\cite{barndorff-nielsen78a} (Theorem~8.1). Indeed,
 the Fisher matrix has a simple structure: $\partial_{\theta_i,\theta_j}^2 \psi(\theta) = g_{ij}(\theta)$. The structure of the projection $\Pi^g$ is simple for exponential families. Finally, alternative coordinates, expectation parameters, defined via $\eta(\theta) = E_\theta [ c] = \nabla \psi =(\partial_{\theta_i} \psi(\theta))_i$ are available, with the two coordinate systems $\eta$ and $\theta$ being bi-orthogonal.
Notice further that
\begin{equation}\label{eq:detagdtheta} \partial_{\theta_i} \eta_j(\theta) = g_{i,j}(\theta), \ \   d \eta(\theta) = g(\theta) d\theta .
\end{equation}
We further have $E_{\theta}\{c_i c_j\}
   = \partial_{ij}^{2} \psi(\theta) + \partial_i \psi(\theta)   \partial_j \psi(\theta)$ and more generally
\begin{displaymath}
   E_{\theta}\{c_{i_1} \cdots c_{i_k}\}
   = \exp[ -\psi(\theta) ]\;
   \frac{\partial^k  \exp[ \psi(\theta) ]}
   {\partial \theta_{i_1} \cdots \partial \theta_{i_k}}\; \ .
\end{displaymath}

To have a well-behaving matrix $g$ and good properties for the map $\theta \mapsto \psi(\theta)$ one typically requires that the sufficient statistics $(c_i)_i$ in the exponential family are linearly independent.

Consider now a special case for the family EF$(c)$.
More specifically, we take the exponential polynomial manifold
EP$(n) := \{p(\cdot,\theta): \theta \in \Theta \subset \mathbb{R}^n\}$,
with $m$ an even positive integer and
with a linear combination of the monomials $x, x^2,\ldots,x^n$
in the exponent:
\begin{equation} \label{exfamily1}
p(x,\theta) = \exp\{\theta_1 x + ... + \theta_n x^n - \psi(\theta)\},
\ \ \theta_n < 0.
\end{equation}
In the PhD dissertation of Brigo (1996) \cite{BrigoPhD} (lemma 3.3.3), later partly published in Brigo and Hanzon (1998) \cite{brigoime}, the following result is introduced.
\begin{theorem}[Expectation-canonical parameters algebraic relation for  EP$(n)$]\label{th:exptocan}
For the family EP$(n)$ with $n$ even positive integer, characterized by
$c_i(x) = x^i$, $i=1,\cdots,n$, $\theta_n < 0$,
  the following recursion formula holds, with $\eta_0(\theta) := 1$.
   For any nonnegative integer $i$
\begin{eqnarray} \label{recexpar}
  &&
\eta_{n+i}(\theta) := E_{\theta}\{x^{n+i}\}
  \\ \nonumber
  && = - \frac{1}{n \theta_n} \left[
\begin{array}{ccccc}
   (i+1) & \theta_1 & 2 \theta_2 & \cdots & (n-1)\theta_{n-1}
\end{array} \right]\; \left[
\begin{array}{c}
   \eta_i(\theta) \\ \eta_{i+1}(\theta) \\ \eta_{i+2}(\theta) \\
   \vdots \\ \eta_{i+n-1}(\theta)
\end{array} \right]\ .
\end{eqnarray}
   Moreover, the entries of the Fisher information matrix satisfy
\begin{equation} \label{magic}
   g_{ij}(\theta)
   = \eta_{i+j}(\theta) - \eta_i(\theta)\, \eta_j(\theta)\ .
\end{equation}
Consequently, by defining the matrix
$M(\eta)$ as follows:
\begin{eqnarray}
   M_{i,j}(\eta) := \eta_{i+j}, \ \ \ i,j=1,2,\ldots,n
\end{eqnarray}
it is easy to verify that \eqref{recexpar} and the related lemma imply the following formula:
\begin{eqnarray} \label{theta_eta}
   \left[
   \begin{array}{c}
       \theta_1\\
       2 \theta_2 \\
       \vdots \\
       n \theta_n
   \end{array}
   \right]
   \ \ = \ \ -M(\eta)^{-1} \ \
   \left[
   \begin{array}{c}
       2 \eta_1\\
       3 \eta_2 \\
       \vdots \\
       (n+1) \eta_n
   \end{array}
   \right].
\end{eqnarray}
From this last equation it follows that we can recover
{\em algebraically} the canonical parameters $\theta$ from the
knowledge of the moments $\eta_1,\ldots,\eta_{2n}$ up to order $2n$.

\end{theorem}

{\sc Proof~:}
The recursion formula~(\ref{recexpar}) is obtained
via integration by parts:
\begin{displaymath}
\begin{array}{rcl}

&&  \int_{-\infty}^{+\infty} x^i\; p(x,\theta)\,dx \\ \\
 && =
   [\frac{x^{i+1}}{i+1}\; p(x,\theta)]_{-\infty}^{+\infty}
   - \int_{-\infty}^{+\infty} \frac{x^{i+1}}{i+1}\;
   [\theta_1 + 2 \theta_2\, x + \cdots + n \theta_n\, x^{n-1}]\;
   p(x,\theta)\,dx \\ \\
 &&  = \displaystyle
   0 - \frac{1}{i+1}\;
   E_{p(\cdot,\theta)}\{\theta_1\, x^{i+1} + 2 \theta_2\, x^{i+2}
   + \cdots + n \theta_n\, x^{i+n} \}\ ,
\end{array}
\end{displaymath}
from which the formula and the other results follow easily. \hfill$\Box$

The above results for EP$(n)$ solve the problem of recovering the density and the canonical parameters $\theta$ from knowledge of the expectation parameters $\eta$.  The opposite direction is straightforward: from (\ref{exfamily1}) it is clear that the canonical
parameters $\theta$ permit to express the densities of $EP(n)$ explicitly.

For a study of such  procedure, in a slightly different context, and for a
comparison with several alternatives, including a Newton method,
see Borwein and Huang (1995)  \cite{BorweinHuang95}.
Further investigations into this so called polynomial moment
problem are called for. Better insight into the geometry of the
manifolds $EP(n)$ is likely to be helpful, especially to understand
the behaviour of the various algorithms at the boundary of the
manifold where $\theta_n$ is close to zero.

This concludes our summary of exponential families results.

\subsection{Simple mixture families}
In dealing with exponential families, we used the Hellinger metric $d_H$ since this resulted in a number of good properties summarized in the previous section. For the direct metric $d_D$, instead, it may be more interesting to  project on simple mixture families. We define a \emph{simple mixture family} as follows. Given $n+1$ fixed squared integrable probability densities  $\hat{q} = [q_1,q_2,\ldots,q_{n+1}]'$, define $\hat{\theta}(\theta) := [ \theta_1, \theta_2, \ldots, \theta_n, 1 - \theta_1- \theta_2 - \ldots - \theta_n]'$ for all $\theta \in \mathbb{R}^n$, and set $q = [q_1,q_2,\ldots,q_{n}]'$.  The simple mixture family (on the simplex) is defined as
\begin{equation} \mbox{SM}(\hat{q}) := \{p(\cdot,\theta) = \hat{\theta}(\theta)' \hat{q}(\cdot), \ \ \theta \in \Theta \subset \mathbb{R}^n\}, \ \
\end{equation}
\[\Theta\ \  \mbox{open subset of}\  \  \{\theta \in \mathbb{R}^n: \theta_i \in [0,1]\ \mbox{for all} \ \ i \ \ \mbox{and} \ \ 0<  \sum_{i=1}^n \theta_i < 1\}.\]

If we consider the $L^2$ based $d_D$ distance with metric $\gamma(\theta)$, the metric $\gamma(\theta)$ itself and the related projection become very simple and do not depend on the point $\theta$. For example, $\frac{\partial p(\cdot,\theta)}{\partial \theta_i} = q_i - q_{n+1}$, $\gamma_{i,j} = \langle q_i-q_{n+1},q_j - q_{n+1} \rangle$.
Accordingly, the tangent space at $p(\cdot,\theta)$ does not depend on $\theta$ and is given by
$\mbox{span}\{ q_1-q_{n+1}, q_2-q_{n+1}, \cdots, q_n-q_{n+1}\}$.

We now introduce expectation parameters for the simple mixture family SM$(\hat{q})$.
Define
\[ m(\theta) = E_\theta[q- q_{n+1} \underline{1} ]\]
where $\underline{1}$ is a vector with $m$ components equal to $1$. The reason for subtracting $q_{n+1}$ will be clear when we consider the metric projection later.
A quick calculation shows that
\begin{equation}\label{dmgammadtheta} m(\theta) = \gamma \theta + \beta, \ \ \beta_i = E_{q_{n+1}}[ q_i - q_{n+1}], \ \ i=1,\ldots,n.
\end{equation}
Note that neither $\gamma$ nor $\beta$ depend on $\theta$, so that
\begin{equation}\label{dmgammadtheta2}
d m(\theta) =  \gamma\  d\theta.
\end{equation}
This is the simple mixture counterpart of the exponential families relationship \eqref{eq:detagdtheta}.

Since $m$ is an alternative parameterization for the mixture family, we will denote the simple mixture density characterized by $m$ with $p(\cdot;m)$.

\subsection{Vector-field tangent projection on mixture \& exponential families}\label{sec:vectorfieldprojhellinger}

Starting from the simple mixture family with the direct metric, we now specialize our tangent-space projection equations to $(SM(\hat{q}),d_D)$.  The first Eq. (\ref{eq:projboth}) specializes to
\begin{eqnarray}\label{eq:projl2mix}
   \dot{ \theta}_t^i
   = \displaystyle
   \sum_{j=1}^n \gamma^{ij}\; \sum_{k=1}^{n+1}
   \hat{\theta}_k(t) \int
   ({\cal L}\left(q_j(x)-q_{m+1}(x)\right)) \  (q_k(x)-q_{m+1}(x) ) dx  , \ \  \theta^i_0 \ , \ \
\end{eqnarray}
 which is a linear equation, see \cite{armstrongbrigomcss} for the more general case of the filtering problem SPDE. The quantity $\theta^i_0$ in the right hand side denotes the given initial condition. See also \cite{brigopistone} for more details, and in particular see \cite{santacroce} for mixture and exponential models in Orlicz spaces. See also Ahmed \cite{ahmedbook}, Chapter 14, Sections 14.3 and 14.4 for a summary of the Galerkin method for the Zakai equation, of which FPK is a special case, and see also \cite{ahmed97}.

For the derivation of Eq. \eqref{eq:projl2mix} to make sense we need to ensure that we are indeed projecting a $L^2$ vector field from the related FPK equation.
 This is ensured if we require that ${\cal L}_t^* p(\cdot,\theta)$ can be tangent-linearly projected via $\Pi^\gamma$ as an $L^2$ vector for $p(\cdot,\theta) \in $SM$(\hat{q})$.

\begin{equation}\label{eq:L2existdirect} \sup_{t \ge 0} \|  {\cal L}_t^\ast p(\cdot,\theta)   \|_2 < \infty \ \ \mbox{for all} \ \theta \in \Theta.
\end{equation}
Given that ${\cal L}_t^* p(\cdot,\theta) =  \hat{\theta}' {\cal L}_t^* \hat{q}$ (where ${\cal L}_t^*$ is meant to be applied component-wise when applied to vectors), it is enough to ensure that each ${\cal L}_t^* q_i$ can be projected via the $L^2$ inner product. By inspection of
${\cal L}_t^* q_i$ we can formulate the following
\begin{proposition}\label{rem:L2existmix}(Sufficient conditions for the direct $L^2$ structure to apply to the mixture case). Sufficient conditions guaranteeing the $L^2$  condition \eqref{eq:L2existdirect}
in SM$(\hat{q})$ are the following: $f_t$ and its first derivatives, $a_t$ and its first and second derivatives, and $\hat{q}$ with their first and second derivatives have at most polynomial growth, and densities $\hat{q}$ together with their first and second derivatives integrate any polynomial.
\end{proposition}

An immediate example of simple mixtures satisfying the above polynomial and integrability conditions are Gaussian mixtures.

Moving to exponential families, we will project in Hellinger distance/Fisher-Rao metric, as hinted above. The tangent linear projection of the FPK equation in Fisher metric has been introduced first in \cite{brigogyor}.
For the second Eq. in \eqref{eq:projgamg} to hold we need to ensure that $({\cal L}_t^\ast p(\cdot,\theta) )/\sqrt{p(\cdot,\theta)}$ is indeed an $L^2$ vector for all $t$ and $\theta$. This holds in turn if the condition
\begin{equation}\label{eq:L2exist} \sup_{t \ge 0} E_\theta \left[ \left| \frac{{\cal L}_t^\ast p(\cdot,\theta) }{p(\cdot,\theta)} \right|^2\right] < \infty \ \ \mbox{for all} \ \theta \in \Theta
\end{equation}
holds. Again by inspection, we have the following

\begin{proposition}\label{rem:L2exist}(Sufficient conditions for the Hellinger $L^2$ structure to apply). Sufficient conditions guaranteeing condition \eqref{eq:L2exist} in EF$(c)$ are the following: $f_t$ and its first derivatives, $a_t$ and its first and second derivatives, and $c$ with its first and second derivatives have at most polynomial growth, and densities in EF$(c)$ integrate any polynomial.
\end{proposition}

See also \cite{brigo99} for a proof and a detailed discussion, including   conditions under which all vector fields are well defined and the tangent projection is well defined, see in particular Theorem 5.4 in \cite{brigo99} in the special case $h=0$.

\begin{eqnarray}\label{as:exist} \mbox{{\bf Assumption}. From now on, we assume sufficient conditions above for}\ \eqref{eq:L2existdirect}\\ \nonumber \mbox{in the mixture case and for}\ \eqref{eq:L2exist}\ \mbox{in the exponential case hold.
For example,} \\ \nonumber \mbox{  we might assume conditions  given in Propositions} \  \ref{rem:L2existmix} \ \mbox{or} \ \ref{rem:L2exist}\ \mbox{to hold.}
\end{eqnarray}

Again for brevity, we write $E_\eta[\varphi] := \int \varphi(x) p(x;\eta(\theta)) dx$.
When projecting on EF$(c)$ the second Eq. (\ref{eq:projboth}) specializes into
\begin{equation}\label{FPEPRO:single}
   \dot{\theta}_t^i
   =  \sum_{j=1}^m g^{ij}(\theta_t)\;
   E_{\theta_t}
   \left[
   {\cal L} c_i  \right]  , \ \theta^i_0 \ , \ \ \  \mbox{or} \ \ \
   \dot{\eta}_t^i
   =
   E_{\eta_t}
   \left[
   {\cal L} c_i  \right]   ,  \ \eta^i_0 \
\end{equation}
where the second equation has been obtained from the first by recalling that  $d \eta(\theta) = g(\theta) d\theta$.

We know that the tangent-linear orthogonal projection is giving us the locally optimal approximation for the chosen metric. A natural question when projecting is how good the projection is locally or, in other terms, how well does the chosen finite dimensional family approximate the infinite dimensional evolution locally? Indeed, one would like to have a measure for how far the projected evolution is, locally, from the original one. We now define a local projection residual as the $L^2$ norm of the FPK infinite dimensional vector field minus its finite-dimensional orthogonal projection. Define the vector field minus its tangent projection and the related norm as
\[ \varepsilon_t(\theta) := \frac{{\cal L}_t^\ast p(\cdot,\theta)}
          { 2 \sqrt{p(\cdot,\theta)}} - \Pi^g_\theta
  \left[\frac{{\cal L}_t^\ast p(\cdot,\theta)}
          {2 \sqrt{p(\cdot,\theta)}}\right], \ \ R^2_t : = {\| \varepsilon_t(\theta) \|_2}^2  \]
The projection residual $R_t$ can be computed jointly with the projected equation evolution \eqref{FPEPRO:single} to have a local measure of the goodness of the approximation involved in the projection.

Monitoring the projection residual and its peaks can be helpful in tracking the local projection method performance, see also  \cite{brigo99} for examples of $L^2$-based projection residuals in the more complex case of the Kushner-Stratonovich equations of nonlinear filtering. However, the projection residual only allows for a local approximation error numerical analysis.
To illustrate this, assume for a moment that time is discrete $1,2,3,\ldots$ and consider again Figure \ref{fig:projections}. To make the point, we are artificially separating tangent projection and propagation and the local and global errors. This is not completely precise but allows us to make an important point on our method. If we start from the manifold with a $p_0 = p(\cdot,\theta^0) \in$ EF$(c)$ (omitting square roots in the notation and with upper indices denoting time), the FPK vector field driven by ${\cal L}^\ast$ will move us out of the manifold as the $L^2$ vector related to ${\cal L}^\ast p^0$  will not be tangent to EF$(c)$. We then project this $L^2$ vector on the tangent space of EF$(c)$ and follow it, obtaining a new $p(\cdot,\theta^1)$ in the manifold. Now we continue, again the FPK vector field driven by ${\cal L}^\ast p(\cdot,\theta^1)$ would bring us out of EF$(c)$, and to avoid this we project it onto the tangent space and follow the projected vector, obtaining $p(\cdot,\theta^2)$. The crucial point here is that this second step was done starting from an approximate point $p(\cdot,\theta^1)$ rather than from the true FPK density $p_1$. This means that, besides the local projection error measured by $R_t$, we have a second error coming from the fact that we start the projection from the wrong point.
If we leave the global approximation error analysis aside for a minute, the big advantage of the above method is that it does not require us to know the true solution of the FPK equation to be implemented. Indeed, Equation \eqref{FPEPRO:single} works perfectly well without knowing the true solution $p_t$.

\section{Globally optimal approximation to FPK}
We now investigate whether it is possible to say something on the globally optimal approximation of the solution of the FPK equation on a family $\{p(\cdot, \theta), \ \theta \in \Theta\}$. This will turn out to be related to a form of moments matching, or matching of expectations. In trying to find related results, we will find a local approximation based on a localized version of the globally optimal approximation via the assumed density approximation, and we will find also a tractable case for the global approximation based on the FPK diffusion operator eigenfunctions.

\subsection{Metric projection}

Now, to study the global error, we introduce a second projection  method. We call this method
{\emph{``metric projection''}}, since here we will project directly the $L^2$ densities or their square roots  onto SM$(\hat{q})$ or EF$(c)$, rather than projecting these densities evolutions. In particular, we will make no use of tangent spaces. In geometry this is called indeed ``metric projection'', as opposed to the tangent space linear projection we used so far. This is illustrated in Figure \ref{fig:mprojections}.

\begin{figure}[htp]
\begin{centering}
\includegraphics[scale=0.5]{mprojfig.png}
\caption{Metric projection of the FPK Equation on the manifold SM$(\hat{q})$ in direct metric. We start from an initial condition $p_0$ in the manifold. $p_0,p_1,p_2$ describes the exact solution in $L^2$, whereas $p_0,p(\cdot,\theta^1), p(\cdot,\theta^2)$ describes the evolution resulting from the metric projection. Notice the difference with Figure \ref{fig:projections}, where we project vectors rather than minimizing the distance from the manifold.}
\label{fig:mprojections}
\end{centering}
\end{figure}

Metric projection will require us to know the true solution, so as an approximation method it will be pointless. However, it will help us with the global error analysis, and a modification of the method based on the assumed density approximation will allow us to find an algorithm for a local approximation that does not require the true solution.

\subsection{Metric projection on exponential families in relative entropy}

The metric projection method for exponential families works as follows. Starting again from the manifold with a $p_0 = p(\cdot,\theta^0) \in$ EF$(c)$, the FPK vector field driven by ${\cal L}^\ast$ will move us out of the manifold; we follow this vector field and reach $p_1$.
To go back to EF$(c)$ we project $p_1$ onto the exponential family by minimizing the relative entropy, or Kullback Leibler information of $p_1$ with respect to EF$(c)$, finding the orthogonal projection of $p_1$ on the chosen manifold. It is well known that the orthogonal projection in relative entropy is obtained by matching the sufficient statistics expectations of the true density. Namely, the projection is the particular exponential density $p(\cdot;\eta^1) \in EF(c)$ with $c$-expectations $\eta^1 = \tilde{\eta}^1$ where $\tilde{\eta}^1 = E_{p_1}[c]$ are the $c$-expectations of the density $p_1$ to be approximated.
          See for example  Kagan et al. (1973) \cite{kagan}, Theorem 13.2.1, or \cite{brigoime} for a quick proof and an application to filtering in discrete time.
For convenience, we briefly prove the theorem below.

\begin{theorem}[Moment matching \& entropy minimization for exponential families]\label{th:mmexp}
Suppose we are given the exponential family EF$(c)$ and a $L^1$ probability density $p$ outside EF$(c)$. Suppose the matrix $g(\theta)$ for EF$(c)$ is positive definite. Then the minimization problem
\[  \min_{\theta \in \Theta} K(p,p(\cdot,\theta)) \]
that consists in finding the density in $p(\cdot;\eta) \in EF(c)$ that is closest, in relative entropy, to $p$ has a unique solution characterized by the ``moment matching'' or ``$c$-expectations matching'' conditions
\[ \eta_i = \tilde{\eta}_i \ \ \mbox{where} \ \ \tilde{\eta}_i= E_p[c_i], \  \eta_i = E_\theta[c_i]  \ \ \mbox{for} \ \ i=1,\ldots,n .\]
In other words, the EF$(c)$ density that is closest to $p$ in relative entropy is the density in EF$(c)$ that shares the $c$-expectations (or $c$-moments) with $p$.
\end{theorem}
The proof is immediate. Write
\[ K(p,p(\cdot,\theta)) = \int (\ln p(x) - \theta' c(x) + \psi(\theta)) p(x) dx \]
and note that the gradient with respect to $\theta$ reads
\[ \frac{\partial K(p,p(\cdot,\theta))}{\partial \theta_i} = \int ( - c_i(x) + \partial_{\theta_i} \psi(\theta)) p(x) dx = - E_p[c_i] + \eta_i(\theta)  \]
where we used the basic property of exponential families introduced above, namely $\partial_{\theta_i} \psi = \eta_i = E_\theta[c_i]$. A necessary condition for minimality is given by setting the gradient to zero. This yields
\[ E_p[c_i] = \eta_i(\theta^*) \ \ \mbox{for} \ \ i=1,\ldots, n.\]
To check that the condition is also sufficient we need to show that the Hessian is positive definite. Compute
\[ \frac{\partial^2 K(p,p(\cdot,\theta))}{\partial \theta_i \partial \theta_j} = \int (  \partial^2_{\theta_i,\theta_j} \psi(\theta)) p(x) dx =    \partial^2_{\theta_i,\theta_j} \psi(\theta) = g_{i,j}(\theta) \]
again by the basic properties of exponential families we saw above.
Since we are assuming $g$ to be positive definite, this concludes the proof.
\hfill$\Box$

          As we hinted above, we know that EF$(c)$, besides $\theta$, admits another important coordinate system, the expectation parameters $\eta$. If one defines
          $\eta(\theta) = E_{p(\theta)} [ c]$ as above,
          then $d \eta(\theta) = g(\theta) d\theta$ where $g$ is the Fisher metric, as we have seen earlier.
          Thus, we can take the $\eta^1$ above coming from the true density $p_1$ and look for the exponential density $p(\cdot;\eta^1)$ sharing these $c$-expectations. This will be the closest in relative entropy to the true $p_1$ in EF$(c)$.
We then continue moving forward in time, iterating this algorithm.

The advantage of this method compared to the previous vector field based one is that we find at every time the best possible approximation (``maximum likelihood'') of the true solution in EF$(c)$. The disadvantage is that in order to compute the projection at every time, such as for example $\eta^1 = E_{p_1}[c]$, we need to know the exact solution $p_1$ at that time.
However,  it turns out that we can get back an algorithm that does not depend on the exact solution if we invoke the assumed density approximation.

\subsection{Back to local: assumed density approximation for exponential families}

 This works as follows.
 Differentiate both sides of $\tilde{\eta}_t = E_{p_t}[c]$ to obtain
 \[ \frac{d}{dt}{\tilde{\eta}}_t = \frac{d}{dt} \int c(x) p_t(x) dx = \int c(x) \frac{\partial p_t(x)}{\partial t} dx = \int c(x) {\cal L}^\ast_t p_t(x) dx  = E_{p_t} [ {\cal L} c] \]
so that
\begin{equation}\label{eq:etatrue}  \frac{d}{dt}{\tilde{\eta}}_t = E_{p_t} [ {\cal L} c].
\end{equation}
This last equation is not a closed equation, since $p_t$ in the right hand side is not characterized by $\tilde{\eta}_t$. Thus, to be solved this equation should be coupled with the original FPK equation for $p_t$ and we would still be in infinite dimension. However, at this point we can close the equation by invoking the assumed density approximation (ADA). Implement the following replacements in Equation \eqref{eq:etatrue}.
\[ \mbox{ADA: \ in Eq. \eqref{eq:etatrue}} \ \  \tilde{\eta}_t \rightarrow \eta_t \ \mbox{(left hand side)}, \ \  \   p_t \rightarrow p(\cdot;\eta_t) \ \mbox{(right hand side)}. \]
We obtain
\begin{equation}\label{eq:etaode} {\dot{\eta}}_t = E_{\eta_t} [ {\cal L} c].
\end{equation}
 This is now a finite dimensional ODE for the expectation parameters. It does not require the true solution to be implemented but the arbitrary replacement implies that we have compromised global optimality. However, perhaps surprisingly, we still have local optimality in the same sense as we had with the tangent space projection. Indeed, this last equation is the same as our earlier vector field based projected equation
 \eqref{FPEPRO:single}. This result had been proven for nonlinear filtering in \cite{brigo99}. Intuitively, the result is related to the fact that the Fisher-Rao metric and relative entropy are infinitesimally equivalent, see Eq \eqref{eq:klig}.
 \begin{theorem}[In EF$(c)$ Hellinger-Fisher-Rao tangent projection = ADA] Assumption
\eqref{as:exist} in force.
Closing the evolution equation for the relative entropy point projection of the FPK solution onto EF$(c)$ by forcing an exponential density on the right hand side is equivalent to the locally optimal Hellinger approximation based on the vector field tangent linear projection in Fisher metric.
 \end{theorem}

\subsection{Global optimality for exponential families: ML \& $\cal L$ eigenfunctions}

We can now attempt an analysis of the error between the best possible projection $\tilde{\eta}_t$ and the vector field based (or equivalently assumed density approximation based) projection ${\eta}$. To do this, write
$\epsilon_t := \tilde{\eta}_t - \eta_t$,
expressing the difference between the best possible approximation and the vector field tangent projection / assumed density one, in expectation coordinates.
Differentiating we see easily that
$\dot{\epsilon}_t =  (E_{p_t} [ {\cal L} c] -  E_{p({\eta}_t)} [ {\cal L} c] )$.
Now suppose that the $c$ statistics in EF$(c)$ are chosen among the eigenfunctions of the operator ${\cal L}$, so that
${\cal L} c =  - \Lambda c$,
where $\Lambda$ is a $n \times n$ diagonal matrix with the eigenvalues corresponding to the chosen eigenfunctions.
Substituting, we obtain
\[ \dot{\epsilon}_t =  - \Lambda (E_{p_t} [  c] -  E_{p({\eta}_t)} [  c] )  \ \ \ \mbox{or} \ \ \  \dot{\epsilon}_t =  - \Lambda  \epsilon_t \Rightarrow  \epsilon_t = \exp(- \Lambda t) \epsilon_0  \]
so that if we start from the manifold ($\epsilon_0=0$) the error is always zero, meaning that the vector field tangent projection gives us the best possible Maximum Likelihood (ML) approximation. If we don't start from the manifold, ie if $p_0$ is outside EF$(c)$, then the difference between the vector field approach and the best possible approximation dies out exponentially fast in time provided we have negative eigenvalues for the chosen eigenfunctions. This leads to the following
\begin{theorem}[Global optimality/ML for FPK  \& Fisher-Rao projection.]
Consider the Fokker-Planck-Kolmogorov equation and an exponential family EF$(c)$.
Assumption \eqref{as:exist} in force.
The vector field tangent linear projection approach leading to the locally optimal approximation \eqref{FPEPRO:single} in Hellinger distance in EF$(c)$ provides also the global optimal approximation of the Fokker-Planck-Kolmogorov equation solution in relative entropy in the family EF$(c)$, provided that the sufficient statistics $c$ are chosen among the eigenfunctions of the adjoint operator $\cal L$ of the original Fokker-Planck-Kolmogorov equation, and provided that EF$(c)$ is an exponential family when using such eigenfunctions. In other words, under such conditions the Fisher Rao vector field projected equation \eqref{FPEPRO:single} provides the exact maximum likelihood density for the solution of the Fokker-Planck-Kolmogorov equation in the related exponential family.
\end{theorem}

Before briefly discussing the eigenfunctions result, we consider an analogous setting and derivation for mixture families

\subsection{Metric projection on mixture families in $L^2$ direct distance}
We begin by deriving the metric projection. Our result is the analogous of Theorem \ref{th:mmexp} for mixture families and direct metric.

\begin{theorem}[Moment matching \& $L^2$ distance minimization for simple mixtures]\label{th:mmmix}
Suppose we are given the simple mixture family SM$(\hat{q})$  and a $L^2$ probability density $p$ outside SM$(\hat{q})$. Suppose the matrix $\gamma$ for the family SM$(\hat{q})$ is positive definite. Then the minimization problem
\[  \min_{\theta \in \Theta} \| p - p(\cdot,\theta) \|_2 \]
that consists in finding the density in SM$(\hat{q})$ that is closest in $L^2$ direct distance $d_D$ to $p$ has a unique solution characterized by the ``$q_i-q_{n+1}$ moment or expectation matching'' conditions
\[\tilde{m} = m \] where
\[ \tilde{m} = E_p \left[ q - \underline{1} q_{n+1} \right] \ \mbox{and} \ \  m = E_\theta \left[ q - \underline{1} q_{n+1} \right] = \gamma \theta + \beta,
 \ \ \beta = E_{q_{n+1}}[ q - q_{n+1} \underline{1}] ,\]
provided that the solution $\theta^* = \gamma^{-1}(m-\beta) \in \Theta$.
\end{theorem}
The proof of the theorem is immediate and similar to the the proof given in Theorem \ref{th:mmexp}. One minimizes
\[ \int (p(x) -\hat{\theta}'(\theta) \hat{q}(x) )^2 dx \]
by taking partial derivatives with respect to $\theta$'s and setting them to zero. The Hessian matrix is $\gamma$ and is by assumption positive definite.

Similarly to what we have seen for exponential families, we point out that the metric projection solution $m$ is difficult to obtain. Compared to the vector field tangent projection method, the advantage is again that we find at every time the best possible approximation of the true solution in SM$(\hat{q})$. The disadvantage is that in order to compute the metric projection at every time, such as for example $m^1 = E_{p_1}[q-q_{m+1}\underline{1}]$, we need to know the exact solution $p_1$ at that time.
However, also in this case it turns out that we can get back an algorithm that does not depend on the exact solution if we invoke the assumed density approximation.

\subsection{Back to local: assumed density approximation for mixture families}

 This works as follows. The metric projection of the FPK equation in $d_D$ metric on SM$(\hat{q})$ is defined by
 \[  \tilde{m}_t = E_{p_t}[q-q_{n+1}\underline{1}] .\]
 Differentiate both sides to obtain
\[ \frac{d}{dt} \tilde{m}_t = \frac{d}{dt} \int [q(x)-q_{n+1}(x)\underline{1}] p_t(x) dx
= \int [q(x)-q_{n+1}(x)\underline{1}] ({\cal L}^*_t p_t)(x) dx = \]\[=
\int ({\cal L}(q-q_{n+1}\underline{1}))(x) p_t(x) dx\]
where we substituted the FPK equation and we used integration by parts.
  We have obtained
\begin{equation}\label{eq:timtrue}
 \frac{d}{dt} \tilde{m}_t = E_{p_t}\left[{\cal L}(q-q_{n+1}\underline{1})\right] .
\end{equation}
This equation is not very helpful as an approximation, since we need to know the exact $p_t$ to compute the right hand side. However, we can apply the ADA in this case too.
\[ \mbox{ADA: \ in Eq. \eqref{eq:timtrue}} \ \  \tilde{m}_t \rightarrow m_t \ \mbox{(left hand side)}, \ \  \   p_t \rightarrow p(\cdot;m_t) \ \mbox{(right hand side)}. \]
We obtain
\begin{equation}\label{eq:etaode} {\dot{m}}_t = E_{m_t} [ {\cal L} (q-q_{n+1}\underline{1})].
\end{equation}
 This is a finite dimensional ODE for the expectation parameters. Also in the mixture case, perhaps surprisingly, we still have local optimality in the same sense as we had with the tangent space projection. Indeed, this last equation is the same as our earlier vector field based projected equation \eqref{eq:projl2mix}, as one notices immediately by applying
$d m(\theta) = \gamma \ d \theta$.
 Intuitively, the result is related to the fact that we are using the same $d_D$ metric both in the vector field projection and in the metric projection.
 \begin{theorem}[In SM$(\hat{q})$ we have $d_D$ tangent projection = ADA] Assumption
\eqref{as:exist} in force.
Closing the evolution equation for the $d_D$ metric projection of the Fokker-Planck-Kolmogorov solution onto SM$(\hat{q})$ by forcing a mixture density on the right hand side is equivalent to the locally optimal $d_D$ approximation based on the vector field tangent linear projection in $L^2$ metric.
 \end{theorem}

\subsection{Global optimality for mixture families: $\cal L$ eigenfunctions and Galerkin methods}

We can now attempt an analysis of the error between the best possible projection $\tilde{m}_t$ and the vector field based (or equivalently assumed density approximation based) projection $m_t$. To do this, write
$\varepsilon_t := \tilde{m}_t - m_t$,
expressing the difference between the best possible approximation and the vector field tangent projection / assumed density one, in expectation coordinates.
Differentiating we see easily that
$\dot{\varepsilon}_t =  (E_{p_t} [ {\cal L}(q-q_{n+1}\underline{1}) ] -  E_{m_t} [{\cal L} (q-q_{n+1}\underline{1}))$.
Now suppose that the $q_i-q_{n+1}$ mixture components of SM$(\hat{q})$ are chosen among the eigenfunctions of the operator ${\cal L}$, so that
${\cal L} (q -q_{n+1}\underline{1}) =  - \Lambda (q -q_{n+1}\underline{1})$,
where $\Lambda$ is a $n \times n$ diagonal matrix with the eigenvalues corresponding to the chosen eigenfunctions.
Substituing, we obtain
\[ \dot{\epsilon}_t =  - \Lambda (E_{p_t} [ q -q_{n+1}\underline{1}] -  E_{m_t} [q -q_{n+1}\underline{1} ] )  \ \ \ \mbox{or} \ \ \  \dot{\epsilon}_t =  - \Lambda  \epsilon_t \Rightarrow  \epsilon_t = \exp(- \Lambda t) \epsilon_0  \]
so that, as in the exponential case, if we start from the manifold ($\epsilon_0=0$) the error is always zero, meaning that the vector field tangent projection gives us the best possible $d_D$ metric projection approximation. If we don't start from the manifold, ie if $p_0$ is outside SM$(\hat{q})$, then the difference between the vector field approach and the best possible approximation dies out exponentially fast in time provided we have negative eigenvalues for the chosen eigenfunctions. This leads to the following

\begin{theorem}[Global optimality for FPK  \& $d_D$ tangent projection]
Consider the Fokker-Planck-Kolmogorov equation and a simple mixture  family SM$(\hat{q})$.
Assumption \eqref{eq:L2existdirect} in force.
The vector field tangent linear projection approach leading to the locally optimal approximation \eqref{eq:projl2mix} in $d_D$ direct $L^2$ metric in SM$(\hat{q})$ provides also the global optimal approximation of the FPK equation solution in $d_D$ in the same family, provided that the mixture components $q-q_{n+1}\underline{1}$  are chosen among the eigenfunctions of the adjoint operator $\cal L$ of the original FPK equation, and provided that SM$(\hat{q})$ is a simple mixture family when using such eigenfunctions.
\end{theorem}

Finally, in \cite{armstrongbrigomcss} we have shown that the vector field projection of the filtering SPDE in direct metric on simple mixtures is equivalent to a Galerkin method with basis functions given by the mixture components. Given that the FPK equation is a special case of the filtering equation, we can immediately derive the analogous result.

The Galerkin approximation is derived by approximating the exact solution $p_t$ of the FPK equation with a linear combination of basis functions $\phi_i(x)$, namely
\begin{equation}\label{eq:galerkin:approx} \tilde{p}_t(x) := \sum_{i=1}^{\ell} c_i(t) \phi_i(x) .
\end{equation}
The idea is that the $\phi_i$ can be extended to indices $\ell+1,\ell+2,\ldots,+\infty$ to form a basis of $L^2$.

Now rewrite the FPK equation in weak form, using test functions:
\[   \langle  -  \partial_t {p}_t + {\cal L}_t^\ast\, {p}_t\ ,  \xi \rangle = 0 \]
for all smooth $L^2$ test functions $\xi$ such that the inner product exists.

We replace this equation  with the equation
\[   \langle  -  \partial_t \tilde{p}_t + {\cal L}_t^\ast\, \tilde{p}_t
   , \ \phi_j \rangle = 0, \ \ j=1,\ldots,\ell . \]

By substituting Equation (\ref{eq:galerkin:approx}) in this last equation, using the linearity of the inner product in each argument and by using integration by parts we obtain easily an equation for the coefficients $c$, namely
\begin{eqnarray}\label{eq:galerkinmix} \sum_{i=1}^\ell \langle \phi_i,\phi_j \rangle \dot{ c}_i =
\sum_{i=1}^\ell \langle \phi_i, {\cal L} \phi_j \rangle c_i
\end{eqnarray}
We see immediately by inspection that this equation  coincides with the vector field projection \eqref{eq:projl2mix} of the FPK equation  if we take
\[\ell = n+1, \  c_i = \theta_i   \ \mbox{and} \  \phi_i = q_i - q_{n+1}   \ \mbox{for} \ i=1,\ldots,n, \ \mbox{and} \  c_{n+1} = 1, \ \phi_{n+1} = q_{n+1}.\]

We have thus proven the following
\begin{theorem}
For simple mixture families SM$(\hat{q})$, the $d_D$ vector field projection approximation (\ref{eq:projl2mix}) coincides with a Galerkin method (\ref{eq:galerkinmix}) where the basis functions are the mixture components~$q$.
\end{theorem}

\section{Conclusions and further work}
We presented several methods to approximate the infinite dimensional solution of the Fokker-Planck-Kolmogorov equation with a finite dimensional probability distribution. The approximations we proposed are either locally or globally optimal, and consist of the vector field projection approximation and of the metric projection approximation. The two turn out to be related in a way that is similar to how the tangent space projection is related to the metric projection in general. Indeed, as the tangent projection is a linearization for small distances of the metric projection, so the vector field projection turns out to be equivalent to a localization of the metric projection based on the assumed density approximation. After exploring this setting for exponential and mixture families and further clarifying their relationship with maximum likelihood and Galerkin methods, we found a special case where one can implement the globally optimal approximation and this turns out to coincide with the vector field projection. This special case is based on the eigenfunctions of the original equation. In this case one obtains the exact maximum likelihood density.
The choice of the specific exponential or mixture family, and the choice or availability of suitable eigenfunctions in particular is not always straightforward and requires further work. In this paper we focused on clarifying the relationship between different methods and on deriving the eigenfunctions result. For an initial discussion on eigenfunctions of the FPK equation see \cite{pavliotis} and \cite{liberzon} for the case of the Fokker-Planck-Kolmogorov equation for a linear SDE. For example, in the one dimensional case $N=1$ where the diffusion is on a bounded domain $[\ell, r]$ with reflecting boundaries and strictly positive diffusion coefficient $\sigma$ then the spectrum of the operator ${\cal L}$ is discrete, there is a stationary density and eigenfunctions can be expressed with respect to this stationary density, that could be taken as background measure, see \cite{brigopistone}.
For the case $N>1$ special types of FPK equations allow for a specific eigenfunctions/eigenvalue analysis, see again \cite{pavliotis}. Further research is needed to explore the eigenfunctions approach in connection with maximum likelihood.
In particular, the present paper might have some links with the variational approach to the Fokker-Planck-Kolmogorov equation started in \cite{otto} and continued for example in \cite{ambrosio}, and with estimation of discretely observed diffusion \cite{bibby}, where eigenfunctions play an important role. We will explore such links in future work.
As a side note, we included in this paper a result showing an algebraic algorithm to calculate the canonical parameters $\theta$ from the expectation parameters $\eta$, which may be needed for implementing Equation \eqref{eq:etaode} in cases where one does not use eigenfunctions for $c$ and resorts to monomial sufficient statistics. It may be worth trying to find closed form solutions or approximations to move from expectation to canonical parameters for more general choices of the sufficient statistics $c$.

Further important generalizations one may study are the  local and global projections of  Stochastic PDEs that extend the FPK PDE, for example in the filtering problem. In this case one has to manage carefully the stochastic part of the PDE. The SPDE is sensitive to the choice of stochastic calculus and the presence of noise, which cannot be differentiated, may compromise the local optimality of the projection, which cannot be assumed to hold pathwise a priori. In \cite{brigo98}, \cite{brigo99} and \cite{armstrongbrigomcss} a Stratonovich version of the infinite dimensional SPDE is considered and projected, but without any rigorous proof of local optimality. One then needs notions of optimality for SPDE that take into account the rough nature of the noise. This has been addressed for SDEs in \cite{armstrongBrigoproj} where optimal ways to project potentially high dimensional SDEs on low dimensional submanifolds have been derived, leading to two new projection methods, the Ito-vector and Ito-jet projections. Such methods have been formally extrapolated and briefly applied to the filtering SPDE in \cite{armstrongBrigoproj} and \cite{armstrongbrigoicms}, and are based on the jet bundle interpretation of Ito SDE's on manifolds \cite{armstrongBrigoJets}. In future work we will study rigorously the lift of the Ito vector and Ito jet projection to the infinite dimensional case of the filtering problem, based on replacing the high dimensional SDE to be projected with the infinite dimensional SPDE.

Finally, one further potential application to filtering with continuous time observations is the following. Find particular systems for which the observation functions are among a set of eigenfunction of the state equation operator and where the optimal filter SPDE is approximated on a finite dimensional family based on said set of eigenfunctions. In this case, with exponential families and Hellinger distance the projection of the rough observations-driven part of the SPDE can be exact, see \cite{brigo99} for the case of exponential families. We could try to extend this result to simple mixtures and also combine it with the optimality in the projection of the drift part of the SPDE coming from generalizing the eigenfunctions result for the FPK equation, to see if we can obtain particularly good finite dimensional filters in this setting.  We might also explore the relationship between the discrete time observation case and the continuous time observations case, to see if the limit of the former can lead to the latter. This could be interesting also in cases where the state equation parameters are to be estimated and are not known \cite{schuppen}, as we find often in social sciences applications.


It's all over,… I arrived (by foot) at the dreaded moment.
Over the past 12 months, I've stuck my head in the sand,… tried gallantly to go on, but each time the regret has been big.
A rare night out,… I've frocked up, slapped on some lippy and slipped into high heels.
Everything is fine at first,… then after arriving at the function… I realize my feet have already started pleading with me to release them from the medieval torture devices I've encased them in.
Now usually a short burst of heels is manageable and I escape with one or two bandaid-ed toes, but if there is more than 100 metres of forward ambulation or any extended period of standing required – its not going to end well.
A little while ago, I went to see the crazy talented Matt O'Kine for the Sydney Comedy Festival with my daughter, our hotel was just a 20 minute walk from the venue. For some stuuuuuupid, misguided 'I wanna look cool in the city' reason… I wore my favourite heels*.
Pretty quickly , each step screeched a new octave of pain at me and when I, finally, hobbled into to the venue I was ready to cut my own feet off with a spatula.
Once seated, I took my gorgeous shoes off and surveyed the damage: peeling skin, mangled heels – like lamb shanks,… with toes.
which I slipped on after the show, bracing myself and begging her to walk slowly.
The problem was, the damage was done and walking (even slowly with socks on) was excruciating.
…I had to walk barefoot.
Unlike days at the races, when it's acceptable to slip off the heels, feel the soft grass underfoot and casually dangle them in one hand with a champers in the other, this felt all kinds of wrong.
Back at the hotel it took 20 minutes to scrub my feet clean in the bath, I felt dirty for days and I was genuinely concerned about catching something nasty through my festy feet.
When you look in your wardrobe and the Converse, Dr Martins, Birkenstocks and sneakers outnumber your heels and you've spent a night absorbing the filth of the city through your soles on a Saturday night,… it's time to face facts.
It's been fun, but soz heels, it's over. I'm too old, I don't care enough to wear you… and comfort has become a 'thing'.
*Grey leather Mary Janes with a fat, wooden 4 inch heal and made even more special because Boofhead** saw them in a shop window and pointed them out to me***.
*** He won't be happy I've told you that.
***** I can't believe I just said that.


Employee work experience details like previous company name, Job title, Job description etc., can be uploaded into the system.
Click on Employee Experience Details under Import/Export tab.
Here we can preview the data. Click on Upload button to import the same in to the Application.