\section{Introduction}

The problem of finding the minimum entropy coupling of two discrete
probability distributions $p,q$, i.e., finding a pair of jointly
distributed random variables $X,Y$ such that $X$ has marginal distribution
$p$, $Y$ has marginal distribution $q$, and the joint entropy $H(X,Y)$
is minimized, has been studied by Vidyasagar \cite{vidyasagar2012metric},
Painsky, Rosset and Feder \cite{painsky2013memoryless,painsky2019innovation},
Kova{\v{c}}evi{\'c}, Stanojevi{\'c} and {\v{S}}enk \cite{kovavcevic2015entropy},
Kocaoglu, Dimakis, Vishwanath and Hassibi \cite{kocaoglu2017entropic,kocaoglu2017greedy},
Cicalese, Gargano and Vaccaro \cite{cicalese2017find,cicalese2019minimum},
Yu and Tan \cite{yu2018asymptotic}, and Rossi \cite{rossi2019greedy}.
Also see \cite{roughgarden2013marginals,han2016mutual,cicalese2016approximating}
for related problems. While it is shown in \cite{vidyasagar2012metric,kovavcevic2015entropy}
that this problem is NP-hard, a polynomial time approximation algorithm
(within 1 bit from the optimum) is given in \cite{cicalese2019minimum}
(also see \cite{kocaoglu2017greedy,rossi2019greedy}).

This problem can be generalized to the coupling of $m$ probability
distributions $p_{1},\ldots,p_{m}$ (i.e., constructing random variables
$X_{1},\ldots,X_{m}$ with marginals $X_{i}\sim p_{i}$), where \cite{cicalese2019minimum}
gives an algorithm for constructing a coupling with entropy $H(X_{1},\ldots,X_{m})$
within $\lceil\log m\rceil$ bits from the optimum (also see \cite{kocaoglu2017greedy}
for another algorithm). More precisely, \cite{cicalese2019minimum}
gives a coupling with entropy at most $H(\bigwedge_{i}p_{i})+\lceil\log m\rceil$
bits, where $\bigwedge_{i}p_{i}$ denotes the greatest lower bound
of $p_{1},\ldots,p_{m}$ with respect to majorization of probability
vectors \cite{marshall1979inequalities}. Since any coupling of $p_{1},\ldots,p_{m}$
has entropy at least $H(\bigwedge_{i}p_{i})$ \cite{cicalese2019minimum},
this gives a construction within $\lceil\log m\rceil$ bits from the
optimum.

In this paper, we improve this result by constructing a coupling of
$p_{1},\ldots,p_{m}$ with entropy at most
\begin{equation}
H\Big(\bigwedge_{i}p_{i}\Big)+2-2^{2-m},\label{eq:main_bd}
\end{equation}
which is at most $2$ bits from the optimum. A more general bound
in terms of R\'enyi entropy \cite{renyi1961measures} can also be
obtained. See Corollary \ref{cor:cp_geom_bd} and Theorem \ref{thm:finite}.
Compared to the $\lceil\log m\rceil$ gap in \cite{cicalese2019minimum},
the gap $2-2^{2-m}\le2$ in our result does not scale with $m$. Also
note that the gap becomes $1$ when $m=2$, the same gap as in \cite{cicalese2019minimum,rossi2019greedy}
for the coupling of two distributions. We describe an algorithm (Algorithm
\ref{alg:finitecp}) for computing a coupling achieving \eqref{eq:main_bd}
with time complexity $O(m^{2}n+mn\log n)$, where we assume the pmf's
$p_{i}$ are over a finite set $\mathcal{X}$ with $|\mathcal{X}|=n$.
If we allow an error at most $\epsilon$ (i.e., changing each $p_{i}$
by at most $\epsilon$ in total variation distance), we can reduce
the time complexity to $O(mn\log(1/\epsilon)+mn\log n)$ (see Remark
\ref{rem:error_cp}).

Moreover, \eqref{eq:main_bd} continues to hold when the collection
of pmf's to be coupled is infinite, or even uncountable (in this case,
$m=\infty$ and $2^{2-m}=0$). The bound in \eqref{eq:main_bd} also
applies to the case where the supports of the distributions are infinite.
These cases are not handled in \cite{kocaoglu2017entropic,kocaoglu2017greedy,cicalese2019minimum,rossi2019greedy}.

Below are some potential applications of a low entropy coupling of
a collection of distributions.

\subsection{Random Number Generation}

It was shown by Knuth and Yao~\cite{knuth1976complexity} that a
discrete random variable $X$ can be generated using an expected number
of fair coin flips no more than $H(X)+2$, indicating that the entropy
$H(X)$ is a measure of the amount of resources (coin flips) needed
to generate the random number $X$ (also see \cite{roche1991,han1997randgen}).
The entropy of a coupling of a collection of distributions $S$ can
be regarded as the amount of resources needed to allow generation
of any distribution in $S$. More precisely, consider the setting
where there is a random number generator device that can output a
random number to the user (who does not have access to random sources
other than the generator). The user wants to generate $X\sim p$ for
a distribution $p\in S$ of the user's choice ($p$ is not fixed a
priori). If the generator is versatile enough to generate any distribution
$p$ at the user's request, then the minimum amount of entropy used
by the generator is $H(p)$. Nevertheless, the generator may not be
programmable or configurable. If we assume the generator is only capable
of generating a random number $Z$ following a fixed distribution
(that depends on the design of the generator, but cannot depend on
the user's choice of $p$), then the user has to apply a mapping $g_{p}$
(depending on the choice of $p$) to obtain the final random number
$X=g_{p}(Z)\sim p$. This induces a coupling $\{g_{p}(Z)\}_{p\in S}$
of the distributions in $S$. Therefore, the minimum entropy coupling
of $S$ corresponds to the distribution of $Z$ that has the minimum
entropy needed to accomplish this task.

Existing hardware random number generators  are capable of generating
a uniformly random integer within a range of integers. While we can
generate from any discrete distribution by repeated and interactive
usages of such generator (e.g. by \cite{knuth1976complexity}), such
interactive communication between the generator and the user may not
be feasible depending on the situation (e.g. delay in generating the
random number and communication). The minimum entropy coupling allows
us to design the generator according to $S$ (with possibly non-uniform
output $Z$) so that we only need to use the generator once per random
number $X\sim p$ obtained by the user.

We will see in the following sections that our construction is similar
to the alias method for random number generation by Walker \cite{walker1977efficient}.
While the alias method only works for discrete distributions with
finite support, and requires an amount of entropy approximately $\log k$
(where $k$ is the size of the support), our construction works for
any discrete distribution (with finite or infinite support), and requires
an amount of entropy close to the theoretical minimum (which can be
much smaller than $\log k$ depending on the collection of distributions
$S$).

A related setting is channel simulation (see \cite{bennett2002entanglement,winter2002compression,cuff2013distributed,bennet2014reverse}
for the asymptotic case, and \cite{harsha2010communication,kumar2014exact,li2018universal}
for the one-shot case), where the encoder observes a distribution
$p\in S$ in a collection of distributions $S$ and transmits a message
$M$ to the decoder (who knows $S$ but does not know $p$ a priori),
so as to allow the decoder to generate $X\sim p$. The aforementioned
random number generation setting corresponds to the one-shot channel
simulation setting where the encoder does not have local randomness,
the communication $M$ from the encoder to the decoder is unlimited,
and our goal is to minimize the amount of local randomness at the
decoder in order to generate $X\sim p$ (we require the distribution
of the local randomness to be fixed).

\subsection{Functional Representation and Entropic Causal Inference}

The functional representation lemma \cite{elgamal2011network} states
that for any pair of random variables $(X,Y)\in\mathcal{X}\times\mathcal{Y}$,
there exists a random variable $Z$ independent of $X$ such that
$Y=g(X,Z)$ is a function of $(X,Z)$. See \cite{hajek1979,willems1985}
for applications of this lemma in information theory. Since $Y_{x}:=g(x,Z)\sim p_{Y|X=x}$,
$\{Y_{x}\}_{x\in\mathcal{X}}$ is a coupling of the conditional distributions
$p_{Y|X=x}$, and hence the problem of finding a functional representation
with the smallest $H(Z)$ is equivalent to the minimum entropy coupling
problem (see \cite{kocaoglu2017entropic,kocaoglu2017greedy}).

Shannon \cite[Fig. 1]{shannon1948mathematical} considers a channel
to be a function mapping the input signal and noise source to the
received signal. Letting the input signal and the received signal
be $X$ and $Y$ respectively, the minimum $H(Z)$ in the functional
representation would be the minimum entropy of the noise source of
the channel. Note that this measure is an inherent property of the
channel, and does not depend on the input distribution $p_{X}$ as
long as $p_{X}(x)>0$ for all $x\in\mathcal{X}$ (since the minimum
$H(Z)$ is the minimum entropy of a coupling of $\{p_{Y|X=x}\}_{x\in\mathcal{X}}$
which does not depend on $p_{X}$).

Kocaoglu, Dimakis, Vishwanath and Hassibi \cite{kocaoglu2017entropic,kocaoglu2017greedy}
consider the problem of identifying the causal direction between $X$
and $Y$, based on the assumption that the correct causal direction
gives a small $H(Z)$. More precisely, the \emph{entropic causal inference}
method declares that $X\to Y$ is the correct direction if $Y=g(X,Z)$
can be achieved with a smaller $H(X)+H(Z)$ compared to the smallest
$H(Y)+H(\tilde{Z})$ satisfying $X=\tilde{g}(Y,\tilde{Z})$. They
have proposed algorithms for minimizing $H(Z)$, or equivalently,
minimizing the entropy of the coupling of $p_{Y|X=x}$ (also see \cite{cicalese2019minimum}
for another algorithm). Nevertheless, these algorithms only work when
$\mathcal{X}$ is finite (or the number of distributions to couple
is finite). The method in this paper works regardless of whether $X$
is a discrete or continuous random variable (though $Y$ must be discrete).
By \eqref{eq:main_bd}, the minimum of $H(Z)$ is closely approximated
by $H(\bigwedge_{x\in\mathcal{X}}p_{Y|X=x})$ (within $2$ bits),
and hence replacing $H(Z)$ by $H(\bigwedge_{x\in\mathcal{X}}p_{Y|X=x})$
(which can be computed in $O(|\mathcal{X}||\mathcal{Y}|\log|\mathcal{Y}|)$
time if $|\mathcal{X}|,|\mathcal{Y}|<\infty$) in the entropic causal
inference method provides a close approximation that can be computed
efficiently (compared to the exact minimization of $H(Z)$ which is
NP-hard \cite{vidyasagar2012metric,kovavcevic2015entropy}). If the
function $g$ is also needed, then it can be computed in $O(|\mathcal{X}|^{2}|\mathcal{Y}|+|\mathcal{X}||\mathcal{Y}|\log|\mathcal{Y}|)$
time using Algorithm \ref{alg:finitecp}.

The problem of minimizing $H(Y|Z)$ (instead of $H(Z)$) was studied
by Li and El Gamal \cite{sfrl_trans}. The strong functional representation
lemma \cite{sfrl_trans} states that for any pair of random variables
$(X,Y)$, there exists a random variable $Z$ independent of $X$
such that $Y$ is a function of $(X,Z)$, and $H(Y|Z)\le I(X;Y)+\log(I(X;Y)+1)+4$
(also see \cite{harsha2010communication,braverman2014public}). The
lemma is applied to show several one-shot variable-length lossy source
coding results, and a short proof of the asymptotic achievability
in the Gelfand-Pinsker theorem \cite{gelfand1980coding}. It is also
used in \cite{li2018minimax} to prove a result on minimax remote
prediction with a communication constraint. The Poisson functional
representation given in \cite{sfrl_trans} (which induces a coupling
of $p_{Y|X=x}$) is also used in \cite{li2019unified} to prove various
results in multi-user information theory. In this paper, we concern
the minimization of $H(Z)$ instead of $H(Y|Z)$ (while \cite{sfrl_trans}
gives a cardinality bound $|\mathcal{Z}|\le|\mathcal{X}|(|\mathcal{Y}|-1)+2$
in addition to the bound on $H(Y|Z)$, this is not the main objective
there).

\subsection{Other Uses of Coupling of Collections of Distributions}

It has been shown that for any collection of distributions $p_{1},\ldots,p_{m}$,
it is possible to find a coupling $X_{1},\ldots,X_{m}$ such that
\begin{equation}
\mathbf{P}(X_{i}\neq X_{j})\le2d_{\mathrm{TV}}(p_{i},p_{j})\label{eq:tv_bd}
\end{equation}
for any $i,j$, where $d_{\mathrm{TV}}$ is the total variation distance.
This was shown in \cite{broder1997resemblance} for uniform distributions,
\cite{kleinberg2002approximation} for discrete distributions, and
\cite{angel2019pairwise,li2019pairwise} for general distributions.
This result was used in locality sensitive hashing \cite{charikar2002similarity}
and randomized rounding algorithms \cite{kleinberg2002approximation,barak2008rounding}.
While a coupling achieving \eqref{eq:tv_bd} is likely to have low
entropy (since many values of $X_{i}$ are the same), a low entropy
coupling does not necessarily have a low $\mathbf{P}(X_{i}\neq X_{j})$
(since whether $X_{i}\neq X_{j}$ is irrelevant in the calculation
of entropy). We also remark that the connection between entropy and
total variation distance has been studied in \cite{sason2013entropy}
using coupling.

In the study of Markov chains, it is often useful to represent the
Markov chain $X_{1},X_{2},\ldots$ using the functional representation
$X_{n}=g(X_{n-1},Z_{n})$, where $Z_{n}\stackrel{iid}{\sim}p_{Z}$.
In the coupling from the past algorithm for sampling from the stationary
distribution of a Markov chain \cite{propp1996exact,propp1998coupling},
the function $g$ is designed so that $g(x,z)$ are likely to be equal
for different values of $x$. This representation is also referred
as innovation representation in \cite{painsky2019innovation}. Since
the minimum entropy of $Z_{n}$ is the entropy of the minimum entropy
coupling of $\{p_{X_{n}|X_{n-1}=x}\}_{x}$, we can apply the coupling
achieving \eqref{eq:main_bd} in this paper to generate $X_{1},X_{2},\ldots$
using a small entropy rate of $Z_{1},Z_{2},\ldots$.

Other works on the coupling of a collection of distributions include
Wasserstein barycenter \cite{agueh2011barycenters} and multi-marginal
optimal transport \cite{kellerer1984duality,gangbo1998optimal,pass2011uniqueness,li2019pairwise}.

\subsection*{Notations}

Throughout this paper, we assume that $\log$ is to base $2$ and
the entropy $H$ is in bits. We write $\mathbb{N}=\{1,2,3,\ldots\}$,
$[n]=\{1,\ldots,n\}$. For a statement $E$, we write $\mathbf{1}\{E\}$
for the indicator function where $\mathbf{1}\{E\}=1$ if $E$ holds,
$\mathbf{1}\{E\}=0$ otherwise.

A right stochastic matrix is a square matrix with nonnegative entries
where each row sums to $1$. The support of a probability mass function
(pmf) $p$ is written as $\mathrm{supp}(p)$. For a pmf $p$ over
$[n]$, its probability vector is denoted as $\vec{p}\in\mathbb{R}^{n}$
(a row vector). For a pmf $p$ over the set $\mathcal{X}$, and a
pmf $q$ over the set $\mathcal{Y}$, the product pmf $p\times q$
is a pmf over $\mathcal{X}\times\mathcal{Y}$ with $(p\times q)(x,y):=p(x)q(y)$.
The pmf of the Bernoulli distribution is denoted as $\mathrm{Bern}_{\gamma}(x):=\mathbf{1}\{x=0\}(1-\gamma)+\mathbf{1}\{x=1\}\gamma$.
The pmf of the geometric distribution over $\mathbb{N}$ is denoted
as $\mathrm{Geom}_{\gamma}(x):=\gamma(1-\gamma)^{x-1}$. The pmf of
the capped geometric distribution over $[k]$ is denoted as
\begin{equation}
\mathrm{CGeom}_{\gamma,k}(x):=\begin{cases}
\gamma(1-\gamma)^{x-1} & \mathrm{if}\;x<k\\
(1-\gamma)^{k-1} & \mathrm{if}\;x=k\\
0 & \mathrm{if}\;x>k.
\end{cases}\label{eq:cgeom}
\end{equation}
The R\'enyi entropy \cite{renyi1961measures} of a pmf $p$ is defined
as
\[
H_{\alpha}(p):=\frac{1}{1-\alpha}\log\sum_{x\in\mathrm{supp}(p)}(p(x))^{\alpha}
\]
for $\alpha\in\mathbb{R}_{\ge0}\backslash\{1\}$. When $\alpha=1$,
$H_{\alpha}(p):=H(p)$ is the Shannon entropy. When $\alpha=\infty$,
$H_{\alpha}(p):=-\log\max_{x}p(x)$.

\medskip{}

\section{Coupling and Majorization\label{sec:mrq}}

We first define a coupling of a set of distributions.
\begin{defn}
For a set of pmf's $S$, we say that an indexed set of random variables
$\{X_{p}\}_{p\in S}$ is a coupling of $S$, written as $\{X_{p}\}_{p\in S}\in\Gamma(S)$,
if $X_{p}$ has marginal distribution $p$ for any $p\in S$. We say
that a pmf $q$ is an \emph{underlying distribution of a coupling}
of $S$, written as $q\in\tilde{\Gamma}(S)$, if there exists $\{X_{p}\}_{p\in S}\in\Gamma(S)$
and random variable $Z\sim q$ such that $X_{p}$ is a function of
$Z$ for all $p\in S$.\footnote{Technically, to make the set $\tilde{\Gamma}(S)$ well-defined, we
can restrict $q$ to be a pmf over $\mathbb{N}$, which will not cause
any loss of generality since the support of a pmf is always countable.
}

Define the \emph{minimum R\'enyi entropy of couplings} of a set of
pmf's $S$ as
\begin{equation}
H_{\alpha}^{*}(S):=\inf_{q\in\tilde{\Gamma}(S)}H_{\alpha}(q)\label{eq:mrec}
\end{equation}
for $\alpha\in\mathbb{R}_{\ge0}\cup\{\infty\}$. We write $H^{*}(S)=H_{1}^{*}(S)$.
It is straightforward to show that when $S=\{p_{1},\ldots,p_{m}\}$
is finite, then $H_{\alpha}^{*}(S)=\inf_{\{X_{p_{i}}\}_{i\in[m]}\in\Gamma(S)}H_{\alpha}(X_{p_{1}},\ldots,X_{p_{m}})$
(to show a one-to-one correspondence between $\tilde{\Gamma}(S)$
and $\Gamma(S)$, simply take $q\in\tilde{\Gamma}(S)$ to be the joint
pmf of $\{X_{p_{i}}\}_{i\in[m]}$). Nevertheless, we define $H_{\alpha}^{*}(S)$
in \eqref{eq:mrec} for general $S$ using $\tilde{\Gamma}(S)$ instead
of $\Gamma(S)$, in order to avoid working with the joint entropy
of an infinite collection of random variables when $|S|=\infty$.
\end{defn}
The goal of this paper is to find or approximate $H_{\alpha}^{*}(S)$.
We present the concept of aggregation in \cite{vidyasagar2012metric,cicalese2016approximating}.
\begin{defn}
For two pmf's $p,q$, we say $p$ is an \emph{aggregation} of $q$,
written as $q\sqsubseteq p$, if there exists a function $g:\mathrm{supp}(q)\to\mathrm{supp}(p)$
(called the \emph{aggregation map}) such that $p$ is the pmf of $g(X)$,
where $X\sim q$.
\end{defn}
It is clear that ``$\sqsubseteq$'' is a transitive relation. If
$p,q$ are pmf's over $[n]$, then $q\sqsubseteq p$ if and only if
there exists a right stochastic matrix $M$ with $\{0,1\}$ entries
such that the probability vectors satisfy $\vec{p}=\vec{q}M$. Note
that $q\in\tilde{\Gamma}(S)$ if and only if $q\sqsubseteq p$ for
any $p\in S$. Therefore, a coupling of $S$ can be specified using
an underlying distribution $q\in\tilde{\Gamma}(S)$ and the set of
aggregation maps $\{g_{p}\}_{p\in S}$, where $g_{p}$ is the aggregation
map for $q\sqsubseteq p$.

We will then show that ``$\sqsubseteq$'' is ``closed under pointwise
limit'' in the following sense:
\begin{prop}
\label{prop:agg_closed}Let $q$ be a pmf over a countable set $\mathcal{X}$,
and $p,p_{1},p_{2},\ldots$ be pmf's over a countable set $\mathcal{Y}$,
such that $p(y)=\lim_{i\to\infty}p_{i}(y)$ for any $y\in\mathcal{Y}$,
and $q\sqsubseteq p_{i}$ for any $i\ge1$, then we have $q\sqsubseteq p$.
\end{prop}
\begin{IEEEproof}
Without loss of generality, assume $\mathcal{X}=\mathcal{Y}=\mathbb{N}$,
and $q(1)\ge q(2)\ge\cdots$. Let $g_{1},g_{2},\ldots$ be functions
from $\mathbb{N}$ to $\mathbb{N}$ such that $g_{i}(X)\sim p_{i}$,
where $X\sim q$. Consider whether $g_{i}(1)=1$. There exists an
increasing sequence $i_{1},i_{2},\ldots$ such that $\mathbf{1}\{g_{i_{j}}(1)=1\}$
are the same for all $j$ (since there are only two possibilities
of $\mathbf{1}\{g_{i_{j}}(1)=1\}\in\{0,1\}$). Let that value of $\mathbf{1}\{g_{i_{j}}(1)=1\}$
be $b_{1,1}$. There exists an increasing subsequence $i'_{1},i'_{2},\ldots$
of $i_{1},i_{2},\ldots$ such that for any $x,y\le2$, $\mathbf{1}\{g_{i'_{j}}(x)=y\}$
are the same for all $j$ (since there are only $2^{4}$ possibilities
of $\{\mathbf{1}\{g_{i'_{j}}(x)=y\}\}_{x,y\le2}$). Let those values
of $\mathbf{1}\{g_{i'_{j}}(x)=y\}$ be $b_{x,y}$ for $x,y\le2$.
Repeat this procedure to define $b_{x,y}$ for any $x,y\in\mathbb{N}$.

Define $g:\mathrm{supp}(q)\to\mathbb{N}$ by $g(x)=y$ if $b_{x,y}=1$.
We now check that $g$ is well-defined and $g(X)\sim p$. It is clear
from the definition that there does not exist $x$ and $y\neq y'$
such that $b_{x,y}=b_{x,y'}=1$ (consider the $\max\{x,y,y'\}$-th
iteration of the above procedure). Fix any $a,y\in\mathbb{N}$ and
$\epsilon>0$, and consider the $\max\{a,y\}$-th iteration of the
above procedure that fixes $b_{x,y}$ for $x\le a$. There exists
an increasing sequence $i_{1},i_{2},\ldots$ such that $\mathbf{1}\{g_{i_{j}}(x)=y\}=b_{x,y}$
for all $j$ and $x\le a$. By $p(y)=\lim_{i\to\infty}p_{i}(y)$,
there exists $j$ such that $|p(y)-p_{i_{j}}(y)|\le\epsilon$. Since
$p_{i_{j}}(y)=\sum_{x}\mathbf{1}\{g_{i_{j}}(x)=y\}q(x)$, we have
\[
\Big|p(y)-\sum_{x\le a}b_{x,y}q(x)\Big|\le\epsilon+\sum_{x>a}q(x).
\]
Taking $a\to\infty$ and $\epsilon\to0$, we have $\sum_{x}b_{x,y}q(x)=p(y)$.
Since $\sum_{x}\sum_{y}b_{x,y}q(x)=\sum_{y}p(y)=1$, for any $x\in\mathrm{supp}(q)$
(where $q(x)>0$), there exists at least one (and thus exactly one)
$y$ such that $b_{x,y}=1$. The result follows.
\end{IEEEproof}
It is demonstrated in \cite{cicalese2019minimum} that majorization
is a useful tool in the study of coupling. Here we present the concept
of majorization which allows infinite sequences or pmf's with infinite
support (e.g. see \cite{marshall1979inequalities}):
\begin{defn}
For two pmf's $p,q$, we say $q$ is \emph{majorized by} $p$, written
as $q\preceq p$, if
\[
\max_{B\subseteq\mathrm{supp}(q):\,|B|\le k}q(B)\le\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k}p(A)
\]
for any $k\in\mathbb{N}$, where we write $p(A):=\sum_{x\in A}p(x)$.
In other words, the sum of the $k$ largest $q(x)$'s is not greater
than the sum of the $k$ largest $p(x)$'s.
\end{defn}
It is clear that ``$\preceq$'' is a transitive relation. It is
shown in \cite{cicalese2016approximating} that $q\sqsubseteq p$
implies $q\preceq p$. If $p,q$ are pmf's over $[n]$, then it has
been shown that $q\preceq p$ if and only if there exists a doubly
stochastic matrix (i.e., square matrix with nonnegative entries where
each row and column sums to $1$) $M$ such that the probability vectors
satisfy $\vec{p}=\vec{q}M$ (e.g. see \cite{marshall1979inequalities}).
Also, if $p,q$ are pmf's over $[n]$ sorted in descending order (i.e.,
$p(1)\ge p(2)\ge\cdots\ge p(n)$ and likewise for $q$), then $q\preceq p$
if and only if there exists a lower triangular right stochastic matrix
$M$ such that $\vec{p}=\vec{q}M$. This property will be strenghtened
in Lemma \ref{lem:alias}.

Also note that R\'enyi entropy is Schur concave \cite{marshall1979inequalities},
i.e., we have $H_{\alpha}(q)\ge H_{\alpha}(p)$ if $q\preceq p$.
We then prove a useful property of majorization and aggregation:
\begin{prop}
\label{prop:kernel_maj}Let $X$ be a random variable with pmf $p_{X}$,
and $Y$ be a random variable with conditional pmf $p_{Y|X=x}$, and
$p_{X,Y}$ be the joint pmf of $(X,Y)$. Define $\tilde{X},p_{\tilde{X}},\tilde{Y},p_{\tilde{Y}|\tilde{X}=x},p_{\tilde{X},\tilde{Y}}$
similarly. We have:
\begin{itemize}
\item If $p_{X}=p_{\tilde{X}}$ and $p_{Y|X=x}\sqsubseteq p_{\tilde{Y}|\tilde{X}=x}$
for all $x$, then $p_{X,Y}\sqsubseteq p_{\tilde{X},\tilde{Y}}$.
\item If $p_{X}=p_{\tilde{X}}$ and $p_{Y|X=x}\preceq p_{\tilde{Y}|\tilde{X}=x}$
for all $x$, then $p_{X,Y}\preceq p_{\tilde{X},\tilde{Y}}$.
\end{itemize}
\end{prop}
\begin{IEEEproof}
Assume $p_{X}=p_{\tilde{X}}$ and $p_{Y|X=x}\sqsubseteq p_{\tilde{Y}|\tilde{X}=x}$
for all $x$. There exists functions $g_{x}$ for $x\in\mathrm{supp}(p_{X})$
such that $(X,g_{X}(Y))\stackrel{d}{=}(\tilde{X},\tilde{Y})$. Hence,
$p_{X,Y}\sqsubseteq p_{\tilde{X},\tilde{Y}}$ with the aggregation
map $(x,y)\mapsto(x,g_{x}(y))$.

Assume $p_{X}=p_{\tilde{X}}$ and $p_{Y|X=x}\preceq p_{\tilde{Y}|\tilde{X}=x}$
for all $x$. Fix any $A\subseteq\mathrm{supp}(p_{X,Y})$. For any
$x$, let $B_{x}$ attains the maximum in $\max_{B\subseteq\mathrm{supp}(p_{\tilde{Y}|\tilde{X}=x}):\,|B|\le|\{y:\,(x,y)\in A\}|}q(B)$.
Since $p_{Y|X=x}\preceq p_{\tilde{Y}|\tilde{X}=x}$, we have
\begin{align*}
p_{X,Y}(A) & =\sum_{x}p_{X}(x)\sum_{y:\,(x,y)\in A}p_{Y|X=x}(y)\\
 & \le\sum_{x}p_{X}(x)p_{\tilde{Y}|\tilde{X}=x}(B_{x})\\
 & =p_{\tilde{X},\tilde{Y}}\left(\left\{ (x,y):\,y\in B_{x}\right\} \right).
\end{align*}
Since $|\{(x,y):\,y\in B_{x}\}|\le|A|$, we have $p_{X,Y}\preceq p_{\tilde{X},\tilde{Y}}$.
\end{IEEEproof}
We then present the definition of the greatest lower bound (see e.g.
\cite{cicalese2002supermodularity} for the finite case):
\begin{defn}
For a set of pmf's $S$ where
\begin{equation}
\lim_{k\to\infty}\inf_{p\in S}\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k}p(A)=1,\label{eq:glb_cond}
\end{equation}
define its \emph{greatest lower bound with respect to majorization},
written as $q=\bigwedge S$, as a pmf $q$ over $\mathbb{N}$ given
by
\[
q(k):=\inf_{p\in S}\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k}p(A)-\inf_{p\in S}\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k-1}p(A).
\]
If \eqref{eq:glb_cond} is not satisfied, then $\bigwedge S$ is undefined.
(Note that \eqref{eq:glb_cond} is always satisfied when $S$ is finite.)
\end{defn}
If $S$ contains pmf's over the set $\mathcal{X}$, and $|S|=m<\infty$,
$|\mathcal{X}|=n<\infty$, then it is clear that $\bigwedge S$ can
be computed in $O(mn\log n)$ time (by sorting the pmf's and computing
partial sums). We give some properties of the greatest lower bound.
Note that the case $|S|=2$ has been shown in \cite{cicalese2002supermodularity}.
While it is straightforward to generalize it to $|S|>2$ and $|S|=\infty$,
we state these properties for the sake of completeness.
\begin{prop}
For a set of pmf's $S$, if $q=\bigwedge S$ exists, then
\begin{enumerate}
\item $q(1)\ge q(2)\ge q(3)\ge\cdots$.
\item $q\preceq p$ for any $p\in S$.
\item For any $\tilde{q}$ such that $\tilde{q}\preceq p$ for any $p\in S$,
we have $\tilde{q}\preceq q$.
\end{enumerate}
\end{prop}
\begin{IEEEproof}
Note that $q(1)\ge q(2)\ge\cdots$ is equivalent to the concavity
of $\inf_{p\in S}\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k}p(A)$
in $k$, which holds because the infimum of concave functions is concave.
We have $\sum_{i=1}^{k}q(i)=\inf_{p\in S}\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k}p(A)$,
and hence $q\preceq p$ for any $p\in S$. For any $\tilde{q}$ such
that $\tilde{q}\preceq p$ for any $p\in S$, we have $\max_{A\subseteq\mathrm{supp}(\tilde{q}):\,|A|\le k}\tilde{q}(A)\le\inf_{p\in S}\max_{A\subseteq\mathrm{supp}(p):\,|A|\le k}p(A)=\sum_{i=1}^{k}q(i)$,
and hence $\tilde{q}\preceq q$.
\end{IEEEproof}
As a result of these properties, if $\bigwedge S$ exists, for any
$q\in\tilde{\Gamma}(S)$, we have $q\preceq\bigwedge S$, and hence
$H_{\alpha}(q)\ge H_{\alpha}(\bigwedge S)$. Therefore, $H_{\alpha}^{*}(S)\ge H_{\alpha}(\bigwedge S)$.

\medskip{}

\section{Coupling by Geometric Splitting\label{sec:geomsplit}}

We now present the main result in this paper, which shows that if
the pmf's $p,q$ satisfy $q\preceq p$, then after splitting each
mass $q(y)$ into a sequence of masses $q(y)/2$, $q(y)/4$, $q(y)/8$,...
(or equivalently, consider the joint pmf of $(Y,Z)$ where $Y\sim q$
is independent of $Z\sim\mathrm{Geom}_{1/2}$), then $p$ will be
an aggregation of the resultant pmf $q\times\mathrm{Geom}_{1/2}$
(``$\times$'' denote the independent product of two pmf's, i.e.,
it is the pmf of $(Y,Z)$ mentioned before; refer to the notation
section for the definition), which we call the \emph{geometric splitting}
of $q$.
\begin{thm}
\label{thm:geomsplit}If $q\preceq p$, then
\[
q\times\mathrm{Geom}_{1/2}\sqsubseteq p.
\]
\end{thm}
\medskip{}

A direct result of this theorem is the following explicit formula
of an underlying distribution of a coupling.
\begin{cor}
\label{cor:cp_geom_bd}For a set of pmf's $S$, if $\bigwedge S$
exists, then
\[
\Big(\bigwedge S\Big)\times\mathrm{Geom}_{1/2}\in\tilde{\Gamma}(S).
\]
As a result, the minimum R\'enyi entropy of couplings of $S$ satisfies
\[
H_{\alpha}\Big(\bigwedge S\Big)\le H_{\alpha}^{*}(S)\le H_{\alpha}\Big(\bigwedge S\Big)+H_{\alpha}(\mathrm{Geom}_{1/2}),
\]
where
\[
H_{\alpha}(\mathrm{Geom}_{1/2})=\begin{cases}
\infty & \mathrm{if}\;\alpha=0\\
2 & \mathrm{if}\;\alpha=1\\
1 & \mathrm{if}\;\alpha=\infty\\
\frac{-\alpha-\log(1-2^{-\alpha})}{1-\alpha} & \mathrm{otherwise}
\end{cases}
\]
is the R\'enyi entropy of $\mathrm{Geom}_{1/2}$.
\end{cor}
\medskip{}

Another way to state Theorem \ref{thm:geomsplit} is that for any
pmf $q$, we have $q\times\mathrm{Geom}_{1/2}\in\tilde{\Gamma}(\{p\;\text{pmf over}\;\mathbb{N}:\,q\preceq p\})$.

Before we prove Theorem \ref{thm:geomsplit}, we present a lemma similar
to the alias method \cite{walker1977efficient}, and is a special
case of the algorithm in \cite{cicalese2019minimum}. We include a
proof of the claim for the sake of completeness, and describe a linear
time algorithm (Algorithm \ref{alg:alias}) which is considerably
simpler than that in \cite{cicalese2019minimum}.
\begin{lem}
\label{lem:alias}For any pmf's $p,q$ over $[n]$ such that $q\preceq p$,
$p(1)\ge p(2)\ge\cdots\ge p(n)$ and $q(1)\ge q(2)\ge\cdots\ge q(n)$,
there exists $a_{x}\in[x-1]$ and $0\le r_{x}\le q(x)$ for $x=2,\ldots,n$
such that
\begin{equation}
p(x)=q(x)-r_{x}+\sum_{y:\,a_{y}=x}r_{y}\label{eq:stoc_tr}
\end{equation}
for any $x=1,\ldots,n$ (we let $r_{1}=0$). Moreover, $a_{x},r_{x}$
can be computed in $O(n)$ time (see Algorithm \ref{alg:alias}).
\end{lem}
Lemma \ref{lem:alias} can be stated in the following more compact
form using matrices. For any pmf's $p,q$ over $[n]$ sorted in descending
order such that $q\preceq p$, there exists a lower triangular right
stochastic matrix $M$ where each row has at most one positive off-diagonal
entry, and the probability vectors satisfy $\vec{p}=\vec{q}M$. Its
equivalence to Lemma \ref{lem:alias} can be shown by letting $M_{x,a_{x}}=r_{x}/q(x)$
and $M_{x,x}=1-r_{x}/q(x)$ for $x\in[n]$ (all other entries of $M$
are zeros).
\begin{IEEEproof}
[Proof of Lemma \ref{lem:alias}]Let $B_{x}:=\{y\in[n]:\,a_{y}=x\}$.
We give $B_{x}$ and $r_{x}$ recursively. Take $B_{n}=\emptyset$,
$r_{n}=q(n)-p(n)$ ($r_{n}\ge0$ since $q\preceq p$). Assume $B_{x+1},\ldots,B_{n}$
and $r_{x+1},\ldots,r_{n}$ are defined and satisfies that $B_{x+1},\ldots,B_{n}$
are disjoint, $B_{x'}\subseteq\{x'+1,\ldots,n\}$ and
\begin{equation}
p(x')=q(x')-r_{x'}+\sum_{y\in B_{x'}}r_{y}\label{eq:ind_hyp}
\end{equation}
for all $x'>x$. We now define $B_{x},r_{x}$. Take
\[
B_{x}=\{t,\ldots,n\}\backslash\bigcup_{y=x+1}^{n}B_{y},
\]
where $t\in\{x+1,\ldots,n+1\}$ such that $\sum_{y\in B_{x}}r_{y}\in[p(x)-q(x),\,p(x)]$.
Such $t$ exists since $r_{y}\le q(y)\le q(x)$ for $y>x$, and
\begin{align*}
 & \sum_{y\in\{x+1,\ldots,n\}\backslash\bigcup_{y'=x+1}^{n}B_{y'}}r_{y}\\
 & =\sum_{y=x+1}^{n}r_{y}-\sum_{y'=x+1}^{n}\sum_{y\in B_{y'}}r_{y}\\
 & \stackrel{(a)}{=}\sum_{y=x+1}^{n}r_{y}-\sum_{y=x+1}^{n}\left(p(y)-q(y)+r_{y}\right)\\
 & =\sum_{y=x+1}^{n}\left(q(y)-p(y)\right)\\
 & =p(x)-q(x)+\sum_{y=x}^{n}\left(q(y)-p(y)\right)\\
 & \ge p(x)-q(x)
\end{align*}
since $q\preceq p$, where (a) is by \eqref{eq:ind_hyp}, and hence
when $t$ decreases from $n+1$ to $x+1$, $\sum_{y\in B_{x}}r_{y}$
increases from $0$ to $\ge p(x)-q(x)$, with step size at most $q(x)$,
and thus there exists $t$ such that $\sum_{y\in B_{x}}r_{y}\in[p(x)-q(x),\,p(x)]$.
In practice, to find $B_{x}$, we only need to scan the elements in
$\bar{B}_{x}:=[n]\backslash\bigcup_{y=x+1}^{n}B_{y}$ in decreasing
order, and add elements from $\bar{B}_{x}$ to $B_{x}$ until $\sum_{y\in B_{x}}r_{y}\ge p(x)-q(x)$.
We then take
\[
r_{x}=q(x)-p(x)+\sum_{y\in B_{x}}r_{y}.
\]
Therefore, we have defined $B_{x},r_{x}$ (and hence $a_{x}$) recursively.

For the running time complexity, note that since $\bar{B}_{x}$ is
decreasing as $x$ decreases, only the elements in $\bar{B}_{x}\backslash\bar{B}_{x-1}$
are relevant to the computation of $B_{x},r_{x}$. Since each $y\in[n]$
can only be removed from $\bar{B}_{x}$ once (i.e., $y\in\bar{B}_{x}\backslash\bar{B}_{x-1}$
for at most one $x$), the overall time complexity is $O(n)$. Also
note that the $B_{x}$ produced by this method must be contiguous
segments of integers, and $\bar{B}_{x}$ must be in the form $\{1,\ldots,|\bar{B}_{x}|\}$,
which allows simpler implementations (e.g. we only need to store $b_{x}:=|\bar{B}_{x}|$
instead of $\bar{B}_{x}$). Refer to Algorithm \ref{alg:alias} (which
we call the \emph{majorized alias} algorithm) for the precise description.
\end{IEEEproof}
\begin{algorithm}[H]
\textbf{$\;\;\;\;$Input:} pmf's $p,q$ over $[n]$ such that $q\preceq p$,
$p(1)\ge\cdots\ge p(n)$ and $q(1)\ge\cdots\ge q(n)$

\textbf{$\;\;\;\;$Output:} $a_{x},r_{x}$ for $x\in\{2,\ldots,n\}$

\smallskip{}

\begin{algorithmic}

\State{$b\leftarrow n$}

\State{$a_{x}\leftarrow1$ for $x=2,\ldots,n$}

\For{$x\leftarrow n,n-1,\ldots,2$}

\State{$r_{x}\leftarrow q(x)-p(x)$}

\While{$r_{x}<0$}

\State{$r_{x}\leftarrow r_{x}+r_{b}$}

\State{$a_{b}\leftarrow x$}

\State{$b\leftarrow b-1$}

\EndWhile

\EndFor

\Return{$\{a_{x}\},\{r_{x}\}$}

\end{algorithmic}

\caption{\label{alg:alias}$\textsc{MajorizedAlias}(p,q)$}
\end{algorithm}

Algorithm \ref{alg:alias} has time complexity $O(n)$ since the block
inside the while loop is executed at most $n$ times ($b$ decreases
each time it is executed). We remark that Algorithm \ref{alg:alias}
reduces to the alias method \cite{walker1977efficient} when $q$
is the uniform distribution. The alias method is an efficient algorithm
that can generate a random number following the distribution $p$
over $[n]$, using a uniformly random integer in $[n]$ and a uniformly
random real number in $[0,1]$. The alias method requires an $O(n)$
(or $O(n\log n)$ if $p$ is unsorted and needs to be sorted first)
precomputation time to compute $a_{x}$ and $r_{x}$ satisfying \eqref{eq:stoc_tr}
(where $q$ is the uniform distribution over $[n]$). After the precomputation,
each sample of $x\sim p$ can be generated in constant time by first
generating $y\sim q$ independent of $z\sim\mathrm{Unif}[0,1]$, and
then outputting $x=y$ if $z\ge r_{y}/q(y)$, $x=a_{y}$ if $z<r_{y}/q(y)$.
While the alias method focuses only on the case where $q$ is uniform
(which guarantees $q\preceq p$), Algorithm \ref{alg:alias} generalizes
it to any $q$ satisfying $q\preceq p$.

Table \ref{tab:alias_eg} shows Algorithm \ref{alg:alias} applied
on $\vec{p}=[0.37,0.36,0.25,0.02,0]$, $\vec{q}=[0.3,0.3,0.2,0.1,0.1]$.
The values of $\{r_{y}\},\{a_{y}\}$ for each iteration $x=5,4,3,2,1$
in the algorithm are given. The red cells are cells with positions
$y$ in the interval $y\in[x..b]$, which are unfinished cells with
$r_{y}$ (the excess amount) computed, but $a_{y}$ is not computed
yet, i.e., it is not yet known where the excess amount $r_{y}$ will
be allocated (while Algorithm \ref{alg:alias} initializes $a_{y}$
to $1$, here we assume $a_{y}$ is initialized to be undefined for
the sake of clarity). The green cells are cells $y$ in the interval
$y\in[b+1\,..\,n]$, which are finished cells with $r_{y},a_{y}$
computed. At each iteration $x$, we keep allocating the the excess
amount of the right-most red cell to the current cell $x$ (and change
the right-most red cell to green), until $q(x)$ plus the total excess
amount allocated to the current cell is at least $p(x)$. The amount
in excess ($q(x)$ plus the total excess amount allocated to the current
cell minus $p(x)$) is written to the $r_{x}$ of the current cell.

\begin{table}
\begin{centering}
\begin{tabular}{|c|c|c|c|c|c|c|}
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
\multicolumn{1}{c|}{} & $y$ & 1 & 2 & 3 & 4 & 5\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
\multicolumn{1}{c|}{} & $p(y)$ & 0.37 & 0.36 & 0.25 & 0.02 & 0\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
\multicolumn{1}{c|}{} & $q(y)$ & 0.3 & 0.3 & 0.2 & 0.1 & 0.1\tabularnewline
\hline
\multirow{2}{*}{Step $x=5$} & $r_{y}$ &  &  &  &  & \cellcolor{lightred}0.1\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
 & $a_{y}$ &  &  &  &  & \cellcolor{lightred}\tabularnewline
\hline
\hline
\multirow{2}{*}{Step $x=4$} & $r_{y}$ &  &  &  & \cellcolor{lightred}0.08 & \cellcolor{lightred}0.1\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
 & $a_{y}$ &  &  &  & \cellcolor{lightred} & \cellcolor{lightred}\tabularnewline
\hline
\hline
\multirow{2}{*}{Step $x=3$} & $r_{y}$ &  &  & \cellcolor{lightred}0.05 & \cellcolor{lightred}0.08 & \cellcolor{lightgreen}0.1\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
 & $a_{y}$ &  &  & \cellcolor{lightred} & \cellcolor{lightred} & \cellcolor{lightgreen}3\tabularnewline
\hline
\hline
\multirow{2}{*}{Step $x=2$} & $r_{y}$ &  & \cellcolor{lightred}0.02 & \cellcolor{lightred}0.05 & \cellcolor{lightgreen}0.08 & \cellcolor{lightgreen}0.1\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
 & $a_{y}$ &  & \cellcolor{lightred} & \cellcolor{lightred} & \cellcolor{lightgreen}2 & \cellcolor{lightgreen}3\tabularnewline
\hline
\hline
\multirow{2}{*}{Step $x=1$} & $r_{y}$ & \cellcolor{lightred}0 & \cellcolor{lightgreen}0.02 & \cellcolor{lightgreen}0.05 & \cellcolor{lightgreen}0.08 & \cellcolor{lightgreen}0.1\tabularnewline
\cline{2-7} \cline{3-7} \cline{4-7} \cline{5-7} \cline{6-7} \cline{7-7}
 & $a_{y}$ & \cellcolor{lightred} & \cellcolor{lightgreen}1 & \cellcolor{lightgreen}1 & \cellcolor{lightgreen}2 & \cellcolor{lightgreen}3\tabularnewline
\hline
\end{tabular}
\par\end{centering}
\medskip{}

\caption{\label{tab:alias_eg}Algorithm \ref{alg:alias} applied on $\vec{p}=[0.37,0.36,0.25,0.02,0]$,
$\vec{q}=[0.3,0.3,0.2,0.1,0.1]$.}

\end{table}

\[
\]
We now give a sketch of the proof of Theorem \ref{thm:geomsplit}.
Let $q\preceq p$ with $p(1)\ge p(2)\ge\cdots$ and $q(1)\ge q(2)\ge\cdots$.
Assume they have finite support for now, and consider them as probability
vectors $\vec{p},\vec{q}\in\mathbb{R}^{n}$. To show $q\times\mathrm{Geom}_{1/2}\sqsubseteq p$,
it is equivalent to show that there exist right stochastic matrices
$M_{1},M_{2},\ldots$ with $\{0,1\}$ entries such that $\vec{p}=\sum_{i=1}^{\infty}2^{-i}\vec{q}M_{i}$.
By Lemma \ref{lem:alias}, we have $\vec{p}=\vec{q}M$ for a stochastic
matrix $M$ where each row has at most one positive off-diagonal entry.
For row $x$ with off-diagonal entry $M_{x,a_{x}}=r_{x}/q(x)$, consider
the binary representation of $r_{x}/q(x)$, and put a ``1'' at the
position $(x,a_{x})$ of $M_{j}$ if the $j$-th digit after the decimal
point of the binary representation is ``1'' (otherwise put a ``1''
at the position $(x,x)$) for $j=1,2,\ldots$. This ensures that $M=\sum_{i=1}^{\infty}2^{-i}M_{i}$,
and hence the requirement is satisfied. The following is the complete
proof for the case where the support size may be infinite.
\begin{IEEEproof}
Without loss of generality, assume $p,q$ are pmf's over $\mathbb{N}$
with $p(1)\ge p(2)\ge\cdots$ and $q(1)\ge q(2)\ge\cdots$. Define
pmf $q_{l}$ by
\[
q_{l}(x)=\begin{cases}
q(x) & \mathrm{if}\;x<l\\
\sum_{y=l}^{\infty}q(y) & \mathrm{if}\;x=l\\
0 & \mathrm{if}\;x>l.
\end{cases}
\]
Define $p_{l}$ similarly. Since $p\sqsubseteq p_{l}$, we have $q\preceq p\preceq p_{l}$.
Fix $l$ and let $n>l$ be large enough that $\sum_{y=n}^{\infty}q(y)\le q(l)$,
and hence the $l$ largest entries of $q_{n}$ are the same as the
$l$ largest entries of $q$. Since $p_{l}$ has at most $l$ nonzero
entries, whether $q\preceq p_{l}$ holds only depend on the $l$ largest
entries of $q$. Hence, we have $q_{n}\preceq p_{l}$. By Lemma \ref{lem:alias},
there exists $a_{x}\in[n]\backslash\{x\}$ (we no longer have $a_{x}<x$
since we have to sort $q_{n}(x)$ in descending order before applying
the lemma) and $r_{x}\in[0,q_{n}(x)]$ for $x=1,\ldots,n$ such that
\[
p_{l}(x)=q_{n}(x)-r_{x}+\sum_{y\in[n]:\,a_{y}=x}r_{y}
\]
for any $x=1,\ldots,n$. Define a mapping $g:\mathrm{supp}(q_{n})\times\mathbb{N}\to[n]$
by
\[
g(x,i)=\begin{cases}
x & \mathrm{if}\;2^{i}r_{x}/q_{n}(x)\;\mathrm{mod}\,2<1\\
a_{x} & \mathrm{if}\;2^{i}r_{x}/q_{n}(x)\;\mathrm{mod}\,2\ge1,
\end{cases}
\]
where $a\,\mathrm{mod}\,b:=a-b\lfloor a/b\rfloor$. Since $\sum_{i=1}^{\infty}2^{-i}\mathbf{1}\{2^{i}r_{x}/q_{n}(x)\;\mathrm{mod}\,2\ge1\}=r_{x}/q_{n}(x)$
is the binary representation of $r_{x}/q_{n}(x)$, we have $\mathbf{P}(g(x,Z)=a_{x})=r_{x}/q_{n}(x)$
and $\mathbf{P}(g(x,Z)=x)=1-r_{x}/q_{n}(x)$, where $Z\sim\mathrm{Geom}_{1/2}$.
Let $X\sim q_{n}$ independent of $Z$, we have
\begin{align*}
 & \mathbf{P}(g(X,Z)=x)\\
 & =q_{n}(x)\mathbf{P}(g(x,Z)=x)+\sum_{y\in[n]:\,a_{y}=x}q_{n}(y)\mathbf{P}(g(y,Z)=a_{y})\\
 & =q_{n}(x)-r_{x}+\sum_{y\in[n]:\,a_{y}=x}r_{y}\\
 & =p_{l}(x),
\end{align*}
and hence $q_{n}\times\mathrm{Geom}_{1/2}\sqsubseteq p_{l}$. Since
$q\sqsubseteq q_{n}$, we have $q\times\mathrm{Geom}_{1/2}\sqsubseteq q_{n}\times\mathrm{Geom}_{1/2}\sqsubseteq p_{l}$
by Proposition \ref{prop:kernel_maj}. Since $p_{l}(x)\to p(x)$ as
$l\to\infty$ for any $x\in\mathbb{N}$, by Proposition \ref{prop:agg_closed},
we have $q\times\mathrm{Geom}_{1/2}\sqsubseteq p$.
\end{IEEEproof}
\[
\]

Note that $(\bigwedge S)\times\mathrm{Geom}_{1/2}$ in Theorem \ref{thm:geomsplit}
has an infinite support size or cardinality. If $S$ is finite and
the pmf's in $S$ are over a set $\mathcal{X}$ which is finite, then
we can reduce the cardinality to $|S|(|\mathcal{X}|-1)+1$ (without
increasing its R\'enyi entropy), as given in the following theorem.
\begin{thm}
\label{thm:finite}For a finite set of pmf's $S$ with $|S|=m$, where
the pmf's in $S$ are over a finite set $\mathcal{X}$ with $|\mathcal{X}|=n$,
there exists a pmf $q\in\tilde{\Gamma}(S)$ with $|\mathrm{supp}(q)|\le m(n-1)+1$
and
\[
\Big(\bigwedge S\Big)\times\mathrm{CGeom}_{1/2,m}\preceq q,
\]
where $\mathrm{CGeom}_{1/2,m}$ is the capped geometric distribution
defined in \eqref{eq:cgeom}. As a result, the minimum R\'enyi entropy
of couplings of $S$ satisfies
\[
H_{\alpha}\Big(\bigwedge S\Big)\le H_{\alpha}^{*}(S)\le H_{\alpha}\Big(\bigwedge S\Big)+H_{\alpha}(\mathrm{CGeom}_{1/2,m}).
\]
Note that $H(\mathrm{CGeom}_{1/2,m})=2-2^{2-m}$. Moreover, $q$ and
the aggregation maps for $q\sqsubseteq p$ for all $p\in S$ can be
computed in $O(m^{2}n+mn\log n)$ time (see Algorithm \ref{alg:finitecp}).
\end{thm}
We remark that the cardinality bound $|\mathrm{supp}(q)|\le m(n-1)+1$
is the same as that in \cite{painsky2013memoryless,kocaoglu2017entropic}.
Therefore, the coupling in Theorem \ref{thm:finite} gives a small
R\'enyi entropy, without penalty on the cardinality.

To prove Theorem \ref{thm:finite}, we first show a lemma about coupling
Bernoulli distributions.
\begin{lem}
\label{lem:bern}For a finite set of pmf's $S$ with $|S|=m$, where
the pmf's in $S$ are over $\{0,1\}$, there exists a pmf $q\in\tilde{\Gamma}(S)$
with $|\mathrm{supp}(q)|\le m+1$ and $\mathrm{CGeom}_{1/2,m+1}\preceq q$.
Moreover, $q$ and the aggregation maps for $q\sqsubseteq p$ for
all $p\in S$ can be computed in $O(m^{2})$ time (see Algorithm \ref{alg:bern}).
\end{lem}
\begin{IEEEproof}
We prove the lemma by induction on $m$. The lemma is true when $m=1$
since $\mathrm{CGeom}_{1/2,2}\preceq p$ for any pmf $p$ over $\{0,1\}$.
We now prove the lemma for $m$, assuming that the lemma is true for
any smaller $m$. Let $S=\{p_{1},\ldots,p_{m}\}$. Without loss of
generality, assume $p_{1}$ attains the minimum of $\max\{p_{i}(0),\,p_{i}(1)\}$
for $i=1,\ldots,m$. Let $\gamma:=\max\{p_{1}(0),\,p_{1}(1)\}$. If
$\gamma=1$, all distributions in $S$ are degenerate, and the lemma
clearly holds, and hence we can assume $\gamma<1$. Let $\tilde{p}_{2},\ldots,\tilde{p}_{m}$
be pmf's over $\{0,1\}$ defined as
\begin{align*}
\tilde{p}_{i}(0) & =\frac{p_{i}(0)-\gamma\mathbf{1}\{p_{i}(0)\ge p_{i}(1)\}}{1-\gamma},\\
\tilde{p}_{i}(1) & =\frac{p_{i}(1)-\gamma\mathbf{1}\{p_{i}(0)<p_{i}(1)\}}{1-\gamma},
\end{align*}
for $i=2,\ldots,m$. Invoke the induction hypothesis to obtain a pmf
$\tilde{q}\in\tilde{\Gamma}(\{\tilde{p}_{2},\ldots,\tilde{p}_{m}\})$
over $[m]$ satisfying $\mathrm{CGeom}_{1/2,m}\preceq\tilde{q}$.
Let $q$ be a pmf over $[m+1]$ with $q(x)=(1-\gamma)\tilde{q}(x)$
for $x\le m$, and $q(m+1)=\gamma$. Since $\gamma\ge1/2$, we have
$\mathrm{CGeom}_{1/2,m+1}\preceq q$. It is left to show that $q\sqsubseteq p_{i}$
for $i=2,\ldots,m$. For $p_{i}$, without loss of generality assume
$p_{i}(0)\ge p_{i}(1)$. Since $\tilde{q}\sqsubseteq\tilde{p_{i}}$,
there exists $A\subseteq[m]$ such that $\tilde{p_{i}}(1)=\sum_{x\in A}\tilde{q}(x)$.
We have
\[
p_{i}(1)=(1-\gamma)\tilde{p}_{i}(1)=\sum_{x\in A}q(x),
\]
and hence $q\sqsubseteq p_{i}$. Refer to Algorithm \ref{alg:bern}
(which we call the \emph{Bernoulli splitting }algorithm) for the precise
description of the algorithm.
\end{IEEEproof}
\begin{algorithm}[H]
\textbf{$\;\;\;\;$Input:} $\rho_{1},\ldots,\rho_{m}\in[0,1]$ (let
$\rho_{i}=p_{i}(1)$)

\textbf{$\;\;\;\;$Output:} $\{q_{x}\}_{x\in[k]}$, $\{g_{i,x}\}_{i\in[m],x\in[k]}$
(where $k\le m+1$)

\textbf{$\;\;\;\;$$\;\;\;\;$}(let $q_{x}=q(x)$, $g_{i,x}=g_{i}(x)\in\{0,1\}$
for the aggregation mapping for $q\sqsubseteq p_{i}$)

\smallskip{}

\begin{algorithmic}

\State{$g_{i,x}\leftarrow0$ for $i\in[m],x\in[k]$}

\State{$c\leftarrow1$}

\State{$k\leftarrow0$}

\While{$c>0$}

\State{$\gamma\leftarrow\min_{i}\max\{\rho_{i},\,c-\rho_{i}\}$}

\State{$k\leftarrow k+1$}

\State{$q_{k}\leftarrow\gamma$}

\For{$i\leftarrow1,\ldots,m$}

\If{$\rho_{i}\ge c/2$}

\State{$g_{i,k}\leftarrow1$}

\State{$\rho_{i}\leftarrow\rho_{i}-\gamma$}

\EndIf

\EndFor

\State{$c\leftarrow c-\gamma$}

\EndWhile

\Return{$\{q_{x}\}$, $\{g_{i,x}\}$}

\end{algorithmic}

\caption{\label{alg:bern}$\textsc{BernoulliSplitting}(\rho_{1},\ldots,\rho_{m})$}
\end{algorithm}

Algorithm \ref{alg:bern} has time complexity $O(m^{2})$, since after
each iteration of the while loop, the number of $i$'s where $\rho_{i}\in\{0,c\}$
increases by at least one (letting $i^{*}=\mathrm{argmin}_{i}\max\{\rho_{i},\,c-\rho_{i}\}$,
then $\rho_{i^{*}}\in\{0,c\}$ after the iteration), and hence the
number of iterations of the while loop is upper bounded by $m+1$.

Figure \ref{fig:bern} shows Algorithm \ref{alg:bern} applied on
$\{\rho_{i}\}_{i}=(0.175,0.35,0.6,0.925)$. The graphs on top are
$\rho_{i}$ at each iteration of the while loop, and the graphs on
the bottom show $q_{x}$ and $g_{i,x}$ at each iteration. We can
consider the problem as finding a set of sticks with lengths $q_{x}$
which sum to $c$ (initially $c=1$), such that every $\rho_{i}$
(we require $0\le\rho_{i}\le c$) is the sum of the lengths of a subset
of sticks. We use the following greedy approach. If the length of
the longest stick is $\gamma$, then every $\rho_{i}$ must satisfy
either $\rho_{i}\ge\gamma$ if we use the stick to form $\rho_{i}$,
or $\rho_{i}\le c-\gamma$ if we do not use the stick to form $\rho_{i}$
(the regions of inadmissible $\rho_{i}$ are shaded in gray on the
graphs on top). Therefore, the longest possible length of the longest
stick is $q_{1}=\gamma=\min_{i}\max\{\rho_{i},\,c-\rho_{i}\}$. For
every $\rho_{i}$ where $\rho_{i}\ge\gamma$ (or equivalently $\rho_{i}\ge c/2$),
we set $g_{i,1}=1$, meaning that we use the first stick to form $\rho_{i}$,
and then set $\rho_{i}\leftarrow\rho_{i}-\gamma$ (the remainder of
the length to be fulfilled by other sticks). We set $g_{i,1}=0$ for
the rest of $\rho_{i}$. Now the remaining total length of sticks
become $c\leftarrow c-\gamma$. Repeat this process until $\rho_{i}=0$
for all $i$.

\begin{figure}
\begin{centering}
\includegraphics[scale=0.85]{fig1}
\par\end{centering}
\caption{\label{fig:bern}Algorithm \ref{alg:bern} applied on $\{\rho_{i}\}_{i}=(0.175,0.35,0.6,0.925)$.}

\end{figure}

\begin{rem}
\label{rem:error}Note that $\sum_{x>L}q(x)\le2^{-L}$ in Lemma \ref{lem:bern}
and Algorithm \ref{alg:bern}. Therefore, stopping Algorithm \ref{alg:bern}
after $L$ steps reduces the time complexity to $O(mL)$, and incurs
an error (in total variation distance) upper bounded by $2^{-L}$,
i.e., it computes a coupling of $\{\mathrm{Bern}(\tilde{\rho}_{i})\}_{i\in[m]}$
instead of $\{\mathrm{Bern}(\rho_{i})\}_{i\in[m]}$, where $|\tilde{\rho}_{i}-\rho_{i}|\le2^{-L}$.
One can also replace ``while $c>0$'' in Algorithm \ref{alg:bern}
to ``while $c>\epsilon$'' (and adjust $q_{x}$ so they sum to 1)
to set the desired error level.
\end{rem}
We now prove Theorem \ref{thm:finite}.
\begin{IEEEproof}
Assume $p_{1},\ldots,p_{m}$ are pmf's over $[n]$ with $p_{i}(1)\ge\cdots\ge p_{i}(n)$.
Let $S=\{p_{1},\ldots,p_{m}\}$, $\bar{q}:=\bigwedge S$. By Lemma
\ref{lem:alias}, let $a_{i,x}\in[x-1]$ and $r_{i,x}\in[0,\bar{q}(x)]$
for $x=2,\ldots,n$, $i=1,\ldots,m$ such that
\begin{equation}
p_{i}(x)=\bar{q}(x)-r_{i,x}+\sum_{x':\,a_{i,x'}=x}r_{i,x'}\label{eq:alias_rec}
\end{equation}
for any $x=1,\ldots,n$, $i=1,\ldots,m$ (let $r_{i,1}=0$).

Fix any $2\le x\le n$. Since
\[
\sum_{z=1}^{x-1}\bar{q}(z)=\min_{i\in[m]}\sum_{z=1}^{x-1}p_{i}(z),
\]
there exists $j$ such that $\sum_{z=1}^{x-1}\bar{q}(z)=\sum_{z=1}^{x-1}p_{j}(z)$,
and hence by \eqref{eq:alias_rec} and $a_{j,x}<x$,
\begin{align*}
0 & =\sum_{z=1}^{x-1}\left(p_{j}(z)-\bar{q}(z)\right)\\
 & =\sum_{z=1}^{x-1}\left(\bar{q}(z)-r_{j,z}+\sum_{x':\,a_{j,x'}=z}r_{j,x'}-\bar{q}(z)\right)\\
 & =-\sum_{z=1}^{x-1}r_{j,z}+\sum_{z=1}^{x-1}\sum_{x':\,a_{j,x'}=z}r_{j,x'}\\
 & =\sum_{z=1}^{x-1}\sum_{x'\ge x:\,a_{j,x'}=z}r_{j,x'}\\
 & \ge r_{j,x},
\end{align*}
which means there exists $j$ such that $r_{j,x}=0$. Applying Lemma
\ref{lem:bern} on $\mathrm{Bern}_{r_{i,x}/\bar{q}(x)}$ for $i\neq j$,
let $\tilde{q}_{x}\in\tilde{\Gamma}(\{\mathrm{Bern}_{r_{i,x}/\bar{q}(x)}\}_{i\in[m]\backslash\{j\}})$
be a pmf over $[m]$ with $\mathrm{CGeom}_{1/2,m}\preceq\tilde{q}_{x}$.
We have $\tilde{q}_{x}\sqsubseteq\mathrm{Bern}_{r_{i,x}/\bar{q}(x)}$
for any $i\in[m]$ (this trivially holds when $i=j$).

For $x=1$, let $\tilde{q}_{1}(1)=1$. Since $r_{i,1}=0$, we have
$\tilde{q}_{1}\in\tilde{\Gamma}(\{\mathrm{Bern}_{r_{i,1}/\bar{q}(1)}\}_{i\in[m]\backslash\{j\}})$
and $\mathrm{CGeom}_{1/2,m}\preceq\tilde{q}_{1}$.

Let $q$ be a pmf over $[n]\times[m]$ defined by $q(x,y):=\bar{q}(x)\tilde{q}_{x}(y)$.
For any $i\in[m]$, since $\tilde{q}_{x}\sqsubseteq\mathrm{Bern}_{r_{i,x}/\bar{q}(x)}$,
there exists $A_{x}\subseteq[m]$ such that $\sum_{y\in A_{x}}\tilde{q}_{x}(y)=r_{i,x}/\bar{q}(x)$.
Hence,
\begin{align*}
p_{i}(x) & =\bar{q}(x)-r_{i,x}+\sum_{x'\in[n]:\,a_{i,x'}=x}r_{i,x'}\\
 & =\bar{q}(x)-\bar{q}(x)\sum_{y\in A_{x}}\tilde{q}_{x}(y)+\sum_{x'\in[n]:\,a_{i,x'}=x}\bar{q}(x')\sum_{y\in A_{x'}}\tilde{q}_{x'}(y)\\
 & =\sum_{y\in[m+1]\backslash A_{x}}q(x,y)+\sum_{x'\in[n]:\,a_{i,x'}=x}\sum_{y\in A_{x'}}q(x',y),
\end{align*}
and thus
\[
g_{i}(x,y):=\begin{cases}
x & \mathrm{if}\;y\notin A_{x}\\
a_{i,x} & \mathrm{if}\;y\in A_{x}
\end{cases}
\]
is an aggregation map for $q\sqsubseteq p_{i}$. Since $|\mathrm{supp}(\tilde{q}_{1})|=1$,
we have $|\mathrm{supp}(q)|\le m(n-1)+1$. We have $\bar{q}\times\mathrm{CGeom}_{1/2,m}\preceq q$
by Proposition \ref{prop:kernel_maj}. Refer to Algorithm \ref{alg:finitecp}
for the precise description of the algorithm.
\end{IEEEproof}
\begin{algorithm}[H]
\textbf{$\;\;\;\;$Input:} pmf's $p_{1},\ldots,p_{m}$ over $[n]$
with $p_{i}(1)\ge\cdots\ge p_{i}(n)$

\textbf{$\;\;\;\;$Output:} $\{q_{x}\}_{x\in[k]}$, $\{g_{i,x}\}_{i\in[m],x\in[k]}$
(where $k\le m(n-1)+1$)

\textbf{$\;\;\;\;$$\;\;\;\;$}(let $q_{x}=q(x)$ for pmf $q$ over
$[k]$,

\textbf{$\;\;\;\;$$\;\;\;\;$ }$g_{i,x}=g_{i}(x)\in[n]$ for the
aggregation mapping for $q\sqsubseteq p_{i}$)

\smallskip{}

\begin{algorithmic}

\State{$\bar{q}\leftarrow\bigwedge_{i\in[m]}p_{i}$}

\For{$i\leftarrow1,\ldots,m$}

\State{$\{a_{i,x}\}_{x=2,\ldots,n},\{r_{i,x}\}_{x=2,\ldots,n}\leftarrow\textsc{MajorizedAlias}(p_{i},\bar{q})$}

\State{$r_{i,1}\leftarrow0$}

\EndFor

\State{$k\leftarrow0$}

\For{$x\leftarrow1,\ldots,n$}

\State{$\{\tilde{q}_{y}\}_{y\in[\tilde{k}]},\{\tilde{g}_{i,y}\}_{i\in[m],y\in[\tilde{k}]}\leftarrow\textsc{BernoulliSplitting}(\{r_{i,x}/\bar{q}(x)\}_{i\in[m]})$}

\State{$q_{k+y}\leftarrow\bar{q}(x)\tilde{q}_{y}$ for $y\in[\tilde{k}]$}

\State{$g_{i,k+y}\leftarrow\mathbf{1}\{\tilde{g}_{i,y}=0\}x+\mathbf{1}\{\tilde{g}_{i,y}=1\}a_{i,x}$
for $i\in[m]$, $y\in[\tilde{k}]$}

\State{$k\leftarrow k+\tilde{k}$}

\EndFor

\Return{$\{q_{x}\},\{g_{i,x}\}$}

\end{algorithmic}

\caption{\label{alg:finitecp}$\textsc{ComputeCoupling}(p_{1},\ldots,p_{m})$}
\end{algorithm}

\medskip{}

\begin{rem}
\label{rem:error_cp}If we perform the modification in Remark \ref{rem:error}
(stopping Algorithm \ref{alg:bern} after $L$ steps), it would reduce
the time complexity of Algorithm \ref{alg:finitecp} to $O(mnL+mn\log n)$,
the support bound to $|\mathrm{supp}(q)|\le(L+1)(n-1)+1$, but incur
an error (in total variation distance on each $p\in S$) upper bounded
by $2^{-L}$, i.e., it computes a coupling of $\{\tilde{p}_{i}\}_{i\in[n]}$
instead of $\{p_{i}\}_{i\in[n]}$, where $d_{\mathrm{TV}}(p_{i},\tilde{p}_{i})\le2^{-L}$.
In practical implementations, setting $L\approx60$ will make the
error negligible compared to floating-point error. Therefore, the
practical running time complexity of Algorithm \ref{alg:finitecp}
is close to $O(mn\log n)$.
\end{rem}

\medskip{}

\section{Acknowledgement}

The author acknowledges support from the Direct Grant for Research,
The Chinese University of Hong Kong. The author would like to thank
the anonymous reviewers for their insightful remarks. In particular,
the author thanks an anonymous reviewer for the suggestion to consider
expressing Lemma \ref{lem:alias} in matrix form.

\medskip{}

\[
\]

\bibliographystyle{IEEEtran}


\section{Introduction}

The loop-erased random walk (LERW) introduced by Lawler in \cite{Lawler1980} and further developed  in \cite{Lawler2006,LawlerLimic2010,Lawler2018} can in two dimensions be described by  Schramm-L\"owner evolution (SLE)
at $\kappa=2$ \cite{LawlerSchrammWerner2004}. It corresponds to a conformal field theory with central charge $c=-2$. When reformulated in terms of loops, the lattice $\ca O(n)$-model can be defined for $-2\le n\le 2$ \cite{Nienhuis1982} (see also \cite{LH2008}, page 187).  In the limit of $n\to -2$ it has the same conformal field theory (CFT), a relation which also holds   off criticality
\cite{BauerBernardKytola2008}, suggesting the description by one   complex fermion.
The latter, however, does not allow one to assess properties of loop-erased random walks after erasure.
 Recently, it was established \cite{WieseFedorenko2018,WieseFedorenko2019} that the equivalence to the $\ca O(n)$ model holds in any dimension $d\ge 2$, by mapping loop-erased random walks onto the $n$-component $\phi^4$-theory for $n\to -2$, or equivalently a theory with two complex fermions, and one complex boson.
 This formulation contains information about the traces of LERWs after erasure, and in particular its Hausdorff dimension.

A rigorous   proof of the equivalence between LERWs and the $\ca O(-2)$ model was given in Ref.~\cite{HelmuthShapira2020}.  The ensuing field theory   uses   {\em nilpotent} bosonic fields, i.e.\ fields which square to zero. Here we give a simplified proof of the equivalence. We also show that the nilpotent bosonic fields can be replaced by standard bosonic ones. The ensuing field theory  has two complex fermions, and one (standard) complex boson, and allows to evaluate, without approximations, observables defined on an arbitrary graph. Keeping only the most relevant terms, it reduces to the $\phi^4$-type theory with two fermions and one boson   given in  Refs.~\cite{WieseFedorenko2018,WieseFedorenko2019}. Interestingly, there this formulation arose naturally in the analysis of  charge-density waves subject to quenched disorder.
As a result, the theories discussed here are linked to such diverse systems as charge-density waves, Abelian sandpiles \cite{NarayanMiddleton1994,FedorenkoLeDoussalWiese2008a}, uniform spanning trees  \cite{Majumdar1992,Dhar2006}, the Potts model \cite{Majumdar1992},   Laplacian walk \cite{LyklemaEvertszPietronero1986,Lawler2006}, and dielectric breakdown \cite{NiemeyerPietroneroWiesmann1984}.

\section{The loop-erased random walk}\label{sec:lerw}

Consider a random walk on a directed graph $\ca G$.  The walk jumps from vertex $x$ to $y$ with rate $\beta_{xy}$, and dies out with rate $\lambda_x = m_x^2$. The coefficients $\{\beta_{xy}\}_{x,y\in \ca G}$ are weights on the graph. In particular, when  $\beta_{xy}$ is positive, $\ca G$ contains an edge from $x$ to $y$. Denote by $r_x = \lambda_x+\sum_y \beta_{xy} $ the total rate at which the walk exits from vertex $x$.

We define a {\em path} to be  a sequence of vertices, denoted $\omega=(\omega_1,\dots,\omega_n)$. We refer to $i=1,...,n$ as {\em time}.
The probability $\bb P(\omega)$ that the random walk selects the path $\omega$ and then stops is
\begin{eqnarray}\label{1}
\bb P(\omega) &=& \frac{\lambda_{\omega_n}}{r_{\omega_n}} q(\omega ) , \\
 q(\omega ) &=& \frac{\beta_{\omega_1 \omega_2}}{r_{\omega_1}} \frac{\beta_{\omega_2 \omega_3}}{r_{\omega_2}}  \dots \frac{\beta_{\omega_{n-1} \omega_n}}{r_{\omega_{n-1}}}.
\end{eqnarray}
We call  $q(\omega)$  the {\em   weight function}.
Note that this weight factorizes: If $\omega^{(a)} = (\omega_1^{(a)},\dots,\omega_n^{(a)})$ and $\omega^{(b)} = (\omega_1^{(b)},\dots,\omega_m^{(b)})$, with $\omega_n^{(a)} = \omega_1^{(b)}$, then the composition
 $ \omega := \omega^{(a)} \circ \omega^{(b)} = (\omega_1^{(a)},\dots,\omega_n^{(a)} =\omega_1^{(b)},\dots,\omega_m^{(b)})$
 of the paths $ \omega^{(a)}$ and  $ \omega^{(b)} $ has weight $q(\omega) = q (\omega^{(a)})  q (\omega^{(b)})  $.

One example to have in mind is \begin{align}\label{eq:SRW}
\ca G &= \bb Z^d, \\
\lambda_x &= \lambda, \\
\beta_{xy} &= \beta_{yx} = 1 _{x \sim y}.
\end{align}
This choice describes the simple symmetric nearest-neighbor random walk on $\bb Z^d$, stopped at a random time with rate $\lambda$;   its length has an exponential distribution with mean $\lambda^{-1}$.

A second important example is a random walk in a finite box stopped on the boundary, described by the choice
\begin{align}
\ca G &= \{-l,\dots,l\}^d, \\
\lambda_x &= \begin{cases}
0 &\text{ if } x \in \{-l+1,\dots,l-1\}^d, \\
\infty &\text{ otherwise,}
\end{cases} \\
\beta_{xy} &= \beta_{yx} = 1 _{x \sim y}.
\end{align}

Given a path $\omega=(\omega_1,\dots,\omega_n)$, we define the \emph{loop-erasure} procedure. The loop erasure is obtained by applying consecutively the \emph{one-loop erasure}: look for the first time $i$ at which the path repeats a vertex, so $\omega_i = \omega_j$ for some $j<i$. The one-loop erasure of $\omega$ is the path $(\omega_1,\dots,\omega_j,\omega_{i+1},\dots,\omega_n)$. We then apply the one-loop erasure  to this new path, and continue until the path has no repeating vertices. The resulting path is the {\em loop erasure} of $\omega$,   denoted $\operatorname{LE}(\omega)$. It is self-avoiding, and when the initial path $\omega$ is a random walk, it is called the {\em loop-erased random walk} (LERW).
If $\gamma$ is a LERW ending at $x$, the probability to generate it is
\begin{equation}
\mathcal P(\gamma ) = \frac{\lambda_x}{r_x} \sum_{\omega:\operatorname{LE}(\omega)=\gamma} q(\omega) \ .
\end{equation}
\textbf{Warning:} a {\em self-avoiding path} is a cominatorial object, i.e., it has no statistics. The loop-erased random walk is one possible distribution on the set of self-avoiding paths. It should not be confused with the {\em self-avoiding walk} or {\em self-avoiding polymer} commonly studied in the physics literature, which is another distribution on the set of self-avoiding paths. We distinguish the combinatorial object from the probabilistic one by using the term  {\em path}.

\section{Viennot's theorem}

The main tool we use is a combinatorial theorem by Viennot \cite{Viennot1986}. It is part of the general theory of {\em heaps of pieces}.  In our case it reduces to a relation between loop-erased random walks, and collections of loops.
It   allows us to calculate the total weight $\mathcal P(\gamma)$ of all paths $\omega$ whose loop erasure is   $\gamma$, and represent the latter as a lattice model.

Before discussing the general case, consider a graph consisting   of two vertices and one edge, and the following path,
\begin{equation} \label{eq:twovertices_example}
\gamma =\,
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1pt},baseline={([yshift=-.5ex]current bounding box.center)}]
	\node[circle,minimum size=15pt, inner sep=0pt,draw] (x) at (0,0) {$x$};
	\node[circle,minimum size=15pt, inner sep=0pt,draw] (y) at (2,0) {$y$};

	\draw[->-=0.6,out=0,in=180] (x) to (y);
\end{tikzpicture}.
\end{equation}
The probability that such a path is the result of a loop-erased random walk leaving from $x$ and arriving at $y$ is   given by a geometric sum,
\begin{align}\nonumber
\ca P(\gamma) &=\,
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1pt},baseline={([yshift=-.5ex]current bounding box.center)}]
	\node[circle,minimum size=15pt, inner sep=0pt,draw] (x) at (0,0) {$x$};
	\node[circle,minimum size=15pt, inner sep=0pt,draw] (y) at (2,0) {$y$};
	\draw[->-=0.6,out=0,in=180] (x) to (y);
\end{tikzpicture}
+
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1pt},baseline={([yshift=-.5ex]current bounding box.center)}]
	\node[circle,minimum size=15pt, inner sep=0pt,draw] (x) at (0,0) {$x$};
	\node[circle,minimum size=15pt, inner sep=0pt,draw] (y) at (2,0) {$y$};
	\draw[->-=0.7,out=40,in=140] (x) to (y);
	\draw[->-=0.6,out=0,in=180] (x) to (y);
    \draw[->-=0.7,out=-140,in=-40] (y) to (x);
\end{tikzpicture}
+\dots  \\
&=  \frac{\lambda_y}{r_y} \left(\frac{\beta_{xy}}{r_x} + \frac{\beta_{xy}}{r_x} \frac{\beta_{yx}}{r_y} \frac{\beta_{xy}}{r_x}+\dots \right) \nonumber\\
&=  \frac{\lambda_y}{r_y} q(\gamma) \, \frac{1}{1-r_x^{-1}\beta_{xy}  r_y^{-1} \beta_{yx}}\;.
\end{align}
We  come back to this example later.

A {\em loop} is a path  $\omega=(\omega_1,\dots,\omega_{n-1},\omega_n=\omega_1)$ where the first and last points are identical. We also require all vertices to be distinct (except $\omega_1$ and $\omega_n$), so it cannot be decomposed into smaller loops. Loops obtained from each other via cyclic permutations (dropping the repeated vertex $\omega_n$) are considered identical.

 By a {\em collection of disjoint loops} we mean a set $L=\{C_1,C_2,\dots\}$, each of whose elements is a loop, and the intersection of any pair of loops in $L$ is empty. We denote the {\em set of all such  collections} by $\cc L$.

In order to formulate the theorem, we   fix a self-avoiding path $\gamma$.
We define the set $\cc L_\gamma$ to consist of the collections of disjoint loops in which no loop intersects $\gamma$.
Then Viennot's theorem can be written as ($|L|$ being the  number of loops)
\begin{equation}\label{eq:Viennot}
\ca A(\gamma):=q(\gamma) \sum_{L\in \cc L_\gamma} (-1)^{|L|}\prod_{C\in L}q(C) = \sum_{\omega:\operatorname{LE}(\omega)=\gamma} q(\omega) \times \sum_{L\in \cc L}(-1)^{|L|}\prod_{C\in L}q(C).
\end{equation}
On the l.h.s.\ one sums over the ensemble of collections of loops which do not intersect $\gamma$, giving each collection a weight $(-1)^{|L|}\prod_{C\in L}q(C)$.
We   later calculate this object using field-theory. The r.h.s.\ contains two factors. The first is the weight to find the LERW path $\gamma$, our object of interest. The second is the partition function
\begin{equation}\label{Z}
{\cal Z }:= \sum_{L\in \cc L}(-1)^{|L|}\prod_{C\in L}q(C)
\end{equation} of the loop model on the left-hand side.
Assuming the walk to stop at $x$, this relation can   be read as
\begin{equation}
\mathcal P(\gamma ) = \frac{\lambda_x}{r_x} \sum_{\omega:\operatorname{LE}(\omega)=\gamma} q(\omega) = \frac{\lambda_x}{r_x} \frac{q(\gamma) \sum_{L\in \cc L_\gamma} (-1)^{|L|}\prod_{C\in L}q(C)}{\cal Z}.
\end{equation}
Let us verify this formula in the elementary example of equation \eqref{eq:twovertices_example}. On the l.h.s, since all loops intersect $\gamma$, the only collection of disjoint loops in $L_\gamma$ is the empty collection that contains no loops. Its size $|L|=0$, so the sign is $1$, and since the empty product equals $1$ we obtain $q(\gamma)$. On the right-hand side, the first term $\sum_{\omega:\operatorname{LE}(\omega)=\gamma} q(\omega)$ is the sum that we have calculated, $q(\gamma) \, \frac{1}{1-r_x^{-1}\beta_{xy}r_y^{-1}\beta_{yx}}$. The second term is the sum over two possible collections: the empty collection, and the one that contains the single loop $L=\{(x,y,x)\}$. The empty collection gives weight $1$, and the single-loop collection has weight $\frac{\beta_{xy}}{r_x}\frac{\beta_{yx}}{r_y}$, with a minus sign   from $(-1)^{|L|}$. Putting everything together,
\begin{equation}\label{7}
q(\gamma) = q(\gamma) \, \frac{1}{1 - r_x^{-1}\beta_{xy} r_y^{-1} \beta_{yx}} \times \left(1 - \frac{\beta_{xy}}{r_x}\frac{\beta_{yx}}{r_y}\right),
\end{equation}
which indeed confirms Viennot's theorem in this specific case.

The idea for the proof of equation~(\ref{eq:Viennot}) is to consider a pair  $ \{\omega, L\}$ constructed as follows: Take a path $\omega$ such that $\operatorname{LE}(\omega)=\gamma$ and an {\em arbitrary} collection $L$ of disjoint loops. Our goal is to construct another pair $\{ \omega', L'\}$ by transferring a loop from $L$ to $\omega$ or vice versa, depending on where the loop originally was. For example,
\begin{equation}
\label{8}
\hspace*{-2cm}{\parbox{12.2cm}{{\begin{tikzpicture}[scale=0.6,every node/.style={scale=1}]
		\node  (1) at (-4.25, 7) {};
		\node  (2) at (-6.75, 7) {};
		\node  (3) at (-1.75, 7) {};
		\node  (5) at (-3, 9.25) {};
		\node  (7) at (-3, 8.5) {};
		\node  (8) at (4.5, 7) {};
		\node  (9) at (2, 7) {};
		\node  (10) at (7, 7) {};
		\node  (11) at (5.75, 9.25) {};
		\node  (12) at (5.75, 8.5) {};
		\node  (13) at (3.25, 7.75) {};
		\node  (14) at (3.25, 9.25) {};
		\node  (15) at (3.25, 7.85) {};
\node  (13-15) at (3.25, 7.8) {};
\fill [green] (13-15) circle (4pt);
\node  (new) at (-5.5, 7.85) {};
\fill [green] (new) circle (4pt);
		\node  at (0,8.25) {$+$};
		\node  at (10, 8.25) {$=0$};
		\node at (-7.2,7) {\color{red}  $\omega$};
		\node at (-2.3,9.45) {\color{blue}  $L$};

		\node at (1.5,7) {\color{red}  $\omega'$};
		\node at (4.6,9) {\color{blue}  $L'$};

		\draw [bend left, looseness=6.25, red] (2.center) to (1.center);
		\draw [bend right, red] (1.center) to (3.center);
		\draw [bend left=90, looseness=1.75, blue] (5.center) to (7.center);
		\draw [bend right=90, looseness=1.50, blue] (5.center) to (7.center);
		\draw [bend right, red] (8.center) to (10.center);
		\draw [bend left=90, looseness=1.75,blue] (11.center) to (12.center);
		\draw [bend right=90, looseness=1.50,blue] (11.center) to (12.center);
		\draw [red] (9.center) to (13.center);
		\draw [red] (13.center) to (8.center);
		\draw [in=-5, out=35, looseness=1.75,blue] (15.center) to (14.center);
		\draw [in=145, out=-175, looseness=1.75,blue] (14.center) to (15.center);
\end{tikzpicture}}}}.
\end{equation}
In the first drawing, the left loop is part of $\omega$, whereas in the second one it is part of $L'$.
These terms cancel, as  $(-1)^{|L|} = - (-1)^{|L'|}$, and all other factors are identical.
After each such pair is canceled, we are left with the terms in which it is impossible to transfer a loop from $\omega$ to $L$ or vice versa. These are exactly the terms in the l.h.s of equation \eqref{eq:Viennot}.

For this procedure to work we need to make sure that we cannot obtain the same pair $\{\omega',L'\}$ starting form two different pairs $\{\omega,L\}$.
In order to  achieve this,  we use the following prescription.
Start walking along $\omega$, until
\begin{enumerate}
\item we reach a vertex $\omega_i$ that belongs to some $C = (\omega_i=c_1,c_2\,\dots,c_m=\omega_i)\in L$,  or
\item we reach a vertex $\omega_i$ that does not belong to any $C$, but that we have already seen before, i.e., $\omega_j=\omega_i$ for $j<i$.
\end{enumerate}
In the first case, we   transfer $C$ to $\omega$, i.e.,
\begin{align}
\omega' &= (\omega_1,\dots,\omega_i,c_2,\dots,c_{m-1},\omega_i,\dots,\omega_n), \text{ and }
L' = L \setminus \{C\}.
\end{align}
In the second case, we apply the one-loop erasure to $\omega$, and transfer the erased loop to $L$,
\begin{align}
\omega' &= (\omega_1,\dots,\omega_j,\omega_{i+1},\dots,\omega_n), \text{ and }
L' = L \cup \{(\omega_j,\omega_{j+1},\dots,\omega_i=\omega_j)\}.
\end{align}
Note that disjointness of the loop collections is  preserved under the transfer, and that the loop erasure of $\omega'$ remains $\gamma$.

Let us consider some special cases:
the first example is $L$ being the empty set. If $\omega$ contains no loops, then $\omega=\gamma$. This term appears in the l.h.s. of equation \eqref{eq:Viennot}. If, on the other hand, $\omega$ does contain a loop, we take $C=(\omega_j,\dots,\omega_i)$ to be the first loop in the loop erasure of $\omega$.
Then $\omega' = (\omega_1,\dots,\omega_j,\omega_{i+1},\dots,\omega_n)$ and $L' = {C}$. The same thing happens when $L$ is not empty, but none of the loops in $L$ intersects $\omega$.
Loops in $L$ that do not intersect $\omega$ act as   spectators, as in  equation~\eqref{8}.

A second example, is when $\omega$ winds $k$ times around a loop $C$ that also belongs to $L$.
If $C$ is the first loop that we encounter, then $L'=L\setminus \{C\}$, and $\omega'$ has the same trace as $\omega$, but it winds $k+1$ times around $C$ rather than $k$ times.

Third, if a loop $C \in L$ intersects $\omega$ in more than one point, the instructions above   determine   how this loop is attached. For example, if
\begin{equation}
\{\omega,L\} =
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1pt},baseline={([yshift=-.5ex]current bounding box.center)}]
	\node[circle] (x) at (0,0) {};
	\node[circle] (y) at (2,0) {};
	\node[circle] (z) at (4,0) {};
	\node[circle] (a) at (1,0) {};
	\node[circle] (b) at (3,0) {};
	\draw[->-=0.6,red] (x.center) to (a.center);
	\draw[->-=0.5,red] (a.center) to (b.center);
	\draw[->-=0.6,red] (b.center) to (z.center);
	\draw[->-=0.6,in=90,out=90,looseness=1.7,blue] (a.center) to (b.center);
	\draw[->-=0.6,in=-90,out=-90,looseness=1.7,blue] (b.center) to (a.center);
\end{tikzpicture},
\end{equation}
then
\begin{equation}
 {\omega}' = {\color{red}
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1pt},baseline={([yshift=-.5ex]current bounding box.center)}]
	\node[circle] (x) at (0,0) {};
	\node[circle] (y) at (2,0) {};
	\node[circle] (z) at (4,0) {};
	\node[circle,  minimum size=6pt, inner sep=0pt] (a) at (1,0) {};
	\node[circle,  minimum size=3pt, inner sep=0pt] (b) at (3,0) {};

	\draw[->-=0.6] (x.center) to (a.west);
	\draw[out=0,in=-95,looseness=1.2] (a.west) to (a.north);
	\draw[->-=0.6,bend left,looseness=1.6,in=90,out=85] (a.north) to (b.center);
	\draw[->-=0.6,in=-85,out=-90,looseness=1.7] (b.center) to (a.south);
	\draw[out=95,in=0,looseness=2.4] (a.south) to (a.east);
	\draw[->-=0.5] (a.east) to (b.west);
	\draw[->-=0.6] (b.east) to (z.center);
\end{tikzpicture}}
\not ={\color{red}
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1pt},baseline={([yshift=-.5ex]current bounding box.center)}]
	\node[circle] (x) at (0,0) {};
	\node[circle] (y) at (2,0) {};
	\node[circle] (z) at (4,0) {};
	\node[circle,  minimum size=2pt, inner sep=0pt] (a) at (1,0) {};
	\node[circle,  minimum size=6pt, inner sep=0pt] (b) at (3,0) {};

	\draw[->-=0.6] (x.center) to (a.center);
	\draw[->-=0.5] (a.center) to (b.west);
	\draw[out=0,in=-95,looseness=2.5] (b.west) to (b.south);
	\draw [->-=0.5, bend left, in=95,out=85,looseness=1.7] (b.south) to (a.south);
	\draw [->-=0.5,bend left,in=95,out=85,looseness=1.7] (a.north) to (b.north);
	\draw[out=-85,in=0,looseness=2.5] (b.north) to (b.east);
	\draw[->-=0.6] (b.east) to (z.center);
\end{tikzpicture}}.
\end{equation}

\section{A lattice action with two complex fermions and one complex boson}
Our   goal is to write a lattice action which generates the l.h.s.\ of \Eq{eq:Viennot}, namely
\begin{equation}
\ca A(\gamma)=q(\gamma) \sum_{L\in \cc L_\gamma} (-1)^{|L|}\prod_{C\in L}q(C).
\end{equation}
This can be achieved with an action based on one pair  $(\phi, \phi^*)$ of complex conjugate fermionic fields.
While this theory   sums over all paths $\gamma$, yielding back  the random-walk propagator, it contains no information on the erasure. In order to answer   whether the resulting loop-erased path passes through a given point $y$ it is necessary to use more fields. The simplest such setting consists of two pairs of complex conjugate fermionic fields $(\phi_1,\phi_1^*)$,
and $(\phi_2,\phi_2^*)$, as well as a pair of complex conjugate bosonic fields $(\phi_3,\phi_3^*)$. When appearing in a loop, the latter   cancels one of the fermions.

In the last section we saw that the loop-erased random walk is tightly connected to the combinatorics of non-intersecting loops and paths. When trying to express such objects using a field theory, the non-intersection property appears naturally with fermionic fields, but not with bosonic ones. In the theory that we construct we   need to have both fermionic and bosonic components. To ensure non-intersection also for bosonic fields, we introduce the \emph{nilpotent bosonic field} in analogy to the Grassmannian fermionic field. Denoting   $g(\phi)$ the {\em grade} of a  field $\phi$, we set $g(\phi)=1$ for   fermionic fields and $g(\phi)=0$ for   bosonic fields,
\begin{eqnarray}
g(\phi_1) &=& g(\phi_1^*) = g(\phi_2) = g(\phi_2^*) = 1, \\
g(\phi_3)&=&g(\phi_3^*) =0.
\end{eqnarray}
The fields are employed with    {\em graded commutation relations}, so that for all $x$ and $y$
\begin{eqnarray}
\phi (x)\psi (y) &=& (-1)^{g(\phi)g(\psi)}\psi (y)  \phi (x), \\
\phi (x)^2 &=& 0.
\end{eqnarray}
The second relation is a consequence of the first one for fermions, but an additional rule for bosons.

The  {\em path integral} for both fermions and bosons is defined as the Berezin integral \cite{Berezin1965},
\begin{equation}\label{Berezin}
 \int_{\phi^*(x)} \int_{\phi(x)} \ca F(\phi, \phi^*) :=  \frac{\mathrm{d} }{\mathrm{d} \phi^*(x)} \frac{\mathrm{d} }{\mathrm{d} \phi(x)}\ca F(\phi, \phi^*) \Big|_{\phi(x) = \phi^*(x) =0}\ .
\end{equation}
This  implies     a   correlation  $1$ for
$ \phi_i(x) \phi_i^*(x) $, in this order.

One way to represent a nilpotent boson is to write it as a product of two fermions \cite{HelmuthShapira2020}.
In this representation, using the Berezin integral for these fermions yields \Eq{Berezin} for the nilpotent bosons, making it applicable for all fields.
As it is not possible to   integrate out these fermionic fields in exchange for a functional determinant, as is done e.g.\ to treat superconductivity \cite{AltlandSimonsBook}, the intuition behind this construction is misleading, and we will not pursue it any further.

We   define our action as
\begin{align}
\label{10}
&  \phi^*(y) \phi(x)   := \sum_{i=1}^3  \phi^*_i(y) \phi_i(x)\ ,\\
\label{eq:expS}
&e^{-\ca S} = \prod_x \bigg\{ \Big[1 + \sum_y \beta_{xy} \phi^*(y) \phi(x) \Big] \prod_{i=1}^3\Big[1 + r_x \phi_i(x) \phi_i^*(x)  \Big] \bigg\}.
\end{align}
We can calculate the partition function $\ca Z = \int e^{-\ca S}$ by expanding the products. Each term of the expansion is represented as a colored multigraph: whenever we encounter a term $\beta_{xy}\phi^*_i(y)\phi_i(x)$, we place an edge from $x$ to $y$ of color $i$;   whenever we encounter a term $r_x \phi_i(x) \phi_i^*(x)$ we place a self-loop of color $i$ at $x$. By \emph{self-loop at $x$} we mean an edge from $x$ to $x$. This representation is
 one-to-one,  since knowing the colored multigraph $G$, we can recover the corresponding term in the expansion by taking $\beta_{xy}\phi^*_i(y)\phi_i(x)$ for each colored edge and $r_x \phi_i(x) \phi_i^*(x)$ for each self-loop.

The multigraphs that contribute to $\ca Z$ must have {\em complete degrees} (i.e., each vertex has incoming and outgoing degree $1$ of each color), and due to the structure of the action each vertex can have at most one outgoing edge, not counting self-loops. By taking outside the self-loops, the graphs  left   are  collections of disjoint colored loops, with  contribution
\begin{align}\label{eq:contributiontoZ}
&\ca Z_0 \, (-1)^{\#\text{fermionic loops}}\prod_{C \text{ loop of }G} q(C). \\
&\ca Z_0 := \int_{\phi^*,\phi}\prod_x \prod_{i=1}^3\Big[1 + r_x \phi_i(x) \phi_i^*(x)  \Big] = \prod_x r_x^3.
\end{align}
The reason for the factor of $(-1)^{\#\text{fermionic loops}}$ is that Berezin integration  requires the fields to come in a certain order. As bosons commute,   we may reorder them as we wish;   reordering fermions may cost a sign. The self-loops   appear in the right order, $\phi_i(x) \phi_i^*(x)$.

In contrast, if the graph contains a fermionic loop $(x_1,\dots,x_n=x_1)$, then the corresponding term in the expansion has edges $(x_{n-1},x_n),(x_{n-2},x_{n-1}),\dots,(x_1,x_2)$, and is given by $\beta_{x_{n-1} x_n}\phi_i^*(x_n)\phi_i(x_{n-1}) \beta_{x_{n-2} x_{n-1}}\phi_i^*(x_{n-1})\phi_i(x_{n-2})\dots \beta_{x_1 x_2}\phi_i^*(x_2)\phi_i(x_1)$. In order to apply Berezin integration,   we need to move the last variable $\phi_i(x_1)$ to the beginning. This requires an odd number of exchanges, hence the minus sign.

Summing equation \eqref{eq:contributiontoZ} over all possible colorings and all graphs, we obtain
\begin{equation}\label{eq:Z}
\ca Z = \ca Z_0 \, \sum_{L\in \cc L} (-1)^{|L|}\prod_{C\in L}q(C).
\end{equation}
This is, up to the prefactor of $\ca Z_0$, the partition function defined in \Eq{Z}.

In order to assess whether a point $b$ belongs to a  loop-erased random walk  from $a$ to $c$ after erasure, we fix the three vertices $a,b$ and $c$ and consider the observable
\begin{equation}\label{eq:U}
U(a,b,c) = \lambda_c r_b \left\langle \phi_2(c) \phi_2^*(b) \phi_1(b) \phi_1^*(a) \right\rangle .
\end{equation}
The graphs that contribute to the integral consist of a self-avoiding path $\gamma$ and a collection $L$ of disjoint self-avoiding colored loops such that (see Figure \ref{fig:Udiagrams}):
\begin{enumerate}
\item $\gamma$ is a path from $a$ to $c$ passing through $b$. The edges of $\gamma$ between $a$ and $b$ have color $1$, and the edges between $b$ and $c$ have color $2$.
\item Fix $C \in L$. If the color of $C$ is $2$ then it cannot intersect $\gamma$. If its color is $1$ or $3$, it can only intersect $\gamma$ at the point $c$.
\end{enumerate}
\begin{figure}
\begin{tikzpicture}[scale=0.8,every node/.style={scale=0.8},every path/.style={line width=1.2pt}]
	\node[circle, draw] (a) at (0,0) {$a$};
	\node[circle, draw]  (b) at (3,3) {$b$};
	\node[circle, draw]  (c) at (6,0) {$c$};

	\node[circle, draw] (x) at (0,3) {$x$};
	\node[circle, draw] (y) at (3,0) {$y$};
	\node[circle, draw] (z) at (6,3) {$z$};
	\node[circle, draw] (w) at (9,3) {$w$};

	\draw[->,bend left=0,blue] (a) to (x);
	\node[blue] at (-0.25,1.5) {\scriptsize $(1)$};

	\draw[->,bend left=0,blue] (x) to (b);
	\node[blue] at (1.5,3.25) {\scriptsize $(1)$};

	\draw[->,bend left=0,green] (b) to (y);
	\node[green] at (2.75,1.5) {\scriptsize $(2)$};

	\draw[->,bend left=0,green] (y) to (c);
	\node[green] at (4.5,0.25) {\scriptsize $(2)$};

	\draw[->,bend right=20, red] (w) to (z);
	\node[red] at (7.5,3.55) {\scriptsize $(3)$};

	\draw[->,bend right=20, red] (z) to (w);
	\node[red] at (7.5,2.4) {\scriptsize $(3)$};

\end{tikzpicture}
\caption{An example of a diagram that contributes to $U$.
Our coloring conventions are blue for   $(\phi_1,\phi_1^*)$, green for $(\phi_2,\phi_2^*)$, and red for $(\phi_3,\phi_3^*)$.
}
\label{fig:Udiagrams}
\end{figure}
In the latter case, the contribution to $\int \phi_2(c) \phi_2^*(b) \phi_1(b) \phi_1^*(a) e^{-\ca S}$ is
\begin{equation}
\ca Z_0 \, (-1)^{\# \text{fermionic loops}} r_b^{-1} r_c^{-1} q(\gamma) \prod_{C \in L} q(C).
\end{equation}
We   now sum over all possible colorings of the loops. Since loops that intersect $c$ may have either color $1$ or $3$, one fermionic and one bosonic, they cancel, leaving only   graphs in which loops do not intersect $\gamma$ at all. The other loops, as before, give a factor of $-1$.
We have therefore established that
\begin{equation} \label{eq:4pntfuntion}
\int_{\phi^*,\phi} \phi_2(c) \phi_2^*(b) \phi_1(b) \phi_1^*(a) e^{- \ca S} = \ca Z_0 \, r_b^{-1} r_c^{-1} \sum_{\substack{\gamma:a \to b \to c \\ \text{ self-avoiding}}} q(\gamma) \sum_{L \in \cc L_\gamma} (-1)^{|L|} \prod_{C \in L} q(C),
\end{equation}
where the sum is over all self-avoiding paths from $a$ to $c$ passing through $b$.

What remains to be done  is to combine  Viennot's theorem \eqref{eq:Viennot} with equations \eqref{eq:Z} and \eqref{eq:4pntfuntion},
\begin{align}\label{eq:PandU}
\sum_{\substack{\gamma:a \to c \\ b\in \gamma}}  \ca P(\gamma) \equiv
\sum_{\substack{\omega:a \to c \\ b\in\operatorname{LE}(\omega)}} \bb P(\omega) &= \sum_{\substack{\gamma:a \to b \to c \\ \text{self-avoiding}}}\sum_{\omega:\operatorname{LE}(\omega)=\gamma} q(\omega) \frac{\lambda_c}{r_c}\nonumber \\
&= \frac{\lambda_c}{r_c} \sum_{\substack{\gamma:a \to b \to c \\ \text{self-avoiding}}}q(\gamma)
\frac{\sum_{L\in \cc L_\gamma} (-1)^{|L|} \prod_{C\in L}q(C)}{\sum_{L\in \cc L} (-1)^{|L|} \prod_{C\in L}q(C)} \nonumber\\
&= U(a,b,c).
\end{align}

Note that the above formalism allows us to consider LERWs starting at $a$,    going through $b_1$ and $b_2$ in this order, and ending at $c$,
\begin{equation}\label{eq:U}
U(a,b_1,b_2,c) = \lambda_c r_{b_1} r_{b_2} \left\langle \phi_3(c) \phi_3^*(b_2) \phi_2(b_2)  \phi_2^*(b_1) \phi_1(b_1) \phi_1^*(a) \right\rangle .
\end{equation}
For   more intermediate points, one  needs  more fields: Each addition of one complex conjugate pair of bosonic fields together with  one    complex conjugate pair of fermionic fields allows us to have two more intermediate points.
We will not pursue this direction any further.

Before concluding this section, let us rewrite the action $\ca S$ explicitly. The starting point is
\Eq{eq:expS},
\begin{equation} \label{action-rewrite}
e^{-\ca S} = \prod_x \bigg\{  \Big[1 + \sum_y \beta_{xy} \phi^*(y) \phi(x) \Big] \prod_{i=1}^3\Big[1 + r_x \phi_i(x) \phi_i^*(x)  \Big]  \bigg\}.
\end{equation}
We need to take the logarithm of this equation.
Due to the nilpotence of the fields, the expansions   stop at the latest at order three, the number of colors,
\begin{equation}
\begin{array}{rll}
\ln\! \Big( 1 + r_x \phi_i(x) \phi_i^*(x)  \Big) &=& r_x \phi_i(x) \phi_i^*(x), \\
\ln\! \Big(1 + \sum_y \beta_{xy} \phi^*(y) \phi(x) \Big) &=& \displaystyle\sum_y \beta_{xy} \phi^*(y) \phi(x)
- \frac{1}{2} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^2 \\
&& \displaystyle\qquad + \frac{1}{3} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^3.
\end{array}
\end{equation}
Putting the fields in the ``natural order'' ($\phi^*_i$ before $\phi_i$) yields
\begin{align}
\nonumber
\ca S &= \sum_x \bigg\{ r_x \Big[ \phi_1^*(x) \phi_1(x) + \phi_2^*(x) \phi_2(x) -\phi_3^*(x) \phi_3(x)  \Big] - \sum_y \beta_{xy} \phi^*(y) \phi(x) \nonumber \\
  & \qquad + \frac{1}{2} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^2 - \frac{1}{3} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^3 \bigg \}   \nonumber \\   \nonumber
  &= \sum_x \bigg\{ r_x \Big[ \phi_1^*(x) \phi_1(x) + \phi_2^*(x) \phi_2(x) -\phi_3^*(x) \phi_3(x)  \Big] - \sum_y \beta_{xy} \Big[\phi^*(y) - \phi^*(x)\Big] \phi(x) \\   \nonumber
  & \qquad - \Big(\sum_y \beta_{xy}\Big) \phi^*(x) \phi(x) + \frac{1}{2} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^2 - \frac{1}{3} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^3\bigg\} \\
  &= \sum_x \bigg\{ m_x^2 \phi_1^*(x) \phi_1(x) + m_x^2 \phi_2^*(x) \phi_2(x) + \mu_x^2 \phi_3^*(x) \phi_3(x) - \nabla_\beta^2 \phi^*(x) \phi(x) \nonumber \\
  & \qquad  + \frac{1}{2} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^2 -  \frac{1}{3} \Big[\sum_y \beta_{xy} \phi^*(y) \phi(x)\Big]^3 \bigg\},
\label{eq:fullaction}
\end{align}
with
\begin{equation}
m_x^2 = r_x - \sum_y \beta_{xy} \ , \qquad \mu_x^2 =  -r_x - \sum_y \beta_{xy}.
\end{equation}
This theory contains two fermions with mass   $m_x^2 = \lambda_x$ and a nilpotent boson with mass   $\mu_x^2 =  -\lambda_x - 2\sum_y \beta_{xy}$. In this formulation, the cancelation between  the nilpotent boson and one of the fermions is not obvious.

\section{Trading nilpotent bosons for standard bosons} \label{sec:nilpotentbosons}

The purpose of this section is to show that
nilpotent bosons behave like interacting standard bosons, and  to rewrite the action with the latter. To simplify our considerations, we do not write the fermions; including them later is straightforward.

The bosonic part of equation~\eqref{eq:fullaction} is
\begin{eqnarray}
\ca S^{(3)} =   \sum_x \Big[ \mu_x^2 \phi_3^*(x) \phi_3(x) -\nabla_\beta^2 \phi_3^*(x) \, \phi_3(x) \Big].
 \label{rewrite}
\end{eqnarray}
As in the previous section, we can expand the corresponding partition function, obtaining
\begin{equation}
\ca Z^{(3)} = \ca Z^{(3)}_0 \sum_{L\in \ca L} \prod_{C\in L} q(C),
\end{equation}
where $\ca Z^{(3)}_0 = \prod_x r_x$.

We   now consider an interacting bosonic field theory with a standard pair of complex conjugate bosonic fields $\{\chi(x)\}_{x\in \ca G}$, and action
\begin{equation} \label{eq:S_boson}
\ca S' = \sum_x  \Big[ r_x \chi^*(x)\chi(x) -\log\Big(1 + \sum_y \beta_{xy} \chi^*(y)\chi(x) \Big) \Big].
\end{equation}
The prime denotes objects evaluated with a standard pair of complex conjugate bosonic fields, and this action.
Consider the corresponding partition function
\begin{equation}
\ca Z' = \int_{\chi^*,\chi} e^{-\ca S'} = \int_{\chi^*,\chi} e^{-\sum_x r_x \chi^*(x)\chi(x)} \prod_x \left(1 + \sum_y \beta_{xy} \chi^*(y)\chi(x) \right).
\end{equation}
This is a Gaussian integral, and we   calculate it by expanding the product over $x$.  As before, we represent terms in the product as a graph, placing an edge from $x$ to $y$ whenever the term $\beta_{xy} \chi^*(y)\chi(x)$ appears. Unlike in the nilpotent case, we  have no self-loops in the graph. Moreover, since the directed edge from $x$ to $y$ appears in only one factor of the product, there are no double edges, yielding a graph rather than a multigraph. One more constraint, coming from the form of the action, is that each vertex of this graph has out-degree at most $1$.

The diagrams that contribute to $\ca Z'$ are those in which at each vertex $x$ the fields appear as $(\chi^*(x)\chi(x))^n$ for some $n$. Since the outgoing degree of $x$ is at most $1$, the power $n$ is at most $1$. Thus each vertex has incoming and outgoing degrees that are either both $1$ or both $0$. We obtain
\begin{eqnarray}
\ca Z' &=& \ca Z'_0 \sum_{L\in \ca L} \prod_{C \in L} q(C), \\
\ca Z'_0 &=& \prod_x \frac{\pi}{r_x}.
\end{eqnarray}
We   see that the diagrams contributing to the (standard) bosonic theory \eqref{eq:S_boson} are the same as those contributing to the nilpotent bosonic field. Care has to be taken when considering correlation functions, since external fields may have a slightly different behavior. This can be done in general, but since   the observable $U(a,b,c)$ defined in equation~\eqref{eq:U} has only fermionic external fields, we can discard these details.

In order to better understand  the action defined in equation~\eqref{eq:S_boson}, let us write it explicitly:
\begin{align}\label{46}
\ca S' &= \sum_x \bigg\{ r_x \chi^*(x)\chi(x) -  \sum_y \beta_{xy} \chi^*(y)\chi(x) + \sum_{k=2}^\infty \frac{(-1)^k}{k} \Big[\sum_y \beta_{xy} \chi^*(y)\chi(x)\Big]^k \bigg\}\nonumber \\
&= \sum_x \bigg\{  m_x^2 \chi^*(x)\chi(x) -  \nabla_\beta^2 \chi^*(x)\chi(x)  + \sum_{k=2}^\infty \frac{(-1)^k}{k} \Big[\Big( \nabla_\beta^2+\sum_y \beta_{xy} \Big)\chi^*(x)\chi(x)\Big]^k\bigg\}.
\end{align}
We   see from this equation that the nilpotent boson with mass $\mu$ is equivalent to an interacting boson with mass $m$. That is, the action of our $\ca O(-2)$ field theory given in equation~\eqref{eq:fullaction} describes a field with two fermionic components and one bosonic one, all with the same mass.

The identities laid out above are exact, and hold for any graph and any set of parameters. If we consider a lattice in dimension $d> 2$, after dropping terms that are irrelevant in the scaling limit, only the term $k=2$ survives. Further dropping the Laplacian in this term,   we are left with a standard $\phi^4$ theory (see, e.g.,~\cite{Zinn}).

To conclude this section, we return to the loop-erased random walk, but this time with a field $\phi$, containing two complex fermions and one complex (\emph{not} nilpotent) boson. The {\em exact} action for the loop-erased random walk is
\begin{equation} \label{eq:stdaction}
\ca S' = \sum_x \bigg\{ m_x^2 \phi^*(x)\phi(x) -  \nabla_\beta^2 \phi^*(x)\phi(x)  + \sum_{k=2}^\infty \frac{(-1)^k}{k} \Big[\Big(\nabla_\beta^2+\sum_y \beta_{xy} \Big)\phi^*(x)\phi(x)\Big]^k\bigg\}.
\end{equation}
We  define $U'$ in the same manner as  in equation~\eqref{eq:U},
\begin{equation}
U'(a,b,c) = \lambda_c r_b \left\langle \phi'_2(c) \phi^{\prime *}_2(b) \phi'_1(b) \phi_1^{\prime *}(a) \right\rangle,
\end{equation}
where $\langle \cdot \rangle$ is understood with respect to $S'$. Then   equation~\eqref{eq:PandU} holds, and
\begin{equation}
\sum_{\substack{\gamma:a \to c \\ b\in \gamma}}  \ca P(\gamma) \equiv
\sum_{\substack{\omega:a \to c \\ b\in \operatorname{LE}(\omega)}}  {\bb P(\omega)} = U'(a,b,c).
\end{equation}
Again, in $d>2$, the scaling limit of this theory is the $\phi^4$ theory. Perturbative calculations  indicate that the theory remains valid in dimension $d=2$ \cite{KompanietsWiese2019}.

\section{Selfavoiding polymers, and other applications}

In the considerations above, we   used two fermionic fields, and one bosonic one. The latter could be chosen  either as a standard bosonic field, or as  nilpotent. We saw that both formulations are exact. There are other physical systems which have the same lattice expansion, albeit with a different number $N_{\rm b}$ of bosonic and $N_{\rm f}$ fermionic fields:

(i) one bosonic $N_{\rm b}=1$ and one fermionic field $N_{\rm f}=1$: Then all loops cancel, and each   self-avoiding path $\gamma$ appears with a weight one. These are self-avoiding polymers. The action \eq{eq:stdaction}, or its resummed version,  are exact. Keeping only the leading term $k=2$, and dropping in the latter the Laplacian, we arrive at weakly self-avoiding walks, which are in the same universality class. This was first observed in 1972 by de Gennes \cite{DeGennes1972}, and later elaborated by many authors, see \cite{DesCloizeauxJannink,McKane1980,ParisiSourlas1980,Zinn,Duplantier1992,WieseHabil} and references therein.
Similar formulas to ours appear in the work by Brydges, Imbrie and Slade \cite{BrydgesImbrieSlade2009}.

(ii) the standard lattice $\ca O(n)$ model, defined by its loop representation which allows one to take $n$ arbitrary,  corresponds to $n = 2( N_{\rm b} - N_{\rm f})$.
In the graph representation,
the factor of $2$ is due to the two possible choices of the direction in a loop. In the field theory $n$ counts the number of real components, and a complex field has both a real and an imaginary part. See e.g.\ \cite{DomanyMukamelNienhuisSchwimmer1981}.

Using \Eq{eq:stdaction} with one real field yields the exact action for the Ising model on a 3-regular graph such as the honeycomb lattice. It allows one to evaluate lattice observables in the field theory exactly. Dropping irrelevant terms   leads to     $\phi^4$ theory, without the need to employ the standard \cite{BrezinBook,Zinn,KardarBook} coarse-graining procedure.

\section{Conclusion and further questions}

We showed  how LERW observables can be expressed as correlation functions of a field theory with two complex fermions, and one complex boson. This mapping is an exact combinatorial identity for a field theory with specific interactions. It explains the result in \cite{WieseFedorenko2018,WieseFedorenko2019}, by observing that, at least for $d>2$, the large-scale limit of this field theory is  the standard $\phi^4$ theory. Our approach has the advantage to give exact equalities on all graphs.

One convenient way to prove this identity was by introducing a field of {\em nilpotent} bosons. While commuting, they are as fermions not allowed to intersect, and   may  be interpreted  as {\em hard-core bosons}, popular in hard condensed matter physics \cite{AltlandSimonsBook}. A nilpotent boson is equivalent to an interacting bosonic field;  the non-intersection property simplifies the combinatorial analysis.
It   suggests a simplifying  alternative for the action of the  $\ca O(n)$ loop-model \cite{Nienhuis1982,LH2008}
  which in those works is only exact
 on $3$-regular graphs  such as the honeycomb lattice.

The results and methods laid out here lead to  various openings: An intriguing problem is the search for a combinatorial object corresponding to the $\ca O(-1)$ theory. Exploring more properties of nilpotent bosons, and studying how they behave in different theories, is another interesting direction.
Finally, the transition rates $\beta_{xy}$ are   general. This allows one to study situations with drift, gauge fields, or disorder.

\section*{Acknowledgements}
We thank Andrei Fedorenko and Tyler Helmuth  for valuable discussions, and a careful reading of the manuscript.

\ifx\doi\undefined
\providecommand{\doi}[2]{\href{http://dx.doi.org/#1}{#2}}
\else
\renewcommand{\doi}[2]{\href{http://dx.doi.org/#1}{#2}}
\fi
\providecommand{\link}[2]{\href{#1}{#2}}
\providecommand{\arxiv}[1]{\href{http://arxiv.org/abs/#1}{#1}}
\providecommand{\hal}[1]{\href{https://hal.archives-ouvertes.fr/hal-#1}{hal-#1}}
\providecommand{\mrnumber}[1]{\href{https://mathscinet.ams.org/mathscinet/search/publdoc.html?pg1=MR&s1=#1&loc=fromreflist}{MR#1}}