---
abstract: 'In nature stationary nonequilibrium systems cannot exist on their own, rather they need to be driven from outside in order to keep them away from equilibrium. While the internal mean entropy of such stationary systems is constant, the external drive will on average increase the entropy in the environment. This external entropy production is usually quantified by a simple formula, stating that each microscopic transition of the system between two configurations $c \to c''$ with rate $w_{c\to c''}$ changes the entropy in the environment by $\Delta S_{\rm env} = {\ln w_{c \to c''}}-{\ln w_{c'' \to c}}$. According to this formula irreversible transitions $c \to c''$ with a vanishing backward rate $w_{c''\to c}=0$ would produce an infinite amount of entropy. However, in experiments designed to mimic such processes, a divergent entropy production, that would cause an infinite increase of heat in the environment, is not seen. The reason is that in an experimental realization the backward process can be suppressed but its rate always remains slightly positive, resulting in a finite entropy production. The paper discusses how this entropy production can be estimated and specifies a lower bound depending on the observation time.'
address:
- '$^1$Bu-Ali Sina University, Physics Department, 65174-4161 Hamedan, Iran'
- '$^2$Universität Würzburg, Fakultät für Physik und Astronomie, 97074 Würzburg, Germany.'
author:
- 'Somayeh Zeraati$^1$, Farhad H. Jafarpour$^1$, and Haye Hinrichsen$^2$'
title: Entropy Production of Nonequilibrium Steady States with Irreversible Transitions
---

\#1[[\#1]{}]{}

Introduction
============

In classical statistical physics, complex systems are often modeled as continuous-time Markov processes in a discrete configuration space. Such systems evolve by spontaneous random transitions from configuration $c$ to configuration $c'$ according to certain transition rates $w_{c\to c'}$. For isolated systems these rates are symmetric so that the model evolves into an equilibrium state with maximal entropy. For open systems, which interact with the surrounding environment, the transition rates are generally asymmetric. In this case the dynamical evolution changes not only the system’s entropy but also the entropy in the environment. The entropy change in the environment is referred to as the *entropy production* of the system.

As shown by Schnakenberg, Andrieux, Gaspard and Seifert [@Schnakenberg; @Gaspard; @Seifert; @Seifert2], the entropy in the environment $S_{\rm env}$ changes discontinuously by $$\label{EntropyProduction}
 \Delta S_{\rm env} \;=\; \ln \frac{w_{c\to c'}}{w_{c'\to c}}\,$$ whenever the system jumps from $c$ to $c'$ (note that we set $k_B=1$ for simplicity). This simple formula is independent of the specific composition and structure of the environment, provided that it equilibrates almost instantaneously between successive transitions of the Markov process [@Kolkata; @SM].

The entropy production formula (\[EntropyProduction\]) requires that for any transition $c \to c'$ with a non-vanishing forward rate $w_{c \to c'}>0$, the corresponding backward rate $w_{c' \to c}$ has to be nonzero as well since otherwise the entropy production would diverge. This means that the definition of entropy production is only meaningful in models with microscopically reversible transitions. Usually it is argued that in realistic physical systems the effective backward rate is always nonzero since the classical description results from a coarse-graining of the underlying quantum-mechanical processes which are intrinsically time-reversible.

However, in statistical physics a large variety of models investigated in the literature involve microscopically irreversible transitions. For some of these models experiments have been suggested or performed. A simple example is the totally asymmetric simple exclusion process, where particles hop only in one direction, which is studied experimentally by optical tweezers [@Korda; @Lacasta; @Dickman]. Another example is directed percolation [@Haye], the standard model of a phase transition from an fluctuating phase into a frozen state, which was recently realized experimentally for the first time [@Takeuchi]. However, a divergent entropy production, which would manifest itself in form of a divergent increase of heat in the environment, has not been reported in these experiments. This suggests that this divergence is a theoretical artifact and needs to be regularized in a meaningful way.

To our knowledge ben-Avraham, Dorosz and Pleimling [@benAvraham] were the first to address this problem in detail. As a possible solution they suggest to coarse-grain the stochastic evolution by interval sampling: Instead of monitoring each transition event separately, they propose to read off the configuration at regular temporal intervals and to use the resulting configuration sequence to define effective transition rates. Even if the backward rate $w_{c'\to c}$ is zero, meaning that direct transitions from $c'$ to $c$ are forbidden, the sampling allows the system to evolve from $c'$ to $c$ through a loop of other intermediate configurations between two consecutive readings. This gives rise to a small but finite effective backward rate in the sampled data, regularizing the entropy production depending on the sampling rate.

In this Letter we propose an alternative regularization method which is more closely related to the question how micro-irreversibility can be implemented in experiments. We start with the assertion that Eq. (\[EntropyProduction\]) is indeed correct and that the preceding argument about the fundamental impossibility of vanishing backward rates in nature remains valid. This means that it is *in principle* impossible to realize micro-irreversible processes experimentally. However, in practice one can approximate irreversible processes very well by designing the experiment in such a way that the backward transition is strongly suppressed. In such an experiment the actual backward rate is positive but so small that the reverse transition practically never takes place during data taking. Nevertheless the positivity of this rate ensures that the entropy produced by the corresponding forward process is still finite.

The aim of this work is to specify a lower bound on the entropy production of a physical system which is designed to approximate an irreversible process over a finite time span $T$. We find that the entropy production rate of such a system can be split into two parts, namely, a conventional constant part stemming from the reversible transitions, and a second part coming from the approximated irreversible transitions which grows logarithmically with $T$.

Lower bound on the entropy production of micro-irreversible transitions
=======================================================================

Starting point is a continuous-time Markov process defined by certain set of configurations $c \in \Omega$. The model evolves by random transitions $c \to c'$ with rates $w_{c \to c'} \geq 0$, where some of the allowed transitions are microscopically irreversible, i.e. $w_{c' \to c}=0$. Suppose that we are able to design an experiment with an identical set of possible configurations which approximately reproduces the Markovian dynamics of the model. In what follows let us distinguish between

> -   the *defining* rates $w_{c\to c'}$ of the original model, and
>
> -   the corresponding *actual* rates $\tilde w_{c\to c'}$ realized in the experiment.
>
However, usually the actual rates in the experiment are not directly accessible, rather they have to be estimated from the observed number of transitions $n_{c \to c'}$ during a finite time span $T$ of data taking. If the system is found in the configuration $c'$ with probability $P_{c'}$, the expectation value of this number is given by $P_{c'} \tilde w_{c'\to c} T$. Observing a vanishing number $n_{c' \to c}=0$ in a single experiment does not necessarily imply that the corresponding actual rate $\tilde w_{c' \to c}$ is zero, it only means that this rate is sufficiently smaller than $(P_{c'}T)^{-1}$ so that this transition did not occur during data taking.

In the following we use this framework to specify a lower bound for the entropy production caused by irreversible transitions in experiments with a finite observation time. The idea is to estimate the actual rate $\tilde w$ of a transition $c \to c'$ in the experiment for a given defining rate $w$ on the basis of the expected count numbers $n_{c \to c'}$ within a given observation time $T$. In this way we want to find a physically motivated conditional probability distribution $P(\tilde w|w)$ of the actual rate $\tilde w$ for a given defining rate $w$.

To determine $P(\tilde w|w)$ let us assume that the transition $c \to c'$ occurs $n$ times during the observation time $T$. As these events are spontaneous and uncorrelated, $n$ is randomly distributed according to a Poisson distribution $$\label{Poisson}
P(n|w)=\frac{(\tau w)^ne^{-\tau w}}{n!}\,,$$ where $\tau = P_{c}T$ is the expected time that the system spends in the configuration $c$. This allows us to express $P(\tilde w|w)$ as $$P(\tilde w|w) = \sum_{n=0}^\infty P(\tilde w|n)P(n|w)\,,$$ where $P(\tilde w|n)$ is the likelihood for the distribution of the actual rate $\tilde w$ for a given number of transitions $n$. According to Bayes rule [@Bayes] this likelihood is given by $$P(\tilde w|n)=\frac{P(n|\tilde w)P(\tilde w)}{P(n)} \,,$$ where $P(\tilde w)$ is the *prior distribution* and $$\label{prior}
P(n)=\int_0^\infty\d \tilde w\,P(n|\tilde w)P(\tilde w)$$ the normalizing marginal likelihood. The prior $P(\tilde w)$ expresses our belief of how the rates are typically distributed and therefore introduces a certain degree of ambiguity in the derivation. However, if no specific information about this distribution is available, it is customary to use the so-called *conjugate prior* which ensures that the posterior $P(\tilde w|n)$ and the prior $P(\tilde w)$ belong to the same family of distributions. The conjugate prior of a Poisson likelihood distribution $P(n|\tilde w)$ is the Gamma distribution $$\label{Prior}
P(\tilde w)=\frac{\tilde\beta^{\tilde\alpha} \tilde w^{\tilde\alpha-1} e^{-{\tilde\beta} \tilde w}}{\Gamma(\tilde\alpha)}$$ which depends on two hyperparameters $\tilde\alpha$ and $\tilde\beta$ (the tilde is used to avoid confusion with rates $\alpha,\beta$ for various models used in the literature). With this prior the posterior is given by $$P(\tilde w|n) \;=\; \frac{\tilde w^{\tilde\alpha +n-1} e^{-(\tilde\beta+\tau)  \tilde w} (\tilde\beta +\tau)^{\tilde\alpha +n}}{\Gamma
   (n+\tilde\alpha )}$$ which allows one to compute the expectation value of the actual rate for the transition $c \to c'$ $$\label{Average}
\langle \tilde w \rangle = \int \d \tilde w \, \tilde w P(\tilde w|w)= \frac{\tau w+\tilde\alpha}{\tau +\tilde\beta}\,.$$ As can be seen, this formula maps a vanishing defining rate $w=0$ onto a nonvanishing actual rate $\langle \tilde w \rangle \propto 1/\tau$. By defining $\tau'=P_{c'} T$ and inserting this expectation value into the entropy production formula (\[EntropyProduction\]) and taking the limit $T \gg 1$ the entropy production for reversible transitions $$\begin{aligned}
\label{EstimateEPR}
\Delta S_{\rm env}^{\rm rev}(c\to c') & = & \ln \frac{\langle \tilde w_{c\to c'} \rangle}{\langle \tilde w_{c'\to c} \rangle}  \nonumber \\
& = &  \ln \frac{(\tau w_{c \to c'}+\tilde\alpha)/(\tau +\tilde\beta)}{(\tau' w_{c' \to c}+\tilde\alpha)/(\tau' +\tilde\beta)} \;\approx\; \ln \frac{w_{c\to c'}}{w_{c'\to c}}\end{aligned}$$ reproduces the known result in Eq. (\[EntropyProduction\]). As our main result, for irreversible transitions we obtain a finite entropy production which grows logarithmically with the observation time $$\begin{aligned}
\label{EstimateEPIR}
\Delta S_{\rm env}^{\rm irr}(c\to c') & = &  \ln \frac{\langle \tilde w_{c\to c'} \rangle}{\tilde\alpha/(\tau+\tilde\beta)} \nonumber \\
& = & \ln \frac{(\tau w_{c \to c'}+\tilde\alpha)/(\tau +\tilde\beta)}{\tilde\alpha/(\tau' +\tilde\beta)} 
\;\approx\; \ln \frac{\tau' w_{c\to c'}}{\tilde\alpha} \,.\end{aligned}$$ Note that we have simplified the derivation by replacing $\langle \ln \tilde w\rangle \to \ln \langle \tilde w \rangle$. However, as shown in Sect. 3 of the Supplemental Material [@SM], apart from a redefinition of $\tilde\alpha$, a correct derivation to lowest order leads to the same result.

The prior distribution (\[Prior\]) depends on two hyperparameters $\tilde\alpha$ and $\tilde\beta$ which determine its shape and scale. Since the entropy production depends on a ratio of rates, the scale hyperparameter $\tilde\beta$ drops out. However, the shape hyperparameter $\tilde\alpha$ appears in the final result and thus it has to be defined in a physically meaningful way. In this regard note that the Gamma distribution (\[Prior\]) evaluated at the origin is finite for $\tilde\alpha=1$, infinite for $\tilde\alpha<1$ and zero for $\tilde\alpha>1$. Therefore, in experiments of models with irreversible transitions, where the likelihood of a vanishing defining rate is expected to be finite, the most natural choice, which we will use from now on, is $\tilde\alpha=1$.

Examples
========

In what follows we study the entropy production in three exemplary nonequilibrium systems with micro-irreversible transitions in their steady state (further examples can be found in the Supplemental Material [@SM]). To this end we first determine the stationary probability distribution $P_c$ to find the system in configuration $c$. The average entropy production is then given by $$\langle \dot S_{\rm env} \rangle \;=\; \sum_{c \neq c'} P_c \, w_{c \to c'} \, \Delta S_{\rm env}(c \to c')\,,$$ where one has to sum over $\Delta S_{\rm env}^{\rm rev}(c\to c')$ or $\Delta S_{\rm env}^{\rm irr}(c\to c')$ depending on whether the transition $c \to c'$ is reversible or irreversible. For lattice models with $L$ sites, we define the average entropy production per site $$\label{AverageEntropyProduction}
\dot s_{\rm env} \;:=\; \frac{1}{L} \langle \dot S_{\rm env} \rangle\,.$$ [**TASEP:**]{}\
The first example is the Totally Asymmetric Simple Exclusion Process (TASEP) on a finite lattice with $L$ sites, where particles are added at the left boundary (removed from the right boundary) with rate $\alpha$ ($\beta$) if the first (last) lattice site is empty (occupied). Since particles hop exclusively to the right, all microscopic transitions are irreversible. The stationary probability distribution of the TASEP can be calculated exactly using the matrix product method [@DEHP], where the stationary weight of each configuration is determined by a product of noncommuting operators corresponding to the actual configuration. Using the exact results of Ref. [@DEHP] in the formulas (\[AverageEntropyProduction\]) and (\[EstimateEPIR\]) it turns out that, to leading order in $T$, the large-$L$ limit of the average entropy production rate $$\label{TASEPEntropyProduction}
\dot s_{\rm env}  \;\approx\; J(\alpha,\beta) \ln T$$ is proportional to the particle current $J(\alpha,\beta)$ in the steady state. Depending on the values of $\alpha$ and $\beta$ the current is equal to $J(\alpha,\beta)=\frac{1}{4}$ for $\alpha,\beta \ge \frac{1}{2}$, $J(\alpha,\beta)=\alpha(1-\alpha)$ for $\alpha<\beta$ and $\alpha<\frac{1}{2}$, and $J(\alpha,\beta)=\beta(1-\beta)$ for $\beta<\alpha$ and $\beta<\frac{1}{2}$. Thus the average entropy production changes continuously in the parameter space and attains its maximum for $\alpha,\beta \ge \frac{1}{2}$ where the particle current in maximal.

![(color online) Branching-Coalescing Process (BCP) at the critical point, exhibiting a diffusing Bernoulli shock. Particles are represented by black pixels while the integrated entropy production at each site is visualized by a periodically changing color scale. \[fig:bcp\]](bcp.pdf){width="100mm"}

[**BCP:**]{}\
The second example is the Branching-Coalescing Process (BCP) on a one-dimensional lattice with $L$ lattice sites and open boundaries. Each lattice site is either empty ($\0$) or occupied by at most one particle ($1$). In the bulk the BCP evolves by the dynamical rules $$\label{RULESBCP}
\begin{array}{l}
\0 1\to 11\;\mbox{with rate}\;w\\
1 1 \to 1\0\;\mbox{with rate}\;w\\
11\to \0 1\;\mbox{with rate}\;1\\
1 \0\to 11\;\mbox{with rate}\;1\\
1 \0\to \01\;\mbox{with rate}\;1.
\end{array}$$ In addition, particles are added at (removed from) the left boundary with rate $\alpha\;$ ($\gamma$) while at the right boundary particles are removed with rate $\beta$.

It is known that the steady state of the BCP can be written as a linear superposition of Bernoulli shock measures provided that $\gamma=\alpha+\frac{w}{2}-1$ [@KJS]. Moreover, under the same constraint it turns out that the model has a matrix product steady state [@Jafarpour]. Varying $w$ the process undergoes a phase transition between a high- and a low-density phase at $w_c=4$. Applying the matrix product method with the two-dimensional representation introduced in [@Jafarpour], it is straightforward to calculate the average entropy production rate per site in the steady state. To leading order in $T$ and in the large-$L$ limit one finds $$\label{APBCP}
\dot s_{\rm env}  \;\approx\; 
\left\{
\begin{array}{ll}
\frac{1}{4}\ln T& \mbox{for} \; 2(1-\alpha) < w<w_{c}, \\[2mm] 
0                                                  & \mbox{for} \; w>w_{c}.
\end{array}
\right.$$ As can be seen, the average entropy production of the BCP changes *discontinuously* at the transition point. The nonzero part of the entropy production in (\[APBCP\]) comes from both reversible and irreversible processes in (\[RULESBCP\]) [@SM]. In Fig. \[fig:bcp\] the time evolution of the entropy production at each lattice site is plotted at the critical point $w=w_{c}$. In this case the last particle in the system (the last black pixel from the left) performs an unbiased random walk on the lattice. It can be seen that only occupied lattice sites contribute to entropy production.

![Average irreversible entropy production in the AKGP as a function of $w_{2}$ for $w_{1}=0.4$, $\alpha=0.3$, $\beta=0.1$, $L=10^{6}$, and $T=10^{6}$. The phase transition occurs at $w_{2}=0.4$.\[fig:akgp\]](akgp.pdf){width="110mm"}

[**AKGP:**]{}\
The third example is the one-dimensional Asymmetric Kawasaki-Glauber Process (AKGP) with the dynamical rules $$\label{RULESAKGP}
\begin{array}{l}
1 \0\to\0\0\;\mbox{with rate}\;w_{1}\\
1 \0\to11\;\mbox{with rate}\;w_{2}\\
\01\to\0\0\;\mbox{with rate}\;w_{3}\\
\01\to11\;\mbox{with rate}\;w_{4}\\
\01\to1\0\;\mbox{with rate}\;w_{5}.
\end{array}$$ In addition, particles are added at (removed from) the left (right) boundary with rate $\alpha\;(\beta)$. In this model all transitions are irreversible. It is known that the AKGP has a matrix product steady state, which can also be written as a linear superposition of Bernoulli shock measures [@KJS], and that it exhibits a phase transition at $w_{1}=w_{2}$ from a low-density to a high-density phase. Using the two-dimensional matrix representation introduced in [@Jafarpour], one can calculate the average entropy production rate in the limit $L\to\infty$ and to leading order in $T$, obtaining $$\langle \dot{S}_{\rm env} \rangle=
\left\{
\begin{array}{ll}
\frac{2\alpha w_{1}}{\alpha+w_{1}-w_{2}}\ln T   
& \mbox{for} \; w_{1} > w_{2}, \\ & \\
\frac{2 \beta w_{2}}{\beta+w_{2}-w_{1}} \ln T
& \mbox{for} \; w_{2} > w_{1}.
\end{array}
\right.$$ Since the probability of a given configuration consisting of a particle in front of an empty lattice site is zero in the steady state, the last three processes in (\[RULESAKGP\]) do not contribute to the average entropy production, explaining why the parameters $\omega_3,\omega_4,\omega_5$ do not appear in the result. In Fig. \[fig:akgp\] the average entropy production rate is plotted as a function of $w_{2}$. In contrast to the BCP, it changes *continuously* at the transition point.

Conclusions
===========

In this Letter we have addressed the problem of entropy production in systems with irreversible transitions. For such systems the Schankenberg formula (\[EntropyProduction\]) predicts an infinite entropy production which is not seen in experiments. We suggest that the finite amount of entropy produced in experiments is related to the fact that vanishing reverse rates are impossible in nature, it is only possible to keep such rates very small. By introducing the concept of a ‘defining rate’ and an ‘actual rate’ and estimating the latter by Bayesian inference, we could specify a lower bound on the entropy production which splits up into a constant contribution for reversible transitions and an additional contribution for irreversible transitions which grows logarithmically with the observation time $T$.

Using the modified entropy production formula, we have calculated the average entropy production rate for three exactly solvable reaction-diffusion models in the steady state (further examples are discussed in the Supplemental Material [@SM]). The steady state of the BCP and AKGP are very similar in the sense that they both can be written as a linear superposition of Bernoulli shock measures, where the shock performs a simple random walk [@KJS]. However, the average entropy production behaves quite differently, namely, discontinuously in the BCP and continuously in the AKGP. This suggests that the irreversible entropy production may be used as an additional tool for the classification of nonequilibrium phase transitions.

References {#references .unnumbered}
==========

[99]{} Schnakenberg J, *Network theory of microscopic and macroscopic behavior of master equation systems,* 1976 Rev. Mod. Phys. [**48**]{}, 571.

Andrieux D and Gaspard P, *Fluctuation theorem and Onsager reciprocity relations*, 2004 J. Chem. Phys. [**121**]{} 6167.

Seifert U, *Entropy production along a stochastic trajectory and an integral fluctuation theorem*, 2005 Phys. Rev. Lett [**95**]{}, 040602.

Seifert U and Speck T, *Fluctuation-dissipation theorem in nonequilibrium steady states*, Europhys. Lett. [**89**]{}, 10007 (2010).

Hinrichsen H, Gogolin C, and Janotta P, *Non-equilibrium dynamics, thermalization and entropy production*, 2011 J. Phys.: Conf. Ser. **297**, 012011.

see attached supplemental material.

Korda PT, Taylor MB, and de Grier G, *Kinetically locked-in colloidal transport in an array of optical tweezers*, Phys. Rev. Lett. **89**, 128301 (2002).

Lacasta AM, Sancho JM, Romero AH, and Lindenberg K, *Sorting on Periodic Surfaces*, Phys. Rev. Lett. **94**, 160601 (2005).

de Oliveira Rodrigues JE and Dickman R, *Asymmetric exclusion process in a system of interacting Brownian particles*, Phys. Rev. E **81**, 061108 (2010).

see e.g. Hinrichsen H, *Nonequilibrium phase transitions*, 2000 Adv. Phys. **49**, 815.

Takeuchi KA, Kuroda M, Chat[é]{} H, and Sano M, *Directed percolation criticality in turbulent liquid crystals*, 2007 Phys. Rev. Lett. 99, 234503.

ben-Avraham D, Dorosz S, and Pleimling M, *Entropy production in nonequilibrium steady states: A different approach and an exactly solvable canonical model*, 2011 Phys. Rev. E 84, 011115.

see e.g. Box GEP and Tiao GC, *Bayesian Inference in Statistical Analysis*, Wiley and Sons, New York (1990).

Derrida B, Evans MR, Hakim V, and Pasquier V, *Exact correlation functions in an asymmetric exclusion model with open boundaries*, 1993 J. Phys. A: Math. Gen. [**26**]{} 1493.

Krebs K, Jafarpour FH, and Schütz GM, *Microscopic structure of travelling wave solutions in a class of stochastic interacting particle systems*, 2003 New J. Phys. [**5**]{} 145.

Jafarpour FH and Masharian SR, *Matrix product steady states as superposition of product shock measures in 1D driven systems*, 2007 J. Stat. Mech: Theor. Exp. P10013.
