content_image/320247.jpg content_image/320246.jpg content_image/320245.jpg content_image/320244.jpg content_image/320243.jpg
cycle constraint ( q, c ) and average power constraint γ can be solved from the following optimization: 

<img src='content_image/320247.jpg'>

In the special case of no transition cost, i.e., c = 0 , the result of (93) is equal to that of (5). 

C. Proof of Theorem 2 

The mutual information expressed by (85) is hard to optimize, even if the input is restricted to Markov processes. To simply the matter, we introduce a lower bound of I ( µ ) , which is given by L ( µ ) in (11). 

Property (a): Using the fact that processing reduce relative entropy and µ is specified as a stationary probability distribution, we have 

<img src='content_image/320246.jpg'>

Therefore 

<img src='content_image/320245.jpg'>

using the fact that the Ces´aro mean of sequence I ( X$_{1}$, X k $_{2}$) is I ( X$_{1}$ ; X ∞ 2 ) . Applying (85), (87) and (97), 

Thus Property (a) is established. 

<img src='content_image/320244.jpg'>

Property (b): For any µ ∈ Λ( γ, q, c ) , which is not Markov in general, its first-order Markov approximation ν is defined by 

<img src='content_image/320243.jpg'>

