## Gradient Flow Decoding

Tadashi Wadayama Nagoya Institute of Technology wadayama@nitech.ac.jp Lantian Wei Nagoya Institute of Technology wei.lantian@nitech.ac.jp

Abstract -This paper presents the Gradient Flow (GF) decoding for LDPC codes. GF decoding, a continuous-time methodology based on gradient flow, employs a potential energy function associated with bipolar codewords of LDPC codes. The decoding process of the GF decoding is concisely defined by an ordinary differential equation and thus it is well suited to an analog circuit implementation. We experimentally demonstrate that the decoding performance of the GF decoding for AWGN channels is comparable to that of the multi-bit mode gradient descent bit flipping algorithm. We further introduce the negative log-likelihood function of the channel for generalizing the GF decoding. The proposed method is shown to be tensor-computable, which means that the gradient of the objective function can be evaluated with the combination of basic tensor computations. This characteristic is well-suited to emerging AI accelerators, potentially applicable in wireless signal processing. The paper assesses the decoding performance of the generalized GF decoding in LDPC-coded MIMO channels. Our numerical experiments reveal that the decoding performance rivals that of established techniques like MMSE+BP. Furthermore, an exploration of score-based channel learning for capturing statistical properties is also provided.

They play a crucial role not only in accelerating the training process of these complex models but also in enhancing the efficiency of inference tasks.

In recent years, optical integrated programmable circuits based on the Mach-Zehnder Interferometers (MZI) have garnered significant interest from researchers [7], [8]. An MZI is an optical component that consists of phase shifters, which alter the phase of the input light beams, and beam splitters, which divide it. As an MZI has a two controllable parameters, it can be seen as a programmable optical component. Optical matrix-vector product (MVP) circuit implemented using MZIs has been actively studied [9]. The programmable MZI-based MVP circuit is a promising candidate of the next generation AI accelerators because of its significant computation speed and power efficiency. A recent example is a neural network implemented in the optical domain [10].

Index Terms -LDPC codes, ordinary differential equations, gradient descent, tensor computation, MIMO channel, MIMO detection

## I. INTRODUCTION

## A. Background

Low-Density Parity-Check (LDPC) codes [3] have emerged as a cornerstone in the ongoing advancement of contemporary wireless communication technologies. Their role is becoming increasingly critical, and LDPC codes are expected to be necessary in ensuring reliable data transmission in the forthcoming 6G systems. The advent of 6G networks represents a substantial leap forward in communication technology, demanding levels of technical innovation. For example, the seamless integration of Artificial Intelligence (AI) with signal processing stands as a pivotal challenge in the realization of 6G networks [4]. This integration is essential not only for enhancing network efficiency and capability but also for unlocking new potentials in wireless communications.

The recent remarkable achievements of Large Language Models (LLMs), such as GPT-4 [5], have significantly influenced the technological landscape, particularly in the field of artificial intelligence. These successes have acted as a catalyst in the development of fast and power-efficient AI accelerators [6], including the next generation of Graphics Processing Units (GPUs). These advanced devices are specifically designed to cater to the demanding computational needs of AI models.

This work was supported by JSPS KAKENHI Grant Number JP22H00514.

RRAM (resistive random-access memory)-based analog computing in electrical domain is another promising technology for analog computing. Wang et al. [12] recently presented a continuous-time resistive memory circuit for solving compressed sensing problem. In [13], RRAM (resistive random-access memory)-based analog computing is applied for implementing MIMO precoding problems. It is demonstrated RRAM-based matrix-vector product circuits achieve high throughput and energy efficiency.

For this direction, a next-generation decoding algorithm needs to be 'tensor-friendly', i.e., each internal process of the decoding algorithm should be efficiently executable on an AI accelerator. This implies that each subprocess must be based on tensor computation such as tensor product, tensor addition, and component-wise function application, which is optimally managed by the AI accelerator. In other words, the core of algorithm should comprise tensor-friendly computations to fully leverage the power of the AI accelerator. A

By offering significant enhancements in speed and energy efficiency, these next-generation AI accelerators could facilitate more extensive and practical applications across various fields, including wireless signal processing. Traditionally, a practical LDPC decoder is often implemented in an Application Specific Integrated Circuit (ASIC) for achieving sufficiently high throughput and energy efficiency. However, the advancements in AI accelerators may be paving the way for the practical implementation of LDPC decoders on AI accelerators in the near future because programability provided by AI accelerators yields significant flexibility in wireless signal processing tasks handling various demands such as rate, latency, code type, and signal format.

tensor-friendly decoding algorithm also facilitates batch based decoding, allowing for the simultaneous handling of multiple codewords. A batch-based algorithm promotes the efficient use of the AI accelerator.

Furthermore, a tensor-friendly algorithm is a differentiable algorithm , i.e., every part of the algorithm is differentiable. This means that an automatic differentiation mechanism including back propagation can be easily and efficiently applied to compute the gradients of the internal trainable variables. We can embed a learnable block or learnable variables into the algorithm. For example, we can immediately use deep unfolding for performance improvement. Such a 'AI-friendly' property is another benefit of a tensor-friendly algorithm.

## B. Contribution

The objective of this study is to devise a tensor-friendly decoding algorithm for LDPC codes. To achieve the goal, we begin by a decoding approach based on a non-convex optimization problem. Utilizing the gradient flow dynamics to minimize the objective function, we can derive a continuous time system for decoding LDPC codes. This paper presents the Gradient Flow (GF) decoding for LDPC codes. GF decoding, a continuous-time methodology based on gradient flow, employs a potential energy function associated with bipolar codewords of LDPC codes. This approach utilizes a potential energy function akin to the objective function employed in the Gradient Descent Bit Flipping (GDBF) algorithm [14].

The key contributions of this paper are outlined as follows:

- · Introduction of the GF decoding method for AWGN channels.
- · Expansion of GF decoding through the inclusion of the channel's negative log-likelihood function, enabling its application to a broad range of channels.
- · Demonstration that GF decoding is tensor-friendly, making it compatible with future AI accelerators.
- · Development of a discretized version of GF decoding, analogous to a gradient descent method, tailored for digital AI accelerators such as GPGPUs (General-Purpose computing on Graphics Processing Units).
- · Exploration on score-based channel learning for capturing statistical property.

The GF decoding can be regarded as the continuous-time counterpart of the GDBF algorithm since both approaches utilize fairly similar objective functions. The simplicity of the method enhances its suitability for implementation on AI accelerators. A notable feature of GF decoding is its differentiability , which allows for smooth integration with other AI components, such as a neural network mimicking a negative log likelihood of the target channel.

## C. Outline

The concept of GF decoding was initially presented in the conference papers [1] and [2]. The current paper enriches the initial presentation by offering a thorough exposition on the detailed derivation of GF decoding, an enhanced treatment of tensor-computability, and extensive experimental validations.

Furthermore, this paper newly introduces the concept of scorebased channel learning. These enhancements are anticipated to substantially bolster the understanding of the proposed method, offering deeper insights.

The structure of this paper is organized as follows: Section II briefly reviews related works and introduces basic notation related to LDPC codes. Section III presents the idea of GF decoding for AWGN channels. Section IV discusses 'tensorcomputability,' a rather simple computation model useful for AI accelerators. Section V broadens the scope of GF decoding to encompass general channels. Section VI shows the results of numerical experiments on MIMO channels. Section VII discusses score-based channel learning suitable for GF decoding. Finally, Section VIII provides a conclusive discussion.

## II. PRELIMINARIES

## A. Related works

The method presented in this paper can be classified as part of the optimization-based decoding algorithms for LDPC codes. A number of works have been developed in this class. The seminal work on Linear Programming (LP) decoding by Feldman [16] clearly formulated the LDPC decoding problem as an LP problem. Applications of interior point methods for solving the LP problem have been discussed in [17] [18]. A gradient descent formulation of a non-convex objective function leads to the GDBF algorithm [14], which introduced optimization perspective into the bit-flipping decoding algorithms. Some variants of the GDBF algorithm, such as the noisy GDBF algorithm [19], have shown considerable improvement in decoding performance over a simple GDBF algorithm. Zhang et al.[20] and Barman et al.[21] applied Alternating Direction Method of Multipliers (ADMM) for solving the Feldman's LP problem, which leads to ADMM decoding. ADMM decoding has been studied by many researchers and has become a decoding algorithm quite competitive with BP in both decoding performance and decoding complexity.

## B. Notation

Assume that a sparse binary parity check matrix H = { H ij } ∈ F m n × 2 is given. The LDPC code ˜ ( C H ) is defined by

The binary to bipolar transform β : F 2 →{ 1 , - } 1 defined by β (0) ≡ 1 and β (1) ≡ -1 transforms ˜ ( C H ) into the bipolar code defined by

$$\tilde { C } ( H ) \equiv \{ b \in \mathbb { F } _ { 2 } ^ { n } \ | \ H b = 0 \}. \quad \ \ ( 1 ) \\ \dots \dots \dots \dots \dots \dots \dots \cdots \cdots \cdots \cdots \cdots \cdots.$$

$$C ( H ) \equiv \{ \beta ( b ) \in \{ 1, - 1 \} ^ { n } \ | \ b \in \tilde { C } ( H ) \}. \quad ( 2 ) \\ \tau _ { 1 } \cdots \dots \dots \dots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots$$

The index sets A i ( ) and B j ( ) are defined as

$$A ( i ) & \equiv \{ j \ | \ j \in [ n ], H _ { i, j } = 1 \}, \quad i \in [ m ], \quad \ ( 3 ) \\ R ( i \, \real \, i \, + \, i \, \equiv [ m ] \ H. \, - \, 1 \, \real \, i \, \equiv [ m ] \quad \ ( \Lambda )$$

respectively. The notation [ n ] denotes the set of consecutive integers { 1 2 , , . . . , n } . The parity check condition for the LDPC code ˜ ( C H ) is

$$B ( j ) \equiv \{ i \ | \ i \in [ m ], \vec { H } _ { i, j } = 1 \}, \quad j \in [ n ], \quad \ \ ( 4 )$$

$$\mathbb { P C } \, \text{code} \, & C ( H ) \text{ is} \\ & \forall x \in \tilde { C } ( H ), \quad \forall i \in [ m ], \sum _ { j \in A ( i ) } x _ { j } = 0. \quad \quad ( 5 )$$

This corresponds to the parity check condition for the bipolar LDPC code C ( H ) , which is

A function g : C → C can be applied to a vector x ∈ R n as

$$\mathcal { U } \cos \omega ( \mathcal { H } ), \, \text{win} \, & \text{We} \\ \forall x \in C ( H ), \quad \forall i \in [ m ], \, \prod _ { j \in A ( i ) } x _ { j } = 1. \quad \ ( 6 ) \quad \text{word} \,. \\ \text{A function } a \cdot \mathbb { C } \to \mathbb { C } \, \text{can be anniled to a vector } x \in \mathbb { R } ^ { n } \quad.$$

$$g ( x ) \equiv ( \begin{matrix} g ( x _ { 1 } ), g ( x _ { 2 } ), \dots, g ( x _ { n } ) ), \\ \vdots \end{matrix} ), \quad \begin{matrix} ( 7 ) \\ \vdots \end{matrix} \, \sim n \, \begin{matrix} \vdots \end{matrix} \, \cdot \end{matrix} \, \cdot \, \cdot \, \cdot \, \cdot \, \cdot \,.$$

where x = ( x , . . . , x 1 n ) ∈ C n . Namely, a scalar function g can be component-wisely applicable to a vector x . For a pair of vectors a = ( a , . . . , a 1 n ) ∈ C n , b ≡ ( b 1 , . . . , b n ) ∈ C n , we define the multiplication and division on two vectors by ab ≡ ( a b , . . . , a 1 1 n n b ) and a b / ≡ ( a /b , . . . , a 1 1 n /b n ) , respectively.

## III. GRADIENT FLOW DECODING FOR AWGN CHANNELS

## A. Gradient flow dynamics

The gradient flow dynamics, also known as steepest descent dynamics, is a continuous-time dynamics defined by an Ordinary Differential Equation (ODE):

B. Gradient flow decoding for AWGN channel

We assume an AWGN channel. A transmitter send a codeword s ∈ C ( H ) and a receiver obtain a received word

$$y = s + n,$$

where n ∼ N ( 0 , σ 2 n I ) .

Let f be a potential energy function defined by

$$f ( x ) \equiv \frac { 1 } { 2 } \| x - y \| ^ { 2 } + h _ { \alpha, \beta } ( x ). \quad \quad ( 1 0 ) \\ \dots \dots \dots \dots \dots$$

The first term of the potential energy function can be regarded as the negative log likelihood function for an AWGN channel. The second term is a penalty term attracting x to a codeword of C ( H ) .

Definition 1: The code potential energy function for C ( H ) is a multivariate polynomial defined as

$$\intertext { N E L S } & \text{ is a multivariate polynomial defined as } \\ \text{lescent} \quad & h _ { \alpha, \beta } ( x ) \equiv \alpha \sum _ { j = 1 } ^ { n } ( x _ { j } ^ { 2 } - 1 ) ^ { 2 } + \beta \sum _ { i = 1 } ^ { m } \left ( \left ( \prod _ { j \in A ( i ) } x _ { j } \right ) - 1 \right ) ^ { 2 }, \\ & \text{ where } x = ( x _ { 1 }, \dots, x _ { n } ) ^ { T } \in \mathbb { R } ^ { n }. \text{ The parameters } \alpha \in \mathbb { R } _ { + } \text{ and }$$

$$\frac { d x ( t ) } { d t } = - \nabla f ( x ( t ) ), \quad \begin{array} { c c c c } & \text{when} \\ & \beta \in \\ & \text{term} \end{array}$$

The first term on the right-hand side of (11) represents the bipolar constraint for x ∈ { +1 , - } 1 n , and the second term corresponds to the parity constraint induced by H , i.e., if x ∈ C ( H ) , we have where x = ( x , . . . , x 1 n ) T ∈ R n . The parameters α ∈ R + and β ∈ R + control the relative strength of the first and second terms.

for any i ∈ [ m ] .

$$\text{ave} \quad \left ( \prod _ { j \in A ( i ) } x _ { j } \right ) - 1 & = 0 \quad \left ( 1 2 \right ) \\ m \right ]. \, \begin{matrix} \quad & ( 1 2 ) \\ \quad & \quad & \quad & \quad & \\ \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \ \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \end{matrix}$$

The code potential energy h α,β ( x ) is inspired by the nonconvex parity constraint function used in the GDBF objective function [14]. The sum-of-squares form of (11) directly implies the most important property of h α,β ( x ) , i.e., the inequality where f : R n → R is a potential energy function. The system's state x : R → R n evolves to reduce the potential energy f as the continuous time variable t increases. This continuous dynamical system can be viewed as the continuous counterpart of the gradient descent method. If the potential function f is strictly convex, the equilibrium point of the dynamics coincides with the minimum point of f . Therefore, the gradient flow dynamics can be seen as a continuous-time minimization process of the potential energy function. If f is a non-convex function, then the gradient flow dynamics finds a stationary point of f as t →∞ . Main idea Main idea A decoding problem is formulated a minimization problem of

$$h _ { \alpha, \beta } ( x ) & \geq 0 \\. \ m n \ e n \quad \ddots \ \cdots \ \cdots \ \cdots \ \cdots \. \ \cdots \. \.$$

Figure 1 presents an example of the gradient flow dynamics for a simple convex energy function E defined on two dimensional Euclidean space. The state vector following the ODE (8) gradually approaches to the minimum point of E . The gradient flow can be considered as the continuous-time counterpart of the gradient descent method. A decoding problem is formulated a minimization problem of a potential function introduction of gradient flow dynamics minimizing the potential function Potential energy function (objective function) a potential function introduction of gradient flow dynamics minimizing the potential function Potential energy function (objective function)

n

m

2 holds for any x ∈ R n . The equality holds if and only if x ∈ C ( H ) .

f

(

x

)

y

x

2

f

(

x

)

≡ ‖

∑

∑



∏

Gradient flow dynamics



Gradient flow dynamics i.e.,

x

A i

( )



∏ A i ( ) x j   -1   2 (1) j   -1   (1) The gradient flow decoding for AWGN channel is simply a gradient flow dynamics based on the potential energy function f that is given by d x ( ) t dt = -∇ f ( x ( )) t . (14)

j

∈

This ODE can be rewritten as follows.

))

(2) (2) Definition 2 (Gradient flow decoding for AWGN channels): The GF decoding is defined by the ODE:

i.e.,

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-fb4b3ea8-fb92-4498-a588-89157b3825e8.jpg)

gradient descent method in continuous-time.

Fig. 1. Minimization process of a convex function in a gradient flow dynamics system: x ( ) t represents the solution to the ordinary differential equation (8). The equilibrium points become the minimum points of the convex function f .

$$\frac { d x } { d t } = - ( x - y + \nabla h _ { \alpha, \beta } ( x ) ) \quad \quad ( 1 5 ) \\ \frac { d x } { d t } = -$$

$$x ( 0 ) = x _ { 0 },$$

where h α,β ( x ) is the code potential energy and x (0) = x 0 ∈ R n is an initial point.

In general, there exist stationary points satisfying

$$f ( x ) > 0, \quad \nabla f ( x ) = 0. \quad \text{ \quad \ \ } ( 1 7 )$$

2

2



Fig. 3. Heat map of h rep ( x ) .

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-8ab323da-afe0-4a01-855f-691c637f7e48.jpg)

0 1 , , . . . , N ) defines the boundaries of the bins where the width of a bin is given by η ≡ T/N. It should be noted that the choice of the bin width η is crucial in order to ensure the stability and the accuracy of the Euler method. By using the Euler method, the ODE (15) can be approximated by

$$x ^ { ( k + 1 ) } = x ^ { ( k ) } - \eta \nabla f ( x ^ { ( k ) } )$$

for k = 0 1 2 , , , . . . , N -1 . The initial condition of this recursion is x (0) = x 0 .

## E. Example of Code potential energy function

These stationary points are called non-codeword stationary points . Additionally, points that satisfy

$$f ( x ) > 0, \nabla f ( x ) \simeq 0 \quad \ \ ( 1 8 ) \\.. \quad. \. \. \. \quad. \quad.$$

rather than equality are called pseudo non-codeword stationary points . When iterative optimization techniques that use gradient information, such as gradient methods, are applied, these non-codeword stationary points or pseudo non-codeword stationary points can cause the search vector's movement speed to drastically reduce near the stationary points, leading to traps at the stationary points. These undesirable stationary points may negatively impact decoding performance because the search points may get trapped near them, preventing them from reaching the codeword stationary points.

## C. Analog circuit implementation

The formulation of GF decoding is useful for continuous time signal processing by an analog circuit with feedback signals. Figure 2 presents a block diagram of the analog circuit realizing the continuous-time dynamics given by the generalized GF-ODE (15). The left box represents an analog circuit evaluating the value of -( x -y + γ ∇ h α,β ( x )) . A similar optical circuit implementing an Ising machine with feedback loop was reported in [11]. In principle, an optical implementation of the analog circuit depicted in Fig.2 would be possible if the gradient part -( x -y + γ ∇ h α,β ( x )) can be implemented with an optical circuit. JOURNAL OF L A T E X CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020

(15) As a simple example, suppose the potential energy function of the b C rep ( n = 2 ) below. The code C rep is



A tensor-friendly implementation of the circuit ed here is given by evaluating ∇ h C α,β ( x ) is to be discussed in Subsection IV-B. The matrixvector product in optical domain could be implemented with e value of ipolar repetition code programmable optical switches when we use a quasi-cyclic plementing given by LDPC codes. Ising left box represents an analog circuit evaluating th -( x -y + γ ∇ ≡ h α,β ( x )) . A similar optical circuit im an machine with feedback loop was reported In principle, an appropriate potential rgy function for ( H )   -1  2 , Hence, the approximate MAP rule consider ˆ x argmin x ∈ R n L ( x y ; ) + γh α,β ( x ) . B. Generalized GF decoding in

[9].

depictede previous subsection, the concept of an approximate this case, the corresponding potential energy function is in OURNAL OF L A T E X CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 Fig.2 would be possible if the gradient part J In th In

∏

x

j

∈

A i

( )

arameters can be implemented with an optical



α

straints.

evaluating

$$\Big | \quad h _ { r e p } ( \boldsymbol x ) = ( x _ { 1 } ^ { 2 } - 1 ) ^ { 2 } + ( x _ { 2 } ^ { 2 } - 1 ) ^ { 2 } + ( x _ { 1 } x _ { 2 } - 1 ) ^ { 2 }, \quad ( 2 1 )$$

in

j

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-8533e2df-97c2-4ab5-9901-18fd5a26e365.jpg)

is inspired by the non-

form directly implies

the

GDBF objec-

In principle,

an optical

∇

it can

0

The shape

at these

be the

two when

takes

a

minimum value of

it ca

(0 0)

0

codewords

Integrator depicted ossibilities ing corresponding to the GFin Fig.2 would imization be possible if gradient part -( x -y + γ ∇ h α,β ( x )) can be implemented with an optical circuit. A tensor-friendly implementation of the circuit evaluating . ∇ h α,β ( x ) is to be discussed in Subsection IV-B. Integrator x ) , i.e., the inequality alog circuit for generalized GF decod e equality holds if and G FOR GENERAL NELS (MAP) decoding imate MAP decodingumerical method is required to solve the ODE. The coding []. Assume thatnumerical method for solving simultaneous nonlinear ) to a given channel. ial density function (PDF), nce ikelihood is defined as r-order | x ) . (9) ufficiently nel is MAP decoding There are many p of opt methods. example, the proximal decoding is to solve this continuous minimization problem by using a proximal gradient descent method [][]. In the following, we will solve this minimizationn problem by using the gradient flow. The ODE for the generalized GF decoding is given by of d x dt = - ∇ ( L ( x y ; ) + γ ∇ h α,β ( x )) (17) x (0) = x 0 . (18) l This formulation is useful for continuous time signal processing by an analog circuit. C. Tensor-computability Fig. rate Analog circuit erformance of the GF decodingected that its movement speed will be extremely slow. 2. e for he generalized nterval GF decoding corresponding optimization techniques such as gradient methods are used, a to the GF-ODE (15). The figure should be revised appropriately. Fig. 3. Heat map ofords in h rep ( x ) . The heatmap representation of the potential energy fun h rep ( x ) is shown in Fig. 3. From this figure, seen that the function Fig. 2. Analog circuit for generalized GF decoding corresponding to the GF-ODE (15). D. Euler method To evaluate th decoding p if (15), e a numerical method is required to solve the ODE. The simplest numerical method for solving simultaneous nonlinear Fig. 3 Heat map of h rep ( x ) . The heatmap representation of the potential energy function h rep ( x ) ises such as gradient methods are used, a shown in Fig. 3. From this figure, seen attracted towards the dir that the function takes ction of the codewords of a minimum value locations other hand, the orig of the codew (+1 +1) ( , , -1 , -1) . of the function holds. h rep ( x ) resembles f a state valley vector with codewords t descent dynamics passes near the origin, that at its base. Therefore, it is expected state vector is attracted towards the direction of the codewords will occur. On the other hand, the origin Fig. 2. An ODE (15). D. Euler method To evaluate the decoding performance of the GF decoding (15), a n simplest different equations is the Euler method [24]. While the converge order of the Euler method is lower than that of highe methods, it is straightforward to use and can provide s accu solutions t time i is sufficiently fin discretized. Therefore, in the present study, we will use the Euler method to solve the ODE (15). The heatmap representation of the potential energy function h rep ( x ) is shown in Fig. 3. From this figure, it can be see that the function takes a minimum value of 0 at the locations of the codewords (+1 +1) ( , , -1 , -1) . The shape the function h rep ( x ) resembles a valley with these two codewords at its base. Therefore, it is expected that when optimization techniqu state vector is wil occur. On the (0 0) , is a maximum point, and ∇ h rep ( 0 ) = 0 Therefore, i following the gradien it is exp

## locations of the the point, and h rep ( 0 ) = 0 F. Example of GF Decoding Process

a

,

holds.

(+1 +1) (

,

,

-

1

,

-

1)

.

The

s

is a maximum

Therefore, if a state

vector sufficiently

) p ( x ) , (10) bove eneral, MAP decoding hus need to consider an Our target hardwares are AI-accelerators including GPU and programable optical circuits based on MZI. These hardware are especially efficient for tensor-based computations. We here GF decoding introduce the concept of tensor-computability. ired D. Euler method To evaluate the decoding performance of the (15), a numerical method is requ he differential equire equationsimulate the dynamics defined While is the Euler method [21]. convergence 5) in the time intervalmethod order of the Euler is lower of higher-order methods, it is straightforward to use and can . Suppose tha provide sufficiently discrete-time accurate solutions if t Suppose that we r t s by the a ODE (1 0 ≤ t ≤ T . The interval is divided into N bins where N denotes the number of discretized intervals. The ticks t k = kη k ( =

to solve the ODE. The hat a transmitted word is of the function h rep ( x ) resembles a valley with these codewords at its base. Therefore, it is expected that optimization techniques such as gradient methods are us state vector is attracted towards the direction of the code than We thatrstfollowing the gradient descent dynamics passes near the origin, time interval . isssume t ∇ it is expected that its movement speed will be extremely slow. fi present a small example to illustrate a decoding process we have a repetition code of ength 2, C rep A x = (1 1) , and that

Definition

1:

f

:

n

n

simplest numerica will occur. On the other hand, the origin

is a maxi fine

discretized.

Therefore, in

the present

study, d on the

code

F.

Example of GF Decoding Process

C

(

H

)

tensor-computable

→

The function l method for solving simultaneous nonlinear

C

C

is said

to be

we will use the Euler method t

Suppose that we re convergence quire

(0 0)

,

t

s

f

differential equations o solve the O

is the

Eu method

combination of the following basic operations:

While the

point,

h

(

0

)

=

0

following the gradient descent dynamics passes near the or rep

holds.

Therefore, if a state

v

process. Suppose that we have a repetition code of length 2, order o of imulate

be ler DE (15). [21

th evaluated

dynam by a ].

We first and t

presen

a

∇

small example to illustrate

a

decoding the

Euler e methodics defined than is

lower that

if the

value of

can evaluating A similar optical circuit implementing ants feedback loop was (16) reported d that by substituting the codewords into in [9]. implementation of the analog circuit the α,β ( x ) . For ential energy value becomes where the const α and β are set to one. It can be easily confirme h rep ( x ) , the pot 0 , and for any x , h rep ( x ) ≥ 0 .

an optical implementation of the analog circuit

$$C _ { r e p } \equiv \{ ( 1, 1 ), ( - 1, - 1 ) \}. \text{ \quad \ \ } ( 2 0 )$$

4

the corresponding received word is y = (0 6027 0 8244) . , . . In this case, the ODE (15) for the GF decoding becomes

$$& \text{this case, the ODE (15) for the $i$ decoding becomes } \quad \text{We } \text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text	$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text
$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text#\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{	$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\test{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\test$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text}$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text'$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text`$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text<$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text1$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text.$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text^{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$
$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text{$\text$$

Fig. 4. Example of solution curve. The repetition code of length 2 is assumed.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-d810059b-c490-4bcf-9a95-c3cf3cdaf61e.jpg)

The initial condition for the ODE is set to x (0) = (0 0) , . Figure 4 shows the solution curve of the ODE, with the white small circle representing the initial point and the black small circle indicating the equilibrium point (0 9642 0 9901) . , . . The solution curve is obtained numerically using the Euler method. In Fig. 4, the contour curves of the potential energy function f ( x ) are also plotted. We can observe that the solution curve is orthogonal to the contour curves because the solution path follows the negative gradient vector field of the potential energy. This means that the curve is a steepest descent curve for f . By rounding the equilibrium point, we obtain ˆ = (1 1) x , , which is the correct estimated word.

We then show solution curves for an LDPC code. Figure 5 displays the solution curves of each element in the state vector x , where x i ( ) t (for i ∈ [204] ) represents the solution of the ODE (15) plotted as a function of time t . Notably, all solution curves are smooth and continuous. It can be seen that the equilibrium point is in proximity to a bipolar vector.

Fig. 5. Solution curves. (3,6)-regular LDPC codes of length 204.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-ad92be39-268b-4eb8-968b-18fa88b63e45.jpg)

.

## G. BER Performance of GF decoding

We evaluated the bit error rate (BER) of GF decoding through computer simulations on LDPC codes with design rate 1/2, the (3 , 6) -regular LDPC codes (96.33.964, 204.33.484, PEGReg252x504, PEGReg504x1008) [25]. We used BP decoding as the baseline and set the maximum iteration of BP to 100. The parameter setting of GF decoding is summarized in Table I.

Figure 6 displays the BER performance of the proposed GF decoding. Compared to BP, the GF decoding has a BER performance that is approximately 2dB worse. Notably, for PEGReg504x1008, the GF decoding performance is almost on par with the multi-mode GDBF algorithm using 100 iterations ([14], Fig. 3). Overall, GF decoding's BER performance is comparable to that of bit flip-type decoding algorithms.

TABLE I PARAMETER SETTING OF GF DECODING.

| Potential energy parameters   | α = 1 , β = 2             |
|-------------------------------|---------------------------|
| Parameters of Euler method    | T = 10 , N = 1000         |
| Sampling time                 | t = 10                    |
| Encoding                      | Uniformly random codeword |

Fig. 6. Bit error rate (BER) of the GF decoding for AWGN channels. The BERs of BP are also included as benchmarks.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-e07f23b4-7f3c-4af2-8e6b-9a5beb0de733.jpg)

## IV. TENSOR-COMPUTABILITY

## A. Computation model

Our target hardware includes AI accelerators, such as GPGPUs, and programmable optical circuits. Such hardware is especially efficient for tensor-based computations. Here, we introduce the concept of tensor-computability to clarify our computation model.

Definition 3: The function f : C n → C n is said to be tensor-computable if the value of f can be evaluated by a combination of the following basic operations:

- · Tensor product, e.g., matrix-vector product.
- · Component-wise function applications to a tensor
- · Scalar multiplication, e.g., α x ( α ∈ C , x ∈ C n ) .
- · Vector addition and subtraction, e.g., a + b a , -b .

A 'tensor-computable' function can be naturally evaluated with parallel computation because its basic operations inherently has parallelism. This concept of tensor-computability could also be viewed as the essential criteria for basic, efficient operations that a future AI accelerator should support. While contemporary AI accelerators have the capacity to handle more complex tensor operations, we restrict the permissible operations to maintain a straightforward and clear definition of tensor-computability.

In the realm of optical computing, MZI-based Matrix-Vector Product (MVP) circuits have already been implemented [8], and the computation of logarithmic and exponential functions in the optical domain has been explored [22]. Optical switches and crossbar can be used for constructing a programmable MVP circuit with a permutation matrix. An implementation of RRAM-based analog matrix computing is discussed in [13]. We may be able to implement a tensor-computable function with such optical or electrical devises, which provides high processing speed and energy efficiency.

An additional advantage of tensor-computable functions is their compatibility with batch processing , crucial for fully leveraging the computational power of GPGPUs. Moreover, this batch processing aligns seamlessly with contemporary neural network frameworks like PyTorch, TensorFlow, and JAX, indicating that tensor-computable functions are wellsuited for AI tasks. In essence, a tensor-computable function is highly AI-friendly as well.

## B. Tensor computability of Gradient of code potential energy

For implementing the GF decoding, we need to evaluate the gradient of the code potential energy. The first order derivative of the potential energy can be easily evaluated as

$$& \text{of the potential energy can be easily evaluated as } \quad \text{vector}. \\ & \frac { \partial } { \partial x _ { k } } h _ { \alpha, \beta } ( x ) \\ & \quad = 4 \alpha ( x _ { k } - 1 ) ( x _ { k } + 1 ) x _ { k } \\ & \quad + 2 \beta \sum _ { i \in B ( k ) } \left ( \left ( \prod _ { j \in A ( i ) } x _ { j } \right ) - 1 \right ) \left ( \prod _ { j \in A ( i ) \langle \{ k \} } x _ { j } \right ). \ \ ( 2 2 ) \quad \text{which} \quad \text{It} \text{ $sh} \quad \text{in Lem} \\ & \text{$\cdot\cdot\cdot\cdot\cdot$} } \quad \text{$\cdot\cdot\cdot\cdot$} } \quad \text{$\cdot\cdot\cdot\cdot$} } \quad \text{$\cdot\cdot\cdot\cdot$} } \quad \text{in Lem}$$

$$\begin{array} { c } \text{$w_{\nu_{\alpha,\beta} \aleph_{v}$ is using $\varepsilon$} \nu_{v}$} \\ \nabla h _ { \alpha, \beta } ( x ) \equiv \left ( \frac { \partial } { \partial x _ { 1 } } h _ { \alpha, \beta } ( x ), \dots, \frac { \partial } { \partial x _ { n } } h _ { \alpha, \beta } ( x ) \right ) ^ { T }. \quad ( 2 3 ) \quad \text{improve} \\ \text{The followin} \ \text{lemma indicates tensor-commitability of the} \end{array} \quad \text{time. $T$}$$

The gradient ∇ h α,β ( x ) is thus given by

The following lemma indicates tensor-computability of the gradient of the code potential energy.

Lemma 1: The gradient ∇ h α,β ( x ) can be evaluated by

$$\nabla h _ { \alpha, \beta } ( x ) = 4 \alpha \exp \left ( z _ { - 1 } + z + z _ { + 1 } \right ) + 2 \beta \exp \left ( w - z \right ), \quad \text{demons} ^ { \prime } \text{ effective}$$

where z z , -1 , z +1 , w are defined by

$$z & \equiv \ln ( x ), & ( 2 5 ) \quad \ln ( x ) \\ \tilde { \ }. & = \ln / \tilde { \omega } - 1 \, \lambda & \tilde { \omega } \quad \dots.$$

$$z _ { + 1 } & \equiv \ln ( x + 1 ), \quad \begin{array} { c c c } ( 2 7 ) & \text{model} \\ \pi = \ln / \mathcal { U } ^ { T } / \omega \nu / \mathcal { U } \nu \cdot \frac { \nu } { \nu } / \mathcal { U } \nu \cdot \frac$$

$$z _ { - 1 } & \equiv \ln ( x - 1 ), & ( 2 6 ) \quad \text{comp} \\ \tilde { \ } \dots & = \ln / \tilde { \alpha } \perp \mathbf 1 \, \searrow \quad \text{mod}$$

$$w \equiv \ln ( H ^ { T } \left ( \exp ( 2 H z ) - \exp ( H z ) ) \right ) \quad ( 2 8 ) \quad \text{Below} _ { \text{parall} }.$$

if x = 0 . This implies that ∇ h α,β ( x ) is tensor-computable.

̸

(Proof) From Eq.(25), we have x i = exp( z i ) , and it results in

$$\frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac { 1 } { 2 } \frac {$$

$$\text{From Exp.} ( 2 ), \, \text{we have } x _ { i } = \exp ( z _ { i } ), \, \text{and it results in} \\ \prod _ { j \in A ( i ) } x _ { j } = \prod _ { j \in A ( i ) } \exp ( z _ { i } ) \\ / \quad \, \, \,$$

It is evident that the following equality holds:

where Q i i ( ∈ [ m ]) is defined by

$$\mathfrak { t } \, \text{that the following equality holds:} \\ \left ( \begin{array} { c } Q _ { 1 } \\ Q _ { 2 } \\ \vdots \\ Q _ { m } \end{array} \right ) \equiv \exp ( H z ), \quad & ( 3 1 ) \\ \vdots \in [ m ] ) \, \text{is defined by} \\ \quad & \quad \end{array} \right )$$

$$\colon [ m ] ) & \text{ is defined by} \\ Q _ { i } \equiv \left ( \prod _ { j \in A ( i ) } x _ { j } \right ). \quad \text{ (32)} \\ \text{ binary matrix } H \text{ is originally defined over } \mathbb { F } _ { 2 } \\ H \text{ is now a mbaded into } \mathbb { D } \text{ Normal to matrix}$$

$$\text{use} \quad \text{vector product} \, H z \, \text{ is carried out over} \, \mathbb { R }. \, By \, using \, ( \text{31}, \, \text{we} \\ \text{low} \quad \text{$4$}. \, \text{in} \quad \text{$2$} \quad \text{$\beta$} ( k ) \, \left ( \left ( \prod _ { j \in A ( i ) } x _ { j } \right ) - 1 \right ) \, \left ( \prod _ { j \in A ( i ) } x _ { j } \right ) \quad ( \text{33} ) \\ \quad \text{$=} \, \frac { 2 \beta } { x _ { k } } \sum _ { i \in B ( k ) } \left ( Q _ { i } ^ { 2 } - Q _ { i } \right ) \quad ( \text{34} ) \\ \text{rgy} \quad \text{$=} \, \frac { 2 \beta } { x _ { k } } \left [ H ^ { T } ( \exp ( 2 H z ) - \exp ( H z ) ) \right ] _ { k }, \quad ( \text{35} ) \\ \text{ive} \quad \text{where the notation} \, [ \cdot ] _ { k } \, \text{ represents the $k$th component of the} \\ \quad \text{vector}. \, \text{We thus have}$$

Note that the binary matrix H is originally defined over F 2 but the matrix H is now embedded into R . Namely, the matrix vector product Hz is carried out over R . By using (31), we have where the notation [ ] · k represents the k th component of the vector. We thus have

$$4 \alpha ( x - 1 ) ( x + 1 ) x = 4 \alpha \exp \left ( z _ { - 1 } + z + z _ { + 1 } \right ), \quad ( 3 6 )$$

$$2 \beta \frac { \exp ( w ) } { x } = 2 \beta \exp \left ( w - z \right ), \quad \quad ( 3 7 )$$

which yields the claim of the lemma.

It should be noted that the evaluation method presented in Lemma 1 may not be optimal in terms of the computing time. The computing time for evaluating ∇ h α,β ( x ) could be improved depending on the basic tensor operations provided by an AI accelerator. For example, some AI accelerators have efficient sparse MVP operation which can make use of the sparseness of the parity check matrix H . While the evaluation approach detailed in Lemma 1 may not be the most efficient for some AI accelerators, Lemma 1 clearly demonstrates that the process of evaluating gradients can be effectively parallelized.

## C. Thread computation model for GPGPU

In estimating the computational complexity and required computational resource with the assumption of GPGPU, a model that takes into account thread parallelism is necessary. Below is a simple computational model that considers thread parallelism for evaluating the gradient of code potential energy function ∇ h α,β ( x ) .

Our basic assumptions are as follows:

- · Assuming batch processing with a batch size of D .
- · A single thread can handle up to O n ( ) processing.
- · The size of H is m × n .

Evaluating the gradient of the code potential energy necessitates computing the matrix-matrix multiplication HZ , where Z ∈ C n × D represents a batch of state vectors. This step is the most demanding part of the gradient evaluation process as described in Lemma 1. It is assumed that each individual thread will handle the computation of the dot product between a row vector from H and a column vector from Z . This dot product operation is carried out sequentially by each thread, and the computational load per thread is O n ( ) . In such a scenario, mD threads are needed to work concurrently in computing HZ . Thus, the parallel computation of the gradient evaluation necessitates the use of mD threads . This is the dominant use of the computational resources of a GPGPU.

A comparison of the required computational resources between GF decoding and tensor-computable BP decoding is presented in the Appendix. GF decoding can be seen as a lighter algorithm that requires fewer GPU computational resources because it involves matrix computations with smaller size matrices.

## V. GENERALIZATION OF GRADIENT FLOW DECODING

A. Approximate maximum a posteriori (MAP) decoding

The GF decoding is defined only for AWGN channel in the previous sections. In this section, we will generalize GF decoding for a general vector channel defined by a conditional probability distribution p ( y x | ) .

We here briefly review the approximate MAP decoding introduced in the paper on proximal decoding [15]. Assume that a sender transmits a codeword of C ( H ) to a given channel. The channel is defined by a probability density function (PDF), p ( y x | )( x ∈ R n , y ∈ R N ) . The negative loglikelihood function is defined as

$$L ( x ; y ) \equiv - \ln p ( y | x ). \quad \$$

The optimal decoding rule for this channel is MAP decoding rule defined as

$$\hat { x } \equiv \arg \max _ { x \in \mathbb { R$$

The ideal prior distribution is based on the code C ( H ) which is given by where p ( x ) is the prior distribution. In general, MAP decoding is computationally intractable and we thus need to consider an approximation of MAP decoding for sufficiently long codes.

$$\Delta \circ \varepsilon \circ \varepsilon \circ \vareps$$

where δ is Dirac's delta function. We here introduce a Gibbs prior distribution

$$\tilde { p } ( x ) \equiv \frac { 1 } { Z } \exp$$

where Z is the normalizing constant and γ is a positive constant. The function h α,β ( x ) takes the value zero if and only if x ∈ C ( H ) and takes a positive value otherwise. This implies that we have

$$\begin{array} {$$

at the limit γ →∞ . This fact suggests the following approximation on the posterior PDF:

$$p ( x | y ) & \in p ( y | x ) p ( x ) \simeq p ( y | x ) \tilde { p } ( x ) \\ & = \exp \left ( - L ( x ; y ) - \gamma h _ { \alpha, \beta } ( x ) \right ). \quad \text{(43)} \\ \text{em this an approximate one of $\partial$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$ } \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \, \text{$\theta$} \.$$

Definition 4: The approximate MAP rule considered here is given by

From this approximate posterior PDF, we can define a following decoding rule.

$$\hat { x } \equiv \arg \min _ { x \in \mathbb { R } ^ { n } } \, \left [ L ( x ; y ) + \gamma h _ { \alpha, \beta } ( x ) \right ], \quad ( 4 4 ) \\. \quad \cdot \quad \cdot \quad \cdot \quad \cdot$$

$$\begin{smallmatrix} \text{be} \quad \text{where $\gamma$ is a positive real number.} \end{smallmatrix}$$

## B. Generalization of GF decoding

In the previous subsection, the concept of an approximate MAP rule was introduced. The objective function to be minimized is

$$F ( x ) \equiv L ( x ; y ) + \gamma h _ { \alpha, \beta } ( x ). \quad \quad ( 4 5 ) \\ \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots.$$

Definition 5 (Generalized GF decoding): The ODE for the generalized GF decoding is given by

There are many possibilities for solving the optimization problem. For example, proximal decoding aims to solve this continuous minimization problem by using a proximal gradient descent method [15]. In the subsequent argument, we will solve this minimization problem by using the continuous-time gradient flow dynamics. The definition of the generalized GF decoding [2] is given as follows.

$$\frac { d x } { d t } = - ( \nabla L ( x ; y ) + \gamma \nabla h _ { \alpha, \beta } ( x ) ), \quad \quad ( 4 6 )$$

where the initial condition is x (0) = x 0 . For example, for AWGN channels, the objective function has the form:

$$F _ { \text{AWGN} } ( x ) \equiv \frac { 1 } { 2 } \| x - y \| ^ { 2 } + \gamma h _ { \alpha, \beta } ( x ), \quad \ \ ( 4 7 ) \\. \quad. \quad. \quad. \ -. \quad. \quad.$$

where y represents the received word. This means that the generalized GF-ODE (46) becomes identical to the original GF-ODE (15).

If the righthand side of the generalized GF-ODE (46),

$$\nabla F ( x ) & = \nabla L ( x ; y ) + \gamma \nabla h _ { \alpha, \beta } ( x ), \quad \quad ( 4 8 ) \\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \ dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots.$$

Theorem 1: If ∇ L ( x y ; ) is tensor-computable, then ∇ F ( x ) is tensor-computable.

is tensor-computable, the whole decoding processes can be efficiently executed in an AI accelerator. The following theorem clarifies the tensor-computability of ∇ F ( x ) .

(Proof) To prove the claim of theorem, it is sufficient to show that ∇ h α,β ( x ) is tensor-computable. Lemma 1 establishes the tensor-computability of the gradient of code potential energy funciton.

## C. Discrete-time version of GF decoding

When implementing GF decoding on a GPGPU, it is necessary to discretize the continuous GF decoding process. The Euler method can be employed to discretize the minimization processes of the generalized GF-ODE (46). This approach yields the recurrence formula for a discrete-time version of GF decoding:

Definition 6 (Discretized GF decoding): The recursive formula of the discretized GF decoding is given by

$$s ^ { ( k + 1 ) } = s ^ { ( k ) } - \eta ( \nabla L ( s ^ { ( k ) } ; \boldsymbol y ) + \gamma \nabla h _ { \alpha, \beta } ( s ^ { ( k ) } ) ), \ k = 0, 1, \dots, \quad \text{In the fo.} \\ ( 4 9 ) \quad \text{algorithm} \, \text{v}$$

where s (0) = x .

0 The iterative process in (49) can be regarded as a gradient descent method for minimizing F ( x ) .

We also refer to the discretized decoding process (49) as GF decoding in this paper. GPGPUs and digital AI accelerators can be used to execute the discrete time iterative process defined in (49). In such cases, computation of ∇ L ( s ( k ) ; y ) should be tensor-computable, as these devices are designed for tensor-friendly computations.

## D. Avoiding Numerical Instability

The discretized GF decoding described above can become numerically unstable when the step size parameter η is relatively large.

Let ξ be a positive constant slightly larger than 1, and let B ξ ≡ -[ ξ, ξ ] n represent an n -dimensional hypercube. Here, [ a, b ] ≡ { x ∈ R | a ≤ x ≤ b } . The norm of the gradient ∥∇ h α,β ( x ) ∥ tends to become very large when x / B ∈ ξ due to the properties of the potential energy function, which can cause numerical instability (oscillations or divergent behavior) in the aforementioned neighborhood decoding process. In such cases, to prevent numerical instability, instead of (49), one can use

The projection operator Π ξ : R n → B ξ represents the projection onto B η , and is defined as

$$\text{use} \quad & \mod c \\ s ^ { ( k + 1 ) } = \Pi _ { \xi } \left ( s ^ { ( k ) } - \eta ( \nabla L ( s ^ { ( k ) } ; y ) + \gamma \nabla h _ { \alpha, \beta } ( s ^ { ( k ) } ) ) \right ). \\ \text{(50)} \quad & \vdots \\ \text{The projection operator } \Pi _ { \epsilon } \ \colon \ \mathbb { R } ^ { n } \ \to \ B _ { \epsilon } \ \text{ represents the} \quad & \vdots$$

$$\Pi _ { \xi } ( x ) \equiv \arg \min _ { x ^ { \prime } \in B _ { \xi } } \| x - x ^ { \prime } \|. \quad \quad ( 5 1 ) \quad \quad ( 5 1 ) \quad \quad \quad \ \Phi _ { \xi } ^ { \Omega }$$

It is experimentally confirmed that this projection operation can stabilize the decoding process.

## E. Deep Unfolding

The recursive equation (49) incorporates multiple internal parameters, such as η , γ , α , and β . It has been confirmed through experiments that the choice of these parameters significantly influences the decoding performance. To fine-tune these parameters, several strategies can be employed, such as conducting preliminary experimental searches, employing random or grid searches, and using Bayesian optimization for hyper-parameter optimization. Nonetheless, pinpointing the ideal parameter setup is a computationally intractable task due to the extensive number of tunable parameters.

Deep unfolding is a method that seeks to boost the performance of iterative algorithms by optimizing the internal parameters via deep learning techniques. In this approach, iterative processes are unfolded into the time direction and each iteration is treated as a layer of a deep neural network. For example, many sparse signal recovery algorithms and MIMO detection algorithms are iterative in nature, applying deep unfolding can lead to significant improvements in both recovery performance and convergence speed [26], [27], [28], [29], [30].

In the following discussion, we will study the GF decoding algorithm with the set of internal trainable parameters:

$$\{ \eta ^ { ( k ) }, \gamma ^ { ( k ) }, \alpha ^ { ( k ) }, \beta ^ { ( k ) } \}$$

for k = 1 2 , , . . . , T . Namely, the recursive formula for the GF decoding becomes as follows.

Definition 7 (GF decoding with trainable parameters):

$$\ s ^ { ( \kappa ) } ; y ) _ { \text{signed} } \quad s ^ { ( k + 1 ) } = s ^ { ( k ) } - \eta ^ { ( k ) } ( \nabla L ( s ^ { ( k ) } ; y ) + \gamma ^ { ( k ) } \nabla h _ { \alpha ^ { ( k ) }, \beta ^ { ( k ) } } ( s ^ { ( k ) } ) ).$$

The decoding process of the trainable discretized GF decoding is summarized in Algorithm 1. Notably, every step in Algorithm 1 is differentiable, meaning that the automatic differentiation mechanism can be seamlessly applied to evaluate the gradients of the trainable parameters.

Applying back-propagation with the Mean Squared Error (MSE) loss function enables the computation of gradients for the trainable parameters within the model. These gradients are essential to know how each parameter needs to be adjusted to minimize the loss. Gradient descent-based optimizers, such as Stochastic Gradient Descent (SGD) or Adam, leverage these gradients to update the trainable parameters. Incremental training is effective Details of how to train the deep unfolded model can be found in [29].

Fig. 7. BER performance of discretized GF decoding over AWGN channels. Left panel: BER as a function of the number of iterations. Right panel: BER as a function of E /N b 0 .

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-d5b41385-662f-40a7-af71-27ac904c45e9.jpg)

## Algorithm 1 GF decoding with trainable parameters

- 1: s (0) := x 0
- 3: Set the following variables:
- 2: for k := 0 to U -1 do

$$\begin{array} { l l l } \text{Set the following variables:} \\ z \coloneqq \ln ( s ^ { ( k ) } ), \\ z _ { - 1 } \coloneqq \ln ( s ^ { ( k ) } - 1 ), \\ z _ { + 1 } \coloneqq \ln ( s ^ { ( k ) } + 1 ), \\ w \coloneqq \ln ( H ^ { T } \left ( \exp ( 2 H z ) - \exp ( H z ) \right ) ), \\ g \coloneqq \nabla L ( s ^ { ( k ) } ; y ) \\. \quad. \quad. \quad. \quad. \quad. \quad. \quad. \end{array}$$

$$\begin{smallmatrix} 4 \colon & h \coloneqq 4 \alpha ^ { ( k ) } \exp \left ( z _ { - 1 } + z + z _ { + 1 } \right ) + 2 \beta ^ { ( k ) } \exp \left ( w - z \right ) & \text{neq } \text{th} \, \text{tra} \\ 5 \colon & s ^ { ( k + 1 ) } \coloneqq s ^ { ( k ) } - \eta ^ { ( k ) } ( g + \gamma ^ { ( k ) } h ) & \text{neq } \text{rad} \\ 6 \colon \text{end for} & & \text{circular} \end{smallmatrix}$$

- 7: ˆ := x sign ( s ( U ) )
- 6: end for
- 8: Output ˆ x

## VI. GF DECODING FOR MIMO CHANNELS

## A. Generalized GF-ODE for MIMO channels

In this section, we discuss an application of GF decoding for LDPC-coded massive MIMO channels. Joint decoding of LDPC codes and MIMO channels is a practically important problem for future wireless systems such as beyond-5G/6G systems.

Let A ∈ R N × n be a channel matrix. Suppose that a received word y ∈ R N is given by

$$y = A x + w, \quad \text{ \quad \ \ } ( 5 3 )$$

where w ∈ R N is a Gaussian noise vector, the components of which follow an i.i.d. Gaussian distribution. The channel input vector x is assumed to be a codeword of C ( H ) , which implies that we assume BPSK modulation.

In this problem setting, the PDF representing the channel is given by

$$\cdot \lambda _ { 2 } \\ p ( y | x ) \equiv a \exp \left ( - b \| y - A x \| ^ { 2 } \right ), \quad \ \ ( 5 4 ) \ \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{  \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{} \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{$$

where a and b are real constants. Our objective function for the approximate MAP rule can be expressed as

$$F _ { \text{MIMO} } ( x ) \equiv \frac { 1 } { 2 } \| y - A x \| ^ { 2 } + \gamma h _ { \alpha, \beta } ( x ). \quad ( 5 5 ) \begin{array} { c c } C. \ N u \\ \ln t \end{array}$$

Corollary 1: ∇ F MIMO ( x ) is tensor-computable. (Proof) The gradient of the first term of F MIMO :

$$\nabla L ( x ; y ) = \nabla \frac { 1 } { 2 } \| y - A x \| ^ { 2 } = A ^ { T } ( A x - y ) \quad ( 5 6 ) \quad \underset { \ell \cdots \, \_ \cdots } { \text{resize} }$$

is tensor-computable. By Theorem 1, we immediately have the claim of the corollary.

The generalized GF-ODE for LDPC-coded MIMO channels thus becomes

$$\frac { d x } { d t } = - ( A ^ { T } ( A x - y ) + \gamma \nabla h _ { \alpha, \beta } ( x ) ), \quad ( 5 7 ) \, \begin{array} { c } \infty \end{array} \cos \\ \text{and} \, \ell \end{array}$$

where the initial condition is given by x (0) = x 0 . In this case, the recursive formula of the discretized GF decoding is given by

$$s ^ { ( k + 1 ) } = s ^ { ( k ) } - \eta ( A ^ { T } ( A s ^ { ( k ) } - y ) + \gamma \nabla h _ { \alpha, \beta } ( s ^ { ( k ) } ) ), \ ( 5 8 ) \\. \quad \, _ { \ell \alpha }$$

where s (0) = x 0 .

## B. Experimental conditions

In this subsection, we will explain the details of the numerical experiments for real-valued MIMO model.

$$\omega = A & x + w \text{ where } A \text{ is given by} \\ & A \equiv \left [ \begin{array} { c c c } \text{Re} ( A ^ { \prime } ) & - \text{Im} ( A ^ { \prime } ) \\ \text{Im} ( A ^ { \prime } ) & \text{Re} ( A ^ { \prime } ) \end{array} \right ] \in \mathbb { R } ^ { \mu \times n }, \\ \text{re} \ \ N \ = \ 2 \imath \text{ and } n \ = \ 2 \imath. \ \text{ The transmitted } \ \text{wor}$$

Let A ′ ≡ { a ′ i,j } ∈ C µ × ν be a channel matrix, where a ′ i,j is the fading coefficient corresponding to the path between the j th transmit antenna and the i th receive antenna. We assume an i.i.d. model such that each component of A ′ follows a complex circular Gaussian distribution CN (0 1) , . A QPSK modulation format is assumed for transmitted signals. An equivalent realvalued MIMO model with BPSK modulation can be defined as y = Ax + w where A is given by where N = 2 µ and n = 2 ν . The transmitted word x is randomly chosen from C ( H ) according to the uniform distribution. Each component of the Gaussian noise vector w ∈ R N follows Gaussian distribution with zero mean and variance σ 2 w / 2 . In this model, σ 2 w is related to the signal-tonoise ratio SNR by σ 2 w = N/ SNR .

In the following experiment, we used the regular (3,6)LDPC code with n = 204 and m = 102 . The step-size parameter ω used in the gradient descent step was set to

$$\omega \equiv \frac { 2 } { \lambda _ { m i n } + \lambda _ { m a x } },$$

where λ min and λ max are the minimum and maximum eigenvalues of A A T , respectively.

For comparison, we exploited the MMSE detector defined as

$$\hat { x } \equiv A ^ { T } ( A A ^ { T } + ( \sigma _ { w } ^ { 2 } / 2 ) I ) ^ { - 1 } y. \quad \quad ( 6 0 )$$

Furthermore, as a baseline for joint detection and decoding, we employed the combination of the MMSE detector and BP decoding which is to be denoted by MMSE + BP.

## C. Numerical experiments

In this subsection, we present experimental results on discretized GF (DGF) decoding for MIMO channels.

From the results, we can find that the BER performance of the plain MMSE detector is far behind other methods that consider LDPC encoding (labeled with 'MMSE+BP', 'DGF', and 'DU-DGF'). With a maximum number of iterations of 100 ,

Figure 8 shows the bit error rate (BER) performances of the proposed 'DGF' and its DU version, 'DU-DGF', alongside baseline schemes. The labels 'DGF-50' and 'DU-DGF-50' represent the BER performance evaluated at the 50 th iteration, while 'DGF-100' and 'DU-DGF-100' represent the BER performance at the 100 th iteration. The maximum number of iterations for BP decoding was set to 100 .

our proposed method ('DGF-100') achieves approximately a 1 6 . dB advantage over 'MMSE+BP' at BER = 10 -4 . Additionally, with DU, the adjustment of the step size for each iteration can speed up the convergence of DGF, further widening the BER performance gap between our proposed method and the 'MMSE+BP' approach. Meanwhile, by comparing the results at the 50 th and 100 th iterations, we also found that the improvement of 'DU-DGF' is more obvious in fewer iterations.

Fig. 8. Bit error rate performances of DGF/DU-DGF decoding and baseline schemes ( n = 204 , m = 102) .

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-5f9738ea-564a-4187-9926-67525a68d743.jpg)

## VII. SCORE-BASED CHANNEL LEARNING

## A. Overview

A trajectory of the search vector of GF decoding is determined by the vector field created by -∇ ln p ( y x | ) and γ ∇ h α,β ( x ) . Essentially, these vector fields dictate the dynamics of the GF decoding process. If the negative log-likelihood function of the channel, ∇ ln p ( y x | ) , is perfectly known and can be computed using tensor-based computation, then implementing GF decoding becomes relatively simple. For example, channels with correlated Gaussian noise and certain nonlinear vector channel fall under this class, as ∇ ln p ( y x | ) for these types of channels can be easily expressed in a simple formula [15]. Conversely, if the negative log-likelihood function for the channel is unknown, it is necessary to learn this function from data. The section discusses channel learning for GF decoding from data on inputs and outputs of the target channel.

For a given probability density function p ( x ) ,

$$s ( x ) \equiv \nabla \ln p ( x ) \quad \ \ \ ( 6 1 )$$

is referred to as the score function of p [31][32]. It is known that by combining the score function s ( x ) with Langevin sampling, it is possible to efficiently generate random vectors that follow p ( x ) . Generative models based on score functions is currently known as score-based diffusion models . In the context of generative models based on score functions, a neural network model approximating s ( x ) is learned from given data. It is known that modeling the score function with a neural network [31] offers more benefits than directly modeling the probability density function p ( x ) itself with a neural network.

If one wishes to use GF decoding in situations where the true likelihood function p ( y x | ) of the channel is unknown, it is anticipated that learning the conditional score function

$$s ( x ; y ) & \equiv \nabla _ { x } \ln p ( y | x ) \\. \. \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot \ \cdot$$

from data is beneficial. After leaning s ( x y ; ) , the model ˆ ( s θ x y ; ) can be used as

$$\begin{array} {$$

in the GF decoding process. Research on learning techniques for score functions, such as score matching methods [32], is actively being pursued in the context of diffusion models, and it is believed that these studies could enable the realization of novel channel learning techniques suitable for GF decoding.

## B. Channel score model

1) Requirements: Modeling the conditional score function s ( x y ; ) with a neural network is an intuitive strategy. The model must meet certain criteria:

- · It should be tensor-computable.
- · To maintain training efficiency, the model should be designed with a limited number of parameters.
- · A shallow network architecture is favored to ensure efficient inference processes.
- · If prior knowledge about the statistical characteristics of the channel exists, the model architecture should reflect this information. For instance, exploiting the memoryless property could greatly simplify the model architecture.
- · There should be an effective method for training the model.

2) Neural network model: Although a variety of neural network models could fulfill the specified requirements described above, for the sake of simplicity, our discussion will focus on employing a simple feedforward neural network. Note that, depending on the characteristics of channels, other models might be more appropriate for channel learning tasks.

In the following discussion, we assume an additive channel where there exists a PDF Q satisfying

$$\begin{array} {$$

for the sake of simplicity of the argument, i.e., the received word y is given by y = x + e , where e ∼ Q e ( ) . We further assume a segment-wise independence and an identically distributed (i.i.d.) memoryless property characterized by the equation:

$$p ( y | x ) = \prod _ { i = 1 } ^ { K } p ($$

$$\underset { \text{he} } { \text{ns} } \quad x \equ$$

where x ∈ R n and y ∈ R N ( n = N ) are segemented into:

where q : R ν → R is a PDF ( n = νK ). In the discussions that follow, we will use boldface letters with a prime symbol to denote vectors that correspond to the segment.

Given these assumptions, the conditional score function can be broken down as follows:

$$s ( x ; y ) = \nabla _ { x } \ln p ( y | x$$

$$\text{se assumptions, the conditional score function can } & \text{$I$} \text{$the} \\ \text{own as follows:} & \quad & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & &  & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & \text{$x;$ y} ) & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & && & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & &
 & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & \\ & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & _ & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & ; & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & - & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & } & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & &. & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & &, & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & ^ & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & ( & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & = & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & &

 & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & b & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & | & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & &$$

where because

$$\nabla _ { x _ { i } ^ { \prime } } \ln q ( x _ { i } ^ { \prime } - y _ { i } ^ { \prime } ) = \nabla _ { e _ { i } ^ { \prime } } \ln q ( e _ { i } ^ { \prime } ) \quad \quad ( 7 3 ) \quad \text{or usi} \\ \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \colon$$

holds when e ′ i = x ′ i -y ′ i . The function ς ( e ′ ) is referred to as the segmented score function .

Next, we delve into utilizing a neural network to approximate the segmented score function ς . We define a feedforward neural network model ˆ ς θ : R ν → R ν as follows:

$$\hat { \varsigma } _ { \theta } ( e ^ { \prime } ) & = h _ { L }, & ( 7 4 ) & \quad \text{needs t}$$

$$h _ { i + 1 } & = g _ { i } ( W _ { i } h _ { i } + b _ { i } ), \quad i = 1, 2, \dots, L - 1, \quad \left ( 7 5 \right ) \quad \overline { \text{Algorit} } \right ) _ { i + 1 \cdot \text{Ini} }$$

$$h _ { 1 } & = e ^ { \prime }, & ( 7 6 ) & \sqrt { 1 \colon \, \text{Ini} }$$

where each g i represents an activation function, and the set θ encompasses all trainable parameters in the model. The W i matrices and b i vectors are the trainable weights and biases, respectively. The aim is for the segmented score model ˆ ς θ to be trained so that effectively modeling the segmented score function for each segment.

$$\text{ned so that} \\ s ( x ; y ) & \approx \left ( \begin{array} { c } \hat { \zeta } _ { \theta } ( x ^ { \prime } _ { 1 } - y ^ { \prime } _ { 1 } ) \\ \vdots \\ \hat { \zeta } _ { \theta } ( x ^ { \$$

## C. Score matching learning

We first briefly outline the idea of the score matching learning according to [31][32]. Assume that we wish to learn the score function of p ( x ) from the data following p ( x ) . It is natural to minimize the expected loss function:

$$\frac { 1 } { 2 } \mathbb { E } _ { x \sim p ( x ) } [ \| s _ { \theta } ( x ) - \nabla \log p ( x ) \| _ { 2 } ^ { 2 } ] \quad \quad ( 7 8 )$$

for tuning the trainable parameter in the model s θ ( x ) but evaluation of ∇ log p ( x ) is impossible as p ( x ) is unknown.

$$= \left ( \begin{array} { c } \varsigma ( x _ { 1 } ^ { \prime } - y _ { 1 } ^ { \prime } ) \\ \vdots \\ \varsigma ( x _ { K } ^ { \prime } - y _ { K } ^ { \prime } ) \end{array} \right ),$$

$$\varsigma ( e ^ { \prime } ) \equiv \nabla _ { e ^ { \prime } } \ln q ( e ^ { \prime } ) \quad \quad \quad ( 7 2 ) \quad \begin{array} { c } \text{ing} \\ \text{(e^{\prime}_{1}},\, \, \, \,$$

The above loss function is known to be equivalent to the following loss function up to a constant:

In principle, the above expectation can be evaluated but is not suitable for learning relatively large model such as deep neural network. In this paper, we use the denoising score matching loss [31] defined by

$$\text{lowing loss function up to a constant: } \\ \frac { 1 } { 2 } \mathbb { E } _ { x \sim p ( x ) } \left [ \text{tr} \left ( \nabla s _ { \theta } ( x ) \right ) + \frac { 1 } { 2 } \| s _ { \theta } ( x ) \$$

$$\text{$loss[31] defined by} \\ \frac { 1 } { 2 } \mathbb { E } _ { x \sim p ( x ) } \mathbb { E } _ { \tilde { x } \sim \mathcal { N } ( x, \sigma ^ { 2 } I ) } \left [ \left \| s _ { \theta } ( \tilde { x } ) + \frac { \tilde { x } - x } { \sigma ^ { 2 } } \right \| _ { 2 } ^ { 2 } \right ]. \quad ( 8 0 ) \\ \text{$It is anticipated that the trained model can precisely approxi$-} \\ \text{$mates the true score function if the noise variance $\sigma^{2}$ is small} \\ \quad. \ \dots$$

It is anticipated that the trained model can precisely approximates the true score function if the noise variance σ 2 is small enough [31].

## D. Details of score-based channel learning

Assume that we have a mini-batch generator producing a random mini-batch for additive error batch: E ≡ ( e ′ 1 , e ′ 2 , . . . , e ′ D ) , where each error vector e ′ i is sampled according to the PDF q . This mini-batch generator could be designed based on a theoretical probabilistic channel model or using actual data collected from the real target channel.

The details of score-based channel learning is summarized in Algorithm 2, which is based on the standard method for denoising socre matching [31]. To update trainable parameters, we can utilize standard optimization techniques like stochastic gradient descent (SGD) and Adam. The noise variance σ 2 specifying the disturbing noise level is a hyperparameter that needs to be determined before the training begins.

## Algorithm 2 Score-based channel learning

- 1: Initialize the trainable parameters in θ .
- 3: Generate a random mini-batch E .
- 2: for k := 1 to U do
- 4: Generate disturbed samples:

$$\tilde { e } _ { d } ^ { \prime } = e _ { d } ^ { \prime } + n _ { d } \text{ \quad \ \ } ( 8 1 )$$

for d ∈ [ D ] where n d ∼ N (0 , σ 2 I ) .

- 5: Evaluate the score matching loss:

$$5 \colon & \text{ evaluate the score matching loss} \colon \\ \mathcal { L } ( \theta ; \mathcal { X }, \mathcal { Y } ) \equiv \frac { 1 } { D } \sum _ { d = 1 } ^ { D } \left \| \hat { \zeta } _ { \theta } ( \tilde { e } _ { d } ^ { \prime } ) + \frac { \tilde { e } _ { d } ^ { \prime } - e _ { d } ^ { \prime } } { \sigma ^ { 2 } } \right \| _ { 2 } ^ { 2 }. \quad ( 8 2 ) \\ \text{Perform back-propagation to evaluate the gradient of } \mathcal { L } \\ \text{reset to the trainable parameters in } \theta.$$

- 7: Update the the trainable parameters in θ .
- 6: Perform back-propagation to evaluate the gradient of L respect to the trainable parameters in θ .
- 8: end for

As a result of the training process, we acquire a trained model denoted by ˆ ς θ ∗ . The gradient of negative log likelihood function L can then be approximated using this trained model as:

$$\begin{array} { c } \dots & \varsigma \\ \\ \nabla L ( x ; y ) \approx \left ( \begin{array} { c } - \hat { \varsigma } _ { \theta ^ { * } } ( x _ { 1 } ^ { \prime } - y _ { 1 } ^ { \prime } ) \\ \vdots \\ - \hat { \varsigma } _ { \theta ^ { * } } ( x _ { K } ^ { \prime } - y _ { K } ^ { \prime } ) \end{array} \right ). \quad \quad ( 8 3 ) \end{array}$$

This approximated negative log likelihood function can be used in the GF decoding process.

There are a couple of strategies to training the whole GF decoding process:

- 1) Serial training : The most straightforward method involves initially training the model ˆ ς θ , and then using deep unfolding to train the trainable parameters specified in (52), while keeping ˆ ς θ fixed.
- 2) Fine tuning : Another approach starts with training the model ˆ ς θ as well, but then combines deep unfolding training to optimize both the set of trainable parameters in (52) and the trainable parameters within ˆ ς θ .

It is crucial to emphasize that, in the training process outlined in Algorithm 2, the disturbed samples are generated using a single noise level. However, the literature on score-based diffusion models [31][32] has established that a gradual change in the noise level is essential for accurately approximating a complex score function. Consequently, further development of Algorithm 2 to incorporate varying noise levels constitutes a significant area for future research, as it has the potential to enhance the accuracy and effectiveness of the score-based channel learning.

## E. Numerical example

In this subsection, we explore a numerical example for GF decoding utilizing the trainable score-based gradient. We focus on a scenario involving artificial two-dimensional correlated noises, i.e., the segment size is ν = 2 . A total of 1000 twodimensional error candidates, e ′ 1 , . . . , e ′ 1000 , are illustrated in Fig. 9. The segment-wise channel model is defined as

$$y ^ { \prime } = x ^ { \prime } + e ^ { \prime }, \quad \ \ \ \ \ ( 8 4 ) \ \ F i g.$$

where e ′ is selected uniformly at random from the error candidates. In this example, we utilize a serial training approach and opt not to employ deep unfolding for additional optimization.

Fig. 9. Correlated error candidates (1000 2D-points). One of the 2D-points is randomly realized in our channel model.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-79a500d6-07b0-439e-bc85-894c7ee068fd.jpg)

Initially, we train the model ˆ ς θ ∗ ( e ′ ) using mini-batches generated randomly in accordance with the probabilistic channel model previously defined. Our model architecture comprises a three-layer ( L = 3 ) feedforward neural network equipped with the ReLU activation function, and we configured each layer of the network to have 64 hidden units. During the training phase, we set the mini-batch size to 100 and carried out 10,000 training iterations. For the adjustment of trainable parameters, the Adam optimizer with learning rate 0 005 . was employed. For generating the disturbed samples, Gaussian noises with σ = 0 3 . were used in Algorithm 2.

Figure 9 presents the vector field generated by the trained model ˆ ς θ ∗ . Upon comparing Figs 9 and 10, it becomes apparent that regions near the error candidate points correspond to attractors in the learned vector field. Specifically, a vector in Fig. 9 points towards the nearest error candidate point which is consistent with the ascending direction of the true segmented score function. This indicates that the trained model ˆ ς θ ∗ ( e ′ ) effectively encapsulates the characteristics of the segmented score function ς ( e ′ ) .

Fig. 10. Learned vector field given by ˆ ς θ ∗ .

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-0f9762d4-abfe-4721-9b50-8c8451519b80.jpg)

We conducted experiments on the discretized GF decoding process using the trained model ˆ ς θ ∗ ( e ′ ) . For these experiments, a (3 , 6) -regular LDPC code with n = 204 and m = 102 was utilized. The configuration for the GF decoder was set as follows: α = β = 1 , η = 0 05 . ,and γ = 1 0 . were chosen. The initial state of the GF decoder was initialized by a random vector x 0 drawn from a normal distribution N ( 0 , 0 1 . 2 I ) .

Figure 11 displays the discrepancy count between ˆ x ( k ) ≡ sign ( s ( k ) ) and the transmitted word x , i.e., the estimated number of errors as a function of the iteration count k . In this experiment, we conducted 100 GF decoding trials, and each curve in Fig. 11 illustrates the change in the number of bit errors for each trial. It is noticeable that all curves exhibit a sharp decrease during the initial few iterations, followed by a more gradual reduction. This indicates that the gradient descent processes are effectively functioning with the learned score function. In all trials, errors were eliminated before reaching 50 iterations.

## F. Segmented linear vector channel

̸

We here briefly discuss the case where the segment sizes are not equal, i.e., N = n . An important channel in this class is a segmented linear vector channel defined by

$$y ^ { \prime } = A x ^ { \prime } + e ^ { \prime },$$

Fig. 11. Number of bit errors as a function of the number of iterations (100 trials, (3,6)-regular LDPC code with n = 204 , m = 102 ).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00293v1-figure-3c3bc487-3308-42ff-a3b2-83f68363b0c6.jpg)

where A ∈ R N × n and e ′ ∼ q ( e ′ ) . For example, a MIMO channel with correlated additive noises are included in this class. In this case, it is expected that the conditional score function s ( x y ; ) can be well approximated by where ˆ ( ς θ e ′ ) is a neural network model approximating the segmented score function ς ( e ′ ) = ∇ e ′ ln q ( e ′ ) .

$$\left | \text{tion } s ( x ; y ) \, \text{can be well approximated by } \quad \text{GF} \, d \text{t} \\ s ( x ; y ) \approx \left ( \begin{array} { c } A ^ { T } \hat { \zeta } _ { \theta } ( A x _ { 1 } ^ { \prime } - y _ { 1 } ^ { \prime } ) \\ A ^ { T } \hat { \zeta } _ { \theta } ( A x _ { 2 } ^ { \prime } - y _ { 2 } ^ { \prime } ) \\ \vdots \\ A ^ { T } \hat { \zeta } _ { \theta } ( A x _ { K } ^ { \prime } - y _ { K } ^ { \prime } ) \end{array} \right ), \quad \text{(86)} \, \text{increa} \\ e r e } \, \hat { \zeta } _ { \theta } ( e ^ { \prime } ) \, \text{is a neural network model approximating the } \text{hardw} \\ \text{mented score function } c ( e ^ { \prime } ) = \nabla _ { \cdot } \ln a ( e ^ { \prime } )$$

This framework is capable of handling multi-valued signal constellations, such as Quadrature Amplitude Modulation (QAM). A straightforward example is as follows: Consider x ′ = ( x , x 1 2 , x 3 , x 4 ) and y ′ = ( y , y 1 2 ) where

$$y _ { 1 } = 2 x _ { 1 } + x _ { 2 }, \ \ y _ { 2 } = 2 x _ { 3 } + x _ { 4 }. \quad \ \ ( 8 7 ) \ \ A I \text{le}.$$

Given the definition, it is evident that y i ( i = 1 2) , takes a value in {-3 , -1 1 3 , , } because x i ∈ { 1 , - } 1 ( i = 1 2) , . Thus, a 16 QAM signal constellation can be obtained through the linear mapping y ′ = Ax ′ , with A defined as

This approach is expected to be applicable to numerous practical wireless communication systems.

$$\text{ping $y^{\prime}=Ax^{\prime}$, with $A$ defined as } & & \text{evol} \\ A \equiv \left ( \begin{array} { c c c c } 2 & 1 & 0 & 0 \\ 0 & 0 & 2 & 1 \end{array} \right ). & & ( 8 8 ) & \text{exp} ^ { \prime } \\ \text{vach is expected to be analytic to numerics}$$

## VIII. CONCLUSION

This paper introduces the GF decoding algorithm, which has a tensor-friendly design, making it particularly well-suited for implementation on next-generation AI accelerators. The underlying principle of GF decoding is remarkably universal, allowing for its application to a wide range of non-trivial channels, provided that the gradient of the negative loglikelihood function can be efficiently evaluated. Another significant advantage of our proposed method lies in its suitability for deep unfolding. This is facilitated by the ease of evaluating the gradient of the internal trainable parameters, which can be efficiently handled using automatic differentiation mechanisms or back-propagation, as every component of the algorithm is differentiable.

In massive MIMO channels encoded with an LDPC code, the proposed algorithm demonstrates competitiveness with established detection methods, such as MMSE + BP. When considering the balance between decoding complexity and decoding performance, GF decoding shows considerable promise for LDPC-coded MIMO channels. Traditionally, signal processing algorithms in a receiver have been designed following a modular design principle. Typically, components like a MIMO signal detector and an LDPC decoder are designed separately and then integrated. However, the principle of GF decoding facilitates a codesign principle [23], potentially enhancing the overall performance of the receiver. The methodology outlined in this paper may pave the way for innovative strategies in developing jointly designed signal processing algorithms within a receiver.

Score-based channel learning offers a novel approach to learn the probabilistic nature of the target channel. A neural network can be employed to approximate a conditional score function, and the resulting model can be directly utilized in GF decoding as the gradient of the negative log-likelihood function. As the importance of handling unknown channels grows, data-driven methods for channel learning will become increasingly crucial in the field of communications.

Error correcting codes and their decoding methods have evolved in tandem with the advancements in foundational hardware technology. In the era when only small-scale integrated circuits were available, algebraic codes were highly valued for their compatibility with the hardware limitations of the time. As technology progressed to enable the assembly of large-scale ASICs, the implementation of decoders for LDPC codes in ASICs became feasible, ushering in a golden era for LDPC codes. Looking ahead, the hardware suited for AI learning and inference processes, which are poised to be ubiquitous in the near future, should not only be capable of supporting the training and inference processes of generative AI models but also be versatile enough to accommodate a wide range of signal processing tasks. As AI hardware continues to evolve, error correcting codes and their decoding methods are expected to follow suit in the 21st century, adapting to the prevailing hardware environments of their times.

## ACKNOWLEDGEMENT

This work was supported by JSPS KAKENHI Grant-inAid for Scientific Research (A) Grant Number JP22H00514. The authors appreciate the inspiring discussions on the initial version of GF decoding with Dr. Ayano Nakai-Kasai and Mr. Kensho Nakajima.

## REFERENCES

- [1] T. Wadayama, K. Nakajima, and A. Nakai-Kasai, 'Gradient flow decoding for LDPC codes,' 2023 International Symposium on Topics in Coding (ISTC2023), Brest, France, 2023.
- [3] R. G. Gallager, 'Low density parity check codes,' MIT Press, 1963.
- [2] T. Wadayama and L. Wei, 'Generalized gradient flow decoding and and Its Tensor-Computability,' International Symposium on Information Theory (ISIT2024), Athens, 2024.

- [4] E. Peltonen et al. , '6G white paper on edge intelligence,' 6G Research Visions, No. 8. University of Oulu, 2020.
- [5] OpenAI, 'GPT-4 technical report, ' arXiv:2303.08774v4, 2023.
- [6] A. Reuther, P. Michaleas, M. Jones, V. Gadepally, S. Samsi, J. Kepner, 'AI and ML accelerator survey and trends,' IEEE High Performance Extreme Computing Conference (HPEC), 2022.
- [7] J. Carolan et al.,'Universal linear optics,' Science, vol.349, no. 6249, pp.711-716, 2015.
- [8] J. Capmany, D. P´ erez, Programmable Integrated Photonics. Oxford University Press, 2020.
- [9] X. Pengfei and Z. Zhou, 'Silicon-based optoelectronics for generalpurpose matrix computation: a review,' Advanced Photonics, vol.4, no. 4, pp.044001-044001, 2022.
- [10] H. Zhang and M. Gu and X. D. Jiang and et al., 'An optical neural chip for implementing complex-valued neural network,' Nature Commun., vol.12, 2021.
- [11] M.Prabhu et al., 'Accelerating recurrent Ising machines in photonic integrated circuits,' Optica, no.7, pp.551-558, 2020.
- [12] S. Wang, Y. Luo, P. Zuo, L. Pan, Y. Li, and Z. Sun, 'In-memory analog solution of compressed sensing recovery in one step,' Science Advances, vol.9, no.50, 2023.
- [13] P. Zuo, Z. Sun, R. Huang, 'Extremely-fast, energy-efficient massive MIMO precoding with analog RRAM matrix computing,' IEEE Transactions on Circuits and Systems II: Express Briefs, vol.7, no.7, pp.23352339, 2023.
- [14] T. Wadayama, K. Nakamura, M. Yagita, Y. Funahashi, S. Usami, I. Takumi, 'Gradient descent bit flipping algorithms for decoding LDPC codes', IEEE Trans. Comm., pp.1610-1614, vol.58, no.6, June (2010)
- [15] T. Wadayama and S. Takabe, 'Proximal decoding for LDPC codes,' IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol. E106-A, no. 3 pp. 359-367 (2023).
- [16] J. Feldman, 'Decoding error-correcting codes via linear programming,' Massachusetts Institute of Technology, Ph. D. thesis, 2003.
- [17] P. O. Vontobel, 'Interior-point algorithms for linear-programming decoding,' IEEE Information Theory and Applications Workshop, 2008.
- [18] T.Wadayama, 'Interior point decoding for linear vector channels based on convex optimization,' IEEE Transactions on Information Theory, vol.56, no.10, pp.4905-4921, 2010.
- [19] G. Sundararajan and C. Winstead and E. Boutillon, 'Noisy gradient descent bit-flip decoding for LDPC codes,' IEEE Transactions on Communications, vol.62, no.10, pp.3385-3400, 2014.
- [20] X. Zhang and P. H. Siegel, 'Efficient iterative LP decoding of LDPC Codes with alternating direction method of multipliers,' IEEE International Symposium on Information Theory (ISIT), 2013.
- [21] S. Barman, X. Liu, and S. C. Draper, and B. Recht, 'Decomposition methods for large scale LP decoding,' IEEE Transactions on Information Theory, vol.59, no.12, pp.7870-7886, 2013.
- [22] Y. Jiang, P. T. S. DeVore, A. Mahjoubfar, and B. Jalali, 'Analog logarithmic computing primitives with silicon photonics,' in Conference on Lasers and Electro-Optics, OSA Technical Digest, 2016.
- [23] A.P. Worthen and W.E. Stark, 'Unified design of iterative receivers using factor graphs,' IEEE Transactions on Information Theory, vol.47, pp.843-849, 2001.
- [24] D. F. Griffiths, and D. J. Higham, 'Numerical methods for ordinary differential equations, ' Springer, 2010.
- [25] D. J. C. MacKay, 'Encyclopedia of sparse graph codes [online],' Available: http://www.inference.phy.cam.ac.uk/mackay/codes/data.html .
- [26] K. Gregor and Y. LeCun, 'Learning fast approximations of sparse coding,' Proc. 27th Int. Conf. Machine Learning , pp. 399-406, 2010.
- [27] A. Balatsoukas-Stimming and C. Studer, 'Deep Unfolding for Communications Systems: A Survey and Some New Directions,' in Proc. 2019 IEEE International Workshop on Signal Processing Systems (SiPS) , 2019, pp. 266-271.
- [28] S. Takabe, M. Imanishi, T. Wadayama, R. Hayakawa, and K. Hayashi, 'Trainable Projected Gradient Detector for Massive Overloaded MIMO Channels: Data-Driven Tuning Approach,' IEEE Access , 2019,vol. 7, pp. 93326-93338.
- [29] D. Ito, S. Takabe, and T. Wadayama, 'Trainable ISTA for Sparse Signal Recovery,' in IEEE Transactions on Signal Processing , vol. 67, no. 12, pp. 3113-3125, 2019.
- [30] H. He, C. -K. Wen, S. Jin and G. Y. Li, 'Model-Driven Deep Learning for MIMO Detection,' in IEEE Transactions on Signal Processing, vol. 68, pp. 1702-1715, 2020.
- [31] Y. Song and S. Ermon, 'Generative modeling by estimating gradients of the data distribution,' NeurIPS 2019.

[32] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, 'Score-based generative modeling through stochastic differential equations,' in Proceedings of the 10th International Conference on Learning Representations (ICLR), pp. 399-406, 2021.

## APPENDIX

Tensor-computable BP decoding

It is possible to formulate log-domain BP decoding in tensor-computable form. Let the set of edges contained in the Tanner graph be defined as

$$E \equiv \{ ( i, j ) \in [ m ] \times [ n ] \ | \ H _ { i j } = 1 \}. \quad \quad ( 8 9 ) \\ \dots \. \. \. \. \. \. \. \. \.$$

Define the matrix U = { U jk } ∈ { 0 1 , } n × e and the matrix V = { V ik } ∈ { 0 1 , } m e × as

In the following, the total number of edges contained in the Tanner graph is denoted by e = | E | . The number of edges e is equal to the number of 1's contained in H . Now, assume that a bijection ϕ : E → [ e ] is given. That is, the edges contained in E are numbered by ϕ . Furthermore, for all k ∈ [ e ] , let p , q k k be defined as the quantities that satisfy ϕ (( p , q k k )) = k .

$$\in \{ U, \, \iota \} \dots \cdots & \text{ as } \\ U _ { j k } & \equiv \begin{cases} 1, & \text{if $q_{k}=j$} \\ 0, & \text{otherwise}, \end{cases}$$

$$V _ { i k } \equiv \begin{cases} 1, & \text{if $p_{k}=i$} \\ 0, & \text{otherwise.} \end{cases}$$

As an example, define the parity-check matrix H as

Define the numbering of edges in the Tanner graph as

$$\text{in example, define the parity-check matrix } H \text{ as } \\ H \equiv \left [ \begin{array} { c c c c } 1 & 1 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 1 & 1 \end{array} \right ]. \quad \text{(92)} \\ \text{: the numbering of edges in the Tanner graph as}$$

$$\text{ition} \quad & \phi ( 1, 1 ) = 1, \phi ( 1, 2 ) = 2, \phi ( 1, 3 ) = 3, \phi ( 2, 3 ) = 4, \\ \text{ition} \quad & \phi ( 2, 4 ) = 5, \phi ( 3, 4 ) = 6, \phi ( 3, 5 ) = 7, \phi ( 3, 6 ) = 8. \quad ( 9 3 )$$

In this case, the matrices U and V are given by:

and

$$U = \left [ \begin{array} { c c c c c c c } 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{array} \right ] \\ \begin{matrix} 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 1 \end{matrix}$$

$$V = \left [ \begin{array} { c c c c c c } 1 & 1 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 &$$

Assume that

These matrices U and V are key matrices in rewriting the belief propagation decoding into a tensor-computable form. The details of tensor-friendly BP decoding are summarized in Algorithm 3. The vectors α and β represent check-to-variable and variable-to-check messages, respectively. The vector λ denotes the log likelihood ratio vector. We can see that a dominant computation of the BP decoding process is a matrixvector product of U U T -I ∈ R e × e and α ∈ R e .

- · Batch processing with the batch size D .
- · In a GPGPU, a single thread handles dot product computations of O e ( ) as in the thread computation model in Subsection IV-C.

In this case, we need eD threads for executing a BP decoding process. Recall that discretized GF decoding requires only mD threads and that each thread handles O n ( ) processing. Because e &gt; n &gt; m , the tensor-computable BP decoding in Algorithm 3 is more resource-demanding than discretized GF decoding.

## Algorithm 3 Tensor-compuatable BP Decoding

- 1: Initialization: Set α ≡ 0 , β ≡ 0
- 3: Variable Node Processing:
- 2: for i = 1 to L do

$$\beta \equiv ( U ^ { T } U - I ) \alpha + U ^ { T } \lambda \text{ \quad \ \ } ( 9 6$$

## 4: Check Node Processing:

$$\alpha _ { \text{sign} } & \equiv \left \{ 1 - 2 V ^ { T } \text{mod} \left ( V ($$

$$& \text{Check Node Processing} \colon \\ & \alpha _ { \text{abs} } \equiv 2 \tanh ^ { - 1 } \$$

$$\alpha \equiv \alpha _ { \text{sign} } \odot \alpha _ { \text{abs} } \quad \quad \quad$$

The function bmod represents the real remainder modulo 2, which is defined by

- 5: end for

$$\mathbf n c { n } & \text{en} \, \text{by} \\ & \text{bmod} ( x ) \equiv x - 2 \left \lfloor \frac { x } { 2 } \right \rfloor. \quad \quad ( 100 )$$

- 6: Calculation of Posterior Probability Ratio:

$$\frac { \gamma \equiv U \alpha + \lambda \quad \text{(10)} } { }$$
Tadashi Wadayama Lantian Wei
Gradient Flow Decoding
This paper presents the Gradient Flow (GF) decoding for LDPC codes. GF
decoding, a continuous-time methodology based on gradient flow, employs a
potential energy function associated with bipolar codewords of LDPC codes. The
decoding process of the GF decoding is concisely defined by an ordinary
differential equation and thus it is well suited to an analog circuit
implementation. We experimentally demonstrate that the decoding performance of
the GF decoding for AWGN channels is comparable to that of the multi-bit mode
gradient descent bit flipping algorithm. We further introduce the negative
log-likelihood function of the channel for generalizing the GF decoding. The
proposed method is shown to be tensor-computable, which means that the gradient
of the objective function can be evaluated with the combination of basic tensor
computations. This characteristic is well-suited to emerging AI accelerators,
potentially applicable in wireless signal processing. The paper assesses the
decoding performance of the generalized GF decoding in LDPC-coded MIMO
channels. Our numerical experiments reveal that the decoding performance rivals
that of established techniques like MMSE + BP. Furthermore, an exploration of
score-based channel learning for capturing statistical properties is also
provided.