CrossMPT: Cross-attention Message-Passing Transformer for Error Correcting Codes
<|reference_start|>CrossMPT: Cross-attention Message-Passing Transformer for Error Correcting Codes: Error correcting codes~(ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer~(CrossMPT). CrossMPT iteratively updates two types of input vectors (i.e., magnitude and syndrome vectors) using two masked cross-attention blocks. The mask matrices in these cross-attention blocks are determined by the code's parity-check matrix that delineates the relationship between magnitude and syndrome vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders, particularly in decoding low-density parity-check codes. Notably, CrossMPT also achieves a significant reduction in computational complexity, achieving over a 50\% decrease in its attention layers compared to the original transformer-based decoder, while retaining the computational complexity of the remaining layers.<|reference_end|>
@article{park2024crossmpt:,
    title={CrossMPT: Cross-attention Message-Passing Transformer for Error
  Correcting Codes},
    author={Seong-Joon Park, Hee-Youl Kwak, Sang-Hyo Kim, Yongjune Kim, Jong-Seon
  No},
    journal={arXiv preprint arXiv:2405.01033},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.01033},
    primaryClass={cs.LG cs.IT math.IT}
}