Seminar: CrossMPT: Cross-Attention Message-Passing Transformer for Error Correcting Codes

Abstract: 

Error-correcting codes (ECCs) are fundamental to achieving reliable transmission in communication systems. Recent advances in deep learning have spurred the development of neural network-based ECC decoders, with transformer architectures demonstrating state-of-the-art decoding performance. This seminar begins by introducing the Error Correction Code Transformer (ECCT), the first transformer-based ECC decoder. Building on this foundation, we present the Cross-Attention Message-Passing Transformer (CrossMPT), a novel architecture that incorporates core principles of conventional message-passing decoders. Unlike prior transformer-based decoders that rely solely on self-attention, CrossMPT updates magnitude and syndrome embeddings separately and iteratively through two masked cross-attention blocks. These mask matrices are derived from the code’s parity-check matrix, enabling the model to explicitly suppress irrelevant interactions between the two types of embeddings. Experimental results demonstrate that CrossMPT consistently outperforms existing neural network-based decoders across multiple code classes. In addition to superior decoding performance, CrossMPT achieves substantial reductions in memory consumption, computational complexity, inference latency, and training time.

Biography:

Yongjune Kim received the B.S. (Hons.) and M.S. degrees in electrical and computer engineering from Seoul National University, Seoul, South Korea, in 2002 and 2004, respectively, and the Ph.D. degree in electrical and computer engineering from Carnegie Mellon University, Pittsburgh, PA, USA, in 2016. From 2016 to 2018, he was a Postdoctoral Research Associate with the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA. From 2018 to 2020, he was with Western Digital Research, Milpitas, CA, USA. From 2020 to 2022, he was an Assistant Professor with the Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, South Korea. Since 2022, he has been with the Department of Electrical and Engineering, Pohang University of Science and Technology (POSTECH), Pohang, South Korea, where he is currently an Associate Professor. His research interests include semantic communications, information theory, and machine learning. He received the IEEE Data Storage Best Student Paper Award, the Best Paper Award of the 2016 IEEE International Conference on Communications, the Best Paper Award (Honorable Mention) of the 2018 IEEE International Symposium on Circuits and Systems, and the Best Paper Award of the 31st Samsung Semiconductor Technology Symposium. He is an Editor of IEEE Transactions on Communications.