CXL-Based Remote Memory for AI Clusters: Rethinking the Memory Hierarchy

Abstract 

The rapid scaling of AI models and accelerator clusters is exposing a growing gap between compute capability and available memory capacity, bandwidth, and data-movement efficiency. Compute Express Link (CXL) introduces the possibility of treating memory as a shared, composable and remotely accessible cluster resource. This talk examines how CXL-based remote memory can reshape the memory hierarchy of AI clusters. We will explore challenges spanning hardware, operating systems, memory management for AI workloads, including a specific problem of false sharing that occurs in AI workloads. The talk centres on open research problems and opportunities for systems researchers in optimizing AI clusters for distributed training.

Biography

Venkatesh is a Professor with the Department of Computer Science and Engineering, Indian Institute of Technology Guwahati. He received Ph.D from Indian Institute of Technology Madras in 2009 and since then hasbeen working at IIT Guwahati. He works broadly in the area of performance evaluation of computer networks and systems. His current areas of interest include data center networks and network telemetry. He is a Senior member of IEEE.