UDON: A case for offloading to general purpose compute on CXL memory
<|reference_start|>UDON: A case for offloading to general purpose compute on CXL memory: Upcoming CXL-based disaggregated memory devices feature special purpose units to offload compute to near-memory. In this paper, we explore opportunities for offloading compute to general purpose cores on CXL memory devices, thereby enabling a greater utility and diversity of offload.   We study two classes of popular memory intensive applications: ML inference and vector database as candidates for computational offload. The study uses Arm AArch64-based dual-socket NUMA systems to emulate CXL type-2 devices.   Our study shows promising results. With our ML inference model partitioning strategy for compute offload, we can place up to 90% data in remote memory with just 20% performance trade-off. Offloading Hierarchical Navigable Small World (HNSW) kernels in vector databases can provide upto 6.87$\times$ performance improvement with under 10% offload overhead.<|reference_end|>
@article{hermes2024udon:,
    title={UDON: A case for offloading to general purpose compute on CXL memory},
    author={Jon Hermes, Josh Minor, Minjun Wu, Adarsh Patil, Eric Van Hensbergen},
    journal={arXiv preprint arXiv:2404.02868},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.02868},
    primaryClass={cs.ET}
}