Leveraging Apache Arrow for Zero-copy, Zero-serialization Cluster Shared Memory
<|reference_start|>Leveraging Apache Arrow for Zero-copy, Zero-serialization Cluster Shared Memory: This paper describes a distributed implementation of Apache Arrow that can leverage cluster-shared load-store addressable memory that is hardware-coherent only within each node. The implementation is built on the ThymesisFlow prototype that leverages the OpenCAPI interface to create a shared address space across a cluster. While Apache Arrow structures are immutable, simplifying their use in a cluster shared memory, this paper creates distributed Apache Arrow tables and makes them accessible in each node.<|reference_end|>
@article{groet2024leveraging,
    title={Leveraging Apache Arrow for Zero-copy, Zero-serialization Cluster Shared
  Memory},
    author={Philip Groet, Joost Hoozemans, Andreas Grapentin, Felix Eberhardt,
  Zaid Al-Ars, H. Peter Hofstee},
    journal={arXiv preprint arXiv:2404.03030},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.03030},
    primaryClass={cs.ET}
}