Towards a Flexible and High-Fidelity Approach to Distributed DNN Training Emulation
<|reference_start|>Towards a Flexible and High-Fidelity Approach to Distributed DNN Training Emulation: We propose NeuronaBox, a flexible, user-friendly, and high-fidelity approach to emulate DNN training workloads. We argue that to accurately observe performance, it is possible to execute the training workload on a subset of real nodes and emulate the networked execution environment along with the collective communication operations. Initial results from a proof-of-concept implementation show that NeuronaBox replicates the behavior of actual systems with high accuracy, with an error margin of less than 1% between the emulated measurements and the real system.<|reference_end|>
@article{liu2024towards,
    title={Towards a Flexible and High-Fidelity Approach to Distributed DNN
  Training Emulation},
    author={Banruo Liu, Mubarak Adetunji Ojewale, Yuhan Ding, Marco Canini},
    journal={arXiv preprint arXiv:2405.02969},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.02969},
    primaryClass={cs.LG cs.DC}
}