SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions
<|reference_start|>SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions: The ubiquity of variable-length integers in data storage and communication necessitates efficient decoding techniques. In this paper, we present SFVInt, a simple and fast approach to decode the prevalent Little Endian Base-128 (LEB128) varints. Our approach effectively utilizes the Bit Manipulation Instruction Set 2 (BMI2) in modern Intel and AMD processors, achieving significant performance improvement while maintaining simplicity and avoiding overengineering. SFVInt, with its generic design, effectively processes both 32-bit and 64-bit unsigned integers using a unified code template, marking a significant leap forward in varint decoding efficiency. We thoroughly evaluate SFVInt's performance across various datasets and scenarios, demonstrating that it achieves up to a 2x increase in decoding speed when compared to varint decoding methods used in established frameworks like Facebook Folly and Google Protobuf.<|reference_end|>
@article{liao2024sfvint:,
    title={SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using
  Bit Manipulation Instructions},
    author={Gang Liao, Ye Liu, Yonghua Ding, Le Cai, Jianjun Chen},
    journal={arXiv preprint arXiv:2403.06898},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06898},
    primaryClass={cs.DB cs.DC}
}