Bidirectional Long-Range Parser for Sequential Data Understanding
<|reference_start|>Bidirectional Long-Range Parser for Sequential Data Understanding: The transformer is a powerful data modelling framework responsible for remarkable performance on a wide range of tasks. However, they are limited in terms of scalability as it is suboptimal and inefficient to process long-sequence data. To this purpose we introduce BLRP (Bidirectional Long-Range Parser), a novel and versatile attention mechanism designed to increase performance and efficiency on long-sequence tasks. It leverages short and long range heuristics in the form of a local sliding window approach combined with a global bidirectional latent space synthesis technique. We show the benefits and versatility of our approach on vision and language domains by demonstrating competitive results against state-of-the-art methods on the Long-Range-Arena and CIFAR benchmarks together with ablations demonstrating the computational efficiency.<|reference_end|>
@article{leotescu2024bidirectional,
    title={Bidirectional Long-Range Parser for Sequential Data Understanding},
    author={George Leotescu, Daniel Voinea, Alin-Ionut Popa},
    journal={arXiv preprint arXiv:2404.05210},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.05210},
    primaryClass={cs.CV cs.CL cs.LG}
}