Universal Approximation of Operators with Transformers and Neural Integral Operators
<|reference_start|>Universal Approximation of Operators with Transformers and Neural Integral Operators: We study the universal approximation properties of transformers and neural integral operators for operators in Banach spaces. In particular, we show that the transformer architecture is a universal approximator of integral operators between H\"older spaces. Moreover, we show that a generalized version of neural integral operators, based on the Gavurin integral, are universal approximators of arbitrary operators between Banach spaces. Lastly, we show that a modified version of transformer, which uses Leray-Schauder mappings, is a universal approximator of operators between arbitrary Banach spaces.<|reference_end|>
@article{zappala2024universal,
    title={Universal Approximation of Operators with Transformers and Neural
  Integral Operators},
    author={Emanuele Zappala and Maryam Bagherian},
    journal={arXiv preprint arXiv:2409.00841},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.00841},
    primaryClass={cs.LG cs.NA math.NA}
}