Ensemble and Mixture-of-Experts DeepONets For Operator Learning
<|reference_start|>Ensemble and Mixture-of-Experts DeepONets For Operator Learning: We present a novel deep operator network (DeepONet) architecture for operator learning, the ensemble DeepONet, that allows for enriching the trunk network of a single DeepONet with multiple distinct trunk networks. This trunk enrichment allows for greater expressivity and generalization capabilities over a range of operator learning problems. We also present a spatial mixture-of-experts (MoE) DeepONet trunk network architecture that utilizes a partition-of-unity (PoU) approximation to promote spatial locality and model sparsity in the operator learning problem. We first prove that both the ensemble and PoU-MoE DeepONets are universal approximators. We then demonstrate that ensemble DeepONets containing a trunk ensemble of a standard trunk, the PoU-MoE trunk, and/or a proper orthogonal decomposition (POD) trunk can achieve 2-4x lower relative $\ell_2$ errors than standard DeepONets and POD-DeepONets on both standard and challenging new operator learning problems involving partial differential equations (PDEs) in two and three dimensions. Our new PoU-MoE formulation provides a natural way to incorporate spatial locality and model sparsity into any neural network architecture, while our new ensemble DeepONet provides a powerful and general framework for incorporating basis enrichment in scientific machine learning architectures for operator learning.<|reference_end|>
@article{sharma2024ensemble,
    title={Ensemble and Mixture-of-Experts DeepONets For Operator Learning},
    author={Ramansh Sharma, Varun Shankar},
    journal={arXiv preprint arXiv:2405.11907},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11907},
    primaryClass={cs.LG}
}