A Blueprint Architecture of Compound AI Systems for Enterprise
<|reference_start|>A Blueprint Architecture of Compound AI Systems for Enterprise: Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency.<|reference_end|>
@article{kandogan2024a,
    title={A Blueprint Architecture of Compound AI Systems for Enterprise},
    author={Eser Kandogan, Sajjadur Rahman, Nikita Bhutani, Dan Zhang, Rafael Li
  Chen, Kushan Mitra, Sairam Gurajada, Pouya Pezeshkpour, Hayate Iso, Yanlin
  Feng, Hannah Kim, Chen Shen, Jin Wang, Estevam Hruschka},
    journal={arXiv preprint arXiv:2406.00584},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.00584},
    primaryClass={cs.DB cs.AI}
}