ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2406.00584
23
1

A Blueprint Architecture of Compound AI Systems for Enterprise

2 June 2024
Eser Kandogan
Sajjadur Rahman
Nikita Bhutani
Dan Zhang
Rafael Li Chen
Kushan Mitra
Sairam Gurajada
Pouya Pezeshkpour
Hayate Iso
Yanlin Feng
H. Kim
Chen Shen
Jin Wang
Estevam R. Hruschka
    LLMAG
ArXivPDFHTML
Abstract

Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency.

View on arXiv
Comments on this paper