title: AlayaDB: The Data Foundation for Efficient and Effective Long-context
    LLM Inference
summary: AlayaDB is a cutting-edge vector database system natively architected for
  efficient and effective long-context inference for Large Language Models (LLMs)
  at AlayaDB AI. Specifically, it decouples the KV cache and attention
  computation from the LLM inference systems, and encapsulates them into a novel
  vector database system. For the Model as a Service providers (MaaS), AlayaDB
  consumes fewer hardware resources and offers higher generation quality for
  various workloads with different kinds of Service Level Objectives (SLOs), when
  comparing with the existing alternative solutions (e.g., KV cache
  disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that
  it abstracts the attention computation and cache management for LLM inference
  into a query processing procedure, and optimizes the performance via a native
  query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via
  (i) three use cases from our industry partners, and (ii) extensive experimental
  results on LLM inference benchmarks.
paper_authors._id: 67ff772061373fdf16ce1d38
paper_authors.name: Yangshen Deng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T11:57:58.302Z
paper_authors._id: 66486ba1640bc89c93bcc8a2
paper_authors.avatarUrl: /avatars/07e07bd768339495e55b7b17f2f8bc59.svg
paper_authors.fullname: Yangshen Deng
paper_authors.type: user
paper_authors.user: YangshenDeng
paper_authors._id: 67ff772061373fdf16ce1d39
paper_authors.name: Zhengxin You
paper_authors._id: 67ff772061373fdf16ce1d3a
paper_authors.name: Long Xiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-17T11:10:34.914Z
paper_authors._id: 647dd5a6becb41a272925b5e
paper_authors.avatarUrl: /avatars/9ec9a3357ac43dcbe350fb3b72f3dbc4.svg
paper_authors.fullname: Long Xiang
paper_authors.type: user
paper_authors.user: BenjaminXIANG
paper_authors._id: 67ff772061373fdf16ce1d3b
paper_authors.name: Qilong Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:16:16.665Z
paper_authors._id: 66d6be0bddf54fd90923c727
paper_authors.avatarUrl: /avatars/7bb82c8c339db944d79d47b3b9b35aa8.svg
paper_authors.fullname: Li
paper_authors.type: user
paper_authors.user: Qilong00
paper_authors._id: 67ff772061373fdf16ce1d3c
paper_authors.name: Peiqi Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:16:22.829Z
paper_authors._id: 66efe50667c4ce2c9024cd45
paper_authors.avatarUrl: /avatars/7b7f650953c371f08a5beecc500b6a43.svg
paper_authors.fullname: peiqiyuan
paper_authors.type: user
paper_authors.user: YuanPeiqi
paper_authors._id: 67ff772061373fdf16ce1d3d
paper_authors.name: Zhaoyang Hong
paper_authors._id: 67ff772061373fdf16ce1d3e
paper_authors.name: Yitao Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:16:36.808Z
paper_authors._id: 66d6c339b61dd11022907252
paper_authors.avatarUrl: /avatars/d2ed3cc003e94b2e5204ce0f8a481dcf.svg
paper_authors.fullname: Yitao Zheng
paper_authors.type: user
paper_authors.user: FeTieTer
paper_authors._id: 67ff772061373fdf16ce1d3f
paper_authors.name: Wanting Li
paper_authors._id: 67ff772061373fdf16ce1d40
paper_authors.name: Runzhong Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:17:02.031Z
paper_authors._id: 671a7bce2b10d343bab18637
paper_authors.avatarUrl: /avatars/af1c10a59236d953b42e67d3955eecc4.svg
paper_authors.fullname: runzhong
paper_authors.type: user
paper_authors.user: runzhongli
paper_authors._id: 67ff772061373fdf16ce1d41
paper_authors.name: Haotian Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:17:25.728Z
paper_authors._id: 63898b61ec1f539adc0f4da2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674248167280-63898b61ec1f539adc0f4da2.jpeg
paper_authors.fullname: Haotian Liu
paper_authors.type: user
paper_authors.user: liuhaotian
paper_authors._id: 67ff772061373fdf16ce1d42
paper_authors.name: Kyriakos Mouratidis
paper_authors._id: 67ff772061373fdf16ce1d43
paper_authors.name: Man Lung Yiu
paper_authors._id: 67ff772061373fdf16ce1d44
paper_authors.name: Huan Li
paper_authors._id: 67ff772061373fdf16ce1d45
paper_authors.name: Qiaomu Shen
paper_authors._id: 67ff772061373fdf16ce1d46
paper_authors.name: Rui Mao
paper_authors._id: 67ff772061373fdf16ce1d47
paper_authors.name: Bo Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T08:18:01.460Z
paper_authors._id: 66b02a2642c34e7a212133c0
paper_authors.avatarUrl: /avatars/737a69b095e8c427ecd08f870b173635.svg
paper_authors.fullname: Bo Tang
paper_authors.type: user
paper_authors.user: BO1022
paper_title: AlayaDB: The Data Foundation for Efficient and Effective Long-context
    LLM Inference
paper_summary: AlayaDB is a cutting-edge vector database system natively architected for
  efficient and effective long-context inference for Large Language Models (LLMs)
  at AlayaDB AI. Specifically, it decouples the KV cache and attention
  computation from the LLM inference systems, and encapsulates them into a novel
  vector database system. For the Model as a Service providers (MaaS), AlayaDB
  consumes fewer hardware resources and offers higher generation quality for
  various workloads with different kinds of Service Level Objectives (SLOs), when
  comparing with the existing alternative solutions (e.g., KV cache
  disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that
  it abstracts the attention computation and cache management for LLM inference
  into a query processing procedure, and optimizes the performance via a native
  query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via
  (i) three use cases from our industry partners, and (ii) extensive experimental
  results on LLM inference benchmarks.
paper_ai_summary: AlayaDB is a vector database system that decouples KV cache and attention computation from LLMs, optimizing inference for higher quality and reduced hardware usage.
paper_ai_keywords: KV cache attention computation vector database Model as a Service (MaaS) Service Level Objectives (SLOs) query processing procedure native query optimizer Large Language Models (LLMs)