title: AERO: Softmax-Only LLMs for Efficient Private Inference
summary: The pervasiveness of proprietary language models has raised privacy concerns
  for users' sensitive data, emphasizing the need for private inference (PI),
  where inference is performed directly on encrypted inputs. However, current PI
  methods face prohibitively higher communication and latency overheads,
  primarily due to nonlinear operations. In this paper, we present a
  comprehensive analysis to understand the role of nonlinearities in
  transformer-based decoder-only language models. We introduce AERO, a four-step
  architectural optimization framework that refines the existing LLM architecture
  for efficient PI by systematically removing nonlinearities such as LayerNorm
  and GELU and reducing FLOPs counts. For the first time, we propose a
  Softmax-only architecture with significantly fewer FLOPs tailored for efficient
  PI. Furthermore, we devise a novel entropy regularization technique to improve
  the performance of Softmax-only models. AERO achieves up to 4.23times
  communication and 1.94times latency reduction. We validate the effectiveness
  of AERO by benchmarking it against the state-of-the-art.
paper_authors._id: 6711c3fea79b431fea2c3b48
paper_authors.name: Nandan Kumar Jha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T16:02:37.993Z
paper_authors._id: 670ec3f6db1a6bcfe832e0a6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-mZNLeLJoXzkwPgYO38lF.png
paper_authors.fullname: Nandan Kumar Jha
paper_authors.type: user
paper_authors.user: nandan523
paper_authors._id: 6711c3fea79b431fea2c3b49
paper_authors.name: Brandon Reagen
paper_title: AERO: Softmax-Only LLMs for Efficient Private Inference
paper_summary: The pervasiveness of proprietary language models has raised privacy concerns
  for users' sensitive data, emphasizing the need for private inference (PI),
  where inference is performed directly on encrypted inputs. However, current PI
  methods face prohibitively higher communication and latency overheads,
  primarily due to nonlinear operations. In this paper, we present a
  comprehensive analysis to understand the role of nonlinearities in
  transformer-based decoder-only language models. We introduce AERO, a four-step
  architectural optimization framework that refines the existing LLM architecture
  for efficient PI by systematically removing nonlinearities such as LayerNorm
  and GELU and reducing FLOPs counts. For the first time, we propose a
  Softmax-only architecture with significantly fewer FLOPs tailored for efficient
  PI. Furthermore, we devise a novel entropy regularization technique to improve
  the performance of Softmax-only models. AERO achieves up to 4.23times
  communication and 1.94times latency reduction. We validate the effectiveness
  of AERO by benchmarking it against the state-of-the-art.
paper_ai_summary: AERO framework optimizes transformer-based LLMs for efficient private inference by removing nonlinearities and using a Softmax-only architecture with reduced FLOPs and improved entropy regularization.
paper_ai_keywords: private inference LLM architecture AERO LayerNorm GELU FLOPs Softmax-only architecture entropy regularization