title: Hardware and Software Platform Inference
summary: It is now a common business practice to buy access to large language model
  (LLM) inference rather than self-host, because of significant upfront hardware
  infrastructure and energy costs. However, as a buyer, there is no mechanism to
  verify the authenticity of the advertised service including the serving
  hardware platform, e.g. that it is actually being served using an NVIDIA H100.
  Furthermore, there are reports suggesting that model providers may deliver
  models that differ slightly from the advertised ones, often to make them run on
  less expensive hardware. That way, a client pays premium for a capable model
  access on more expensive hardware, yet ends up being served by a (potentially
  less capable) cheaper model on cheaper hardware. In this paper we introduce
  \textbf{hardware and software platform inference (HSPI)} -- a method
  for identifying the underlying  architecture and software stack of a
  (black-box) machine learning model solely based on its input-output behavior.
  Our method leverages the inherent differences of various  architectures
  and compilers to distinguish between different  types and software
  stacks. By analyzing the numerical patterns in the model's outputs, we propose
  a classification framework capable of accurately identifying the  used
  for model inference as well as the underlying software configuration. Our
  findings demonstrate the feasibility of inferring  type from black-box
  models. We evaluate HSPI against models served on different real hardware and
  find that in a white-box setting we can distinguish between different s
  with between 83.9% and 100% accuracy. Even in a black-box setting we are
  able to achieve results that are up to three times higher than random guess
  accuracy.
paper_authors._id: 67349ba576c7f079f723a657
paper_authors.name: Cheng Zhang
paper_authors._id: 67349ba576c7f079f723a658
paper_authors.name: Hanna Foerster
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:06:09.812Z
paper_authors._id: 666c1544de375f3114d590c0
paper_authors.avatarUrl: /avatars/de6c4d3395160603b0bf68c9bca398df.svg
paper_authors.fullname: Hanna Foerster
paper_authors.type: user
paper_authors.user: aprilflower
paper_authors._id: 67349ba576c7f079f723a659
paper_authors.name: Robert D. Mullins
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:06:17.645Z
paper_authors._id: 66349b8dc53115681c534d07
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Suu9bZJecfCcJccjRsrqH.png
paper_authors.fullname: Robert Mullins
paper_authors.type: user
paper_authors.user: rcmullins
paper_authors._id: 67349ba576c7f079f723a65a
paper_authors.name: Yiren Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:06:23.792Z
paper_authors._id: 6654464df0c8c891ca528618
paper_authors.avatarUrl: /avatars/26aa1f673da92e3632576954bcee1b28.svg
paper_authors.fullname: Yiren Zhao
paper_authors.type: user
paper_authors.user: YirenZH
paper_authors._id: 67349ba576c7f079f723a65b
paper_authors.name: Ilia Shumailov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T08:07:12.854Z
paper_authors._id: 6475c2794766357252e69e9f
paper_authors.avatarUrl: /avatars/757ed789423113369868e972f21ce559.svg
paper_authors.fullname: i
paper_authors.type: user
paper_authors.user: iliashum
paper_title: Hardware and Software Platform Inference
paper_summary: It is now a common business practice to buy access to large language model
  (LLM) inference rather than self-host, because of significant upfront hardware
  infrastructure and energy costs. However, as a buyer, there is no mechanism to
  verify the authenticity of the advertised service including the serving
  hardware platform, e.g. that it is actually being served using an NVIDIA H100.
  Furthermore, there are reports suggesting that model providers may deliver
  models that differ slightly from the advertised ones, often to make them run on
  less expensive hardware. That way, a client pays premium for a capable model
  access on more expensive hardware, yet ends up being served by a (potentially
  less capable) cheaper model on cheaper hardware. In this paper we introduce
  \textbf{hardware and software platform inference (HSPI)} -- a method
  for identifying the underlying  architecture and software stack of a
  (black-box) machine learning model solely based on its input-output behavior.
  Our method leverages the inherent differences of various  architectures
  and compilers to distinguish between different  types and software
  stacks. By analyzing the numerical patterns in the model's outputs, we propose
  a classification framework capable of accurately identifying the  used
  for model inference as well as the underlying software configuration. Our
  findings demonstrate the feasibility of inferring  type from black-box
  models. We evaluate HSPI against models served on different real hardware and
  find that in a white-box setting we can distinguish between different s
  with between 83.9% and 100% accuracy. Even in a black-box setting we are
  able to achieve results that are up to three times higher than random guess
  accuracy.
paper_ai_summary: A method, HSPI, is introduced to identify the GPU architecture and software stack of a machine learning model based on its input-output behavior, providing a means to verify the authenticity of LLM inference services.
paper_ai_keywords: GPU LLM input-output behavior numerical patterns classification framework white-box setting black-box setting