title: Large Language Models are Locally Linear Mappings
summary: We demonstrate that the inference operations of several open-weight large
  language models (LLMs) can be mapped to an exactly equivalent linear system for
  an input sequence without modifying the model weights or altering output
  predictions. Extending techniques from image diffusion models that exhibit
  local or piecewise linearity, we strategically alter the gradient computation
  with respect to a given input sequence for a next-token prediction such that
  the Jacobian of the model nearly exactly reproduces the forward prediction with
  a linear system. We demonstrate this approach across models (Llama 3, Gemma 3,
  Qwen 3, Phi 4, Mistral Ministral and OLMo 2, up to Llama 3.3 70B Q4) and show
  through the singular value decomposition of the detached Jacobian that these
  LLMs operate in extremely low-dimensional subspaces where many of the largest
  singular vectors decode to concepts related to the most-likely output token.
  This approach also allows us to examine the operation of each successive layer
  (and its attention and MLP components) as nearly-exact linear systems and
  observe the emergence of semantic concepts. Despite their expressive power and
  global nonlinearity, modern LLMs can be interpreted through nearly-exact
  locally linear decompositions that provide insights into their internal
  representations and reveal interpretable semantic structures in the next-token
  prediction process.
paper_authors._id: 683d01ec446fd0c8ff323010
paper_authors.name: James R. Golden
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-06-02T02:38:10.635Z
paper_authors._id: 6658f863ce1b283888625af3
paper_authors.avatarUrl: /avatars/e4b2c7df0f398eb68c0566031ceac99e.svg
paper_authors.fullname: James Golden
paper_authors.type: user
paper_authors.user: jamesgolden1
paper_title: Large Language Models are Locally Linear Mappings
paper_summary: We demonstrate that the inference operations of several open-weight large
  language models (LLMs) can be mapped to an exactly equivalent linear system for
  an input sequence without modifying the model weights or altering output
  predictions. Extending techniques from image diffusion models that exhibit
  local or piecewise linearity, we strategically alter the gradient computation
  with respect to a given input sequence for a next-token prediction such that
  the Jacobian of the model nearly exactly reproduces the forward prediction with
  a linear system. We demonstrate this approach across models (Llama 3, Gemma 3,
  Qwen 3, Phi 4, Mistral Ministral and OLMo 2, up to Llama 3.3 70B Q4) and show
  through the singular value decomposition of the detached Jacobian that these
  LLMs operate in extremely low-dimensional subspaces where many of the largest
  singular vectors decode to concepts related to the most-likely output token.
  This approach also allows us to examine the operation of each successive layer
  (and its attention and MLP components) as nearly-exact linear systems and
  observe the emergence of semantic concepts. Despite their expressive power and
  global nonlinearity, modern LLMs can be interpreted through nearly-exact
  locally linear decompositions that provide insights into their internal
  representations and reveal interpretable semantic structures in the next-token
  prediction process.
paper_ai_summary: LLMs can be approximated as linear systems for inference, offering insights into their internal representations and semantic structures without altering the models or their predictions.
paper_ai_keywords: large language models (LLMs) inference operations linear system gradient computation Jacobian singular value decomposition low-dimensional subspaces semantic concepts attention components MLP components locally linear decompositions