title: Tokenization Constraints in LLMs: A Study of Symbolic and Arithmetic
    Reasoning Limits
summary: Tokenization is the first - and often underappreciated - layer of computation
  in language models. While Chain-of-Thought (CoT) prompting enables transformer
  models to approximate recurrent computation by externalizing intermediate
  steps, we show that the success of such reasoning is fundamentally bounded by
  the structure of tokenized inputs. This work presents a theoretical and
  empirical investigation into how tokenization schemes, particularly
  subword-based methods like byte-pair encoding (BPE), impede symbolic
  computation by merging or obscuring atomic reasoning units. We introduce the
  notion of Token Awareness to formalize how poor token granularity disrupts
  logical alignment and prevents models from generalizing symbolic procedures.
  Through systematic evaluation on arithmetic and symbolic tasks, we demonstrate
  that token structure dramatically affect reasoning performance, causing failure
  even with CoT, while atomically-aligned formats unlock strong generalization,
  allowing small models (e.g., GPT-4o-mini) to outperform larger systems (e.g.,
  o1) in structured reasoning. Our findings reveal that symbolic reasoning
  ability in LLMs is not purely architectural, but deeply conditioned on
  token-level representations.
paper_authors._id: 682d388c57686b8c44ede70b
paper_authors.name: Xiang Zhang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-21T02:21:00.902Z
paper_authors._id: 656553d89bf6665f10e3a92d
paper_authors.avatarUrl: /avatars/f55b09e249c48ef3fe484afa33a182ae.svg
paper_authors.fullname: xiang wyatt zhang
paper_authors.type: user
paper_authors.user: Wyattz23
paper_authors._id: 682d388c57686b8c44ede70c
paper_authors.name: Juntai Cao
paper_authors._id: 682d388c57686b8c44ede70d
paper_authors.name: Jiaqi Wei
paper_authors._id: 682d388c57686b8c44ede70e
paper_authors.name: Yiwei Xu
paper_authors._id: 682d388c57686b8c44ede70f
paper_authors.name: Chenyu You
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-21T02:30:12.849Z
paper_authors._id: 6466d463060756d2854ab3e1
paper_authors.avatarUrl: /avatars/4401387180c16472a6823f78aaa86d54.svg
paper_authors.fullname: Chenyu You
paper_authors.type: user
paper_authors.user: Charlesyooo
paper_title: Tokenization Constraints in LLMs: A Study of Symbolic and Arithmetic
    Reasoning Limits
paper_summary: Tokenization is the first - and often underappreciated - layer of computation
  in language models. While Chain-of-Thought (CoT) prompting enables transformer
  models to approximate recurrent computation by externalizing intermediate
  steps, we show that the success of such reasoning is fundamentally bounded by
  the structure of tokenized inputs. This work presents a theoretical and
  empirical investigation into how tokenization schemes, particularly
  subword-based methods like byte-pair encoding (BPE), impede symbolic
  computation by merging or obscuring atomic reasoning units. We introduce the
  notion of Token Awareness to formalize how poor token granularity disrupts
  logical alignment and prevents models from generalizing symbolic procedures.
  Through systematic evaluation on arithmetic and symbolic tasks, we demonstrate
  that token structure dramatically affect reasoning performance, causing failure
  even with CoT, while atomically-aligned formats unlock strong generalization,
  allowing small models (e.g., GPT-4o-mini) to outperform larger systems (e.g.,
  o1) in structured reasoning. Our findings reveal that symbolic reasoning
  ability in LLMs is not purely architectural, but deeply conditioned on
  token-level representations.
paper_ai_summary: Tokenization structure significantly impacts symbolic reasoning in language models by affecting token granularity and alignment, influencing performance even with Chain-of-Thought prompting.
paper_ai_keywords: tokenization Chain-of-Thought (CoT) transformer models subword-based methods byte-pair encoding (BPE) Token Awareness symbolic computation logical alignment structured reasoning large language models (LLMs)