Position of Uncertainty: A Cross-Linguistic Study of Positional Bias in
  Large Language Models
Large language models exhibit positional bias -- systematic neglect of
information at specific context positions -- yet its interplay with linguistic
diversity remains poorly understood. We present a cross-linguistic study across
five typologically distinct languages (English, Russian, German, Hindi,
Vietnamese), examining how positional bias interacts with model uncertainty,
syntax, and prompting. Key findings: (1) Positional bias is model-driven, with
language-specific variations -- Qwen2.5-7B favors late positions, challenging
assumptions of early-token bias; (2) Explicit positional guidance (e.g.,
correct context is at position X) reduces accuracy across languages,
undermining prompt-engineering practices; (3) Aligning context with positional
bias increases entropy, yet minimal entropy does not predict accuracy. (4) We
further uncover that LLMs differently impose dominant word order in
free-word-order languages like Hindi.
6834833c672ae46abcdc244d
Menschikov Mikhail
6834833c672ae46abcdc244e
Alexander Kharitonov
claimed_verified
2025-05-27T07:53:48.385Z
65265ffb193ab5e06b5c3bc1
/avatars/d02fd8bd4ea27be2c4f7699697d922bd.svg
Alexander Kharitonov
user
mathamateur
6834833c672ae46abcdc244f
Maiia Kotyga
claimed_verified
2025-05-27T07:53:50.495Z
634eb2f279980a15ac66646c
https://cdn-avatars.huggingface.co/v1/production/uploads/634eb2f279980a15ac66646c/hGOrxySznCWkZCEAof41n.jpeg
Maiia Kotyga
user
kotmayyaka
6834833c672ae46abcdc2450
Vadim Porvatov
6834833c672ae46abcdc2451
Anna Zhukovskaya
6834833c672ae46abcdc2452
David Kagramanyan
claimed_verified
2025-05-27T07:53:45.724Z
6359b0b6548e5c7e6e69398c
https://cdn-avatars.huggingface.co/v1/production/uploads/1668424684378-6359b0b6548e5c7e6e69398c.jpeg
David Kagramanyan
user
dkagramanyan
6834833c672ae46abcdc2453
Egor Shvetsov
6834833c672ae46abcdc2454
Evgeny Burnaev
Position of Uncertainty: A Cross-Linguistic Study of Positional Bias in
  Large Language Models
Large language models exhibit positional bias -- systematic neglect of
information at specific context positions -- yet its interplay with linguistic
diversity remains poorly understood. We present a cross-linguistic study across
five typologically distinct languages (English, Russian, German, Hindi,
Vietnamese), examining how positional bias interacts with model uncertainty,
syntax, and prompting. Key findings: (1) Positional bias is model-driven, with
language-specific variations -- Qwen2.5-7B favors late positions, challenging
assumptions of early-token bias; (2) Explicit positional guidance (e.g.,
correct context is at position X) reduces accuracy across languages,
undermining prompt-engineering practices; (3) Aligning context with positional
bias increases entropy, yet minimal entropy does not predict accuracy. (4) We
further uncover that LLMs differently impose dominant word order in
free-word-order languages like Hindi.
LLMs display positional bias across different languages, which affects model uncertainty, syntax, and prompting, revealing that explicit positional guidance can reduce accuracy.
positional bias model uncertainty syntax prompting positional guidance entropy dominant word order free-word-order languages