title: Teaching Metric Distance to Autoregressive Multimodal Foundational
    Models
summary: As large language models expand beyond natural language to domains such as
  mathematics, multimodal understanding, and embodied agents, tokens increasingly
  reflect metric relationships rather than purely linguistic meaning. We
  introduce DIST2Loss, a distance-aware framework designed to train
  autoregressive discrete models by leveraging predefined distance relationships
  among output tokens. At its core, DIST2Loss transforms continuous exponential
  family distributions derived from inherent distance metrics into discrete,
  categorical optimization targets compatible with the models' architectures.
  This approach enables the models to learn and preserve meaningful distance
  relationships during token generation while maintaining compatibility with
  existing architectures. Empirical evaluations show consistent performance gains
  in diverse multimodal applications, including visual grounding, robotic
  manipulation, generative reward modeling, and image generation using
  vector-quantized features. These improvements are pronounced in cases of
  limited training data, highlighting DIST2Loss's effectiveness in
  resource-constrained settings.
paper_authors._id: 67c7acfb3c08da184dd8463e
paper_authors.name: Jiwan Chung
paper_authors._id: 67c7acfb3c08da184dd8463f
paper_authors.name: Saejin Kim
paper_authors._id: 67c7acfb3c08da184dd84640
paper_authors.name: Yongrae Jo
paper_authors._id: 67c7acfb3c08da184dd84641
paper_authors.name: Jaewoo Park
paper_authors._id: 67c7acfb3c08da184dd84642
paper_authors.name: Dongjun Min
paper_authors._id: 67c7acfb3c08da184dd84643
paper_authors.name: Youngjae Yu
paper_title: Teaching Metric Distance to Autoregressive Multimodal Foundational
    Models
paper_summary: As large language models expand beyond natural language to domains such as
  mathematics, multimodal understanding, and embodied agents, tokens increasingly
  reflect metric relationships rather than purely linguistic meaning. We
  introduce DIST2Loss, a distance-aware framework designed to train
  autoregressive discrete models by leveraging predefined distance relationships
  among output tokens. At its core, DIST2Loss transforms continuous exponential
  family distributions derived from inherent distance metrics into discrete,
  categorical optimization targets compatible with the models' architectures.
  This approach enables the models to learn and preserve meaningful distance
  relationships during token generation while maintaining compatibility with
  existing architectures. Empirical evaluations show consistent performance gains
  in diverse multimodal applications, including visual grounding, robotic
  manipulation, generative reward modeling, and image generation using
  vector-quantized features. These improvements are pronounced in cases of
  limited training data, highlighting DIST2Loss's effectiveness in
  resource-constrained settings.
paper_ai_keywords: autoregressive discrete models DIST2Loss distance-aware framework exponential family distributions discrete, categorical optimization targets token generation visual grounding robotic manipulation generative reward modeling image generation vector-quantized features resource-constrained settings