title: GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for
    Autonomous Driving
summary: Self-supervised pre-training based on next-token prediction has enabled large
  language models to capture the underlying structure of text, and has led to
  unprecedented performance on a large array of tasks when applied at scale.
  Similarly, autonomous driving generates vast amounts of spatiotemporal data,
  alluding to the possibility of harnessing scale to learn the underlying
  geometric and semantic structure of the environment and its evolution over
  time. In this direction, we propose a geometric and semantic self-supervised
  pre-training method, GASP, that learns a unified representation by predicting,
  at any queried future point in spacetime, (1) general occupancy, capturing the
  evolving structure of the 3D scene; (2) ego occupancy, modeling the ego vehicle
  path through the environment; and (3) distilled high-level features from a
  vision foundation model. By modeling geometric and semantic 4D occupancy fields
  instead of raw sensor measurements, the model learns a structured,
  generalizable representation of the environment and its evolution through time.
  We validate GASP on multiple autonomous driving benchmarks, demonstrating
  significant improvements in semantic occupancy forecasting, online mapping, and
  ego trajectory prediction. Our results demonstrate that continuous 4D geometric
  and semantic occupancy prediction provides a scalable and effective
  pre-training paradigm for autonomous driving. For code and additional
  visualizations, see \href{https://research.zenseact.com/publications/gasp/.
paper_authors._id: 67ddab768f06eb5bf437487a
paper_authors.name: William Ljungbergh
paper_authors._id: 67ddab768f06eb5bf437487b
paper_authors.name: Adam Lilja
paper_authors._id: 67ddab768f06eb5bf437487c
paper_authors.name: Adam Tonderski. Arvid Laveno Ling
paper_authors._id: 67ddab768f06eb5bf437487d
paper_authors.name: Carl Lindström
paper_authors._id: 67ddab768f06eb5bf437487e
paper_authors.name: Willem Verbeke
paper_authors._id: 67ddab768f06eb5bf437487f
paper_authors.name: Junsheng Fu
paper_authors._id: 67ddab768f06eb5bf4374880
paper_authors.name: Christoffer Petersson
paper_authors._id: 67ddab768f06eb5bf4374881
paper_authors.name: Lars Hammarstrand
paper_authors._id: 67ddab768f06eb5bf4374882
paper_authors.name: Michael Felsberg
paper_title: GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for
    Autonomous Driving
paper_summary: Self-supervised pre-training based on next-token prediction has enabled large
  language models to capture the underlying structure of text, and has led to
  unprecedented performance on a large array of tasks when applied at scale.
  Similarly, autonomous driving generates vast amounts of spatiotemporal data,
  alluding to the possibility of harnessing scale to learn the underlying
  geometric and semantic structure of the environment and its evolution over
  time. In this direction, we propose a geometric and semantic self-supervised
  pre-training method, GASP, that learns a unified representation by predicting,
  at any queried future point in spacetime, (1) general occupancy, capturing the
  evolving structure of the 3D scene; (2) ego occupancy, modeling the ego vehicle
  path through the environment; and (3) distilled high-level features from a
  vision foundation model. By modeling geometric and semantic 4D occupancy fields
  instead of raw sensor measurements, the model learns a structured,
  generalizable representation of the environment and its evolution through time.
  We validate GASP on multiple autonomous driving benchmarks, demonstrating
  significant improvements in semantic occupancy forecasting, online mapping, and
  ego trajectory prediction. Our results demonstrate that continuous 4D geometric
  and semantic occupancy prediction provides a scalable and effective
  pre-training paradigm for autonomous driving. For code and additional
  visualizations, see \href{https://research.zenseact.com/publications/gasp/.
paper_ai_summary: GASP, a geometric and semantic self-supervised pre-training method for autonomous driving, significantly improves 4D occupancy forecasting, online mapping, and trajectory prediction by learning structured representations from driving data.
paper_ai_keywords: self-supervised pre-training next-token prediction large language models autonomous driving geometric and semantic structure unified representation general occupancy ego occupancy vision foundation model 4D occupancy fields structured representation semantic occupancy forecasting online mapping ego trajectory prediction pre-training paradigm