GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for
  Autonomous Driving
Self-supervised pre-training based on next-token prediction has enabled large
language models to capture the underlying structure of text, and has led to
unprecedented performance on a large array of tasks when applied at scale.
Similarly, autonomous driving generates vast amounts of spatiotemporal data,
alluding to the possibility of harnessing scale to learn the underlying
geometric and semantic structure of the environment and its evolution over
time. In this direction, we propose a geometric and semantic self-supervised
pre-training method, GASP, that learns a unified representation by predicting,
at any queried future point in spacetime, (1) general occupancy, capturing the
evolving structure of the 3D scene; (2) ego occupancy, modeling the ego vehicle
path through the environment; and (3) distilled high-level features from a
vision foundation model. By modeling geometric and semantic 4D occupancy fields
instead of raw sensor measurements, the model learns a structured,
generalizable representation of the environment and its evolution through time.
We validate GASP on multiple autonomous driving benchmarks, demonstrating
significant improvements in semantic occupancy forecasting, online mapping, and
ego trajectory prediction. Our results demonstrate that continuous 4D geometric
and semantic occupancy prediction provides a scalable and effective
pre-training paradigm for autonomous driving. For code and additional
visualizations, see \href{https://research.zenseact.com/publications/gasp/.
67ddab768f06eb5bf437487a
William Ljungbergh
67ddab768f06eb5bf437487b
Adam Lilja
67ddab768f06eb5bf437487c
Adam Tonderski. Arvid Laveno Ling
67ddab768f06eb5bf437487d
Carl Lindström
67ddab768f06eb5bf437487e
Willem Verbeke
67ddab768f06eb5bf437487f
Junsheng Fu
67ddab768f06eb5bf4374880
Christoffer Petersson
67ddab768f06eb5bf4374881
Lars Hammarstrand
67ddab768f06eb5bf4374882
Michael Felsberg
GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for
  Autonomous Driving
Self-supervised pre-training based on next-token prediction has enabled large
language models to capture the underlying structure of text, and has led to
unprecedented performance on a large array of tasks when applied at scale.
Similarly, autonomous driving generates vast amounts of spatiotemporal data,
alluding to the possibility of harnessing scale to learn the underlying
geometric and semantic structure of the environment and its evolution over
time. In this direction, we propose a geometric and semantic self-supervised
pre-training method, GASP, that learns a unified representation by predicting,
at any queried future point in spacetime, (1) general occupancy, capturing the
evolving structure of the 3D scene; (2) ego occupancy, modeling the ego vehicle
path through the environment; and (3) distilled high-level features from a
vision foundation model. By modeling geometric and semantic 4D occupancy fields
instead of raw sensor measurements, the model learns a structured,
generalizable representation of the environment and its evolution through time.
We validate GASP on multiple autonomous driving benchmarks, demonstrating
significant improvements in semantic occupancy forecasting, online mapping, and
ego trajectory prediction. Our results demonstrate that continuous 4D geometric
and semantic occupancy prediction provides a scalable and effective
pre-training paradigm for autonomous driving. For code and additional
visualizations, see \href{https://research.zenseact.com/publications/gasp/.
GASP, a geometric and semantic self-supervised pre-training method for autonomous driving, significantly improves 4D occupancy forecasting, online mapping, and trajectory prediction by learning structured representations from driving data.
self-supervised pre-training next-token prediction large language models autonomous driving geometric and semantic structure unified representation general occupancy ego occupancy vision foundation model 4D occupancy fields structured representation semantic occupancy forecasting online mapping ego trajectory prediction pre-training paradigm