HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for
  Autonomous Driving
In the past few decades, autonomous driving algorithms have made significant
progress in perception, planning, and control. However, evaluating individual
components does not fully reflect the performance of entire systems,
highlighting the need for more holistic assessment methods. This motivates the
development of HUGSIM, a closed-loop, photo-realistic, and real-time simulator
for evaluating autonomous driving algorithms. We achieve this by lifting
captured 2D RGB images into the 3D space via 3D Gaussian Splatting, improving
the rendering quality for closed-loop scenarios, and building the closed-loop
environment. In terms of rendering, We tackle challenges of novel view
synthesis in closed-loop scenarios, including viewpoint extrapolation and
360-degree vehicle rendering. Beyond novel view synthesis, HUGSIM further
enables the full closed simulation loop, dynamically updating the ego and actor
states and observations based on control commands. Moreover, HUGSIM offers a
comprehensive benchmark across more than 70 sequences from KITTI-360, Waymo,
nuScenes, and PandaSet, along with over 400 varying scenarios, providing a fair
and realistic evaluation platform for existing autonomous driving algorithms.
HUGSIM not only serves as an intuitive evaluation benchmark but also unlocks
the potential for fine-tuning autonomous driving algorithms in a photorealistic
closed-loop setting.
674e7f8d5a971a957da459b3
Hongyu Zhou
claimed_verified
2024-12-03T11:15:10.817Z
664d9d1eafe6e8c3a986ef67
/avatars/88535076724e97f412ecd362a38904db.svg
hyzhou
user
hyzhou404
674e7f8d5a971a957da459b4
Longzhong Lin
674e7f8d5a971a957da459b5
Jiabao Wang
674e7f8d5a971a957da459b6
Yichong Lu
674e7f8d5a971a957da459b7
Dongfeng Bai
674e7f8d5a971a957da459b8
Bingbing Liu
674e7f8d5a971a957da459b9
Yue Wang
674e7f8d5a971a957da459ba
Andreas Geiger
674e7f8d5a971a957da459bb
Yiyi Liao
HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for
  Autonomous Driving
In the past few decades, autonomous driving algorithms have made significant
progress in perception, planning, and control. However, evaluating individual
components does not fully reflect the performance of entire systems,
highlighting the need for more holistic assessment methods. This motivates the
development of HUGSIM, a closed-loop, photo-realistic, and real-time simulator
for evaluating autonomous driving algorithms. We achieve this by lifting
captured 2D RGB images into the 3D space via 3D Gaussian Splatting, improving
the rendering quality for closed-loop scenarios, and building the closed-loop
environment. In terms of rendering, We tackle challenges of novel view
synthesis in closed-loop scenarios, including viewpoint extrapolation and
360-degree vehicle rendering. Beyond novel view synthesis, HUGSIM further
enables the full closed simulation loop, dynamically updating the ego and actor
states and observations based on control commands. Moreover, HUGSIM offers a
comprehensive benchmark across more than 70 sequences from KITTI-360, Waymo,
nuScenes, and PandaSet, along with over 400 varying scenarios, providing a fair
and realistic evaluation platform for existing autonomous driving algorithms.
HUGSIM not only serves as an intuitive evaluation benchmark but also unlocks
the potential for fine-tuning autonomous driving algorithms in a photorealistic
closed-loop setting.
HUGSIM, a closed-loop, photo-realistic, and real-time simulator, enhances the evaluation of autonomous driving algorithms through 3D Gaussian Splatting and a comprehensive benchmark.
3D Gaussian Splatting closed-loop photo-realistic real-time simulator novel view synthesis viewpoint extrapolation 360-degree vehicle rendering autonomous driving algorithms KITTI-360 Waymo nuScenes PandaSet