title: NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged
    Information Guidance
summary: Learning navigation in dynamic open-world environments is an important yet
  challenging skill for robots. Most previous methods rely on precise
  localization and mapping or learn from expensive real-world demonstrations. In
  this paper, we propose the Navigation Diffusion Policy (NavDP), an end-to-end
  framework trained solely in simulation and can zero-shot transfer to different
  embodiments in diverse real-world environments. The key ingredient of NavDP's
  network is the combination of diffusion-based trajectory generation and a
  critic function for trajectory selection, which are conditioned on only local
  observation tokens encoded from a shared policy transformer. Given the
  privileged information of the global environment in simulation, we scale up the
  demonstrations of good quality to train the diffusion policy and formulate the
  critic value function targets with contrastive negative samples. Our
  demonstration generation approach achieves about 2,500 trajectories/GPU per
  day, 20times more efficient than real-world data collection, and results in
  a large-scale navigation dataset with 363.2km trajectories across 1244 scenes.
  Trained with this simulation dataset, NavDP achieves state-of-the-art
  performance and consistently outstanding generalization capability on
  quadruped, wheeled, and humanoid robots in diverse indoor and outdoor
  environments. In addition, we present a preliminary attempt at using Gaussian
  Splatting to make in-domain real-to-sim fine-tuning to further bridge the
  sim-to-real gap. Experiments show that adding such real-to-sim data can improve
  the success rate by 30\% without hurting its generalization capability.
paper_authors._id: 682451c487e04e8c4ee5d13b
paper_authors.name: Wenzhe Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-14T10:12:04.232Z
paper_authors._id: 66a347adb839c8994e6cb641
paper_authors.avatarUrl: /avatars/efdeff32628b6c531109e047b45b2627.svg
paper_authors.fullname: Wenzhe Cai
paper_authors.type: user
paper_authors.user: WadeCai
paper_authors._id: 682451c487e04e8c4ee5d13c
paper_authors.name: Jiaqi Peng
paper_authors._id: 682451c487e04e8c4ee5d13d
paper_authors.name: Yuqiang Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-14T10:12:17.389Z
paper_authors._id: 670bbd8541e624a441f76306
paper_authors.avatarUrl: /avatars/b606cabd30f1374b1ffa82ff1b7e9ae6.svg
paper_authors.fullname: yuqiang yang
paper_authors.type: user
paper_authors.user: fulifuli666
paper_authors._id: 682451c487e04e8c4ee5d13e
paper_authors.name: Yujian Zhang
paper_authors._id: 682451c487e04e8c4ee5d13f
paper_authors.name: Meng Wei
paper_authors._id: 682451c487e04e8c4ee5d140
paper_authors.name: Hanqing Wang
paper_authors._id: 682451c487e04e8c4ee5d141
paper_authors.name: Yilun Chen
paper_authors._id: 682451c487e04e8c4ee5d142
paper_authors.name: Tai Wang
paper_authors._id: 682451c487e04e8c4ee5d143
paper_authors.name: Jiangmiao Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-14T10:13:19.562Z
paper_authors._id: 65783ee6ee33d547aecc3ffc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
paper_authors.fullname: Jiangmiao Pang
paper_authors.type: user
paper_authors.user: Jiangmiao
paper_title: NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged
    Information Guidance
paper_summary: Learning navigation in dynamic open-world environments is an important yet
  challenging skill for robots. Most previous methods rely on precise
  localization and mapping or learn from expensive real-world demonstrations. In
  this paper, we propose the Navigation Diffusion Policy (NavDP), an end-to-end
  framework trained solely in simulation and can zero-shot transfer to different
  embodiments in diverse real-world environments. The key ingredient of NavDP's
  network is the combination of diffusion-based trajectory generation and a
  critic function for trajectory selection, which are conditioned on only local
  observation tokens encoded from a shared policy transformer. Given the
  privileged information of the global environment in simulation, we scale up the
  demonstrations of good quality to train the diffusion policy and formulate the
  critic value function targets with contrastive negative samples. Our
  demonstration generation approach achieves about 2,500 trajectories/GPU per
  day, 20times more efficient than real-world data collection, and results in
  a large-scale navigation dataset with 363.2km trajectories across 1244 scenes.
  Trained with this simulation dataset, NavDP achieves state-of-the-art
  performance and consistently outstanding generalization capability on
  quadruped, wheeled, and humanoid robots in diverse indoor and outdoor
  environments. In addition, we present a preliminary attempt at using Gaussian
  Splatting to make in-domain real-to-sim fine-tuning to further bridge the
  sim-to-real gap. Experiments show that adding such real-to-sim data can improve
  the success rate by 30\% without hurting its generalization capability.
paper_ai_summary: NavDP, an end-to-end diffusion policy framework trained in simulation, achieves state-of-the-art performance and generalization in diverse real-world environments through efficient data generation and real-to-sim fine-tuning using Gaussian Splatting.
paper_ai_keywords: navigation diffusion policy NavDP diffusion-based trajectory generation critic function trajectory selection local observation tokens policy transformer global environment contrastive negative samples Gaussian Splatting sim-to-real gap