title: DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car
    Reconstruction
summary: Self-driving industries usually employ professional artists to build
  exquisite 3D cars. However, it is expensive to craft large-scale digital
  assets. Since there are already numerous datasets available that contain a vast
  number of images of cars, we focus on reconstructing high-quality 3D car models
  from these datasets. However, these datasets only contain one side of cars in
  the forward-moving scene. We try to use the existing generative models to
  provide more supervision information, but they struggle to generalize well in
  cars since they are trained on synthetic datasets not car-specific. In
  addition, The reconstructed 3D car texture misaligns due to a large error in
  camera pose estimation when dealing with in-the-wild images. These restrictions
  make it challenging for previous methods to reconstruct complete 3D cars. To
  address these problems, we propose a novel method, named DreamCar, which can
  reconstruct high-quality 3D cars given a few images even a single image. To
  generalize the generative model, we collect a car dataset, named Car360, with
  over 5,600 vehicles. With this dataset, we make the generative model more
  robust to cars. We use this generative prior specific to the car to guide its
  reconstruction via Score Distillation Sampling. To further complement the
  supervision information, we utilize the geometric and appearance symmetry of
  cars. Finally, we propose a pose optimization method that rectifies poses to
  tackle texture misalignment. Extensive experiments demonstrate that our method
  significantly outperforms existing methods in reconstructing high-quality 3D
  cars. https://xiaobiaodu.github.io/dreamcar-project/{Our code is
  available.}
paper_authors._id: 66a1b1438f0d2327e0a19314
paper_authors.name: Xiaobiao Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-25T07:38:30.366Z
paper_authors._id: 64ad086c5d48838462e2eee1
paper_authors.avatarUrl: /avatars/735b29d01bae05599e7d5ce61b223153.svg
paper_authors.fullname: Du
paper_authors.type: user
paper_authors.user: xiaobiaodu
paper_authors._id: 66a1b1438f0d2327e0a19315
paper_authors.name: Haiyang Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-25T12:25:59.881Z
paper_authors._id: 639cc617b0496dc19b6f326f
paper_authors.avatarUrl: /avatars/0bfabcb1f973716b604c9d733e499242.svg
paper_authors.fullname: Haiyang Sun
paper_authors.type: user
paper_authors.user: haiyangsun
paper_authors._id: 66a1b1438f0d2327e0a19316
paper_authors.name: Ming Lu
paper_authors._id: 66a1b1438f0d2327e0a19317
paper_authors.name: Tianqing Zhu
paper_authors._id: 66a1b1438f0d2327e0a19318
paper_authors.name: Xin Yu
paper_title: DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car
    Reconstruction
paper_summary: Self-driving industries usually employ professional artists to build
  exquisite 3D cars. However, it is expensive to craft large-scale digital
  assets. Since there are already numerous datasets available that contain a vast
  number of images of cars, we focus on reconstructing high-quality 3D car models
  from these datasets. However, these datasets only contain one side of cars in
  the forward-moving scene. We try to use the existing generative models to
  provide more supervision information, but they struggle to generalize well in
  cars since they are trained on synthetic datasets not car-specific. In
  addition, The reconstructed 3D car texture misaligns due to a large error in
  camera pose estimation when dealing with in-the-wild images. These restrictions
  make it challenging for previous methods to reconstruct complete 3D cars. To
  address these problems, we propose a novel method, named DreamCar, which can
  reconstruct high-quality 3D cars given a few images even a single image. To
  generalize the generative model, we collect a car dataset, named Car360, with
  over 5,600 vehicles. With this dataset, we make the generative model more
  robust to cars. We use this generative prior specific to the car to guide its
  reconstruction via Score Distillation Sampling. To further complement the
  supervision information, we utilize the geometric and appearance symmetry of
  cars. Finally, we propose a pose optimization method that rectifies poses to
  tackle texture misalignment. Extensive experiments demonstrate that our method
  significantly outperforms existing methods in reconstructing high-quality 3D
  cars. https://xiaobiaodu.github.io/dreamcar-project/{Our code is
  available.}
paper_ai_keywords: DreamCar Score Distillation Sampling Car360 dataset pose optimization texture misalignment