title: AerialMegaDepth: Learning Aerial-Ground Reconstruction and View
    Synthesis
summary: We explore the task of geometric reconstruction of images captured from a
  mixture of ground and aerial views. Current state-of-the-art learning-based
  approaches fail to handle the extreme viewpoint variation between aerial-ground
  image pairs. Our hypothesis is that the lack of high-quality, co-registered
  aerial-ground datasets for training is a key reason for this failure. Such data
  is difficult to assemble precisely because it is difficult to reconstruct in a
  scalable way. To overcome this challenge, we propose a scalable framework
  combining pseudo-synthetic renderings from 3D city-wide meshes (e.g., Google
  Earth) with real, ground-level crowd-sourced images (e.g., MegaDepth). The
  pseudo-synthetic data simulates a wide range of aerial viewpoints, while the
  real, crowd-sourced images help improve visual fidelity for ground-level images
  where mesh-based renderings lack sufficient detail, effectively bridging the
  domain gap between real images and pseudo-synthetic renderings. Using this
  hybrid dataset, we fine-tune several state-of-the-art algorithms and achieve
  significant improvements on real-world, zero-shot aerial-ground tasks. For
  example, we observe that baseline DUSt3R localizes fewer than 5% of
  aerial-ground pairs within 5 degrees of camera rotation error, while
  fine-tuning with our data raises accuracy to nearly 56%, addressing a major
  failure point in handling large viewpoint changes. Beyond camera estimation and
  scene reconstruction, our dataset also improves performance on downstream tasks
  like novel-view synthesis in challenging aerial-ground scenarios, demonstrating
  the practical value of our approach in real-world applications.
paper_authors._id: 6804392129303a3402c4f38e
paper_authors.name: Khiem Vuong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-20T15:01:35.224Z
paper_authors._id: 631bfb21f6bc4be4a6592afc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631bfb21f6bc4be4a6592afc/FRgc7nwHylQZ9QURrr88y.jpeg
paper_authors.fullname: Khiem Vuong
paper_authors.type: user
paper_authors.user: kvuong2711
paper_authors._id: 6804392129303a3402c4f38f
paper_authors.name: Anurag Ghosh
paper_authors._id: 6804392129303a3402c4f390
paper_authors.name: Deva Ramanan
paper_authors._id: 6804392129303a3402c4f391
paper_authors.name: Srinivasa Narasimhan
paper_authors._id: 6804392129303a3402c4f392
paper_authors.name: Shubham Tulsiani
paper_title: AerialMegaDepth: Learning Aerial-Ground Reconstruction and View
    Synthesis
paper_summary: We explore the task of geometric reconstruction of images captured from a
  mixture of ground and aerial views. Current state-of-the-art learning-based
  approaches fail to handle the extreme viewpoint variation between aerial-ground
  image pairs. Our hypothesis is that the lack of high-quality, co-registered
  aerial-ground datasets for training is a key reason for this failure. Such data
  is difficult to assemble precisely because it is difficult to reconstruct in a
  scalable way. To overcome this challenge, we propose a scalable framework
  combining pseudo-synthetic renderings from 3D city-wide meshes (e.g., Google
  Earth) with real, ground-level crowd-sourced images (e.g., MegaDepth). The
  pseudo-synthetic data simulates a wide range of aerial viewpoints, while the
  real, crowd-sourced images help improve visual fidelity for ground-level images
  where mesh-based renderings lack sufficient detail, effectively bridging the
  domain gap between real images and pseudo-synthetic renderings. Using this
  hybrid dataset, we fine-tune several state-of-the-art algorithms and achieve
  significant improvements on real-world, zero-shot aerial-ground tasks. For
  example, we observe that baseline DUSt3R localizes fewer than 5% of
  aerial-ground pairs within 5 degrees of camera rotation error, while
  fine-tuning with our data raises accuracy to nearly 56%, addressing a major
  failure point in handling large viewpoint changes. Beyond camera estimation and
  scene reconstruction, our dataset also improves performance on downstream tasks
  like novel-view synthesis in challenging aerial-ground scenarios, demonstrating
  the practical value of our approach in real-world applications.
paper_ai_summary: A scalable hybrid dataset combining pseudo-synthetic renderings and real ground-level images improves performance in geometric reconstruction and novel-view synthesis across aerial-ground viewpoints.
paper_ai_keywords: geometric reconstruction aerial-ground views viewpoint variation co-registered datasets pseudo-synthetic renderings 3D city-wide meshes crowd-sourced images visual fidelity domain gap baseline DUSt3R camera rotation error novel-view synthesis