title: GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
summary: Recent advances in large multimodal models (LMMs) have recognized
  fine-grained grounding as an imperative factor of visual understanding and
  dialogue. However, the benefits of such representation in LMMs are limited to
  the natural image domain, and these models perform poorly for remote sensing
  (RS). The distinct overhead viewpoint, scale variation, and presence of small
  objects in high-resolution RS imagery present a unique challenge in
  region-level comprehension. Moreover, the development of the grounding
  conversation capability of LMMs within RS is hindered by the lack of granular,
  RS domain-specific grounded data. Addressing these limitations, we propose
  GeoPixel - the first end-to-end high resolution RS-LMM that supports
  pixel-level grounding. This capability allows fine-grained visual perception by
  generating interleaved masks in conversation. GeoPixel supports up to 4K HD
  resolution in any aspect ratio, ideal for high-precision RS image analysis. To
  support the grounded conversation generation (GCG) in RS imagery, we curate a
  visually grounded dataset GeoPixelD through a semi-automated pipeline that
  utilizes set-of-marks prompting and spatial priors tailored for RS data to
  methodically control the data generation process. GeoPixel demonstrates
  superior performance in pixel-level comprehension, surpassing existing LMMs in
  both single-target and multi-target segmentation tasks. Our methodological
  ablation studies validate the effectiveness of each component in the overall
  architecture. Our code and data will be publicly released.
paper_authors._id: 679789af4a10be7109a28675
paper_authors.name: Akashah Shabbir
paper_authors._id: 679789af4a10be7109a28676
paper_authors.name: Mohammed Zumri
paper_authors._id: 679789af4a10be7109a28677
paper_authors.name: Mohammed Bennamoun
paper_authors._id: 679789af4a10be7109a28678
paper_authors.name: Fahad S. Khan
paper_authors._id: 679789af4a10be7109a28679
paper_authors.name: Salman Khan
paper_title: GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
paper_summary: Recent advances in large multimodal models (LMMs) have recognized
  fine-grained grounding as an imperative factor of visual understanding and
  dialogue. However, the benefits of such representation in LMMs are limited to
  the natural image domain, and these models perform poorly for remote sensing
  (RS). The distinct overhead viewpoint, scale variation, and presence of small
  objects in high-resolution RS imagery present a unique challenge in
  region-level comprehension. Moreover, the development of the grounding
  conversation capability of LMMs within RS is hindered by the lack of granular,
  RS domain-specific grounded data. Addressing these limitations, we propose
  GeoPixel - the first end-to-end high resolution RS-LMM that supports
  pixel-level grounding. This capability allows fine-grained visual perception by
  generating interleaved masks in conversation. GeoPixel supports up to 4K HD
  resolution in any aspect ratio, ideal for high-precision RS image analysis. To
  support the grounded conversation generation (GCG) in RS imagery, we curate a
  visually grounded dataset GeoPixelD through a semi-automated pipeline that
  utilizes set-of-marks prompting and spatial priors tailored for RS data to
  methodically control the data generation process. GeoPixel demonstrates
  superior performance in pixel-level comprehension, surpassing existing LMMs in
  both single-target and multi-target segmentation tasks. Our methodological
  ablation studies validate the effectiveness of each component in the overall
  architecture. Our code and data will be publicly released.
paper_ai_keywords: GeoPixel large multimodal models (LMMs) fine-grained grounding visual understanding dialogue remote sensing (RS) region-level comprehension grounding conversation capability pixel-level grounding high resolution RS-LMM visually grounded dataset semi-automated pipeline set-of-marks prompting spatial priors pixel-level comprehension single-target segmentation multi-target segmentation