title: GAEA: A Geolocation Aware Conversational Model
summary: Image geolocalization, in which, traditionally, an AI model predicts the
  precise GPS coordinates of an image is a challenging task with many downstream
  applications. However, the user cannot utilize the model to further their
  knowledge other than the GPS coordinate; the model lacks an understanding of
  the location and the conversational ability to communicate with the user. In
  recent days, with tremendous progress of large multimodal models (LMMs)
  proprietary and open-source researchers have attempted to geolocalize images
  via LMMs. However, the issues remain unaddressed; beyond general tasks, for
  more specialized downstream tasks, one of which is geolocalization, LMMs
  struggle. In this work, we propose to solve this problem by introducing a
  conversational model GAEA that can provide information regarding the location
  of an image, as required by a user. No large-scale dataset enabling the
  training of such a model exists. Thus we propose a comprehensive dataset GAEA
  with 800K images and around 1.6M question answer pairs constructed by
  leveraging OpenStreetMap (OSM) attributes and geographical context clues. For
  quantitative evaluation, we propose a diverse benchmark comprising 4K
  image-text pairs to evaluate conversational capabilities equipped with diverse
  question types. We consider 11 state-of-the-art open-source and proprietary
  LMMs and demonstrate that GAEA significantly outperforms the best open-source
  model, LLaVA-OneVision by 25.69% and the best proprietary model, GPT-4o by
  8.28%. Our dataset, model and codes are available
paper_authors._id: 67def53e2c86b9aaee43f306
paper_authors.name: Ron Campos
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-24T14:49:32.851Z
paper_authors._id: 63dd76b63880ace5ca41c133
paper_authors.avatarUrl: /avatars/d8593fc89c00eeadc7941be040174d35.svg
paper_authors.fullname: Ron Campos
paper_authors.type: user
paper_authors.user: rjccv
paper_authors._id: 67def53e2c86b9aaee43f307
paper_authors.name: Ashmal Vayani
paper_authors._id: 67def53e2c86b9aaee43f308
paper_authors.name: Parth Parag Kulkarni
paper_authors._id: 67def53e2c86b9aaee43f309
paper_authors.name: Rohit Gupta
paper_authors._id: 67def53e2c86b9aaee43f30a
paper_authors.name: Aritra Dutta
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-24T08:52:11.893Z
paper_authors._id: 67d58d156db0e6f0c33c0f60
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67d58d156db0e6f0c33c0f60/9KiFw0iEMZQHcHnm1k0ED.jpeg
paper_authors.fullname: Aritra Dutta
paper_authors.type: user
paper_authors.user: aritradutta
paper_authors._id: 67def53e2c86b9aaee43f30b
paper_authors.name: Mubarak Shah
paper_title: GAEA: A Geolocation Aware Conversational Model
paper_summary: Image geolocalization, in which, traditionally, an AI model predicts the
  precise GPS coordinates of an image is a challenging task with many downstream
  applications. However, the user cannot utilize the model to further their
  knowledge other than the GPS coordinate; the model lacks an understanding of
  the location and the conversational ability to communicate with the user. In
  recent days, with tremendous progress of large multimodal models (LMMs)
  proprietary and open-source researchers have attempted to geolocalize images
  via LMMs. However, the issues remain unaddressed; beyond general tasks, for
  more specialized downstream tasks, one of which is geolocalization, LMMs
  struggle. In this work, we propose to solve this problem by introducing a
  conversational model GAEA that can provide information regarding the location
  of an image, as required by a user. No large-scale dataset enabling the
  training of such a model exists. Thus we propose a comprehensive dataset GAEA
  with 800K images and around 1.6M question answer pairs constructed by
  leveraging OpenStreetMap (OSM) attributes and geographical context clues. For
  quantitative evaluation, we propose a diverse benchmark comprising 4K
  image-text pairs to evaluate conversational capabilities equipped with diverse
  question types. We consider 11 state-of-the-art open-source and proprietary
  LMMs and demonstrate that GAEA significantly outperforms the best open-source
  model, LLaVA-OneVision by 25.69% and the best proprietary model, GPT-4o by
  8.28%. Our dataset, model and codes are available
paper_ai_summary: A conversational model named GAEA addresses geolocalization by providing detailed location information, outperforming state-of-the-art multimodal models.
paper_ai_keywords: multimodal models conversational model geolocalization dataset OpenStreetMap geographical context benchmark LMMs LLaVA-OneVision GPT-4o