GAEA: A Geolocation Aware Conversational Model
Image geolocalization, in which, traditionally, an AI model predicts the
precise GPS coordinates of an image is a challenging task with many downstream
applications. However, the user cannot utilize the model to further their
knowledge other than the GPS coordinate; the model lacks an understanding of
the location and the conversational ability to communicate with the user. In
recent days, with tremendous progress of large multimodal models (LMMs)
proprietary and open-source researchers have attempted to geolocalize images
via LMMs. However, the issues remain unaddressed; beyond general tasks, for
more specialized downstream tasks, one of which is geolocalization, LMMs
struggle. In this work, we propose to solve this problem by introducing a
conversational model GAEA that can provide information regarding the location
of an image, as required by a user. No large-scale dataset enabling the
training of such a model exists. Thus we propose a comprehensive dataset GAEA
with 800K images and around 1.6M question answer pairs constructed by
leveraging OpenStreetMap (OSM) attributes and geographical context clues. For
quantitative evaluation, we propose a diverse benchmark comprising 4K
image-text pairs to evaluate conversational capabilities equipped with diverse
question types. We consider 11 state-of-the-art open-source and proprietary
LMMs and demonstrate that GAEA significantly outperforms the best open-source
model, LLaVA-OneVision by 25.69% and the best proprietary model, GPT-4o by
8.28%. Our dataset, model and codes are available
67def53e2c86b9aaee43f306
Ron Campos
claimed_verified
2025-03-24T14:49:32.851Z
63dd76b63880ace5ca41c133
/avatars/d8593fc89c00eeadc7941be040174d35.svg
Ron Campos
user
rjccv
67def53e2c86b9aaee43f307
Ashmal Vayani
67def53e2c86b9aaee43f308
Parth Parag Kulkarni
67def53e2c86b9aaee43f309
Rohit Gupta
67def53e2c86b9aaee43f30a
Aritra Dutta
claimed_verified
2025-03-24T08:52:11.893Z
67d58d156db0e6f0c33c0f60
https://cdn-avatars.huggingface.co/v1/production/uploads/67d58d156db0e6f0c33c0f60/9KiFw0iEMZQHcHnm1k0ED.jpeg
Aritra Dutta
user
aritradutta
67def53e2c86b9aaee43f30b
Mubarak Shah
GAEA: A Geolocation Aware Conversational Model
Image geolocalization, in which, traditionally, an AI model predicts the
precise GPS coordinates of an image is a challenging task with many downstream
applications. However, the user cannot utilize the model to further their
knowledge other than the GPS coordinate; the model lacks an understanding of
the location and the conversational ability to communicate with the user. In
recent days, with tremendous progress of large multimodal models (LMMs)
proprietary and open-source researchers have attempted to geolocalize images
via LMMs. However, the issues remain unaddressed; beyond general tasks, for
more specialized downstream tasks, one of which is geolocalization, LMMs
struggle. In this work, we propose to solve this problem by introducing a
conversational model GAEA that can provide information regarding the location
of an image, as required by a user. No large-scale dataset enabling the
training of such a model exists. Thus we propose a comprehensive dataset GAEA
with 800K images and around 1.6M question answer pairs constructed by
leveraging OpenStreetMap (OSM) attributes and geographical context clues. For
quantitative evaluation, we propose a diverse benchmark comprising 4K
image-text pairs to evaluate conversational capabilities equipped with diverse
question types. We consider 11 state-of-the-art open-source and proprietary
LMMs and demonstrate that GAEA significantly outperforms the best open-source
model, LLaVA-OneVision by 25.69% and the best proprietary model, GPT-4o by
8.28%. Our dataset, model and codes are available
A conversational model named GAEA addresses geolocalization by providing detailed location information, outperforming state-of-the-art multimodal models.
multimodal models conversational model geolocalization dataset OpenStreetMap geographical context benchmark LMMs LLaVA-OneVision GPT-4o