title: MapQaTor: A System for Efficient Annotation of Map Query Datasets
summary: Mapping and navigation services like Google Maps, Apple Maps, Openstreet
  Maps, are essential for accessing various location-based data, yet they often
  struggle to handle natural language geospatial queries. Recent advancements in
  Large Language Models (LLMs) show promise in question answering (QA), but
  creating reliable geospatial QA datasets from map services remains challenging.
  We introduce MapQaTor, a web application that streamlines the creation of
  reproducible, traceable map-based QA datasets. With its plug-and-play
  architecture, MapQaTor enables seamless integration with any maps API, allowing
  users to gather and visualize data from diverse sources with minimal setup. By
  caching API responses, the platform ensures consistent ground truth, enhancing
  the reliability of the data even as real-world information evolves. MapQaTor
  centralizes data retrieval, annotation, and visualization within a single
  platform, offering a unique opportunity to evaluate the current state of
  LLM-based geospatial reasoning while advancing their capabilities for improved
  geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the
  annotation process by at least 30 times compared to manual methods,
  underscoring its potential for developing geospatial resources, such as complex
  map reasoning datasets. The website is live at: https://mapqator.github.io/ and
  a demo video is available at: https://youtu.be/7_aV9Wmhs6Q.
paper_authors._id: 67740b5f846a267749caed44
paper_authors.name: Mahir Labib Dihan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-01T20:12:31.356Z
paper_authors._id: 6565a60e0ff3292512bae26d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6565a60e0ff3292512bae26d/5P6wgEhCI0ndFVgX9KX8k.jpeg
paper_authors.fullname: Mahir Labib Dihan
paper_authors.type: user
paper_authors.user: mahirlabibdihan
paper_authors._id: 67740b5f846a267749caed45
paper_authors.name: Mohammed Eunus Ali
paper_authors._id: 67740b5f846a267749caed46
paper_authors.name: Md Rizwan Parvez
paper_title: MapQaTor: A System for Efficient Annotation of Map Query Datasets
paper_summary: Mapping and navigation services like Google Maps, Apple Maps, Openstreet
  Maps, are essential for accessing various location-based data, yet they often
  struggle to handle natural language geospatial queries. Recent advancements in
  Large Language Models (LLMs) show promise in question answering (QA), but
  creating reliable geospatial QA datasets from map services remains challenging.
  We introduce MapQaTor, a web application that streamlines the creation of
  reproducible, traceable map-based QA datasets. With its plug-and-play
  architecture, MapQaTor enables seamless integration with any maps API, allowing
  users to gather and visualize data from diverse sources with minimal setup. By
  caching API responses, the platform ensures consistent ground truth, enhancing
  the reliability of the data even as real-world information evolves. MapQaTor
  centralizes data retrieval, annotation, and visualization within a single
  platform, offering a unique opportunity to evaluate the current state of
  LLM-based geospatial reasoning while advancing their capabilities for improved
  geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the
  annotation process by at least 30 times compared to manual methods,
  underscoring its potential for developing geospatial resources, such as complex
  map reasoning datasets. The website is live at: https://mapqator.github.io/ and
  a demo video is available at: https://youtu.be/7_aV9Wmhs6Q.