title: Is GPT-4 a Good Data Analyst?
summary: As large language models (LLMs) have demonstrated their powerful capabilities
  in plenty of domains and tasks, including context understanding, code
  generation, language generation, data storytelling, etc., many data analysts
  may raise concerns if their jobs will be replaced by AI. This controversial
  topic has drawn a lot of attention in public. However, we are still at a stage
  of divergent opinions without any definitive conclusion. Motivated by this, we
  raise the research question of "is GPT-4 a good data analyst?" in this work and
  aim to answer it by conducting head-to-head comparative studies. In detail, we
  regard GPT-4 as a data analyst to perform end-to-end data analysis with
  databases from a wide range of domains. We propose a framework to tackle the
  problems by carefully designing the prompts for GPT-4 to conduct experiments.
  We also design several task-specific evaluation metrics to systematically
  compare the performance between several professional human data analysts and
  GPT-4. Experimental results show that GPT-4 can achieve comparable performance
  to humans. We also provide in-depth discussions about our results to shed light
  on further studies before we reach the conclusion that GPT-4 can replace data
  analysts.
paper_authors._id: 646f0f22153a68efd10f1444
paper_authors.name: Liying Cheng
paper_authors._id: 646f0f22153a68efd10f1445
paper_authors.name: Xingxuan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-26T08:10:30.716Z
paper_authors._id: 6362a77dd3be91534c2e9213
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6362a77dd3be91534c2e9213/3uUM3B1m2CFMukbkA0yDv.png
paper_authors.fullname: Xingxuan Li
paper_authors.type: user
paper_authors.user: veggiebird
paper_authors._id: 646f0f22153a68efd10f1446
paper_authors.name: Lidong Bing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-26T08:11:00.990Z
paper_authors._id: 6454685a548f22be598414c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
paper_authors.fullname: Lidong Bing
paper_authors.type: user
paper_authors.user: LidongBing
paper_title: Is GPT-4 a Good Data Analyst?
paper_summary: As large language models (LLMs) have demonstrated their powerful capabilities
  in plenty of domains and tasks, including context understanding, code
  generation, language generation, data storytelling, etc., many data analysts
  may raise concerns if their jobs will be replaced by AI. This controversial
  topic has drawn a lot of attention in public. However, we are still at a stage
  of divergent opinions without any definitive conclusion. Motivated by this, we
  raise the research question of "is GPT-4 a good data analyst?" in this work and
  aim to answer it by conducting head-to-head comparative studies. In detail, we
  regard GPT-4 as a data analyst to perform end-to-end data analysis with
  databases from a wide range of domains. We propose a framework to tackle the
  problems by carefully designing the prompts for GPT-4 to conduct experiments.
  We also design several task-specific evaluation metrics to systematically
  compare the performance between several professional human data analysts and
  GPT-4. Experimental results show that GPT-4 can achieve comparable performance
  to humans. We also provide in-depth discussions about our results to shed light
  on further studies before we reach the conclusion that GPT-4 can replace data
  analysts.
paper_ai_keywords: large language models (LLMs) context understanding code generation language generation data storytelling GPT-4 data analysis head-to-head comparative studies end-to-end data analysis prompts task-specific evaluation metrics human data analysts end-to-end data analysis