title: Comparing Software Developers with ChatGPT: An Empirical Investigation
summary: The advent of automation in particular Software Engineering (SE) tasks has
  transitioned from theory to reality. Numerous scholarly articles have
  documented the successful application of Artificial Intelligence to address
  issues in areas such as project management, modeling, testing, and development.
  A recent innovation is the introduction of ChatGPT, an ML-infused chatbot,
  touted as a resource proficient in generating programming codes and formulating
  software testing strategies for developers and testers respectively. Although
  there is speculation that AI-based computation can increase productivity and
  even substitute software engineers in software development, there is currently
  a lack of empirical evidence to verify this. Moreover, despite the primary
  focus on enhancing the accuracy of AI systems, non-functional requirements
  including energy efficiency, vulnerability, fairness (i.e., human bias), and
  safety frequently receive insufficient attention. This paper posits that a
  comprehensive comparison of software engineers and AI-based solutions,
  considering various evaluation criteria, is pivotal in fostering human-machine
  collaboration, enhancing the reliability of AI-based methods, and understanding
  task suitability for humans or AI. Furthermore, it facilitates the effective
  implementation of cooperative work structures and human-in-the-loop processes.
  This paper conducts an empirical investigation, contrasting the performance of
  software engineers and AI systems, like ChatGPT, across different evaluation
  metrics. The empirical study includes a case of assessing ChatGPT-generated
  code versus code produced by developers and uploaded in Leetcode.
paper_authors._id: 646ac70935c7a57f936c2802
paper_authors.name: Nathalia Nascimento
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-03-24T19:50:24.627Z
paper_authors._id: 660083406aa2a661dd0c504f
paper_authors.avatarUrl: /avatars/55e09db1ed079e5619ee312898a3d96a.svg
paper_authors.fullname: Nathalia Nascimento
paper_authors.type: user
paper_authors.user: nathyecomp
paper_authors._id: 646ac70935c7a57f936c2803
paper_authors.name: Paulo Alencar
paper_authors._id: 646ac70935c7a57f936c2804
paper_authors.name: Donald Cowan
paper_title: Comparing Software Developers with ChatGPT: An Empirical Investigation
paper_summary: The advent of automation in particular Software Engineering (SE) tasks has
  transitioned from theory to reality. Numerous scholarly articles have
  documented the successful application of Artificial Intelligence to address
  issues in areas such as project management, modeling, testing, and development.
  A recent innovation is the introduction of ChatGPT, an ML-infused chatbot,
  touted as a resource proficient in generating programming codes and formulating
  software testing strategies for developers and testers respectively. Although
  there is speculation that AI-based computation can increase productivity and
  even substitute software engineers in software development, there is currently
  a lack of empirical evidence to verify this. Moreover, despite the primary
  focus on enhancing the accuracy of AI systems, non-functional requirements
  including energy efficiency, vulnerability, fairness (i.e., human bias), and
  safety frequently receive insufficient attention. This paper posits that a
  comprehensive comparison of software engineers and AI-based solutions,
  considering various evaluation criteria, is pivotal in fostering human-machine
  collaboration, enhancing the reliability of AI-based methods, and understanding
  task suitability for humans or AI. Furthermore, it facilitates the effective
  implementation of cooperative work structures and human-in-the-loop processes.
  This paper conducts an empirical investigation, contrasting the performance of
  software engineers and AI systems, like ChatGPT, across different evaluation
  metrics. The empirical study includes a case of assessing ChatGPT-generated
  code versus code produced by developers and uploaded in Leetcode.
paper_ai_summary: An empirical comparison of software engineers and AI systems leverages evaluation metrics to assess the suitability and performance of AI in software development tasks.
paper_ai_keywords: Machine Learning ML-infused chatbot ChatGPT AI-based computation human-machine collaboration cooperative work structures human-in-the-loop processes empirical investigation evaluation metrics Leetcode