Assessing Good, Bad and Ugly Arguments Generated by ChatGPT: a New Dataset, its Methodology and Associated Tasks
<|reference_start|>Assessing Good, Bad and Ugly Arguments Generated by ChatGPT: a New Dataset, its Methodology and Associated Tasks: The recent success of Large Language Models (LLMs) has sparked concerns about their potential to spread misinformation. As a result, there is a pressing need for tools to identify ``fake arguments'' generated by such models. To create these tools, examples of texts generated by LLMs are needed. This paper introduces a methodology to obtain good, bad and ugly arguments from argumentative essays produced by ChatGPT, OpenAI's LLM. We then describe a novel dataset containing a set of diverse arguments, ArGPT. We assess the effectiveness of our dataset and establish baselines for several argumentation-related tasks. Finally, we show that the artificially generated data relates well to human argumentation and thus is useful as a tool to train and test systems for the defined tasks.<|reference_end|>
@article{rocha2024assessing,
    title={Assessing Good, Bad and Ugly Arguments Generated by ChatGPT: a New
  Dataset, its Methodology and Associated Tasks},
    author={Victor Hugo Nascimento Rocha, Igor Cataneo Silveira, Paulo Pirozelli,
  Denis Deratani Mau'a, Fabio Gagliardi Cozman},
    journal={Progress in Artificial Intelligence (EPIA 2023)},
    year={2024},
    doi={10.1007/978-3-031-49008-8_34},
    archivePrefix={arXiv},
    eprint={2406.15130},
    primaryClass={cs.CL cs.AI}
}