DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM
  Workflows
Large language models (LLMs) have become a dominant and important tool for
NLP researchers in a wide range of tasks. Today, many researchers use LLMs in
synthetic data generation, task evaluation, fine-tuning, distillation, and
other model-in-the-loop research workflows. However, challenges arise when
using these models that stem from their scale, their closed source nature, and
the lack of standardized tooling for these new and emerging workflows. The
rapid rise to prominence of these models and these unique challenges has had
immediate adverse impacts on open science and on the reproducibility of work
that uses them. In this paper, we introduce DataDreamer, an open source Python
library that allows researchers to write simple code to implement powerful LLM
workflows. DataDreamer also helps researchers adhere to best practices that we
propose to encourage open science and reproducibility. The library and
documentation are available at https://github.com/datadreamer-dev/DataDreamer .
65d2d88d6236ca85a40313c8
Ajay Patel
extracted_confirmed
2024-02-19T04:29:30.810Z
61c40eeb727d1257bf3cf5ba
https://cdn-avatars.huggingface.co/v1/production/uploads/61c40eeb727d1257bf3cf5ba/hVNbcFjsvwWqWarcGTOdI.jpeg
Ajay Patel
user
AjayP13
65d2d88d6236ca85a40313c9
Colin Raffel
extracted_pending
2024-02-19T04:26:53.817Z
6079c29765b9d0165cb18392
https://cdn-avatars.huggingface.co/v1/production/uploads/1618592397610-noauth.jpeg
Colin Raffel
user
craffel
65d2d88d6236ca85a40313ca
Chris Callison-Burch
extracted_confirmed
2024-02-19T12:02:44.275Z
6303ce25fc783bfc744216af
/avatars/09f5e87c1f56a1b7f6ef9c5037682285.svg
Chris Callison-Burch
user
CCB
DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM
  Workflows
Large language models (LLMs) have become a dominant and important tool for
NLP researchers in a wide range of tasks. Today, many researchers use LLMs in
synthetic data generation, task evaluation, fine-tuning, distillation, and
other model-in-the-loop research workflows. However, challenges arise when
using these models that stem from their scale, their closed source nature, and
the lack of standardized tooling for these new and emerging workflows. The
rapid rise to prominence of these models and these unique challenges has had
immediate adverse impacts on open science and on the reproducibility of work
that uses them. In this paper, we introduce DataDreamer, an open source Python
library that allows researchers to write simple code to implement powerful LLM
workflows. DataDreamer also helps researchers adhere to best practices that we
propose to encourage open science and reproducibility. The library and
documentation are available at https://github.com/datadreamer-dev/DataDreamer .
DataDreamer is an open-source Python library facilitating LLM workflows and promoting open science and reproducibility in NLP research.
Large language models LLMs NLP synthetic data generation task evaluation fine-tuning distillation open source open science reproducibility