title: Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation
    for Generative AI
summary: In the dynamic landscape of generative NLP, traditional text processing
  pipelines limit research flexibility and reproducibility, as they are tailored
  to specific dataset, task, and model combinations. The escalating complexity,
  involving system prompts, model-specific formats, instructions, and more, calls
  for a shift to a structured, modular, and customizable solution. Addressing
  this need, we present Unitxt, an innovative library for customizable textual
  data preparation and evaluation tailored to generative language models. Unitxt
  natively integrates with common libraries like HuggingFace and LM-eval-harness
  and deconstructs processing flows into modular components, enabling easy
  customization and sharing between practitioners. These components encompass
  model-specific formats, task prompts, and many other comprehensive dataset
  processing definitions. The Unitxt-Catalog centralizes these components,
  fostering collaboration and exploration in modern textual data workflows.
  Beyond being a tool, Unitxt is a community-driven platform, empowering users to
  build, share, and advance their pipelines collaboratively. Join the Unitxt
  community at https://github.com/IBM/unitxt!
paper_authors._id: 65b31aa18627e4cca319973e
paper_authors.name: Elron Bandel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:52:55.318Z
paper_authors._id: 5fc0292de45c5468456e022b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5fc0292de45c5468456e022b/MEuHFEf6LtTbcptc0heoM.png
paper_authors.fullname: Elron Bandel
paper_authors.type: user
paper_authors.user: Elron
paper_authors._id: 65b31aa18627e4cca319973f
paper_authors.name: Yotam Perlitz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:53:07.694Z
paper_authors._id: 61e01a92a24b774dac1f32d7
paper_authors.avatarUrl: /avatars/307aac0c6cfe1879f6a5e5e1d6c10413.svg
paper_authors.fullname: Yotam Perlitz
paper_authors.type: user
paper_authors.user: per
paper_authors._id: 65b31aa18627e4cca3199740
paper_authors.name: Elad Venezian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:53:14.387Z
paper_authors._id: 62fcd609c1588e1d4c6526ea
paper_authors.avatarUrl: /avatars/11eb8818ed43e14584e3d42930308fc8.svg
paper_authors.fullname: Elad Venezian
paper_authors.type: user
paper_authors.user: eladven
paper_authors._id: 65b31aa18627e4cca3199741
paper_authors.name: Roni Friedman-Melamed
paper_authors._id: 65b31aa18627e4cca3199742
paper_authors.name: Ofir Arviv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:53:39.090Z
paper_authors._id: 632ae2606cb20ba0ae8007d8
paper_authors.avatarUrl: /avatars/023eb4536e9bfcebbcc42eaea7167596.svg
paper_authors.fullname: Ofir Arviv
paper_authors.type: user
paper_authors.user: OfirArviv
paper_authors._id: 65b31aa18627e4cca3199743
paper_authors.name: Matan Orbach
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-29T08:26:16.417Z
paper_authors._id: 65b60c17729ea1912ef74718
paper_authors.avatarUrl: /avatars/49880599a977a45ce03bc5112edf8a1a.svg
paper_authors.fullname: matan orbach
paper_authors.type: user
paper_authors.user: matanor
paper_authors._id: 65b31aa18627e4cca3199744
paper_authors.name: Shachar Don-Yehyia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-26T10:16:09.060Z
paper_authors._id: 652e3800f7431c19a99c91e9
paper_authors.avatarUrl: /avatars/df1c1d66e52b06af4dbe5791c4819782.svg
paper_authors.fullname: Shachar Don-Yehiya
paper_authors.type: user
paper_authors.user: shachar-don
paper_authors._id: 65b31aa18627e4cca3199745
paper_authors.name: Dafna Sheinwald
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:53:57.470Z
paper_authors._id: 654a153ef39b362cffa91eac
paper_authors.avatarUrl: /avatars/6e8940c6c8e13e78a31cecf140b185ab.svg
paper_authors.fullname: Dafna Sheinwald
paper_authors.type: user
paper_authors.user: dafnashein
paper_authors._id: 65b31aa18627e4cca3199746
paper_authors.name: Ariel Gera
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:54:04.079Z
paper_authors._id: 6415ebd2107962562e9a0712
paper_authors.avatarUrl: /avatars/dfd2e6d42c6110bb64b6a73c96eeb3f3.svg
paper_authors.fullname: Ariel Gera
paper_authors.type: user
paper_authors.user: arielgera
paper_authors._id: 65b31aa18627e4cca3199747
paper_authors.name: Leshem Choshen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-26T09:54:09.866Z
paper_authors._id: 61bf40824b4300d0fb0acf59
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644224872623-61bf40824b4300d0fb0acf59.jpeg
paper_authors.fullname: Leshem Choshen
paper_authors.type: user
paper_authors.user: borgr
paper_authors._id: 65b31aa18627e4cca3199748
paper_authors.name: Michal Shmueli-Scheuer
paper_authors._id: 65b31aa18627e4cca3199749
paper_authors.name: Yoav Katz
paper_title: Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation
    for Generative AI
paper_summary: In the dynamic landscape of generative NLP, traditional text processing
  pipelines limit research flexibility and reproducibility, as they are tailored
  to specific dataset, task, and model combinations. The escalating complexity,
  involving system prompts, model-specific formats, instructions, and more, calls
  for a shift to a structured, modular, and customizable solution. Addressing
  this need, we present Unitxt, an innovative library for customizable textual
  data preparation and evaluation tailored to generative language models. Unitxt
  natively integrates with common libraries like HuggingFace and LM-eval-harness
  and deconstructs processing flows into modular components, enabling easy
  customization and sharing between practitioners. These components encompass
  model-specific formats, task prompts, and many other comprehensive dataset
  processing definitions. The Unitxt-Catalog centralizes these components,
  fostering collaboration and exploration in modern textual data workflows.
  Beyond being a tool, Unitxt is a community-driven platform, empowering users to
  build, share, and advance their pipelines collaboratively. Join the Unitxt
  community at https://github.com/IBM/unitxt!
paper_ai_keywords: generative NLP textual data preparation evaluation generative language models HuggingFace LM-eval-harness modular components model-specific formats task prompts dataset processing definitions Unitxt-Catalog community-driven platform