title: Generating Skyline Datasets for Data Science Models
summary: Preparing high-quality datasets required by various data-driven AI and
  machine learning models has become a cornerstone task in data-driven analysis.
  Conventional data discovery methods typically integrate datasets towards a
  single pre-defined quality measure that may lead to bias for downstream tasks.
  This paper introduces MODis, a framework that discovers datasets by optimizing
  multiple user-defined, model-performance measures. Given a set of data sources
  and a model, MODis selects and integrates data sources into a skyline dataset,
  over which the model is expected to have the desired performance in all the
  performance measures. We formulate MODis as a multi-goal finite state
  transducer, and derive three feasible algorithms to generate skyline datasets.
  Our first algorithm adopts a "reduce-from-universal" strategy, that starts with
  a universal schema and iteratively prunes unpromising data. Our second
  algorithm further reduces the cost with a bi-directional strategy that
  interleaves data augmentation and reduction. We also introduce a
  diversification algorithm to mitigate the bias in skyline datasets. We
  experimentally verify the efficiency and effectiveness of our skyline data
  discovery algorithms, and showcase their applications in optimizing data
  science pipelines.
paper_authors._id: 67b68ee59076bb9959a6fd6e
paper_authors.name: Mengying Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T09:36:16.105Z
paper_authors._id: 63f05764f1a47aaea5bcdee0
paper_authors.avatarUrl: /avatars/3a58d6f0439dce32d2010499f321fe9d.svg
paper_authors.fullname: Mengying Wang
paper_authors.type: user
paper_authors.user: wmying
paper_authors._id: 67b68ee59076bb9959a6fd6f
paper_authors.name: Hanchao Ma
paper_authors._id: 67b68ee59076bb9959a6fd70
paper_authors.name: Yiyang Bian
paper_authors._id: 67b68ee59076bb9959a6fd71
paper_authors.name: Yangxin Fan
paper_authors._id: 67b68ee59076bb9959a6fd72
paper_authors.name: Yinghui Wu
paper_title: Generating Skyline Datasets for Data Science Models
paper_summary: Preparing high-quality datasets required by various data-driven AI and
  machine learning models has become a cornerstone task in data-driven analysis.
  Conventional data discovery methods typically integrate datasets towards a
  single pre-defined quality measure that may lead to bias for downstream tasks.
  This paper introduces MODis, a framework that discovers datasets by optimizing
  multiple user-defined, model-performance measures. Given a set of data sources
  and a model, MODis selects and integrates data sources into a skyline dataset,
  over which the model is expected to have the desired performance in all the
  performance measures. We formulate MODis as a multi-goal finite state
  transducer, and derive three feasible algorithms to generate skyline datasets.
  Our first algorithm adopts a "reduce-from-universal" strategy, that starts with
  a universal schema and iteratively prunes unpromising data. Our second
  algorithm further reduces the cost with a bi-directional strategy that
  interleaves data augmentation and reduction. We also introduce a
  diversification algorithm to mitigate the bias in skyline datasets. We
  experimentally verify the efficiency and effectiveness of our skyline data
  discovery algorithms, and showcase their applications in optimizing data
  science pipelines.