Techniques for strategically crafting a ML dataset For a supervised machine learning application where the input features can be readily calculated and the corresponding labels are the result of a somewhat time-consuming simulation using the inputs, it is advantageous to "craft" the best dataset with the fewest data. We have several ideas to this end, a few are:

*

*Use unsupervised learning techniques to build an initial set of inputs that maximally spans the feature space.

*After an iteration of training, generate new inputs and add to the training set if the trained model is uncertain of the input class (predicted probability near 0.5 for a binary system).

*Add new inputs to the training set based on whether they are similar to previous inputs that were misclassified (using unsupervised learning)

I am wondering if any of these are valid approaches and more broadly if there is a name for techniques likes this where a ML dataset is strategically crafted and improved in an iterative manner?
This is different than most data augmentation techniques I'm familiar with because rather than processing existing inputs (e.g. rotating, translating images) in this application it's trivial to generate entirely new inputs.

This is clearly an open and active research question. To my knowledge it is relevant in a few different contexts:
1/ In a supervised learning setting, it can be extremely tedious (for humans) to label the training data. In that case it is preferable to label the samples that "matter the most".
2/ In Scientific Machine Learning, ground truth data is usually generated by computationally expensive ODE or PDE solvers, which limits the number of available samples.
3/ In sensor-based ML, such sensors (cameras, satellites, thermometers, etc...) can be expensive/difficult to monitor, which limit the number of available sensors.
You may take a look at a few recent papers:

*

*Increasing Data Diversity with Iterative Sampling to Improve Performance

*Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning

*Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities

*Exploring Representativeness and Informativeness for Active Learning

To my knowledge, the problem of "crafting" small but representative datasets is very popular in the domain of Active Learning as well as data-centric AI.
