Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks
<|reference_start|>Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks: Existing subset selection methods for efficient learning predominantly employ discrete combinatorial and model-specific approaches which lack generalizability. For an unseen architecture, one cannot use the subset chosen for a different model. To tackle this problem, we propose $\texttt{SubSelNet}$, a trainable subset selection framework, that generalizes across architectures. Here, we first introduce an attention-based neural gadget that leverages the graph structure of architectures and acts as a surrogate to trained deep neural networks for quick model prediction. Then, we use these predictions to build subset samplers. This naturally provides us two variants of $\texttt{SubSelNet}$. The first variant is transductive (called as Transductive-$\texttt{SubSelNet}$) which computes the subset separately for each model by solving a small optimization problem. Such an optimization is still super fast, thanks to the replacement of explicit model training by the model approximator. The second variant is inductive (called as Inductive-$\texttt{SubSelNet}$) which computes the subset using a trained subset selector, without any optimization. Our experiments show that our model outperforms several methods across several real datasets<|reference_end|>
@article{jain2024efficient,
    title={Efficient Data Subset Selection to Generalize Training Across Models:
  Transductive and Inductive Networks},
    author={Eeshaan Jain, Tushar Nandy, Gaurav Aggarwal, Ashish Tendulkar, Rishabh
  Iyer, Abir De},
    journal={Advances in Neural Information Processing Systems, 36 (2024)},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.12255},
    primaryClass={cs.LG stat.ML}
}