Towards Automated Machine Learning Research
<|reference_start|>Towards Automated Machine Learning Research: This paper explores a top-down approach to automating incremental advances in machine learning research through component-level innovation, facilitated by Large Language Models (LLMs). Our framework systematically generates novel components, validates their feasibility, and evaluates their performance against existing baselines. A key distinction of this approach lies in how these novel components are generated. Unlike traditional AutoML and NAS methods, which often rely on a bottom-up combinatorial search over predefined, hardcoded base components, our method leverages the cross-domain knowledge embedded in LLMs to propose new components that may not be confined to any hard-coded predefined set. By incorporating a reward model to prioritize promising hypotheses, we aim to improve the efficiency of the hypothesis generation and evaluation process. We hope this approach offers a new avenue for exploration and contributes to the ongoing dialogue in the field.<|reference_end|>
@article{ardeshir2024towards,
    title={Towards Automated Machine Learning Research},
    author={Shervin Ardeshir},
    journal={arXiv preprint arXiv:2409.05258},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.05258},
    primaryClass={cs.LG cs.AI cs.CV}
}