LICO: Large Language Models for In-Context Molecular Optimization
<|reference_start|>LICO: Large Language Models for In-Context Molecular Optimization: Optimizing black-box functions is a fundamental problem in science and engineering. To solve this problem, many approaches learn a surrogate function that estimates the underlying objective from limited historical evaluations. Large Language Models (LLMs), with their strong pattern-matching capabilities via pretraining on vast amounts of data, stand out as a potential candidate for surrogate modeling. However, directly prompting a pretrained language model to produce predictions is not feasible in many scientific domains due to the scarcity of domain-specific data in the pretraining corpora and the challenges of articulating complex problems in natural language. In this work, we introduce LICO, a general-purpose model that extends arbitrary base LLMs for black-box optimization, with a particular application to the molecular domain. To achieve this, we equip the language model with a separate embedding layer and prediction layer, and train the model to perform in-context predictions on a diverse set of functions defined over the domain. Once trained, LICO can generalize to unseen molecule properties simply via in-context prompting. LICO achieves state-of-the-art performance on PMO, a challenging molecular optimization benchmark comprising over 20 objective functions.<|reference_end|>
@article{nguyen2024lico:,
    title={LICO: Large Language Models for In-Context Molecular Optimization},
    author={Tung Nguyen and Aditya Grover},
    journal={arXiv preprint arXiv:2406.18851},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.18851},
    primaryClass={cs.LG cs.AI physics.chem-ph q-bio.BM q-bio.QM}
}