LM4OPT: Unveiling the Potential of Large Language Models in Formulating Mathematical Optimization Problems
<|reference_start|>LM4OPT: Unveiling the Potential of Large Language Models in Formulating Mathematical Optimization Problems: In the rapidly evolving field of natural language processing, the translation of linguistic descriptions into mathematical formulation of optimization problems presents a formidable challenge, demanding intricate understanding and processing capabilities from Large Language Models (LLMs). This study compares prominent LLMs, including GPT-3.5, GPT-4, and Llama-2-7b, in zero-shot and one-shot settings for this task. Our findings show GPT-4's superior performance, particularly in the one-shot scenario. A central part of this research is the introduction of `LM4OPT,' a progressive fine-tuning framework for Llama-2-7b that utilizes noisy embeddings and specialized datasets. However, this research highlights a notable gap in the contextual understanding capabilities of smaller models such as Llama-2-7b compared to larger counterparts, especially in processing lengthy and complex input contexts. Our empirical investigation, utilizing the NL4Opt dataset, unveils that GPT-4 surpasses the baseline performance established by previous research, achieving an F1-score of 0.63, solely based on the problem description in natural language, and without relying on any additional named entity information. GPT-3.5 follows closely, both outperforming the fine-tuned Llama-2-7b. These findings not only benchmark the current capabilities of LLMs in a novel application area but also lay the groundwork for future improvements in mathematical formulation of optimization problems from natural language input.<|reference_end|>
@article{ahmed2024lm4opt:,
    title={LM4OPT: Unveiling the Potential of Large Language Models in Formulating
  Mathematical Optimization Problems},
    author={Tasnim Ahmed, Salimur Choudhury},
    journal={arXiv preprint arXiv:2403.01342},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.01342},
    primaryClass={cs.CL cs.IR}
}