In this section, we present MLCopilot, with the formulation of the main problem and the overall
architecture of our method. Then, we will describe some key components of MLCopilot in detail,
including task description, retrieval, canonicalization, and knowledge elicitation.

4 MLCopilot

4.1 Overall Framework

MLCopilot contains two stages, including ofﬂine and online parts, both of which have been visually
illustrated in Figure 1. In the ofﬂine stage, LLM has been incorporated to analyze the canonicalized
historical experience data and elicit useful knowledge. And in the online stage, the user will query
MLCopilot, which is also built upon LLM, to obtain a suitable ML solution for the novel task.

3

Figure 1: Overview of MLCopilot.

4.1.1 Ofﬂine Stage: Understanding and Reasoning
MLCopilot- Unleashing the Power of Large Language Models in Solving Machine Learning Tasks