With the canonicalized experience stored in pool PE, MLCopilot can then elicit knowledge, to better
support the online stage for solving novel ML tasks. It is important to note that knowledge elicitation
occurs ofﬂine, prior to serving user tasks. The approach involves constructing a prompt that consists
of a subset of the experience pool, along with an inquiry that asks for analysis and summary. This
prompt is then sent to the LLM to generate a knowledge “candidate”. The candidate is subsequently
validated using the experience pool. The ﬂow of this process is illustrated in the right part of Figure 2.
To prevent knowledge bias towards speciﬁc tasks, we adopt a random sampling approach to select
several experiences from the experience pool PE, without replacement, until the prompt is full.
However, it is worth noting that state-of-the-art LLMs can only handle about 10 entries of experiences,
MLCopilot- Unleashing the Power of Large Language Models in Solving Machine Learning Tasks