challenges on the way to building an advanced AI system. We discuss them from these aspects: 1)
Limited to the input and output forms of text generation, current LLMs lack the ability to process
complex information such as vision and speech, regardless of their signiﬁcant achievements in NLP
tasks; 2) In real-world scenarios, some complex tasks are usually composed of multiple sub-tasks, and
thus require the scheduling and cooperation of multiple models, which are also beyond the capability
of language models; 3) For some challenging tasks, LLMs demonstrate excellent results in zero-shot
∗ The ﬁrst two authors have equal contributions. This work was done when the ﬁrst author was an intern at
HuggingGPT- Solving AI Tasks with ChatGPT and its Friends in Hugging Face