HuggingGPT inevitably suffers from some limitations. One of the limitations we are most concerned
about is efﬁciency. The bottleneck of efﬁciency lies in the inference of the large language model. For
each round of user requests, HuggingGPT requires at least one interaction with the large language
model during the task planning, model selection, and response generation stages. These interactions
greatly increase the response latency and lead to a degradation of user experience. The second is
the limitation of the maximum context length. Limited by the maximum number of tokens that the
LLM can accept, HuggingGPT also faces a limitation on the maximum context length. We have
used the conversation window and only tracked the conversation context in the task planning stage to
alleviate it. The third is system stability, which includes two aspects. One is the rebellion that occurs
during the inference of large language models. Large language models occasionally fail to conform
HuggingGPT- Solving AI Tasks with ChatGPT and its Friends in Hugging Face