3.4 Response Generation

After all task executions are completed, HuggingGPT enters the response generation stage. In this
stage, HuggingGPT integrates all the information from the previous three stages (task planning,
model selection, and task execution) into a concise summary, including the list of planned tasks, the
selected models for the tasks, and the inference results of the models.
Most important among them are the inference results, which are the supports for HuggingGPT to
make the ﬁnal decisions. These inference results appear in a structured format, such as bounding
boxes with detection probabilities in the object detection model, answer distributions in the question-
answering model, etc. HuggingGPT allows LLM to receive these structured inference results as
input and generate responses in the form of friendly human language. Moreover, instead of simply

8
HuggingGPT- Solving AI Tasks with ChatGPT and its Friends in Hugging Face