Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond

5

that are optimized for specific tasks such as code generation, BloombergGPT [117] for the financial domain. The recent
breakthrough is ChatGPT, which refines GPT-3 specifically for conversational tasks, resulting in more interactive,
coherent, and context-aware conversational for various real-world applications.

3 PRACTICAL GUIDE FOR DATA
In this section, we’ll be discussing the critical role that data plays in selecting appropriate models for downstream
tasks. The impact of data on the models’ effectiveness starts during the pre-training stage and continues through to the
training and inference stages.

Remark 1
(1) LLMs generalize better than fine-tuned models in downstream tasks facing out-of-distribution data, such as

adversarial examples and domain shifts.

(2) LLMs are preferable to fine-tuned models when working with limited annotated data, and both can be
Harnessing the Power of LLMs in Practice- A Survey on ChatGPT and Beyond