Llm, Knowledge Distillation, Quantization, Online Advertising, AI.
“teacher”). This is achieved by using larger parameter LLMs to generate training data for a smaller student LLM to fine-tune on. To control the quality of machine generated training data, we could consider model confidence among other domain constraints to selectively sample training data generated