title: Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
summary: Classifier-Free Guidance (CFG) significantly enhances controllability in
  generative models by interpolating conditional and unconditional predictions.
  However, standard CFG often employs a static unconditional input, which can be
  suboptimal for iterative generation processes where model uncertainty varies
  dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel
  method that tailors the unconditional input by leveraging the model's
  instantaneous predictive confidence. At each step of an iterative (masked)
  diffusion language model, A-CFG identifies tokens in the currently generated
  sequence for which the model exhibits low confidence. These tokens are
  temporarily re-masked to create a dynamic, localized unconditional input. This
  focuses CFG's corrective influence precisely on areas of ambiguity, leading to
  more effective guidance. We integrate A-CFG into a state-of-the-art masked
  diffusion language model and demonstrate its efficacy. Experiments on diverse
  language generation benchmarks show that A-CFG yields substantial improvements
  over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work
  highlights the benefit of dynamically adapting guidance mechanisms to model
  uncertainty in iterative generation.
paper_authors._id: 683919611186f2cbf3ed2267
paper_authors.name: Pengxiang Li
paper_authors._id: 683919611186f2cbf3ed2268
paper_authors.name: Shilin Yan
paper_authors._id: 683919611186f2cbf3ed2269
paper_authors.name: Joey Tsai
paper_authors._id: 683919611186f2cbf3ed226a
paper_authors.name: Renrui Zhang
paper_authors._id: 683919611186f2cbf3ed226b
paper_authors.name: Ruichuan An
paper_authors._id: 683919611186f2cbf3ed226c
paper_authors.name: Ziyu Guo
paper_authors._id: 683919611186f2cbf3ed226d
paper_authors.name: Xiaowei Gao
paper_title: Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
paper_summary: Classifier-Free Guidance (CFG) significantly enhances controllability in
  generative models by interpolating conditional and unconditional predictions.
  However, standard CFG often employs a static unconditional input, which can be
  suboptimal for iterative generation processes where model uncertainty varies
  dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel
  method that tailors the unconditional input by leveraging the model's
  instantaneous predictive confidence. At each step of an iterative (masked)
  diffusion language model, A-CFG identifies tokens in the currently generated
  sequence for which the model exhibits low confidence. These tokens are
  temporarily re-masked to create a dynamic, localized unconditional input. This
  focuses CFG's corrective influence precisely on areas of ambiguity, leading to
  more effective guidance. We integrate A-CFG into a state-of-the-art masked
  diffusion language model and demonstrate its efficacy. Experiments on diverse
  language generation benchmarks show that A-CFG yields substantial improvements
  over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work
  highlights the benefit of dynamically adapting guidance mechanisms to model
  uncertainty in iterative generation.
paper_ai_summary: Adaptive Classifier-Free Guidance (A-CFG) dynamically adjusts the guidance in masked diffusion language models by focusing on areas of low model confidence, leading to significant improvements in language generation performance.
paper_ai_keywords: Classifier-Free Guidance (CFG) Adaptive Classifier-Free Guidance (A-CFG) masked diffusion language model predictive confidence GPQA