Chain-of-Thought Augmentation with Logit Contrast for Enhanced Reasoning in Language Models
<|reference_start|>Chain-of-Thought Augmentation with Logit Contrast for Enhanced Reasoning in Language Models: Rapidly increasing model scales coupled with steering methods such as chain-of-thought prompting have led to drastic improvements in language model reasoning. At the same time, models struggle with compositional generalization and are far from human performance on many reasoning-based benchmarks. Leveraging the success of chain-of-thought prompting, and also taking inspiration from context-aware decoding (CAD), we explore input-based contrasting methods to further encourage the type of reasoning induced by chain-of-thought prompting. While work remains to stabilize these results across datasets and models, the improvements we find warrant further investigation into input-based steering methods for context-aware reasoning.<|reference_end|>
@article{shim2024chain-of-thought,
    title={Chain-of-Thought Augmentation with Logit Contrast for Enhanced Reasoning
  in Language Models},
    author={Jay Shim, Grant Kruttschnitt, Alyssa Ma, Daniel Kim, Benjamin Chek,
  Athul Anand, Kevin Zhu, Sean O'Brien},
    journal={arXiv preprint arXiv:2407.03600},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.03600},
    primaryClass={cs.CL}
}