SimpleStrat: Diversifying Language Model Generation with Stratification
Generating diverse responses from large language models (LLMs) is crucial for
applications such as planning/search and synthetic data generation, where
diversity provides distinct answers across generations. Prior approaches rely
on increasing temperature to increase diversity. However, contrary to popular
belief, we show not only does this approach produce lower quality individual
generations as temperature increases, but it depends on model's next-token
probabilities being similar to the true distribution of answers. We propose
, an alternative approach that uses the language model itself to
partition the space into strata. At inference, a random stratum is selected and
a sample drawn from within the strata. To measure diversity, we introduce
CoverageQA, a dataset of underspecified questions with multiple equally
plausible answers, and assess diversity by measuring KL Divergence between the
output distribution and uniform distribution over valid ground truth answers.
As computing probability per response/solution for proprietary models is
infeasible, we measure recall on ground truth solutions. Our evaluation show
using SimpleStrat achieves higher recall by 0.05 compared to GPT-4o and 0.36
average reduction in KL Divergence compared to Llama 3.
670d5d3db0c427993f8748b8
Justin Wong
670d5d3db0c427993f8748b9
Yury Orlovskiy
670d5d3db0c427993f8748ba
Michael Luo
claimed_verified
2025-02-24T09:27:19.292Z
654037be97949fd2304aab7f
https://cdn-avatars.huggingface.co/v1/production/uploads/654037be97949fd2304aab7f/2cSME81gcwYa2OTeVlq5Q.jpeg
Michael Luo
user
michaelzhiluo
670d5d3db0c427993f8748bb
Sanjit A. Seshia
670d5d3db0c427993f8748bc
Joseph E. Gonzalez
SimpleStrat: Diversifying Language Model Generation with Stratification
Generating diverse responses from large language models (LLMs) is crucial for
applications such as planning/search and synthetic data generation, where
diversity provides distinct answers across generations. Prior approaches rely
on increasing temperature to increase diversity. However, contrary to popular
belief, we show not only does this approach produce lower quality individual
generations as temperature increases, but it depends on model's next-token
probabilities being similar to the true distribution of answers. We propose
, an alternative approach that uses the language model itself to
partition the space into strata. At inference, a random stratum is selected and
a sample drawn from within the strata. To measure diversity, we introduce
CoverageQA, a dataset of underspecified questions with multiple equally
plausible answers, and assess diversity by measuring KL Divergence between the
output distribution and uniform distribution over valid ground truth answers.
As computing probability per response/solution for proprietary models is
infeasible, we measure recall on ground truth solutions. Our evaluation show
using SimpleStrat achieves higher recall by 0.05 compared to GPT-4o and 0.36
average reduction in KL Divergence compared to Llama 3.
A method using language models to partition the response space increases diversity and quality by selecting random strata, outperforming prior methods based on temperature adjustment and existing models.
language models strata inference CoverageQA KL Divergence recall SimpleStrat GPT-4o Llama 3