Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for
  Political Text
Social scientists quickly adopted large language models due to their ability
to annotate documents without supervised training, an ability known as
zero-shot learning. However, due to their compute demands, cost, and often
proprietary nature, these models are often at odds with replication and open
science standards. This paper introduces the Political DEBATE (DeBERTa
Algorithm for Textual Entailment) language models for zero-shot and few-shot
classification of political documents. These models are not only as good, or
better than, state-of-the art large language models at zero and few-shot
classification, but are orders of magnitude more efficient and completely open
source. By training the models on a simple random sample of 10-25 documents,
they can outperform supervised classifiers trained on hundreds or thousands of
documents and state-of-the-art generative models with complex, engineered
prompts. Additionally, we release the PolNLI dataset used to train these models
-- a corpus of over 200,000 political documents with highly accurate labels
across over 800 classification tasks.
66d969ff36aa505569569194
Michael Burnham
66d969ff36aa505569569195
Kayla Kahn
admin_assigned
2024-09-05T08:55:42.160Z
66902066a4d0df7e518d9bf2
/avatars/d4973a2693b146e756847d081709b792.svg
Kayla Kahn
user
kaylakahn
66d969ff36aa505569569196
Ryan Yank Wang
66d969ff36aa505569569197
Rachel X. Peng
Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for
  Political Text
Social scientists quickly adopted large language models due to their ability
to annotate documents without supervised training, an ability known as
zero-shot learning. However, due to their compute demands, cost, and often
proprietary nature, these models are often at odds with replication and open
science standards. This paper introduces the Political DEBATE (DeBERTa
Algorithm for Textual Entailment) language models for zero-shot and few-shot
classification of political documents. These models are not only as good, or
better than, state-of-the art large language models at zero and few-shot
classification, but are orders of magnitude more efficient and completely open
source. By training the models on a simple random sample of 10-25 documents,
they can outperform supervised classifiers trained on hundreds or thousands of
documents and state-of-the-art generative models with complex, engineered
prompts. Additionally, we release the PolNLI dataset used to train these models
-- a corpus of over 200,000 political documents with highly accurate labels
across over 800 classification tasks.
Political DEBATE models achieve high efficiency and accuracy in zero-shot and few-shot political document classification using minimal training data and are openly sourced.
zero-shot learning few-shot learning Political DEBATE DeBERTa textual entailment PolNLI dataset