
Deep Ignorance
This collection contains the model and data artifacts from O'Brien et al. (2025). Code: github.com/EleutherAI/deep-ignorance
7B • Updated • 94Note Fully Trained — Unfiltered Baseline Model - Pretraining Filtering: None - Annealing Filtering: None - Results Location: Main Paper
EleutherAI/deep-ignorance-e2e-strong-filter
Text Generation • 7B • Updated • 21Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Results Location: Main Paper (Strong Filter)
EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal
Text Generation • 7B • Updated • 22Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Results Location: Main Paper (Weak Filter)
EleutherAI/deep-ignorance-e2e-weak-filter
Text Generation • 7B • Updated • 26Note Fully Trained - Pretraining Filtering: Weak Filter - Annealing Filtering: Weak Filter - Results Location: Appendix
EleutherAI/deep-ignorance-weak-filter-pt-strong-filter-anneal
Text Generation • 7B • Updated • 20Note Fully Trained - Pretraining Filtering: Weak Filter - Annealing Filtering: Strong Filter
EleutherAI/deep-ignorance-pretraining-stage-unfiltered
Text Generation • 7B • Updated • 368Note Pretrained model that has not undergone annealing or any data filtering. - Pretraining Filtering: None - Results Location: Not Included
EleutherAI/deep-ignorance-pretraining-stage-strong-filter
Text Generation • 7B • Updated • 368Note Pretrained model that has not undergone annealing. - Pretraining Filtering: Strong Filter - Results Location: Not Included
EleutherAI/deep-ignorance-pretraining-stage-weak-filter
Text Generation • 7B • Updated • 127Note Pretrained model which has not undergone annealing. - Pretraining Filtering: Weak Filter - Results Location: Not Included
EleutherAI/deep-ignorance-pretraining-stage-extra-weak-filter
Updated • 78Note Pretrained model that has not undergone annealing. - Pretraining Filtering: Extra Weak Filter - Results Location: Not Included
EleutherAI/deep-ignorance-e2e-strong-filter-cb-lat
Text Generation • 7B • Updated • 19Note Fully Trained with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (Strong Filter + CB + LAT)
EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal-cb-lat
Text Generation • 7B • Updated • 17Note Fully Trained with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (Weak Filter + CB + LAT)
EleutherAI/deep-ignorance-unfiltered-cb
Text Generation • 7B • Updated • 13Note Fully Trained — Unfiltered Baseline Model with Circuit Breaking - Pretraining Filtering: None - Annealing Filtering: None - Post-training: Circuit Breaking - Results Location: Main Paper (CB)
EleutherAI/deep-ignorance-unfiltered-cb-lat
Text Generation • 7B • Updated • 15Note Fully Trained — Unfiltered Baseline Model with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: None - Annealing Filtering: None - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (CB + LAT)
EleutherAI/deep-ignorance-e2e-strong-filter-cb
Text Generation • 7B • Updated • 16Note Fully Trained with Circuit Breaking - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Circuit Breaking - Results Location: Main Paper (Strong Filter + CB)
EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal-cb
Text Generation • 7B • Updated • 16Note Fully Trained with Circuit Breaking - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Post-training: Circuit Breaking - Results Location: Main Paper (Weak Filter + CB)
EleutherAI/deep-ignorance-e2e-extra-weak-filter
Text Generation • 7B • Updated • 115Note Fully Trained - Pretraining Filtering: Extra Weak Filter - Annealing Filtering: Extra Weak Filter - Results Location: Not Included
EleutherAI/deep-ignorance-e2e-strong-filter-weak-knowledge-corrupted
Text Generation • 7B • Updated • 43Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Weak Knowledge Corruption via Synthetic Document Fine-Tuning - Results Location: Main Paper & Appendix
EleutherAI/deep-ignorance-e2e-strong-filter-strong-knowledge-corrupted
Text Generation • 7B • Updated • 52Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Strong Knowledge Corruption via Synthetic Document Fine-Tuning - Results Location: Main Paper & Appendix
EleutherAI/wmdp_bio_cloze
Viewer • Updated • 1.27k • 757Note All prompts from WMDP-Bio that can be evaluated using a cloze-style prompt.
EleutherAI/wmdp_bio_robust_mcqa
Viewer • Updated • 1.27k • 212Note WMDP-Bio, where data is broken down by topic category and whether it contains likely shortcuts.
EleutherAI/mmlu_test_task_training_mix
Viewer • Updated • 200k • 36Note General knowledge multiple-choice and cloze-style prompts that are used to ensure that models are familiar with the MCQA test benchmarks, like WMDP and MMLU.
EleutherAI/deep-ignorance-annealing-mix
Viewer • Updated • 89M • 520Note The original annealing dataset for training the LLMs. This dataset is not filtered.
EleutherAI/deep-ignorance-pretraining-mix
Viewer • Updated • 410M • 742 • 2Note The original pretraining dataset for training the LLMs. This dataset is not filtered.