Gutenberg (PG-19)
OpenSubtitles
Wikipedia (en)
DM Mathematics
Ubuntu IRC
BookCorpus2
EuroParl

HackerNews
YoutubeSubtitles
PhilPapers
NIH ExPorter
Enron Emails

Topic #1
Generic
Cells
Unknown
US Politics
Data
Unknown
Appeals
Software
Data
Organ Trans-
plant
Unknown
Unknown
Education

Calculation
Bugs
Unknown
International
Politics
Generic
Unknown
Logic
Cells
Email

Dis-

Topic #9
Education
Cells
Unknown
Drugs
Dynamics
Unknown
Legal
Programming
Imaging
Human
ease
Unknown
Unknown
Sports
Calculation
Software
Unknown
International
Politics
Generic
Unknown
Epistemology
Cells
Energy

Topic #2
Politics
Cells
Unknown
Law
Math
Programming
Appeals
Unknown
Electronics
Nervous
System
Unknown
Unknown
International
Politics
Probability
Pull Requests
Unknown
International
Politics
Software
Unknown
Science
Disease
Email

Topic #10
Politics
Cells
Unknown
Sports
Math
HTML/CSS
Legal
HTML/CSS
Electronics
Research
The Pile- An 800GB Dataset of Diverse Text for Language Modeling