Announcing the Common Pile and Comma v0.1
•
15
Common Pile
Enterprise
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
Organization Card
The Common Pile
We are a group of researchers working together to collect and curate openly licensed and public domain data for training large language models. So far, we have released:
- The Common Pile v0.1, an 8 TB dataset of text from over 30 diverse sources
- Our paper: The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
- Comma v0.1-1T and Comma v0.1-2T, 7B parameter LLMs trained on text from the Common Pile v0.1
- The training dataset used to train the Comma v0.1 models
- Our code for collecting data from each source
If you're interested in contributing, please open an issue on GitHub!
datasets
62
common-pile/comma_v0.1_training_dataset
Viewer
•
Updated
•
784M
•
12.8k
•
31
common-pile/youtube_filtered
Viewer
•
Updated
•
986k
•
467
•
3
common-pile/youtube
Viewer
•
Updated
•
1.13M
•
465
•
7
common-pile/wikiteam_filtered
Viewer
•
Updated
•
10.2M
•
338
common-pile/wikiteam
Viewer
•
Updated
•
552M
•
885
•
2
common-pile/wikimedia_filtered
Viewer
•
Updated
•
12.9M
•
812
•
3
common-pile/wikimedia
Viewer
•
Updated
•
78.1M
•
577
•
4
common-pile/uspto_filtered
Viewer
•
Updated
•
14.4M
•
1.67k
•
1
common-pile/uspto
Viewer
•
Updated
•
16.2M
•
1.17k
common-pile/usgpo_filtered
Viewer
•
Updated
•
2.34M
•
312