AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
Recent Activity
View all activity
-
open-sci/open-sci-ref-v0.01-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 4
Research baseline models trained on various open reference datasets
Open-sci-ref: reference baselines releases
-
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000-lr0.006-2
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000
0.4B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000-lr0.004-2
0.4B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000
0.1B • Updated • 6
-
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-lr0.004-2
2B • Updated • 1 -
open-sci/open-sci-ref-v0.01-0.13b-nemotron-hq-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-nemotron-hq-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-nemotron-hq-300B-4096
1B • Updated • 4
openMammut models trained on various datasets (Re-LAION, DataComp, DFN)
Materials related to OpenThoughts and OpenThinker releases
-
open-sci/open-sci-ref-v0.01-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 4
-
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000-lr0.006-2
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000
0.4B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000-lr0.004-2
0.4B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000
0.1B • Updated • 6
-
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-lr0.004-2
2B • Updated • 1 -
open-sci/open-sci-ref-v0.01-0.13b-nemotron-hq-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-nemotron-hq-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-nemotron-hq-300B-4096
1B • Updated • 4
Research baseline models trained on various open reference datasets
openMammut models trained on various datasets (Re-LAION, DataComp, DFN)
Open-sci-ref: reference baselines releases
Materials related to OpenThoughts and OpenThinker releases