Pythia 160M, Mamba 130M, and RWKV 169M models trained on OpenWebText for 4000 steps (context window: 1024; effective batch size: 512). 6 seeds each.
James Michaelov
jmichaelov
AI & ML interests
None yet
Recent Activity
updated
a model
3 days ago
jmichaelov/pythia-160m-512k-10mtok-mr20-decay80
updated
a model
3 days ago
jmichaelov/pythia-160m-512k-10mtok-mr90-decay90
updated
a model
3 days ago
jmichaelov/pythia-160m-512k-10mtok-mr90-decay90
Organizations
models
43
jmichaelov/pythia-160m-512k-10mtok-mr95-decay90
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay10
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay20
Updated
jmichaelov/pythia-160m-512k-10mtok-mr0-decay100
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay80
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay40
Updated
jmichaelov/pythia-160m-512k-10mtok-mr20-decay10
Updated
jmichaelov/pythia-160m-512k-10mtok-mr40-decay40
Updated
jmichaelov/pythia-160m-512k-10mtok-mr40-decay20
Updated
jmichaelov/pythia-160m-512k-10mtok-mr80-decay10
Updated
datasets
10
jmichaelov/inverse_scaling_prize-hindsight_neglect
Viewer
•
Updated
•
315
•
38
jmichaelov/inverse_scaling_prize-memo_trap
Viewer
•
Updated
•
936
•
21
jmichaelov/inverse_scaling_prize-neqa
Viewer
•
Updated
•
300
•
14
jmichaelov/inverse_scaling_prize-redefine
Viewer
•
Updated
•
1.24k
•
17
jmichaelov/inverse_scaling_prize-into_the_unknown
Viewer
•
Updated
•
1.82k
•
13
jmichaelov/inverse_scaling_prize-modus_tollens
Viewer
•
Updated
•
1.24k
•
16
jmichaelov/inverse_scaling_prize-pattern_matching_suppression
Viewer
•
Updated
•
1.43k
•
14
jmichaelov/inverse_scaling_prize-repetitive_algebra
Viewer
•
Updated
•
1k
•
18
jmichaelov/inverse_scaling_prize-resisting_correction
Viewer
•
Updated
•
7.34k
•
14
jmichaelov/inverse_scaling_prize-sig_figs
Viewer
•
Updated
•
20.9k
•
30