BabyLM-community
/

babylm-baseline-10m-gpt-bert-masked-focus

babylm-baseline

Model card Files Files and versions Community

lgcharpe commited on 11 days ago

Commit

4c500ea

·

verified ·

1 Parent(s): 918e2d3

Uploading patch

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -17,5 +17,5 @@
     "num_attention_heads": 6,
     "num_hidden_layers": 12,
     "vocab_size": 16384,
-    "layer_norm_eps": 1.0e-5
   }

     "num_attention_heads": 6,
     "num_hidden_layers": 12,
     "vocab_size": 16384,
+    "layer_norm_eps": 1.0e-7
   }