convaiinnovations
/

hindi-foundational-model-base

Text Generation

hindi_causal_lm

Model card Files Files and versions Community

convaiinnovations commited on 29 days ago

Commit

c88b467

·

verified ·

1 Parent(s): 70a712a

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -12,7 +12,7 @@
   "bos_token_id": 1,
   "eos_token_id": 2,
   "tie_word_embeddings": true,
-  "activation_function": "swiglu",
   "normalization_layer": "rmsnorm",
   "positional_encoding_type": "rope",
   "unk_token_id": 3,

   "bos_token_id": 1,
   "eos_token_id": 2,
   "tie_word_embeddings": true,
+  "hidden_act": "silu",
   "normalization_layer": "rmsnorm",
   "positional_encoding_type": "rope",
   "unk_token_id": 3,