Upload folder using huggingface_hub

README.md

@@ -0,0 +1,167 @@
+---
+library_name: transformers
+tags:
+- falcon-h1
+- unsloth
+license: other
+license_name: falcon-llm-license
+license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
+base_model:
+- tiiuae/Falcon-H1-3B-Instruct
+inference: true
+---
+> [!NOTE]
+>  Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
+>
+
+<div>
+<p style="margin-top: 0;margin-bottom: 0;">
+    <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
+  </p>
+  <div style="display: flex; gap: 5px; align-items: center; ">
+    <a href="https://github.com/unslothai/unsloth/">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
+    </a>
+    <a href="https://discord.gg/unsloth">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
+    </a>
+    <a href="https://docs.unsloth.ai/">
+      <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
+    </a>
+  </div>
+</div>
+
+
+<img src="https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/falcon_mamba/falcon-h1-logo.png" alt="drawing" width="800"/>
+
+#  Table of Contents
+
+0. [TL;DR](#TL;DR)
+1. [Model Details](#model-details)
+2. [Training Details](#training-details)
+3. [Usage](#usage)
+4. [Evaluation](#evaluation)
+5. [Citation](#citation)
+
+# TL;DR
+
+# Model Details
+
+## Model Description
+
+- **Developed by:** [https://www.tii.ae](https://www.tii.ae)
+- **Model type:** Causal decoder-only
+- **Architecture:** Hybrid Transformers + Mamba architecture
+- **Language(s) (NLP):** English, Multilingual
+- **License:** Falcon-LLM License
+
+# Training details
+
+For more details about the training protocol of this model, please refer to the [Falcon-H1 technical blogpost](https://falcon-lm.github.io/blog/falcon-h1/).
+
+# Usage
+
+Currently to use this model you can either rely on Hugging Face `transformers`, `vLLM` or `llama.cpp` library.
+
+## Inference
+
+Make sure to install the latest version of `transformers` or `vllm`, eventually install these packages from source:
+
+```bash
+pip install git+https://github.com/huggingface/transformers.git
+```
+
+For vLLM, make sure to install `vllm>=0.9.0`:
+
+```bash
+pip install "vllm>=0.9.0"
+```
+
+### 🤗 transformers
+
+Refer to the snippet below to run H1 models using 🤗 transformers:
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "tiiuae/Falcon-H1-1B-Base"
+
+model = AutoModelForCausalLM.from_pretrained(
+  model_id,
+  torch_dtype=torch.bfloat16,
+  device_map="auto"
+)
+
+# Perform text generation
+```
+
+### vLLM
+
+For vLLM, simply start a server by executing the command below:
+
+```
+# pip install vllm
+vllm serve tiiuae/Falcon-H1-1B-Instruct --tensor-parallel-size 2 --data-parallel-size 1
+```
+
+### `llama.cpp`
+
+You can find all GGUF files under [our official collection](https://huggingface.co/collections/tiiuae/falcon-h1-6819f2795bc406da60fab8df)
+
+# Evaluation
+
+Falcon-H1 series perform very well on a variety of tasks, including reasoning tasks. 
+
+| Tasks | Falcon-H1-3B | Qwen3-4B | Qwen2.5-3B | Gemma3-4B | Llama3.2-3B | Falcon3-3B |
+| --- | --- | --- | --- | --- | --- | --- |
+| **General**  | | | | | |
+| BBH | **53.69** | 51.07 | 46.55 | 50.01 | 41.47 | 45.02 |
+| ARC-C | **49.57** | 37.71 | 43.77 | 44.88 | 44.88 | 48.21 |
+| TruthfulQA | 53.19 | 51.75 | **58.11** | 51.68 | 50.27 | 50.06 |
+| HellaSwag | **69.85** | 55.31 | 64.21 | 47.68 | 63.74 | 64.24 |
+| MMLU | **68.3** | 67.01 | 65.09 | 59.53 | 61.74 | 56.76 |
+| **Math**  | | | | | |
+| GSM8k | **84.76** | 80.44 | 57.54 | 77.41 | 77.26 | 74.68 |
+| MATH-500 | 74.2 | **85.0** | 64.2 | 76.4 | 41.2 | 54.2 |
+| AMC-23 | 55.63 | **66.88** | 39.84 | 48.12 | 22.66 | 29.69 |
+| AIME-24 | 11.88 | **22.29** | 6.25 | 6.67 | 11.67 | 3.96 |
+| AIME-25 | 13.33 | **18.96** | 3.96 | 13.33 | 0.21 | 2.29 |
+| **Science**  | | | | | |
+| GPQA | **33.89** | 28.02 | 28.69 | 29.19 | 28.94 | 28.69 |
+| GPQA_Diamond | 38.72 | **40.74** | 35.69 | 28.62 | 29.97 | 29.29 |
+| MMLU-Pro | **43.69** | 29.75 | 32.76 | 29.71 | 27.44 | 29.71 |
+| MMLU-stem | **69.93** | 67.46 | 59.78 | 52.17 | 51.92 | 56.11 |
+| **Code**  | | | | | |
+| HumanEval | 76.83 | **84.15** | 73.78 | 67.07 | 54.27 | 52.44 |
+| HumanEval+ | 70.73 | **76.83** | 68.29 | 61.59 | 50.0 | 45.73 |
+| MBPP | **79.63** | 68.78 | 72.75 | 77.78 | 62.17 | 61.9 |
+| MBPP+ | **67.46** | 59.79 | 60.85 | 66.93 | 50.53 | 55.29 |
+| LiveCodeBench | 26.81 | **39.92** | 11.74 | 21.14 | 2.74 | 3.13 |
+| CRUXEval | 56.25 | **69.63** | 43.26 | 52.13 | 17.75 | 44.38 |
+| **Instruction Following**  | | | | | |
+| IFEval | **85.05** | 84.01 | 64.26 | 77.01 | 74.0 | 69.1 |
+| Alpaca-Eval | 31.09 | 36.51 | 17.37 | **39.64** | 19.69 | 14.82 |
+| MTBench | **8.72** | 8.45 | 7.79 | 8.24 | 7.96 | 7.79 |
+| LiveBench | 36.86 | **51.34** | 27.32 | 36.7 | 26.37 | 26.01 |
+
+You can check more in detail on our [our release blogpost](https://falcon-lm.github.io/blog/falcon-h1/), detailed benchmarks.
+
+# Useful links
+
+- View [our release blogpost](https://falcon-lm.github.io/blog/falcon-h1/).
+- Feel free to join [our discord server](https://discord.gg/trwMYP9PYm) if you have any questions or to interact with our researchers and developers.
+
+# Citation
+
+If the Falcon-H1 family of models were helpful to your work, feel free to give us a cite.
+
+```
+@misc{tiifalconh1,
+    title = {Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance},
+    url = {https://falcon-lm.github.io/blog/falcon-h1},
+    author = {Falcon-LLM Team},
+    month = {May},
+    year = {2025}
+}
+```

chat_template.jinja

@@ -0,0 +1,17 @@
+{{bos_token}}
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "You are a function calling AI model. You are provided with function signature within <tools> </tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.\n<tools>\n" }}
+    {%- for tool in tools %}[{{- tool | tojson }}]{%- endfor %}
+    {{- "\n</tools>\nFor each function call, return a json object with function name and arguments within <tool_call> </tool_call> tags with the following schema:\n<tool_call>\n{'arguments': <args-dict>, 'name': <function-name>}\n</tool_call>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}{% for message in messages %}{%- if message.role != 'system' %}{{'<|im_start|>' + message['role'] + '
+' + message['content'] + '<|im_end|>' + '
+'}}{%- endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
+' }}{% endif %}

config.json

@@ -0,0 +1,65 @@
+{
+  "architectures": [
+    "FalconH1ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attention_in_multiplier": 1.0,
+  "attention_out_multiplier": 0.15,
+  "attn_layer_indices": null,
+  "bos_token_id": 1,
+  "embedding_multiplier": 5.656854249492381,
+  "eos_token_id": 11,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "key_multiplier": 0.044194173824159216,
+  "lm_head_multiplier": 0.015625,
+  "mamba_chunk_size": 128,
+  "mamba_conv_bias": true,
+  "mamba_d_conv": 4,
+  "mamba_d_head": 128,
+  "mamba_d_ssm": 4096,
+  "mamba_d_state": 256,
+  "mamba_expand": 2,
+  "mamba_n_groups": 1,
+  "mamba_n_heads": 32,
+  "mamba_norm_before_gate": false,
+  "mamba_proj_bias": false,
+  "mamba_rms_norm": true,
+  "mamba_use_mlp": true,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "mlp_expansion_factor": 8,
+  "mlp_multipliers": [
+    0.3535533905932738,
+    0.078125
+  ],
+  "model_type": "falcon_h1",
+  "num_attention_heads": 10,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 2,
+  "num_logits_to_keep": 1,
+  "pad_token_id": 65536,
+  "projectors_bias": false,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 100000000000.0,
+  "ssm_in_multiplier": 0.5,
+  "ssm_multipliers": [
+    0.3535533905932738,
+    0.25,
+    0.1767766952966369,
+    0.5,
+    0.3535533905932738
+  ],
+  "ssm_out_multiplier": 0.08838834764831845,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.53.1",
+  "unsloth_fixed": true,
+  "use_cache": true,
+  "vocab_size": 65537
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": [11, 228],
+  "pad_token_id": 0,
+  "transformers_version": "4.52.0.dev0"
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:977e5fada35522cebf1c2a046dcbd52fbe019f5c4ff77c49840d063aaf0cdbd6
+size 4971004584

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca9b659bbd9be3b77b3d148e9933d3f3bcceceff565afd33a61861c56ddf7677
+size 1327841504

model.safetensors.index.json

@@ -0,0 +1,554 @@
+{
+  "metadata": {
+    "total_size": 6298784768
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00002-of-00002.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.final_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.0.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.feed_forward.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.A_log": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.D": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.dt_bias": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.pre_ff_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.feed_forward.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.feed_forward.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.feed_forward.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.A_log": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.D": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.dt_bias": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.pre_ff_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.feed_forward.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.feed_forward.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.feed_forward.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.A_log": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.D": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.dt_bias": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.pre_ff_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.feed_forward.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.feed_forward.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.feed_forward.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.A_log": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.D": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.dt_bias": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.pre_ff_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.3.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.30.feed_forward.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.feed_forward.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.feed_forward.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.A_log": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.D": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.dt_bias": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.pre_ff_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.feed_forward.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.feed_forward.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.feed_forward.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.A_log": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.D": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.conv1d.bias": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.conv1d.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.dt_bias": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.in_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mamba.out_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.pre_ff_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.4.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.feed_forward.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.feed_forward.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.feed_forward.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.A_log": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.D": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.conv1d.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.conv1d.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.dt_bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.in_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mamba.out_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.pre_ff_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,356 @@
+{
+  "additional_special_tokens": [
+    "<|pad|>",
+    ">>ABSTRACT<<",
+    ">>INTRODUCTION<<",
+    ">>SUMMARY<<",
+    ">>COMMENT<<",
+    ">>ANSWER<<",
+    ">>QUESTION<<",
+    ">>DOMAIN<<",
+    ">>PREFIX<<",
+    ">>SUFFIX<<",
+    ">>MIDDLE<<",
+    "<|finetune_right_pad_id|>",
+    "<|start_header_id|>",
+    "<|end_header_id|>",
+    "<|eom_id|>",
+    "<|eot_id|>",
+    "<|begin_of_text|>",
+    ">>TITLE<<",
+    "<tool_response>",
+    "</tool_response>",
+    "<tool_call>",
+    "</tool_call>",
+    "<schema>",
+    "</schema>",
+    "<scratch_pad>",
+    "</scratch_pad>",
+    "<thinking>",
+    "</thinking>",
+    "<explanation>",
+    "</explanation>",
+    "<file_sep>",
+    "<repo_name>",
+    "<|im_end|>",
+    "<|im_start|>",
+    ">>UNUSED_119<<",
+    ">>UNUSED_120<<",
+    "<|system|>",
+    ">>UNUSED_218<<",
+    ">>UNUSED_219<<",
+    ">>UNUSED_220<<",
+    ">>UNUSED_221<<",
+    ">>UNUSED_222<<",
+    ">>UNUSED_223<<",
+    ">>UNUSED_224<<",
+    ">>UNUSED_225<<",
+    ">>UNUSED_226<<",
+    ">>UNUSED_227<<",
+    ">>UNUSED_228<<",
+    ">>UNUSED_229<<",
+    ">>UNUSED_230<<",
+    ">>UNUSED_231<<",
+    ">>UNUSED_232<<",
+    ">>UNUSED_233<<",
+    ">>UNUSED_234<<",
+    ">>UNUSED_235<<",
+    ">>UNUSED_236<<",
+    ">>UNUSED_237<<",
+    ">>UNUSED_238<<",
+    ">>UNUSED_239<<",
+    ">>UNUSED_240<<",
+    ">>UNUSED_241<<",
+    ">>UNUSED_242<<",
+    ">>UNUSED_243<<",
+    ">>UNUSED_244<<",
+    ">>UNUSED_245<<",
+    ">>UNUSED_246<<",
+    ">>UNUSED_247<<",
+    ">>UNUSED_248<<",
+    ">>UNUSED_249<<",
+    ">>UNUSED_250<<",
+    ">>UNUSED_251<<",
+    ">>UNUSED_252<<",
+    ">>UNUSED_253<<",
+    ">>UNUSED_254<<",
+    ">>UNUSED_255<<",
+    ">>UNUSED_256<<",
+    ">>UNUSED_257<<",
+    ">>UNUSED_258<<",
+    ">>UNUSED_259<<",
+    ">>UNUSED_260<<",
+    ">>UNUSED_261<<",
+    ">>UNUSED_262<<",
+    ">>UNUSED_263<<",
+    ">>UNUSED_264<<",
+    ">>UNUSED_265<<",
+    ">>UNUSED_266<<",
+    ">>UNUSED_267<<",
+    ">>UNUSED_268<<",
+    ">>UNUSED_269<<",
+    ">>UNUSED_270<<",
+    ">>UNUSED_271<<",
+    ">>UNUSED_272<<",
+    ">>UNUSED_273<<",
+    ">>UNUSED_274<<",
+    ">>UNUSED_275<<",
+    ">>UNUSED_276<<",
+    ">>UNUSED_277<<",
+    ">>UNUSED_278<<",
+    ">>UNUSED_279<<",
+    ">>UNUSED_280<<",
+    ">>UNUSED_281<<",
+    ">>UNUSED_282<<",
+    ">>UNUSED_283<<",
+    ">>UNUSED_284<<",
+    ">>UNUSED_285<<",
+    ">>UNUSED_286<<",
+    ">>UNUSED_287<<",
+    ">>UNUSED_288<<",
+    ">>UNUSED_289<<",
+    ">>UNUSED_290<<",
+    ">>UNUSED_291<<",
+    ">>UNUSED_292<<",
+    ">>UNUSED_293<<",
+    ">>UNUSED_294<<",
+    ">>UNUSED_295<<",
+    ">>UNUSED_296<<",
+    ">>UNUSED_297<<",
+    ">>UNUSED_298<<",
+    ">>UNUSED_299<<",
+    ">>UNUSED_300<<",
+    ">>UNUSED_301<<",
+    ">>UNUSED_302<<",
+    ">>UNUSED_303<<",
+    ">>UNUSED_304<<",
+    ">>UNUSED_305<<",
+    ">>UNUSED_306<<",
+    ">>UNUSED_307<<",
+    ">>UNUSED_308<<",
+    ">>UNUSED_309<<",
+    ">>UNUSED_310<<",
+    ">>UNUSED_311<<",
+    ">>UNUSED_312<<",
+    ">>UNUSED_313<<",
+    ">>UNUSED_314<<",
+    ">>UNUSED_315<<",
+    ">>UNUSED_316<<",
+    ">>UNUSED_317<<",
+    ">>UNUSED_318<<",
+    ">>UNUSED_319<<",
+    ">>UNUSED_320<<",
+    ">>UNUSED_321<<",
+    ">>UNUSED_322<<",
+    ">>UNUSED_323<<",
+    ">>UNUSED_324<<",
+    ">>UNUSED_325<<",
+    ">>UNUSED_326<<",
+    ">>UNUSED_327<<",
+    ">>UNUSED_328<<",
+    ">>UNUSED_329<<",
+    ">>UNUSED_330<<",
+    ">>UNUSED_331<<",
+    ">>UNUSED_332<<",
+    ">>UNUSED_333<<",
+    ">>UNUSED_334<<",
+    ">>UNUSED_335<<",
+    ">>UNUSED_336<<",
+    ">>UNUSED_337<<",
+    ">>UNUSED_338<<",
+    ">>UNUSED_339<<",
+    ">>UNUSED_340<<",
+    ">>UNUSED_341<<",
+    ">>UNUSED_342<<",
+    ">>UNUSED_343<<",
+    ">>UNUSED_344<<",
+    ">>UNUSED_345<<",
+    ">>UNUSED_346<<",
+    ">>UNUSED_347<<",
+    ">>UNUSED_348<<",
+    ">>UNUSED_349<<",
+    ">>UNUSED_350<<",
+    ">>UNUSED_351<<",
+    ">>UNUSED_352<<",
+    ">>UNUSED_353<<",
+    ">>UNUSED_354<<",
+    ">>UNUSED_355<<",
+    ">>UNUSED_356<<",
+    ">>UNUSED_357<<",
+    ">>UNUSED_358<<",
+    ">>UNUSED_359<<",
+    ">>UNUSED_360<<",
+    ">>UNUSED_361<<",
+    ">>UNUSED_362<<",
+    ">>UNUSED_363<<",
+    ">>UNUSED_364<<",
+    ">>UNUSED_365<<",
+    ">>UNUSED_366<<",
+    ">>UNUSED_367<<",
+    ">>UNUSED_368<<",
+    ">>UNUSED_369<<",
+    ">>UNUSED_370<<",
+    ">>UNUSED_371<<",
+    ">>UNUSED_372<<",
+    ">>UNUSED_373<<",
+    ">>UNUSED_374<<",
+    ">>UNUSED_375<<",
+    ">>UNUSED_376<<",
+    ">>UNUSED_377<<",
+    ">>UNUSED_378<<",
+    ">>UNUSED_379<<",
+    ">>UNUSED_380<<",
+    ">>UNUSED_381<<",
+    ">>UNUSED_382<<",
+    ">>UNUSED_383<<",
+    ">>UNUSED_384<<",
+    ">>UNUSED_385<<",
+    ">>UNUSED_386<<",
+    ">>UNUSED_387<<",
+    ">>UNUSED_388<<",
+    ">>UNUSED_389<<",
+    ">>UNUSED_390<<",
+    ">>UNUSED_391<<",
+    ">>UNUSED_392<<",
+    ">>UNUSED_393<<",
+    ">>UNUSED_394<<",
+    ">>UNUSED_395<<",
+    ">>UNUSED_396<<",
+    ">>UNUSED_397<<",
+    ">>UNUSED_398<<",
+    ">>UNUSED_399<<",
+    ">>UNUSED_400<<",
+    ">>UNUSED_401<<",
+    ">>UNUSED_402<<",
+    ">>UNUSED_403<<",
+    ">>UNUSED_404<<",
+    ">>UNUSED_405<<",
+    ">>UNUSED_406<<",
+    ">>UNUSED_407<<",
+    ">>UNUSED_408<<",
+    ">>UNUSED_409<<",
+    ">>UNUSED_410<<",
+    ">>UNUSED_411<<",
+    ">>UNUSED_412<<",
+    ">>UNUSED_413<<",
+    ">>UNUSED_414<<",
+    ">>UNUSED_415<<",
+    ">>UNUSED_416<<",
+    ">>UNUSED_417<<",
+    ">>UNUSED_418<<",
+    ">>UNUSED_419<<",
+    ">>UNUSED_420<<",
+    ">>UNUSED_421<<",
+    ">>UNUSED_422<<",
+    ">>UNUSED_423<<",
+    ">>UNUSED_424<<",
+    ">>UNUSED_425<<",
+    ">>UNUSED_426<<",
+    ">>UNUSED_427<<",
+    ">>UNUSED_428<<",
+    ">>UNUSED_429<<",
+    ">>UNUSED_430<<",
+    ">>UNUSED_431<<",
+    ">>UNUSED_432<<",
+    ">>UNUSED_433<<",
+    ">>UNUSED_434<<",
+    ">>UNUSED_435<<",
+    ">>UNUSED_436<<",
+    ">>UNUSED_437<<",
+    ">>UNUSED_438<<",
+    ">>UNUSED_439<<",
+    ">>UNUSED_440<<",
+    ">>UNUSED_441<<",
+    ">>UNUSED_442<<",
+    ">>UNUSED_443<<",
+    ">>UNUSED_444<<",
+    ">>UNUSED_445<<",
+    ">>UNUSED_446<<",
+    ">>UNUSED_447<<",
+    ">>UNUSED_448<<",
+    ">>UNUSED_449<<",
+    ">>UNUSED_450<<",
+    ">>UNUSED_451<<",
+    ">>UNUSED_452<<",
+    ">>UNUSED_453<<",
+    ">>UNUSED_454<<",
+    ">>UNUSED_455<<",
+    ">>UNUSED_456<<",
+    ">>UNUSED_457<<",
+    ">>UNUSED_458<<",
+    ">>UNUSED_459<<",
+    ">>UNUSED_460<<",
+    ">>UNUSED_461<<",
+    ">>UNUSED_462<<",
+    ">>UNUSED_463<<",
+    ">>UNUSED_464<<",
+    ">>UNUSED_465<<",
+    ">>UNUSED_466<<",
+    ">>UNUSED_467<<",
+    ">>UNUSED_468<<",
+    ">>UNUSED_469<<",
+    ">>UNUSED_470<<",
+    ">>UNUSED_471<<",
+    ">>UNUSED_472<<",
+    ">>UNUSED_473<<",
+    ">>UNUSED_474<<",
+    ">>UNUSED_475<<",
+    ">>UNUSED_476<<",
+    ">>UNUSED_477<<",
+    ">>UNUSED_478<<",
+    ">>UNUSED_479<<",
+    ">>UNUSED_480<<",
+    ">>UNUSED_481<<",
+    ">>UNUSED_482<<",
+    ">>UNUSED_483<<",
+    ">>UNUSED_484<<",
+    ">>UNUSED_485<<",
+    ">>UNUSED_486<<",
+    ">>UNUSED_487<<",
+    ">>UNUSED_488<<",
+    ">>UNUSED_489<<",
+    ">>UNUSED_490<<",
+    ">>UNUSED_491<<",
+    ">>UNUSED_492<<",
+    ">>UNUSED_493<<",
+    ">>UNUSED_494<<",
+    ">>UNUSED_495<<",
+    ">>UNUSED_496<<",
+    ">>UNUSED_497<<",
+    ">>UNUSED_498<<",
+    ">>UNUSED_499<<",
+    ">>UNUSED_500<<",
+    ">>UNUSED_501<<",
+    ">>UNUSED_502<<",
+    ">>UNUSED_503<<",
+    ">>UNUSED_504<<",
+    ">>UNUSED_505<<",
+    ">>UNUSED_506<<",
+    ">>UNUSED_507<<",
+    ">>UNUSED_508<<",
+    ">>UNUSED_509<<",
+    ">>UNUSED_510<<",
+    ">>UNUSED_511<<"
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}