danielhanchen commited on
Commit
565e1a2
·
verified ·
1 Parent(s): 6e0608d

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. README.md +282 -0
  3. chat_template.jinja +123 -0
  4. config.json +37 -0
  5. images/README +1 -0
  6. images/cogito-v2-405b-benchmarks.png +3 -0
  7. images/deep-cogito-logo.png +0 -0
  8. model-00001-of-00191.safetensors +3 -0
  9. model-00002-of-00191.safetensors +3 -0
  10. model-00003-of-00191.safetensors +3 -0
  11. model-00004-of-00191.safetensors +3 -0
  12. model-00005-of-00191.safetensors +3 -0
  13. model-00006-of-00191.safetensors +3 -0
  14. model-00007-of-00191.safetensors +3 -0
  15. model-00008-of-00191.safetensors +3 -0
  16. model-00009-of-00191.safetensors +3 -0
  17. model-00010-of-00191.safetensors +3 -0
  18. model-00011-of-00191.safetensors +3 -0
  19. model-00012-of-00191.safetensors +3 -0
  20. model-00013-of-00191.safetensors +3 -0
  21. model-00014-of-00191.safetensors +3 -0
  22. model-00015-of-00191.safetensors +3 -0
  23. model-00016-of-00191.safetensors +3 -0
  24. model-00017-of-00191.safetensors +3 -0
  25. model-00018-of-00191.safetensors +3 -0
  26. model-00019-of-00191.safetensors +3 -0
  27. model-00020-of-00191.safetensors +3 -0
  28. model-00021-of-00191.safetensors +3 -0
  29. model-00022-of-00191.safetensors +3 -0
  30. model-00023-of-00191.safetensors +3 -0
  31. model-00024-of-00191.safetensors +3 -0
  32. model-00025-of-00191.safetensors +3 -0
  33. model-00026-of-00191.safetensors +3 -0
  34. model-00027-of-00191.safetensors +3 -0
  35. model-00028-of-00191.safetensors +3 -0
  36. model-00029-of-00191.safetensors +3 -0
  37. model-00030-of-00191.safetensors +3 -0
  38. model-00031-of-00191.safetensors +3 -0
  39. model-00032-of-00191.safetensors +3 -0
  40. model-00033-of-00191.safetensors +3 -0
  41. model-00034-of-00191.safetensors +3 -0
  42. model-00035-of-00191.safetensors +3 -0
  43. model-00036-of-00191.safetensors +3 -0
  44. model-00037-of-00191.safetensors +3 -0
  45. model-00038-of-00191.safetensors +3 -0
  46. model-00039-of-00191.safetensors +3 -0
  47. model-00040-of-00191.safetensors +3 -0
  48. model-00041-of-00191.safetensors +3 -0
  49. model-00042-of-00191.safetensors +3 -0
  50. model-00043-of-00191.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ images/cogito-v2-405b-benchmarks.png filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,282 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - unsloth
4
+ license: llama3.1
5
+ library_name: transformers
6
+ base_model:
7
+ - deepcogito/cogito-v2-preview-llama-405B
8
+ pipeline_tag: text-generation
9
+ ---
10
+ > [!NOTE]
11
+ > Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
12
+ >
13
+
14
+ <div>
15
+ <p style="margin-top: 0;margin-bottom: 0;">
16
+ <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
17
+ </p>
18
+ <div style="display: flex; gap: 5px; align-items: center; ">
19
+ <a href="https://github.com/unslothai/unsloth/">
20
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
21
+ </a>
22
+ <a href="https://discord.gg/unsloth">
23
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
24
+ </a>
25
+ <a href="https://docs.unsloth.ai/">
26
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
27
+ </a>
28
+ </div>
29
+ </div>
30
+
31
+
32
+ <p align="center">
33
+ <img src="images/deep-cogito-logo.png" alt="Logo" width="40%">
34
+ </p>
35
+
36
+
37
+ # Cogito v2 preview - 405B
38
+
39
+ [Blog Post](https://www.deepcogito.com/research/cogito-v2-preview)
40
+
41
+ The Cogito v2 LLMs are instruction tuned generative models. All models are released under an open license for commercial use.
42
+
43
+ - Cogito v2 models are hybrid reasoning models. Each model can answer directly (standard LLM), or self-reflect before answering (like reasoning models).
44
+ - The LLMs are trained using **Iterated Distillation and Amplification (IDA)** - an scalable and efficient alignment strategy for superintelligence using iterative self-improvement.
45
+ - The models have been optimized for coding, STEM, instruction following and general helpfulness, and have significantly higher multilingual, coding and tool calling capabilities than size equivalent counterparts.
46
+ - In both standard and reasoning modes, Cogito v2-preview models outperform their size equivalent counterparts on common industry benchmarks.
47
+ - This model is trained in over 30 languages and supports a context length of 128k.
48
+
49
+ # Evaluations
50
+ Here is the model performance on some standard industry benchmarks:
51
+
52
+ <p align="left">
53
+ <img src="images/cogito-v2-405b-benchmarks.png" alt="Logo" width="90%">
54
+ </p>
55
+
56
+ For detailed evaluations, please refer to the [Blog Post](https://www.deepcogito.com/research/cogito-v2-preview).
57
+
58
+ # Usage
59
+ Here is a snippet below for usage with Transformers:
60
+
61
+ ```python
62
+ import transformers
63
+ import torch
64
+
65
+ model_id = "deepcogito/cogito-v2-preview-llama-405B"
66
+
67
+ pipeline = transformers.pipeline(
68
+ "text-generation",
69
+ model=model_id,
70
+ model_kwargs={"torch_dtype": torch.bfloat16},
71
+ device_map="auto",
72
+ )
73
+
74
+ messages = [
75
+ {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
76
+ {"role": "user", "content": "Give me a short introduction to LLMs."},
77
+ ]
78
+
79
+ outputs = pipeline(
80
+ messages,
81
+ max_new_tokens=512,
82
+ )
83
+
84
+ print(outputs[0]["generated_text"][-1])
85
+ ```
86
+
87
+
88
+
89
+ ## Implementing extended thinking
90
+ - By default, the model will answer in the standard mode.
91
+ - To enable thinking, you can do any one of the two methods:
92
+ - Set `enable_thinking=True` while applying the chat template.
93
+ - Add a specific system prompt, along with prefilling the response with "\<think\>\n".
94
+
95
+ **NOTE: Unlike Cogito v1 models, we initiate the response with "\<think\>\n" at the beginning of every output when reasoning is enabled. This is because hybrid models can be brittle at times (<0.1% of the cases), and adding a "\<think\>\n" ensures that the model does indeed respect thinking.**
96
+
97
+ ### Method 1 - Set enable_thinking=True in the tokenizer
98
+ If you are using Huggingface tokenizers, then you can simply use add the argument `enable_thinking=True` to the tokenization (this option is added to the chat template).
99
+
100
+ Here is an example -
101
+ ```python
102
+ from transformers import AutoModelForCausalLM, AutoTokenizer
103
+
104
+ model_name = "deepcogito/cogito-v2-preview-llama-405B"
105
+
106
+ model = AutoModelForCausalLM.from_pretrained(
107
+ model_name,
108
+ torch_dtype="auto",
109
+ device_map="auto"
110
+ )
111
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
112
+
113
+ prompt = "Give me a short introduction to LLMs."
114
+ messages = [
115
+ {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
116
+ {"role": "user", "content": prompt}
117
+ ]
118
+
119
+ text = tokenizer.apply_chat_template(
120
+ messages,
121
+ tokenize=False,
122
+ add_generation_prompt=True,
123
+ enable_thinking=True
124
+ )
125
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
126
+
127
+ generated_ids = model.generate(
128
+ **model_inputs,
129
+ max_new_tokens=512
130
+ )
131
+ generated_ids = [
132
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
133
+ ]
134
+
135
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
136
+ print(response)
137
+ ```
138
+
139
+ ### Method 2 - Add a specific system prompt, along with prefilling the response with "\<think\>\n".
140
+ To enable thinking using this method, you need to do two parts -
141
+
142
+
143
+ Step 1 - Simply use this in the system prompt `system_instruction = 'Enable deep thinking subroutine.'`
144
+
145
+ If you already have a system_instruction, then use `system_instruction = 'Enable deep thinking subroutine.' + '\n\n' + system_instruction`.
146
+
147
+ Step 2 - Prefil the response with the tokens `"<think>\n"`.
148
+
149
+ Here is an example -
150
+
151
+ ```python
152
+ import transformers
153
+ import torch
154
+
155
+ model_name = "deepcogito/cogito-v2-preview-llama-405B"
156
+
157
+ model = AutoModelForCausalLM.from_pretrained(
158
+ model_name,
159
+ torch_dtype="auto",
160
+ device_map="auto"
161
+ )
162
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
163
+
164
+ # Step 1 - Add deep thinking instruction.
165
+ DEEP_THINKING_INSTRUCTION = "Enable deep thinking subroutine."
166
+
167
+ messages = [
168
+ {"role": "system", "content": DEEP_THINKING_INSTRUCTION},
169
+ {"role": "user", "content": "Write a bash script that takes a matrix represented as a string with format '[1,2],[3,4],[5,6]' and prints the transpose in the same format."},
170
+ ]
171
+
172
+ text = tokenizer.apply_chat_template(
173
+ messages,
174
+ tokenize=False,
175
+ add_generation_prompt=True
176
+ )
177
+
178
+ # Step 2 - Prefill response with "<think>\n".
179
+ text += "<think>\n"
180
+
181
+ # Now, continue as usual.
182
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
183
+
184
+ generated_ids = model.generate(
185
+ **model_inputs,
186
+ max_new_tokens=512
187
+ )
188
+ generated_ids = [
189
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
190
+ ]
191
+
192
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
193
+ print(response)
194
+ ```
195
+
196
+
197
+ Similarly, if you have a system prompt, you can append the `DEEP_THINKING_INSTRUCTION` to the beginning in this way -
198
+
199
+ ```python
200
+ DEEP_THINKING_INSTRUCTION = "Enable deep thinking subroutine."
201
+
202
+ system_prompt = "Reply to each prompt with only the actual code - no explanations."
203
+ prompt = "Write a bash script that takes a matrix represented as a string with format '[1,2],[3,4],[5,6]' and prints the transpose in the same format."
204
+
205
+ messages = [
206
+ {"role": "system", "content": DEEP_THINKING_INSTRUCTION + '\n\n' + system_prompt},
207
+ {"role": "user", "content": prompt}
208
+ ]
209
+ ```
210
+
211
+
212
+ # Tool Calling
213
+ Cogito models support tool calling (single, parallel, multiple and parallel_multiple) both in standard and extended thinking mode.
214
+
215
+ Here is a snippet -
216
+
217
+ ```python
218
+ # First, define a tool
219
+ def get_current_temperature(location: str) -> float:
220
+ """
221
+ Get the current temperature at a location.
222
+
223
+ Args:
224
+ location: The location to get the temperature for, in the format "City, Country"
225
+ Returns:
226
+ The current temperature at the specified location in the specified units, as a float.
227
+ """
228
+ return 22. # A real function should probably actually get the temperature!
229
+
230
+ # Next, create a chat and apply the chat template
231
+ messages = [
232
+ {"role": "user", "content": "Hey, what's the temperature in Paris right now?"}
233
+ ]
234
+
235
+ model_inputs = tokenizer.apply_chat_template(messages, tools=[get_current_temperature], add_generation_prompt=True)
236
+
237
+ text = tokenizer.apply_chat_template(messages, tools=[get_current_temperature], add_generation_prompt=True, tokenize=False)
238
+ inputs = tokenizer(text, return_tensors="pt", add_special_tokens=False).to(model.device)
239
+ outputs = model.generate(**inputs, max_new_tokens=512)
240
+ output_text = tokenizer.batch_decode(outputs)[0][len(text):]
241
+ print(output_text)
242
+ ```
243
+
244
+ This will result in the output -
245
+ ```
246
+ <tool_call>
247
+ {"name": "get_current_temperature", "arguments": {"location": "Paris, France"}}
248
+ </tool_call><|eot_id|>
249
+ ```
250
+
251
+ You can then generate text from this input as normal. If the model generates a tool call, you should add it to the chat like so:
252
+
253
+ ```python
254
+ tool_call = {"name": "get_current_temperature", "arguments": {"location": "Paris, France"}}
255
+ messages.append({"role": "assistant", "tool_calls": [{"type": "function", "function": tool_call}]})
256
+ ```
257
+
258
+ and then call the tool and append the result, with the `tool` role, like so:
259
+
260
+ ```python
261
+ messages.append({"role": "tool", "name": "get_current_temperature", "content": "22.0"})
262
+ ```
263
+
264
+ After that, you can `generate()` again to let the model use the tool result in the chat:
265
+
266
+ ```python
267
+ text = tokenizer.apply_chat_template(messages, tools=[get_current_temperature], add_generation_prompt=True, tokenize=False)
268
+ inputs = tokenizer(text, return_tensors="pt", add_special_tokens=False).to(model.device)
269
+ outputs = model.generate(**inputs, max_new_tokens=512)
270
+ output_text = tokenizer.batch_decode(outputs)[0][len(text):]
271
+ ```
272
+
273
+ This should result in the string -
274
+ ```
275
+ 'The current temperature in Paris is 22.0 degrees.<|eot_id|>'
276
+ ```
277
+
278
+ ## License
279
+ This repository and the model weights are licensed under the [Llama 3.1 Community License Agreement](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE) (Llama models' default license agreement).
280
+
281
+ ## Contact
282
+ If you would like to reach out to our team, send an email to [[email protected]]([email protected]).
chat_template.jinja ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{- bos_token }}
2
+ {%- if not tools is defined %}
3
+ {%- set tools = none %}
4
+ {%- endif %}
5
+ {%- if not enable_thinking is defined %}
6
+ {%- set enable_thinking = false %}
7
+ {%- endif %}
8
+ {#- This block extracts the system message, so we can slot it into the right place. #}
9
+ {%- if messages[0]['role'] == 'system' %}
10
+ {%- set system_message = messages[0]['content']|trim %}
11
+ {%- set messages = messages[1:] %}
12
+ {%- else %}
13
+ {%- set system_message = "" %}
14
+ {%- endif %}
15
+ {#- Set the system message. If enable_thinking is true, add the "Enable deep thinking subroutine." #}
16
+ {%- if enable_thinking %}
17
+ {%- if system_message != "" %}
18
+ {%- set system_message = "Enable deep thinking subroutine.
19
+
20
+ " ~ system_message %}
21
+ {%- else %}
22
+ {%- set system_message = "Enable deep thinking subroutine." %}
23
+ {%- endif %}
24
+ {%- endif %}
25
+ {#- Set the system message. In case there are tools present, add them to the system message. #}
26
+ {%- if tools is not none or system_message != '' %}
27
+ {{- "<|start_header_id|>system<|end_header_id|>
28
+
29
+ " }}
30
+ {{- system_message }}
31
+ {%- if tools is not none %}
32
+ {%- if system_message != "" %}
33
+ {{- "
34
+
35
+ " }}
36
+ {%- endif %}
37
+ {{- "Available Tools:
38
+ " }}
39
+ {%- for t in tools %}
40
+ {{- t | tojson(indent=4) }}
41
+ {{- "
42
+
43
+ " }}
44
+ {%- endfor %}
45
+ {%- endif %}
46
+ {{- "<|eot_id|>" }}
47
+ {%- endif %}
48
+
49
+ {#- Rest of the messages #}
50
+ {%- for message in messages %}
51
+ {#- The special cases are when the message is from a tool (via role ipython/tool/tool_results) or when the message is from the assistant, but has "tool_calls". If not, we add the message directly as usual. #}
52
+ {#- Case 1 - Usual, non tool related message. #}
53
+ {%- if not (message.role == "ipython" or message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
54
+ {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>
55
+
56
+ ' }}
57
+ {%- if message['content'] is string %}
58
+ {{- message['content'] | trim }}
59
+ {%- else %}
60
+ {%- for item in message['content'] %}
61
+ {%- if item.type == 'text' %}
62
+ {{- item.text | trim }}
63
+ {%- endif %}
64
+ {%- endfor %}
65
+ {%- endif %}
66
+ {{- '<|eot_id|>' }}
67
+
68
+ {#- Case 2 - the response is from the assistant, but has a tool call returned. The assistant may also have returned some content along with the tool call. #}
69
+ {%- elif message.tool_calls is defined and message.tool_calls is not none %}
70
+ {{- "<|start_header_id|>assistant<|end_header_id|>
71
+
72
+ " }}
73
+ {%- if message['content'] is string %}
74
+ {{- message['content'] | trim }}
75
+ {%- else %}
76
+ {%- for item in message['content'] %}
77
+ {%- if item.type == 'text' %}
78
+ {{- item.text | trim }}
79
+ {%- if item.text | trim != "" %}
80
+ {{- "
81
+
82
+ " }}
83
+ {%- endif %}
84
+ {%- endif %}
85
+ {%- endfor %}
86
+ {%- endif %}
87
+ {{- "[" }}
88
+ {%- for tool_call in message.tool_calls %}
89
+ {%- set out = tool_call.function|tojson %}
90
+ {%- if not tool_call.id is defined %}
91
+ {{- out }}
92
+ {%- else %}
93
+ {{- out[:-1] }}
94
+ {{- ', "id": "' + tool_call.id + '"}' }}
95
+ {%- endif %}
96
+ {%- if not loop.last %}
97
+ {{- ", " }}
98
+ {%- else %}
99
+ {{- "]<|eot_id|>" }}
100
+ {%- endif %}
101
+ {%- endfor %}
102
+
103
+ {#- Case 3 - the response is from a tool call. The tool call may have an id associated with it as well. If it does, we add it to the prompt. #}
104
+ {%- elif message.role == "ipython" or message["role"] == "tool_results" or message["role"] == "tool" %}
105
+ {{- "<|start_header_id|>ipython<|end_header_id|>
106
+
107
+ " }}
108
+ {%- if message.tool_call_id is defined and message.tool_call_id != '' %}
109
+ {{- '{"content": ' + (message.content | tojson) + ', "call_id": "' + message.tool_call_id + '"}' }}
110
+ {%- else %}
111
+ {{- '{"content": ' + (message.content | tojson) + '}' }}
112
+ {%- endif %}
113
+ {{- "<|eot_id|>" }}
114
+ {%- endif %}
115
+ {%- endfor %}
116
+ {%- if add_generation_prompt %}
117
+ {{- '<|start_header_id|>assistant<|end_header_id|>
118
+
119
+ ' }}
120
+ {%- if enable_thinking %}
121
+ {{- '<think>\n' }}
122
+ {%- endif %}
123
+ {%- endif %}
config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LlamaForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 128000,
8
+ "eos_token_id": 128001,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 16384,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 53248,
14
+ "max_position_embeddings": 131072,
15
+ "mlp_bias": false,
16
+ "model_type": "llama",
17
+ "num_attention_heads": 128,
18
+ "num_hidden_layers": 126,
19
+ "num_key_value_heads": 8,
20
+ "pad_token_id": 128004,
21
+ "pretraining_tp": 1,
22
+ "rms_norm_eps": 1e-05,
23
+ "rope_scaling": {
24
+ "factor": 8.0,
25
+ "high_freq_factor": 4.0,
26
+ "low_freq_factor": 1.0,
27
+ "original_max_position_embeddings": 8192,
28
+ "rope_type": "llama3"
29
+ },
30
+ "rope_theta": 500000.0,
31
+ "tie_word_embeddings": false,
32
+ "torch_dtype": "bfloat16",
33
+ "transformers_version": "4.54.1",
34
+ "unsloth_fixed": true,
35
+ "use_cache": true,
36
+ "vocab_size": 128256
37
+ }
images/README ADDED
@@ -0,0 +1 @@
 
 
1
+ Directory for images associated with the model.
images/cogito-v2-405b-benchmarks.png ADDED

Git LFS Details

  • SHA256: 618ca9a3374e3c5cc5995788f78d7f1b4f3007ed051606e856b0c61d6c2b06f3
  • Pointer size: 131 Bytes
  • Size of remote file: 266 kB
images/deep-cogito-logo.png ADDED
model-00001-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da4fcf2c07107d219e61518b3554b8b8487e60c4f72769cc1c0c35ce36df6e92
3
+ size 4202692736
model-00002-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3a135fe37c739441ac99f987561094c602676dcd35ba5b40eacc7210dd85304
3
+ size 4202725632
model-00003-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f7e4aac30dc96813431d601a4c790205d595da3d34f199f670c7ddcd21e4519
3
+ size 3489661192
model-00004-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb4dd8714572e5385993288d53d360c84bf03b0ac815e1f29da1a999a8bcf7b3
3
+ size 4630578120
model-00005-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4b27d8bed5e0bdac8446319a447f85cc4cb35135dc7409bb1e31c3fbacf1bef
3
+ size 4630578128
model-00006-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec7117f7539c3e627aa56c26436de62cafd82b75abd058e097bc7b1207c7e664
3
+ size 3489661200
model-00007-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc9f428e34fb33a8be29b5571284db6c4e8541bcc11bf483b95ce26f72687175
3
+ size 4630578136
model-00008-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb6f9bc2c61a29c08a53ebebce2d70a06b52b8b2054bb064ee60491cf4a3fbca
3
+ size 4630578136
model-00009-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:baf7597d17b5846f1d8160ef7ec8c80653d4a4a4649da93ddf2aba772c25057c
3
+ size 3489661200
model-00010-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1493a5ba226da3a0d2fce9bbc74d9b2884ce4815d1930d71652677c755951f5
3
+ size 4630578136
model-00011-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b1ebe3a8c4ee0e721b9fbdcdd54b21c57b0b809343fad7bc31c61ea1e00251b
3
+ size 4630578136
model-00012-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:366926d3047c0bacd272ed15c55ce86be399cf99dba82f0eb302b588ce3f747f
3
+ size 3489661200
model-00013-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:586114ddd75056f4792edec1ee75c7af39079178bbb90e0b27798c2e1f63524f
3
+ size 4630578136
model-00014-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40390d7018375855a1a04d5e562cab5ca0bb68d3bd51a0e86e367f2c423e3e0b
3
+ size 4630578136
model-00015-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96b773560e39bbaf4f9c5d55843ec8ef84ff986fe5a73c9e7a98a0509a980c2a
3
+ size 3489661200
model-00016-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:341a98443418b77676c3131a96b756f3bb428d032d61067a13d36cf6c52994a2
3
+ size 4630578136
model-00017-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d33df86c8ebefee0c39ed1f96c26db509ddcd85ae505d70e16a34d6adf2c3d7
3
+ size 4630578136
model-00018-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac83bd51c6d077e4ab0794e02c7ae9f4621766f0b519b6c6e085e7fe65a71603
3
+ size 3489661200
model-00019-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:139a7491200398b94dcdbae5f8058ace5c108d3a63934d89b6f1738ce277a52e
3
+ size 4630578136
model-00020-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7885e2bddfd7bbd3a434b51275738063c3d5c12376c8b187a92c7398875e8111
3
+ size 4630578136
model-00021-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14cff14385de974fa593d585556da391e5167a52110464d6f8db24b72743b6ac
3
+ size 3489661200
model-00022-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7e6d59503b8b46f8b673a7d7f50bdcd3e918edeec5fd65d841364f3736896e5
3
+ size 4630578136
model-00023-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb1e1bd040ff1f02c985928efb5a489624c8b1ab7519140cca774df299417c03
3
+ size 4630578136
model-00024-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9d4af519dc268f65973d6dd9c05bd807aaab947f974aa034503a23908129d68
3
+ size 3489661200
model-00025-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d89f8867a6197b799c8071f92fdc299835f0d6433024ebdfde5d2348c2823c5f
3
+ size 4630578136
model-00026-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fee4fa1fa8acf4e0858ef233a9e7e2401cb363c05b1cae531b737fcdf5cb7ed9
3
+ size 4630578136
model-00027-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d09202cd8a34ad2ab2f69a1b3db10735cea24db8df88a139f0a843873f5b07b0
3
+ size 3489661200
model-00028-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7c15d87433374725722796f0f958c255cda51239f9f101a1f29fdefdda454e0
3
+ size 4630578136
model-00029-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec264e604a8521068743454b72c55e0c64e3ce3c0a82162a6c4505ce1c440c43
3
+ size 4630578136
model-00030-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:672f353bbdabbac38030b599f352dd54f3bc5a60582d1d6849e5f41cba6651e5
3
+ size 3489661200
model-00031-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e6a99410a0a731ac150fca9bbb9784d17bd725624a682fd8ffe20a881e9af2d
3
+ size 4630578136
model-00032-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da98f02616eb867d3a7b488690ff8503f3a4f345f5dd809f1a4647ecd2221340
3
+ size 4630578136
model-00033-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea0e4d1677e17bdb346d43621098849236f0bfd96461ca75a5b7fb419a6f01b1
3
+ size 3489661200
model-00034-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30110887b9c9fda6bac9d47f05de8527395c6fa7e9d6b4486bebbe60cd3aecac
3
+ size 4630578136
model-00035-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c37b39aa057388f2903d1f56fe3f2f092ed1f76158cdfd05b1da762784acfe0
3
+ size 4630578136
model-00036-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74b73f7094ea241f79df6119880291776df1e4ab76820bf630a13ddad568868a
3
+ size 3489661200
model-00037-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5239e1e4e6386dbb6209738fbd8d5b02840a1997cec1ec0bb9ca466033779f3
3
+ size 4630578136
model-00038-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09af40136899dfdce4cab00f74cb48aa244c5bac99fd457bede59ce360292bb2
3
+ size 4630578136
model-00039-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c23206fa7c9c051ed121b4031b7c78b33d3f29c9828e783013ac24652ea547e
3
+ size 3489661200
model-00040-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:283766bd72e27d35bf79650202c55ff7e97e9050796f95cc6007cce1404e5d21
3
+ size 4630578136
model-00041-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42c47881d517142e961761bf78483fd496f65db4a0142ba4feaa35479f2ff8c1
3
+ size 4630578136
model-00042-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:088b827b5beb35b1d61c38156695db8ee2063250beeaaa390a139f48142dab4c
3
+ size 3489661200
model-00043-of-00191.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e5c09d07dccda230b7f51873b4f27c74ff5fe83d609f4bc907b40391184185b
3
+ size 4630578136