Improve language tag
#1
by
lbourdois
- opened
README.md
CHANGED
|
@@ -1,66 +1,77 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
datasets:
|
| 4 |
-
- Azure99/blossom-v6-sft-stage1
|
| 5 |
-
- Azure99/blossom-v6-sft-stage2
|
| 6 |
-
language:
|
| 7 |
-
-
|
| 8 |
-
-
|
| 9 |
-
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
model
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
```
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
datasets:
|
| 4 |
+
- Azure99/blossom-v6-sft-stage1
|
| 5 |
+
- Azure99/blossom-v6-sft-stage2
|
| 6 |
+
language:
|
| 7 |
+
- zho
|
| 8 |
+
- eng
|
| 9 |
+
- fra
|
| 10 |
+
- spa
|
| 11 |
+
- por
|
| 12 |
+
- deu
|
| 13 |
+
- ita
|
| 14 |
+
- rus
|
| 15 |
+
- jpn
|
| 16 |
+
- kor
|
| 17 |
+
- vie
|
| 18 |
+
- tha
|
| 19 |
+
- ara
|
| 20 |
+
base_model:
|
| 21 |
+
- Qwen/Qwen2.5-32B
|
| 22 |
+
---
|
| 23 |
+
# **BLOSSOM-V6-32B**
|
| 24 |
+
|
| 25 |
+
[💻Github](https://github.com/Azure99/BlossomLM) • [🚀Blossom Chat Demo](https://blossom-chat.com/)
|
| 26 |
+
|
| 27 |
+
### Introduction
|
| 28 |
+
|
| 29 |
+
Blossom is a powerful open-source conversational large language model that provides reproducible post-training data, dedicated to delivering an open, powerful, and cost-effective locally accessible general-purpose model for everyone.
|
| 30 |
+
|
| 31 |
+
| Chat Model | Resource | Base Model |
|
| 32 |
+
| ------------------------------------------------------------ | ------------------------------------------------------------ | ----------- |
|
| 33 |
+
| [Blossom-V6-32B](https://huggingface.co/Azure99/Blossom-V6-32B) | [Demo](https://huggingface.co/spaces/Azure99/Blossom-V6-32B-AWQ-Demo) [AWQ](https://huggingface.co/Azure99/Blossom-V6-32B-AWQ) [GGUF](https://huggingface.co/Azure99/Blossom-V6-32B-GGUF) [Ollama](https://ollama.com/azure99/blossom-v6:32b) | Qwen2.5-32B |
|
| 34 |
+
| [Blossom-V6-14B](https://huggingface.co/Azure99/Blossom-V6-14B) | [Demo](https://huggingface.co/spaces/Azure99/Blossom-V6-14B-Demo) [AWQ](https://huggingface.co/Azure99/Blossom-V6-14B-AWQ) [GGUF](https://huggingface.co/Azure99/Blossom-V6-14B-GGUF) [Ollama](https://ollama.com/azure99/blossom-v6:14b) | Qwen2.5-14B |
|
| 35 |
+
| [Blossom-V6-7B](https://huggingface.co/Azure99/Blossom-V6-7B) | [Demo](https://huggingface.co/spaces/Azure99/Blossom-V6-7B-Demo) [AWQ](https://huggingface.co/Azure99/Blossom-V6-7B-AWQ) [GGUF](https://huggingface.co/Azure99/Blossom-V6-7B-GGUF) [Ollama](https://ollama.com/azure99/blossom-v6:7b) | Qwen2.5-7B |
|
| 36 |
+
|
| 37 |
+
You can find the training data here: [Blossom-V6-SFT-Stage1](https://huggingface.co/datasets/Azure99/blossom-v6-sft-stage1) (1 epoch)、[Blossom-V6-SFT-Stage2](https://huggingface.co/datasets/Azure99/blossom-v6-sft-stage2) (3 epoch)。
|
| 38 |
+
|
| 39 |
+
### **Data Synthesis Workflow Overview**
|
| 40 |
+
|
| 41 |
+
Primarily employs three cost-effective models—Yi-Lightning, Deepseek-V2.5, and Doubao-Pro-32K (denoted as A, B, C)—to regenerate responses under different scenarios using tailored synthesis strategies.
|
| 42 |
+
|
| 43 |
+
For example:
|
| 44 |
+
|
| 45 |
+
- In objective scenarios like mathematics (where answers are unique), Model A first generates responses as a "teacher." If reference answers exist in the source data, Model B verifies the correctness of A's responses against them. If no reference answers exist, Model C generates a second response, and Model B checks consistency between A and C's outputs. Inconsistent responses are filtered out.
|
| 46 |
+
- For subjective scenarios, three models cross-evaluate each other. For instance, Models A and B generate responses to a question, and Model C evaluates which is better. The superior response may be retained as training data or used for preference data construction. To mitigate model bias, roles (respondent/evaluator) are randomly assigned to A, B, and C in each instance.
|
| 47 |
+
|
| 48 |
+
Additional rule-based filtering is applied, such as:
|
| 49 |
+
|
| 50 |
+
- N-Gram filtering to remove data with many repetitions.
|
| 51 |
+
- Discarding questions containing toxic content that triggers teacher model refusals.
|
| 52 |
+
|
| 53 |
+
Further technical details will be released in the future. The data is synthesized by the [🌸BlossomData](https://github.com/Azure99/BlossomData) framework.
|
| 54 |
+
|
| 55 |
+
### Inference
|
| 56 |
+
|
| 57 |
+
```python
|
| 58 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 59 |
+
|
| 60 |
+
MODEL = "Azure99/Blossom-V6-32B"
|
| 61 |
+
|
| 62 |
+
model = AutoModelForCausalLM.from_pretrained(MODEL)
|
| 63 |
+
tokenizer = AutoTokenizer.from_pretrained(MODEL)
|
| 64 |
+
|
| 65 |
+
messages = [
|
| 66 |
+
{"role": "user", "content": "北京有什么好吃的"}
|
| 67 |
+
]
|
| 68 |
+
|
| 69 |
+
formatted_input = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 70 |
+
input_ids = tokenizer([formatted_input], return_tensors="pt").to(model.device).input_ids
|
| 71 |
+
generated_ids = model.generate(input_ids, max_new_tokens=512)
|
| 72 |
+
generated_ids = [
|
| 73 |
+
output_ids[len(input_ids):] for input_ids, output_ids in zip(input_ids, generated_ids)
|
| 74 |
+
]
|
| 75 |
+
|
| 76 |
+
print(tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0])
|
| 77 |
```
|