Update README.md
Browse files
README.md
CHANGED
@@ -25,90 +25,90 @@ pipeline_tag: text-generation
|
|
25 |
|
26 |
# 🤖 gama-4b
|
27 |
|
28 |
-
**gama-4b**
|
29 |
|
30 |
-
## 📋
|
31 |
|
32 |
-
|
33 |
|
34 |
-
### 🌟
|
35 |
|
36 |
-
- **💬
|
37 |
-
- **⚡
|
38 |
-
- **🔧
|
39 |
|
40 |
-
### 🔧
|
41 |
|
42 |
-
|
43 |
|
44 |
-
- **[CEIA-UFG/Gemma-3-Gaia-PT-BR-4b-it](https://huggingface.co/CEIA-UFG/Gemma-3-Gaia-PT-BR-4b-it)**
|
45 |
-
- **[soob3123/Veiled-Calla-4B](https://huggingface.co/soob3123/Veiled-Calla-4B)**
|
46 |
-
- **[soob3123/amoral-gemma3-4B-v2-qat](https://huggingface.co/soob3123/amoral-gemma3-4B-v2-qat)**
|
47 |
|
48 |
-
### 🛠️
|
49 |
|
50 |
-
|
51 |
|
52 |
-
## ⚙️
|
53 |
|
54 |
-
###
|
55 |
|
56 |
```yaml
|
57 |
-
models:
|
58 |
- model: CEIA-UFG/Gemma-3-Gaia-PT-BR-4b-it
|
59 |
parameters:
|
60 |
-
density: 0.6
|
61 |
weight: 0.34
|
62 |
-
|
63 |
- model: soob3123/Veiled-Calla-4B
|
64 |
parameters:
|
65 |
-
density: 0.6
|
66 |
weight: 0.33
|
67 |
-
|
68 |
- model: soob3123/amoral-gemma3-4B-v2-qat
|
69 |
parameters:
|
70 |
density: 0.6
|
71 |
-
weight: 0.33
|
72 |
|
73 |
merge_method: dare_ties
|
74 |
base_model: unsloth/gemma-3-4b-it-qat
|
75 |
|
76 |
parameters:
|
77 |
-
normalize: true
|
78 |
-
int8_mask: true
|
79 |
-
|
80 |
dtype: bfloat16
|
81 |
```
|
82 |
|
83 |
-
###
|
84 |
|
85 |
-
- **
|
86 |
-
- **
|
87 |
-
- **
|
88 |
-
- **
|
89 |
-
- **
|
90 |
-
- **
|
91 |
-
- **
|
92 |
|
93 |
-
## 💻
|
94 |
|
95 |
-
###
|
96 |
|
97 |
```bash
|
98 |
pip install -qU transformers accelerate torch
|
99 |
```
|
100 |
|
101 |
-
###
|
102 |
|
103 |
```python
|
104 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
105 |
import transformers
|
106 |
import torch
|
107 |
|
108 |
-
#
|
109 |
model_name = "rodrigomt/gama-4b"
|
110 |
|
111 |
-
#
|
112 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
113 |
model = AutoModelForCausalLM.from_pretrained(
|
114 |
model_name,
|
@@ -117,24 +117,24 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
117 |
trust_remote_code=True
|
118 |
)
|
119 |
|
120 |
-
#
|
121 |
messages_pt = [
|
122 |
-
{"role": "user", "content": "
|
123 |
]
|
124 |
|
125 |
-
#
|
126 |
messages_en = [
|
127 |
{"role": "user", "content": "What is a large language model?"}
|
128 |
]
|
129 |
|
130 |
-
#
|
131 |
prompt = tokenizer.apply_chat_template(
|
132 |
-
messages_pt,
|
133 |
-
tokenize=False,
|
134 |
add_generation_prompt=True
|
135 |
)
|
136 |
|
137 |
-
#
|
138 |
pipeline = transformers.pipeline(
|
139 |
"text-generation",
|
140 |
model=model,
|
@@ -143,7 +143,7 @@ pipeline = transformers.pipeline(
|
|
143 |
device_map="auto",
|
144 |
)
|
145 |
|
146 |
-
#
|
147 |
outputs = pipeline(
|
148 |
prompt,
|
149 |
max_new_tokens=256,
|
@@ -157,13 +157,13 @@ outputs = pipeline(
|
|
157 |
print(outputs[0]["generated_text"])
|
158 |
```
|
159 |
|
160 |
-
###
|
161 |
|
162 |
```python
|
163 |
-
#
|
164 |
conversation = [
|
165 |
-
{"role": "user", "content": "
|
166 |
-
{"role": "assistant", "content": "
|
167 |
{"role": "user", "content": "Can you switch to English?"},
|
168 |
{"role": "assistant", "content": "Of course! I can communicate in both Portuguese and English. How can I help you?"}
|
169 |
]
|
@@ -173,14 +173,14 @@ outputs = pipeline(prompt, max_new_tokens=128, temperature=0.7)
|
|
173 |
print(outputs[0]["generated_text"])
|
174 |
```
|
175 |
|
176 |
-
###
|
177 |
|
178 |
```python
|
179 |
-
#
|
180 |
def generate_response(prompt_text, max_tokens=256, temperature=0.7):
|
181 |
inputs = tokenizer.encode(prompt_text, return_tensors="pt")
|
182 |
attention_mask = inputs.ne(tokenizer.pad_token_id)
|
183 |
-
|
184 |
with torch.no_grad():
|
185 |
outputs = model.generate(
|
186 |
inputs,
|
@@ -194,51 +194,56 @@ def generate_response(prompt_text, max_tokens=256, temperature=0.7):
|
|
194 |
pad_token_id=tokenizer.eos_token_id,
|
195 |
eos_token_id=tokenizer.eos_token_id
|
196 |
)
|
197 |
-
|
198 |
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
199 |
return response
|
200 |
|
201 |
-
#
|
202 |
-
response = generate_response("
|
203 |
print(response)
|
204 |
```
|
205 |
|
206 |
-
## ⚠️
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
207 |
|
208 |
-
###
|
209 |
-
- **RAM:** 16GB
|
210 |
-
- **VRAM:** 8GB (GPU)
|
211 |
-
- **Armazenamento:** 20GB disponíveis
|
212 |
-
- **GPU:** GTX 3070 ou superior
|
213 |
|
214 |
-
|
215 |
-
- **
|
216 |
-
- **
|
217 |
-
- **
|
218 |
-
- **CPU:** Processador moderno multi-core
|
219 |
|
220 |
-
### Deployment
|
221 |
-
- **RAM:** 32GB+
|
222 |
-
- **VRAM:** 24GB+ (GPU)
|
223 |
-
- **GPU:** A6000, A100 ou superior para alta concorrência
|
224 |
|
225 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
226 |
|
227 |
-
### Ajuste de Temperatura
|
228 |
```python
|
229 |
-
#
|
230 |
outputs = pipeline(prompt, temperature=0.9, top_p=0.95)
|
231 |
|
232 |
-
#
|
233 |
outputs = pipeline(prompt, temperature=0.3, top_k=30)
|
234 |
```
|
235 |
|
236 |
-
###
|
|
|
237 |
```python
|
238 |
-
#
|
239 |
outputs = pipeline(prompt, repetition_penalty=1.2, no_repeat_ngram_size=3)
|
240 |
```
|
241 |
|
242 |
-
## 📝
|
243 |
|
244 |
-
|
|
|
25 |
|
26 |
# 🤖 gama-4b
|
27 |
|
28 |
+
**gama-4b** is an efficient 4-billion parameter language model, specially optimized for **multilingual** conversation with a focus on **Portuguese and English**. This model combines specialized capabilities through a strategic merge of complementary models.
|
29 |
|
30 |
+
## 📋 Overview
|
31 |
|
32 |
+
This model was developed using the **DARE TIES** (Drop And REscale with Ties-Elimination) technique, combining specialized models to create a compact and versatile solution for conversational applications in Portuguese and English.
|
33 |
|
34 |
+
### 🌟 Key Features
|
35 |
|
36 |
+
- **💬 Bilingual:** Optimized for Brazilian Portuguese and English
|
37 |
+
- **⚡ Efficient:** Only 4B parameters for fast deployment
|
38 |
+
- **🔧 Quantized:** QAT for better performance/size
|
39 |
|
40 |
+
### 🔧 Base Models Used
|
41 |
|
42 |
+
**gama-4b** is the result of a strategic merge of the following models:
|
43 |
|
44 |
+
- **[CEIA-UFG/Gemma-3-Gaia-PT-BR-4b-it](https://huggingface.co/CEIA-UFG/Gemma-3-Gaia-PT-BR-4b-it)**
|
45 |
+
- **[soob3123/Veiled-Calla-4B](https://huggingface.co/soob3123/Veiled-Calla-4B)**
|
46 |
+
- **[soob3123/amoral-gemma3-4B-v2-qat](https://huggingface.co/soob3123/amoral-gemma3-4B-v2-qat)**
|
47 |
|
48 |
+
### 🛠️ Merge Tool
|
49 |
|
50 |
+
The merge was performed using **[LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing)**, facilitating the process of merging language models with advanced configurations.
|
51 |
|
52 |
+
## ⚙️ Technical Configuration
|
53 |
|
54 |
+
### Merge Parameters
|
55 |
|
56 |
```yaml
|
57 |
+
models:
|
58 |
- model: CEIA-UFG/Gemma-3-Gaia-PT-BR-4b-it
|
59 |
parameters:
|
60 |
+
density: 0.6
|
61 |
weight: 0.34
|
62 |
+
|
63 |
- model: soob3123/Veiled-Calla-4B
|
64 |
parameters:
|
65 |
+
density: 0.6
|
66 |
weight: 0.33
|
67 |
+
|
68 |
- model: soob3123/amoral-gemma3-4B-v2-qat
|
69 |
parameters:
|
70 |
density: 0.6
|
71 |
+
weight: 0.33
|
72 |
|
73 |
merge_method: dare_ties
|
74 |
base_model: unsloth/gemma-3-4b-it-qat
|
75 |
|
76 |
parameters:
|
77 |
+
normalize: true
|
78 |
+
int8_mask: true
|
79 |
+
|
80 |
dtype: bfloat16
|
81 |
```
|
82 |
|
83 |
+
### Technical Specifications
|
84 |
|
85 |
+
- **Architecture:** Gemma-3 4B
|
86 |
+
- **Merge Method:** DARE TIES
|
87 |
+
- **Precision:** BFloat16
|
88 |
+
- **Quantization:** QAT (Quantization Aware Training)
|
89 |
+
- **Normalization:** Enabled
|
90 |
+
- **Int8 Mask:** Enabled
|
91 |
+
- **Languages:** Portuguese (PT-BR) and English
|
92 |
|
93 |
+
## 💻 How to Use
|
94 |
|
95 |
+
### Installing Dependencies
|
96 |
|
97 |
```bash
|
98 |
pip install -qU transformers accelerate torch
|
99 |
```
|
100 |
|
101 |
+
### Basic Usage Example
|
102 |
|
103 |
```python
|
104 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
105 |
import transformers
|
106 |
import torch
|
107 |
|
108 |
+
# Model configuration
|
109 |
model_name = "rodrigomt/gama-4b"
|
110 |
|
111 |
+
# Load tokenizer and model
|
112 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
113 |
model = AutoModelForCausalLM.from_pretrained(
|
114 |
model_name,
|
|
|
117 |
trust_remote_code=True
|
118 |
)
|
119 |
|
120 |
+
# Example in Portuguese
|
121 |
messages_pt = [
|
122 |
+
{"role": "user", "content": "What is a large language model?"}
|
123 |
]
|
124 |
|
125 |
+
# Example in English
|
126 |
messages_en = [
|
127 |
{"role": "user", "content": "What is a large language model?"}
|
128 |
]
|
129 |
|
130 |
+
# Apply chat template
|
131 |
prompt = tokenizer.apply_chat_template(
|
132 |
+
messages_pt,
|
133 |
+
tokenize=False,
|
134 |
add_generation_prompt=True
|
135 |
)
|
136 |
|
137 |
+
# Pipeline configuration
|
138 |
pipeline = transformers.pipeline(
|
139 |
"text-generation",
|
140 |
model=model,
|
|
|
143 |
device_map="auto",
|
144 |
)
|
145 |
|
146 |
+
# Text generation
|
147 |
outputs = pipeline(
|
148 |
prompt,
|
149 |
max_new_tokens=256,
|
|
|
157 |
print(outputs[0]["generated_text"])
|
158 |
```
|
159 |
|
160 |
+
### Multilingual Usage Example
|
161 |
|
162 |
```python
|
163 |
+
# Conversation switching languages
|
164 |
conversation = [
|
165 |
+
{"role": "user", "content": "Hello! How are you?"},
|
166 |
+
{"role": "assistant", "content": "Hello! I'm doing well, thank you for asking. How can I help you today?"},
|
167 |
{"role": "user", "content": "Can you switch to English?"},
|
168 |
{"role": "assistant", "content": "Of course! I can communicate in both Portuguese and English. How can I help you?"}
|
169 |
]
|
|
|
173 |
print(outputs[0]["generated_text"])
|
174 |
```
|
175 |
|
176 |
+
### Advanced Usage Example
|
177 |
|
178 |
```python
|
179 |
+
# For more granular control over generation
|
180 |
def generate_response(prompt_text, max_tokens=256, temperature=0.7):
|
181 |
inputs = tokenizer.encode(prompt_text, return_tensors="pt")
|
182 |
attention_mask = inputs.ne(tokenizer.pad_token_id)
|
183 |
+
|
184 |
with torch.no_grad():
|
185 |
outputs = model.generate(
|
186 |
inputs,
|
|
|
194 |
pad_token_id=tokenizer.eos_token_id,
|
195 |
eos_token_id=tokenizer.eos_token_id
|
196 |
)
|
197 |
+
|
198 |
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
199 |
return response
|
200 |
|
201 |
+
# Using the function
|
202 |
+
response = generate_response("Explain machine learning in simple terms:")
|
203 |
print(response)
|
204 |
```
|
205 |
|
206 |
+
## ⚠️ System Requirements
|
207 |
+
|
208 |
+
### Minimum Configuration
|
209 |
+
|
210 |
+
- **RAM:** 16GB
|
211 |
+
- **VRAM:** 8GB (GPU)
|
212 |
+
- **Storage:** 20GB available
|
213 |
+
- **GPU:** GTX 3070 or higher
|
214 |
|
215 |
+
### Recommended Configuration
|
|
|
|
|
|
|
|
|
216 |
|
217 |
+
- **RAM:** 32GB
|
218 |
+
- **VRAM:** 16GB (GPU)
|
219 |
+
- **GPU:** RTX 4070, A4000 or higher
|
220 |
+
- **CPU:** Modern multi-core processor
|
|
|
221 |
|
222 |
+
### Production Deployment
|
|
|
|
|
|
|
223 |
|
224 |
+
- **RAM:** 32GB+
|
225 |
+
- **VRAM:** 24GB+ (GPU)
|
226 |
+
- **GPU:** A6000, A100 or higher for high concurrency
|
227 |
+
|
228 |
+
## 🔧 Advanced Settings
|
229 |
+
|
230 |
+
### Temperature Adjustment
|
231 |
|
|
|
232 |
```python
|
233 |
+
# More creative responses
|
234 |
outputs = pipeline(prompt, temperature=0.9, top_p=0.95)
|
235 |
|
236 |
+
# More conservative responses
|
237 |
outputs = pipeline(prompt, temperature=0.3, top_k=30)
|
238 |
```
|
239 |
|
240 |
+
### Repetition Control
|
241 |
+
|
242 |
```python
|
243 |
+
# Reduce repetitions
|
244 |
outputs = pipeline(prompt, repetition_penalty=1.2, no_repeat_ngram_size=3)
|
245 |
```
|
246 |
|
247 |
+
## 📝 License
|
248 |
|
249 |
+
This model is licensed under the **Gemma License**.
|