Update README.md
Browse files
README.md
CHANGED
@@ -26,12 +26,16 @@ tags:
|
|
26 |
|
27 |
<!-- Provide a quick summary of what the model is/does. -->
|
28 |
|
29 |
-
This is a quantized version of `Llama 3.1
|
30 |
|
31 |
-
- **Developed by:**
|
32 |
- **License:** llama3.1
|
33 |
- **Base Model [optional]:** meta-llama/Meta-Llama-3.1-8B-Instruct
|
34 |
|
|
|
|
|
|
|
|
|
35 |
```python
|
36 |
# Use a pipeline as a high-level helper
|
37 |
from transformers import pipeline
|
@@ -40,17 +44,21 @@ messages = [
|
|
40 |
{"role": "user", "content": "Who are you?"},
|
41 |
]
|
42 |
pipe = pipeline("text-generation", model="meta-llama/Meta-Llama-3.1-8B-Instruct")
|
43 |
-
pipe(messages)
|
44 |
```
|
45 |
|
|
|
|
|
|
|
46 |
```python
|
|
|
47 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
48 |
|
49 |
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
|
50 |
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
|
51 |
```
|
52 |
|
53 |
-
The model information can be found in the original [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
|
54 |
|
55 |
|
56 |
|
|
|
26 |
|
27 |
<!-- Provide a quick summary of what the model is/does. -->
|
28 |
|
29 |
+
This is a quantized version of `Llama 3.1 8B Instruct`. Quantized to **4-bit** using `bistandbytes` and `accelerate`.
|
30 |
|
31 |
+
- **Developed by:** Farid Saud @ DSRS
|
32 |
- **License:** llama3.1
|
33 |
- **Base Model [optional]:** meta-llama/Meta-Llama-3.1-8B-Instruct
|
34 |
|
35 |
+
## Use this model
|
36 |
+
|
37 |
+
|
38 |
+
Use a pipeline as a high-level helper:
|
39 |
```python
|
40 |
# Use a pipeline as a high-level helper
|
41 |
from transformers import pipeline
|
|
|
44 |
{"role": "user", "content": "Who are you?"},
|
45 |
]
|
46 |
pipe = pipeline("text-generation", model="meta-llama/Meta-Llama-3.1-8B-Instruct")
|
47 |
+
pipe(messages)
|
48 |
```
|
49 |
|
50 |
+
|
51 |
+
|
52 |
+
Load model directly
|
53 |
```python
|
54 |
+
# Load model directly
|
55 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
56 |
|
57 |
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
|
58 |
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
|
59 |
```
|
60 |
|
61 |
+
The base model information can be found in the original [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
|
62 |
|
63 |
|
64 |
|