Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -1,16 +1,15 @@
|
|
1 |
---
|
2 |
library_name: pytorch
|
3 |
-
license:
|
4 |
tags:
|
5 |
- llm
|
6 |
- generative_ai
|
7 |
-
- quantized
|
8 |
- android
|
9 |
pipeline_tag: text-generation
|
10 |
|
11 |
---
|
12 |
|
13 |
-
.
|
22 |
|
23 |
|
24 |
-
More details on model performance across various devices, can be found [here](https://aihub.qualcomm.com/models/
|
25 |
|
26 |
### Model Details
|
27 |
|
28 |
-
- **Model Type:**
|
29 |
- **Model Stats:**
|
30 |
- Input sequence length for Prompt Processor: 1024
|
31 |
- Context length: 1024
|
@@ -45,11 +44,11 @@ This model is an implementation of Llama-v2-7B-Chat found [here](https://hugging
|
|
45 |
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. For Llama-v2-7B-Chat, both values in the range are the same since prompt length is the full context length (1024 tokens).
|
46 |
- Response Rate: Rate of response generation after the first response token.
|
47 |
|
48 |
-
| Model | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
49 |
|---|---|---|---|---|---|
|
50 |
-
| Llama-v2-7B-Chat | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 12.85 | 1.49583 - 1.49583 | -- | -- |
|
51 |
-
| Llama-v2-7B-Chat | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 11.2 | 1.
|
52 |
-
| Llama-v2-7B-Chat | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 17.94 | 1.44 - 1.44 | -- | -- |
|
53 |
|
54 |
## Deploying Llama 2 on-device
|
55 |
|
|
|
1 |
---
|
2 |
library_name: pytorch
|
3 |
+
license: other
|
4 |
tags:
|
5 |
- llm
|
6 |
- generative_ai
|
|
|
7 |
- android
|
8 |
pipeline_tag: text-generation
|
9 |
|
10 |
---
|
11 |
|
12 |
+

|
13 |
|
14 |
# Llama-v2-7B-Chat: Optimized for Mobile Deployment
|
15 |
## State-of-the-art large language model useful on a variety of language understanding and generation tasks
|
|
|
20 |
This model is an implementation of Llama-v2-7B-Chat found [here](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).
|
21 |
|
22 |
|
23 |
+
More details on model performance across various devices, can be found [here](https://aihub.qualcomm.com/models/llama_v2_7b_chat).
|
24 |
|
25 |
### Model Details
|
26 |
|
27 |
+
- **Model Type:** Model_use_case.text_generation
|
28 |
- **Model Stats:**
|
29 |
- Input sequence length for Prompt Processor: 1024
|
30 |
- Context length: 1024
|
|
|
44 |
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. For Llama-v2-7B-Chat, both values in the range are the same since prompt length is the full context length (1024 tokens).
|
45 |
- Response Rate: Rate of response generation after the first response token.
|
46 |
|
47 |
+
| Model | Precision | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
48 |
|---|---|---|---|---|---|
|
49 |
+
| Llama-v2-7B-Chat | w4a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN | 12.85 | 1.49583 - 1.49583 | -- | -- |
|
50 |
+
| Llama-v2-7B-Chat | w4a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 11.2 | 1.919 - 1.919 | -- | -- |
|
51 |
+
| Llama-v2-7B-Chat | w4a16 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN | 17.94 | 1.44 - 1.44 | -- | -- |
|
52 |
|
53 |
## Deploying Llama 2 on-device
|
54 |
|