qaihm-bot commited on
Commit
4ca2006
·
verified ·
1 Parent(s): b9325a1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -9
README.md CHANGED
@@ -1,16 +1,15 @@
1
  ---
2
  library_name: pytorch
3
- license: llama2
4
  tags:
5
  - llm
6
  - generative_ai
7
- - quantized
8
  - android
9
  pipeline_tag: text-generation
10
 
11
  ---
12
 
13
- ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/llama_v2_7b_chat_quantized/web-assets/model_demo.png)
14
 
15
  # Llama-v2-7B-Chat: Optimized for Mobile Deployment
16
  ## State-of-the-art large language model useful on a variety of language understanding and generation tasks
@@ -21,11 +20,11 @@ Llama 2 is a family of LLMs. The "Chat" at the end indicates that the model is o
21
  This model is an implementation of Llama-v2-7B-Chat found [here](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).
22
 
23
 
24
- More details on model performance across various devices, can be found [here](https://aihub.qualcomm.com/models/llama_v2_7b_chat_quantized).
25
 
26
  ### Model Details
27
 
28
- - **Model Type:** Text generation
29
  - **Model Stats:**
30
  - Input sequence length for Prompt Processor: 1024
31
  - Context length: 1024
@@ -45,11 +44,11 @@ This model is an implementation of Llama-v2-7B-Chat found [here](https://hugging
45
  - TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. For Llama-v2-7B-Chat, both values in the range are the same since prompt length is the full context length (1024 tokens).
46
  - Response Rate: Rate of response generation after the first response token.
47
 
48
- | Model | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
49
  |---|---|---|---|---|---|
50
- | Llama-v2-7B-Chat | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 12.85 | 1.49583 - 1.49583 | -- | -- |
51
- | Llama-v2-7B-Chat | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 11.2 | 1.9189999999999998 - 1.9189999999999998 | -- | -- |
52
- | Llama-v2-7B-Chat | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 17.94 | 1.44 - 1.44 | -- | -- |
53
 
54
  ## Deploying Llama 2 on-device
55
 
 
1
  ---
2
  library_name: pytorch
3
+ license: other
4
  tags:
5
  - llm
6
  - generative_ai
 
7
  - android
8
  pipeline_tag: text-generation
9
 
10
  ---
11
 
12
+ ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/llama_v2_7b_chat/web-assets/model_demo.png)
13
 
14
  # Llama-v2-7B-Chat: Optimized for Mobile Deployment
15
  ## State-of-the-art large language model useful on a variety of language understanding and generation tasks
 
20
  This model is an implementation of Llama-v2-7B-Chat found [here](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).
21
 
22
 
23
+ More details on model performance across various devices, can be found [here](https://aihub.qualcomm.com/models/llama_v2_7b_chat).
24
 
25
  ### Model Details
26
 
27
+ - **Model Type:** Model_use_case.text_generation
28
  - **Model Stats:**
29
  - Input sequence length for Prompt Processor: 1024
30
  - Context length: 1024
 
44
  - TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. For Llama-v2-7B-Chat, both values in the range are the same since prompt length is the full context length (1024 tokens).
45
  - Response Rate: Rate of response generation after the first response token.
46
 
47
+ | Model | Precision | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
48
  |---|---|---|---|---|---|
49
+ | Llama-v2-7B-Chat | w4a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN | 12.85 | 1.49583 - 1.49583 | -- | -- |
50
+ | Llama-v2-7B-Chat | w4a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 11.2 | 1.919 - 1.919 | -- | -- |
51
+ | Llama-v2-7B-Chat | w4a16 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN | 17.94 | 1.44 - 1.44 | -- | -- |
52
 
53
  ## Deploying Llama 2 on-device
54