Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -1,10 +1,9 @@
|
|
1 |
---
|
2 |
library_name: pytorch
|
3 |
-
license:
|
4 |
tags:
|
5 |
- llm
|
6 |
- generative_ai
|
7 |
-
- quantized
|
8 |
- android
|
9 |
pipeline_tag: text-generation
|
10 |
|
@@ -25,7 +24,7 @@ This model is an implementation of IBM-Granite-v3.1-8B-Instruct found [here](htt
|
|
25 |
|
26 |
### Model Details
|
27 |
|
28 |
-
- **Model Type:**
|
29 |
- **Model Stats:**
|
30 |
- Input sequence length for Prompt Processor: 128
|
31 |
- Context length: 4096
|
@@ -50,10 +49,10 @@ This model is an implementation of IBM-Granite-v3.1-8B-Instruct found [here](htt
|
|
50 |
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (2048 tokens).
|
51 |
- Response Rate: Rate of response generation after the first response token.
|
52 |
|
53 |
-
| Model | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
54 |
|---|---|---|---|---|---|
|
55 |
-
| IBM-Granite-v3.1-8B-Instruct | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 11.01293 | 0.19679249999999998 - 6.297359999999999 | -- | -- |
|
56 |
-
| IBM-Granite-v3.1-8B-Instruct | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 8.01724 | 0.2953902 - 9.4524864 | -- | -- |
|
57 |
|
58 |
## Deploying IBM Granite 3.1 on-device
|
59 |
|
|
|
1 |
---
|
2 |
library_name: pytorch
|
3 |
+
license: other
|
4 |
tags:
|
5 |
- llm
|
6 |
- generative_ai
|
|
|
7 |
- android
|
8 |
pipeline_tag: text-generation
|
9 |
|
|
|
24 |
|
25 |
### Model Details
|
26 |
|
27 |
+
- **Model Type:** Model_use_case.text_generation
|
28 |
- **Model Stats:**
|
29 |
- Input sequence length for Prompt Processor: 128
|
30 |
- Context length: 4096
|
|
|
49 |
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (2048 tokens).
|
50 |
- Response Rate: Rate of response generation after the first response token.
|
51 |
|
52 |
+
| Model | Precision | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
53 |
|---|---|---|---|---|---|
|
54 |
+
| IBM-Granite-v3.1-8B-Instruct | w4a16 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN | 11.01293 | 0.19679249999999998 - 6.297359999999999 | -- | -- |
|
55 |
+
| IBM-Granite-v3.1-8B-Instruct | w4a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 8.01724 | 0.2953902 - 9.4524864 | -- | -- |
|
56 |
|
57 |
## Deploying IBM Granite 3.1 on-device
|
58 |
|