Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -4,7 +4,6 @@ license: other
|
|
| 4 |
tags:
|
| 5 |
- llm
|
| 6 |
- generative_ai
|
| 7 |
-
- quantized
|
| 8 |
- android
|
| 9 |
pipeline_tag: text-generation
|
| 10 |
|
|
@@ -25,7 +24,7 @@ Please contact us to purchase this model. More details on model performance acro
|
|
| 25 |
|
| 26 |
### Model Details
|
| 27 |
|
| 28 |
-
- **Model Type:**
|
| 29 |
- **Model Stats:**
|
| 30 |
- Input sequence length for Prompt Processor: 128
|
| 31 |
- Max context length: 4096
|
|
@@ -37,9 +36,9 @@ Please contact us to purchase this model. More details on model performance acro
|
|
| 37 |
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
|
| 38 |
- Response Rate: Rate of response generation after the first response token.
|
| 39 |
|
| 40 |
-
| Model | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
| 41 |
|---|---|---|---|---|---|
|
| 42 |
-
| Ministral-3B | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 18.79867 | 0.
|
| 43 |
|
| 44 |
## Deploying Ministral 3B on-device
|
| 45 |
|
|
|
|
| 4 |
tags:
|
| 5 |
- llm
|
| 6 |
- generative_ai
|
|
|
|
| 7 |
- android
|
| 8 |
pipeline_tag: text-generation
|
| 9 |
|
|
|
|
| 24 |
|
| 25 |
### Model Details
|
| 26 |
|
| 27 |
+
- **Model Type:** Model_use_case.text_generation
|
| 28 |
- **Model Stats:**
|
| 29 |
- Input sequence length for Prompt Processor: 128
|
| 30 |
- Max context length: 4096
|
|
|
|
| 36 |
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
|
| 37 |
- Response Rate: Rate of response generation after the first response token.
|
| 38 |
|
| 39 |
+
| Model | Precision | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
| 40 |
|---|---|---|---|---|---|
|
| 41 |
+
| Ministral-3B | w8a8 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN | 18.79867 | 0.10237900000000001 - 3.2761280000000004 | -- | -- |
|
| 42 |
|
| 43 |
## Deploying Ministral 3B on-device
|
| 44 |
|