Update README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,7 @@ Limitation: Due to SmallThinker's current limitations in instruction following,
|
|
21 |
SmallThinker is designed for the following use cases:
|
22 |
|
23 |
1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
|
24 |
-
2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model.
|
25 |
|
26 |
## Limitations & Disclaimer
|
27 |
|
|
|
21 |
SmallThinker is designed for the following use cases:
|
22 |
|
23 |
1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
|
24 |
+
2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model. From my test, in llama.cpp we can get 70% speedup (from 40 tokens/s to 70 tokens/s).
|
25 |
|
26 |
## Limitations & Disclaimer
|
27 |
|