yixinsong commited on
Commit
b71f3e0
·
verified ·
1 Parent(s): 011743e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -21,7 +21,7 @@ Limitation: Due to SmallThinker's current limitations in instruction following,
21
  SmallThinker is designed for the following use cases:
22
 
23
  1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
24
- 2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model.
25
 
26
  ## Limitations & Disclaimer
27
 
 
21
  SmallThinker is designed for the following use cases:
22
 
23
  1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
24
+ 2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model. From my test, in llama.cpp we can get 70% speedup (from 40 tokens/s to 70 tokens/s).
25
 
26
  ## Limitations & Disclaimer
27