DeepSeek-V4-Flash-0731-REAP-DSpark
This is the complete 0xSero/DeepSeek-V4-Flash-0731-REAP checkpoint with its three DSpark MTP draft blocks (mtp.0/1/2) replaced by the fine-tuned head s3recap-p25-b0.1. It is meant for single-stream speculative decoding with the author's pure-CUDA engine on one NVIDIA Jetson AGX Thor. Measured there, the 8-prompt suite decodes at 28.38 tok/s, against 14.61 tok/s for plain autoregressive decode in the same run (1.94x).
The 43-layer backbone is unchanged. Shards 1–45, config.json, the tokenizer files, REAP_MANIFEST.json, reap_plan.json, LICENSE and model.safetensors.index.json are byte-identical to the base repo (same LFS SHA-256). Only shards 46–48, which hold the MTP blocks, differ.
Model overview
| Upstream model | deepseek-ai/DeepSeek-V4-Flash-0731 @ 9e165c30e2704aec5d9d593cce3eebd58bbef1cb (MIT) |
| Direct base | 0xSero/DeepSeek-V4-Flash-0731-REAP: REAP prune to 160 of 256 routed experts per MoE layer |
| What this repo changes | The DSpark MTP draft blocks only (shards 46–48) |
| Architecture | DeepseekV4ForCausalLM, model_type: deepseek_v4 |
| Layers | 43 backbone MoE layers + 3 chained DSpark MTP blocks tapping layers 40, 41 and 42 |
| Routed experts | 160 per layer (from 256), top-6, plus 1 shared expert |
| Attention | MLA: 64 heads, head_dim 512, Q/O LoRA rank 1024, sliding window 128, DSA indexer (top-512), KV compression ratios 4/128 |
| Hyper-connections | hc_mult 4, 20 Sinkhorn iterations |
| Parameters (from safetensors headers) | ~193.0 B total: ~180.4 B backbone + ~12.6 B MTP blocks. Packed FP4 bytes are counted as two weights each; scale tensors are excluded. |
| Active parameters | ~13.26 B per token, computed from tensor shapes in the engine's docs/ROOFLINE.md (not a measured figure) |
| Precision | Routed experts are OCP MXFP4 (I8-packed E2M1 with F8_E8M0 block-32 scales). Attention, shared expert and dense projections are FP8 E4M3 with 128x128 UE8M0 block scales. Norms, embedding, lm_head and Markov head are BF16 or FP32. |
| Draft block width | 5 (dspark_block_size in config.json) |
| Context | max_position_embeddings 1,048,576 (YaRN, factor 16 over 65,536). 131,072 tokens have been served on Thor; see Hardware and memory. |
| Size on disk | 48 shards, 107,808,354,360 bytes (100.40 GiB). Tensor payload is 107,803,320,952 bytes, 45,821 tensors. |
| Licence | MIT |
Which version should I use
| Repo | What it is | Use it when |
|---|---|---|
| This repo | Full checkpoint with the s3recap-p25-b0.1 head installed |
You want to download one thing and run the CUDA engine (recommended) |
0xSero/DeepSeek-V4-Flash-0731-REAP |
Same backbone, stock (never fine-tuned) DSpark head | You need the original head, or the encoding/ and inference/ reference code, which this repo does not include |
dspark-mtp-draft-head-s3recap-p25-b0.1 |
The installed head as a 1.03 GB BF16 training master, plus 23 other trained heads and their logs | You already have the base checkpoint, or want to compare heads |
deepseek-v4-flash-0731-REAP-calibrated-draft-head |
The earlier s3 head, measured at draft width 6 |
Historical comparison only |
Quickstart
The checkpoint has been run only with patrickbdevaney/deepseek-v4-flash-0731-cuda, a single-request engine that targets sm_110a. The commands below are that repo's documented launch path. The author ran them on a Jetson AGX Thor (CUDA 13.0) against a local checkpoint whose shards are byte-identical to this repo's.
huggingface-cli download patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark --local-dir ~/models/DeepSeek-V4-Flash-0731-REAP-DSpark
git clone https://github.com/patrickbdevaney/deepseek-v4-flash-0731-cuda && cd deepseek-v4-flash-0731-cuda
bash scripts/build_server.sh # unit gates, server, terminal client
CKPT=~/models/DeepSeek-V4-Flash-0731-REAP-DSpark bash scripts/serve.sh # OpenAI-compatible, port 8080
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Write a binary search in Python."}]}'
Run nothing else that loads a model on the same machine: the weights occupy 100.4 GiB of the 122.8 GiB unified pool. Transformers, vLLM and SGLang have not been tested with this checkpoint. The base repo documents a vLLM preview smoke test on DGX Spark that needed a router patch for 160 experts, with DSpark disabled.
Evaluation
All measurements were taken on one Jetson AGX Thor with the weights as shipped. Logs are in the engine repo under evidence/.
Speculative decoding speed
The protocol is the engine's frozen 8-prompt suite, one prompt per category, with at least 200 generated tokens per prompt, greedy decoding and draft width 5. τ is the number of tokens committed per target forward. At width 5 it can reach 6 (five drafted tokens plus the correction token). Throughput is the suite mean.
| τ | tok/s | Base AR in the same run | Source | |
|---|---|---|---|---|
| Stock DSpark head, width 5 | 3.2550 | 23.25 | 13.86 | evidence/run0_tau_blk5.log |
This head (s3recap-p25-b0.1), width 5 |
3.8413 | 28.38 | 14.61 | evidence/s3recap-p25-b0.1_eval.log |
- Against its own run's AR decode, the speed-up is 28.38 / 14.61 = 1.94x. Per-prompt ratios range from 1.44x (
code_gen) to 2.33x (multi_turn). - The stock head was 22.66 tok/s at width 6 on an earlier engine revision (
HEAD_REGISTRY.md). The project quotes the move from 22.66 to 28.38 (+25.3%) as its end-to-end gain. That figure combines the head, the width change and engine work. - On three copy-heavy probe prompts built for a suffix-drafting experiment, this head ran at 32.38, 31.38 and 34.97 tok/s (2.25–2.51x against 13.94 tok/s AR in that run;
evidence/s6_probe.log). - Speculation verifies greedily, so output is the same as plain greedy decode by construction. Each measurement run also checks the first 8 speculative tokens against base AR (
LOSSLESS GATE ... PASS).
Acceptance by category (τ at width 5). The head was selected on the suite mean. That mean hides a trade:
| Category | Stock head | This head | Change |
|---|---|---|---|
| long_context | 5.00 | 4.53 | −0.47 |
| agentic_format | 4.41 | 4.16 | −0.25 |
| code_edit | 4.08 | 4.18 | +0.10 |
| multi_turn | 4.06 | 4.83 | +0.77 |
| short_factual | 3.11 | 3.87 | +0.76 |
| code_gen | 1.86 | 2.59 | +0.73 |
| reasoning | 1.82 | 3.57 | +1.75 |
| explanation | 1.70 | 3.00 | +1.30 |
| Suite mean | 3.2550 | 3.8413 | +18.0% |
Under the project's own release rule, this head fails 3 of 6 category floors: long_context, agentic_format and reasoning (docs/HEAD_REGISTRY.md). It was promoted and deployed on the mean. If your workload is mostly long-context reconstruction, the stock head may draft better.
Task accuracy
These scores measure the pruned checkpoint as served by this engine. They were taken with reasoning_effort=low, temperature 1.0 and top-p 0.95. Brackets are 95% intervals: Wilson for single-sample tasks, cluster bootstrap over problems for AIME and SciCode. A truncated item has no extractable answer and is scored wrong. The draft head does not change these numbers: the target weights are unchanged, and the acceptance rule draws every token from the target. Items were scored under more than one head (docs/EVALS.md).
| Benchmark | Items | Budget | Accuracy | Truncated |
|---|---|---|---|---|
| MATH-500 (subset) | 100 | 8K | 95.0 [88.8, 97.8] | 1 |
| HumanEval | 164 | 8K | 95.1 [90.7, 97.5] | 6 |
| AIME 2024 (30 problems, avg@2) | 60 | 24K | 91.7 [80.0, 100.0] | 6 |
| GPQA Diamond | 198 | 24K | 83.8 [78.1, 88.3] | 19 |
| MMLU-Pro (seeded subset) | 150 | 24K | 74.7 [67.2, 81.0] | 4 |
BFCL v3, 4 exec_* categories, AST |
240 | 2K | 86.2 [81.3, 90.0] | 3 |
BFCL v3, live_* categories, AST |
508 | 2K | 78.7 [75.0, 82.1] | 5 |
LiveCodeBench test6 window |
175 | 8K | 46.9 [39.6, 54.2] | 104 (59%): budget-limited, not a capability number |
| SciCode, subproblem pass rate | 291 | 8K | 30.2 [24.1, 36.9] | 54 (19%): budget-limited |
- GPQA Diamond missed its pre-registered prediction. Before the 24K run, the project predicted 87–93% and set 85% as the falsification line. The run scored 83.8%. Of the 51 items that had hit the 8K cap, 19 still had not terminated at 24K, and the 32 that did terminate scored 78.1%. Items that finished inside 8K scored 93.9% (
dev/archive/EVAL_NEXT_STEPS.md). No matched-harness run of the unpruned model exists, so the effect has not been attributed to pruning. - Budget changes results. At 8K, GPQA Diamond reads 72.7% with 51 items truncated.
- AIME 2025 at 24K is incomplete (54 of 60 items) and is not reported.
- Not measured: the unpruned model under the same harness, long-context benchmarks, multi-turn agentic benchmarks, and any perplexity or agreement comparison against the unpruned model.
Hardware and memory
Everything was measured on one NVIDIA Jetson AGX Thor (sm_110a, 20 SMs, 122.8 GiB unified memory, CUDA 13.0, 240 GB/s measured streaming bandwidth). The server processes one request at a time.
| Measured | |
|---|---|
| Weights resident | 100.4 GiB, mapped zero-copy |
| Server ready, 32,768 context, FP32 KV rows (default) | 119.9 / 122.8 GiB |
| Server ready, 131,072 context, packed KV (FP8 + UE8M0, 720 B/row instead of 2,048) | 117.0 / 122.8 GiB |
| 262,144 context, packed | Allocates at 121.8 / 122.8 GiB. That leaves too little headroom to serve, so it is not supported. |
| Prefill | 108.1 tok/s (bit-exact against the earlier path) |
| KV packing cost | About 12% of prefill (107.9 to 94.8 tok/s). Tokens are identical; packing turns on automatically above 32,768. |
The decode budget is dominated by bytes per token. The engine's roofline computes 11.20 GB read per decoded token. MLA projections account for 41.1% of that and routed experts for 30.8%. At 200 GB/s this gives a computed AR ceiling of 17.85 tok/s (docs/ROOFLINE.md). That ceiling is an estimate, not a measurement. Measured AR decode is 14.61 tok/s on the suite.
How it was made
- Pruning, by 0xSero. REAP expert rankings were transferred from an observation run on the earlier
deepseek-ai/DeepSeek-V4-Flashrevision60d8d707(21,289 rows across coding, math, science, tool calling and agentic trajectories). Router alignment was checked against0731: every old expert's nearest current row had the same ID, and the minimum layer-mean cosine was 0.9924. Then 160 of 256 experts were kept in all 46 MoE scopes, including the three MTP blocks. Details are inreap_plan.json,REAP_MANIFEST.jsonand the base card. - Draft-head fine-tune, by the author. Training covered the non-expert tensors of the three MTP blocks: 72 tensors, 517,318,215 parameters, covering attention, router, shared expert, hyper-connections,
main_proj, Markov and confidence heads. Routed experts were copied untouched. The data was one epoch over 1,472 sequences, with 64 held out, generated by the base model itself from 1,536 prompts balanced over 8 categories; theagentic_formatprompts came from BFCL. The loss mixed cross-entropy and total variation at 0.1/0.9, deficit weighting was clamped at 3.0, a KL anchor with β = 0.1 pulled toward the pre-training head, and the peak learning rate was 5e-5. Full recipe:docs/BEST_SETUP.md. - Install. The BF16 master was re-quantised to the checkpoint's own formats, FP8 E4M3 with freshly computed power-of-two E8M0 block scales, by
tools/build_trained_head.py. That script refuses any tensor whose round-trip error exceeds 0.10. The result was written into shards 46–48: 2,977 MTP tensors, 7,010,423,612 bytes (6.53 GiB).
Limitations
- One machine, one engine. All speed and accuracy numbers come from one Jetson AGX Thor running the custom CUDA engine. No other runtime has been tested.
- Single stream. The engine has no batching across clients.
- Head is checkpoint-specific. It was trained against this 160-expert REAP and is not expected to transfer to another pruning or quantisation.
- Held-out drafting. Trained heads in this programme win the frozen suite but have lost to the untrained head on held-out continuation drafting against a paired control (
wiki/draft-head-finetuning.md). The β anchor was added to address this. The effect is not shown to be eliminated. - Pruned model. 37.5% of routed experts are removed and rankings were transferred, not re-observed on
0731. Accuracy versus the unpruned model is unmeasured under a matched harness. SHA256SUMSwas replaced on 2026-11-06. The earlier file was the base repo's: it listed 21 files this repo does not contain and base-checkpoint hashes for shards 46–48. The new file covers every file in this repo exceptREADME.md(which changes with card edits), using the Hub's LFS SHA-256 for large files. Shards 46–48 match the engine'shead_card.json(790139019c60…,335d72f9624f…,80f11aa77095…). Check a download withsha256sum -c SHA256SUMS.- No chat-template code here. This repo omits the base repo's
encoding/directory. The CUDA engine has its own encoder; other runtimes should takeencoding/from the base repo.
Changelog and corrections
- 2026-08-27: first upload, with the full 48-shard checkpoint and the
s3recap-p25-b0.1head in shards 46–48. - 2026-11-06: card rewritten and verified against repo files and build artifacts; a 2026-10-26 automated rewrite contained errors.
License and attribution
MIT, following the upstream and base repos. The DeepSeek copyright notice is retained in LICENSE.
- Upstream model: DeepSeek,
deepseek-ai/DeepSeek-V4-Flash-0731, MIT. - REAP pruning: 0xSero,
0xSero/DeepSeek-V4-Flash-0731-REAP, MIT. - REAP method: Lasby et al., 2025, arXiv:2510.13999.
- This repo adds only the fine-tuned draft-head tensors in shards 46–48.
Citation
@misc{devaney2026dsv4dspark,
title = {DeepSeek-V4-Flash-0731-REAP-DSpark: a fine-tuned DSpark draft head for lossless speculative decoding on Jetson AGX Thor},
author = {Devaney, Patrick},
year = {2026},
url = {https://huggingface.co/patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark}
}
- Downloads last month
- 604
Model tree for patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark
Base model
deepseek-ai/DeepSeek-V4-Flash-0731