DeepSeek-V4-Flash-0731-REAP-DSpark

This is the complete 0xSero/DeepSeek-V4-Flash-0731-REAP checkpoint with its three DSpark MTP draft blocks (mtp.0/1/2) replaced by the fine-tuned head s3recap-p25-b0.1. It is meant for single-stream speculative decoding with the author's pure-CUDA engine on one NVIDIA Jetson AGX Thor. Measured there, the 8-prompt suite decodes at 28.38 tok/s, against 14.61 tok/s for plain autoregressive decode in the same run (1.94x).

The 43-layer backbone is unchanged. Shards 1–45, config.json, the tokenizer files, REAP_MANIFEST.json, reap_plan.json, LICENSE and model.safetensors.index.json are byte-identical to the base repo (same LFS SHA-256). Only shards 46–48, which hold the MTP blocks, differ.

Model overview

Upstream model deepseek-ai/DeepSeek-V4-Flash-0731 @ 9e165c30e2704aec5d9d593cce3eebd58bbef1cb (MIT)
Direct base 0xSero/DeepSeek-V4-Flash-0731-REAP: REAP prune to 160 of 256 routed experts per MoE layer
What this repo changes The DSpark MTP draft blocks only (shards 46–48)
Architecture DeepseekV4ForCausalLM, model_type: deepseek_v4
Layers 43 backbone MoE layers + 3 chained DSpark MTP blocks tapping layers 40, 41 and 42
Routed experts 160 per layer (from 256), top-6, plus 1 shared expert
Attention MLA: 64 heads, head_dim 512, Q/O LoRA rank 1024, sliding window 128, DSA indexer (top-512), KV compression ratios 4/128
Hyper-connections hc_mult 4, 20 Sinkhorn iterations
Parameters (from safetensors headers) ~193.0 B total: ~180.4 B backbone + ~12.6 B MTP blocks. Packed FP4 bytes are counted as two weights each; scale tensors are excluded.
Active parameters ~13.26 B per token, computed from tensor shapes in the engine's docs/ROOFLINE.md (not a measured figure)
Precision Routed experts are OCP MXFP4 (I8-packed E2M1 with F8_E8M0 block-32 scales). Attention, shared expert and dense projections are FP8 E4M3 with 128x128 UE8M0 block scales. Norms, embedding, lm_head and Markov head are BF16 or FP32.
Draft block width 5 (dspark_block_size in config.json)
Context max_position_embeddings 1,048,576 (YaRN, factor 16 over 65,536). 131,072 tokens have been served on Thor; see Hardware and memory.
Size on disk 48 shards, 107,808,354,360 bytes (100.40 GiB). Tensor payload is 107,803,320,952 bytes, 45,821 tensors.
Licence MIT

Which version should I use

Repo What it is Use it when
This repo Full checkpoint with the s3recap-p25-b0.1 head installed You want to download one thing and run the CUDA engine (recommended)
0xSero/DeepSeek-V4-Flash-0731-REAP Same backbone, stock (never fine-tuned) DSpark head You need the original head, or the encoding/ and inference/ reference code, which this repo does not include
dspark-mtp-draft-head-s3recap-p25-b0.1 The installed head as a 1.03 GB BF16 training master, plus 23 other trained heads and their logs You already have the base checkpoint, or want to compare heads
deepseek-v4-flash-0731-REAP-calibrated-draft-head The earlier s3 head, measured at draft width 6 Historical comparison only

Quickstart

The checkpoint has been run only with patrickbdevaney/deepseek-v4-flash-0731-cuda, a single-request engine that targets sm_110a. The commands below are that repo's documented launch path. The author ran them on a Jetson AGX Thor (CUDA 13.0) against a local checkpoint whose shards are byte-identical to this repo's.

huggingface-cli download patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark --local-dir ~/models/DeepSeek-V4-Flash-0731-REAP-DSpark

git clone https://github.com/patrickbdevaney/deepseek-v4-flash-0731-cuda && cd deepseek-v4-flash-0731-cuda
bash scripts/build_server.sh                     # unit gates, server, terminal client
CKPT=~/models/DeepSeek-V4-Flash-0731-REAP-DSpark bash scripts/serve.sh   # OpenAI-compatible, port 8080
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Write a binary search in Python."}]}'

Run nothing else that loads a model on the same machine: the weights occupy 100.4 GiB of the 122.8 GiB unified pool. Transformers, vLLM and SGLang have not been tested with this checkpoint. The base repo documents a vLLM preview smoke test on DGX Spark that needed a router patch for 160 experts, with DSpark disabled.

Evaluation

All measurements were taken on one Jetson AGX Thor with the weights as shipped. Logs are in the engine repo under evidence/.

Speculative decoding speed

The protocol is the engine's frozen 8-prompt suite, one prompt per category, with at least 200 generated tokens per prompt, greedy decoding and draft width 5. τ is the number of tokens committed per target forward. At width 5 it can reach 6 (five drafted tokens plus the correction token). Throughput is the suite mean.

τ tok/s Base AR in the same run Source
Stock DSpark head, width 5 3.2550 23.25 13.86 evidence/run0_tau_blk5.log
This head (s3recap-p25-b0.1), width 5 3.8413 28.38 14.61 evidence/s3recap-p25-b0.1_eval.log
  • Against its own run's AR decode, the speed-up is 28.38 / 14.61 = 1.94x. Per-prompt ratios range from 1.44x (code_gen) to 2.33x (multi_turn).
  • The stock head was 22.66 tok/s at width 6 on an earlier engine revision (HEAD_REGISTRY.md). The project quotes the move from 22.66 to 28.38 (+25.3%) as its end-to-end gain. That figure combines the head, the width change and engine work.
  • On three copy-heavy probe prompts built for a suffix-drafting experiment, this head ran at 32.38, 31.38 and 34.97 tok/s (2.25–2.51x against 13.94 tok/s AR in that run; evidence/s6_probe.log).
  • Speculation verifies greedily, so output is the same as plain greedy decode by construction. Each measurement run also checks the first 8 speculative tokens against base AR (LOSSLESS GATE ... PASS).

Acceptance by category (τ at width 5). The head was selected on the suite mean. That mean hides a trade:

Category Stock head This head Change
long_context 5.00 4.53 −0.47
agentic_format 4.41 4.16 −0.25
code_edit 4.08 4.18 +0.10
multi_turn 4.06 4.83 +0.77
short_factual 3.11 3.87 +0.76
code_gen 1.86 2.59 +0.73
reasoning 1.82 3.57 +1.75
explanation 1.70 3.00 +1.30
Suite mean 3.2550 3.8413 +18.0%

Under the project's own release rule, this head fails 3 of 6 category floors: long_context, agentic_format and reasoning (docs/HEAD_REGISTRY.md). It was promoted and deployed on the mean. If your workload is mostly long-context reconstruction, the stock head may draft better.

Task accuracy

These scores measure the pruned checkpoint as served by this engine. They were taken with reasoning_effort=low, temperature 1.0 and top-p 0.95. Brackets are 95% intervals: Wilson for single-sample tasks, cluster bootstrap over problems for AIME and SciCode. A truncated item has no extractable answer and is scored wrong. The draft head does not change these numbers: the target weights are unchanged, and the acceptance rule draws every token from the target. Items were scored under more than one head (docs/EVALS.md).

Benchmark Items Budget Accuracy Truncated
MATH-500 (subset) 100 8K 95.0 [88.8, 97.8] 1
HumanEval 164 8K 95.1 [90.7, 97.5] 6
AIME 2024 (30 problems, avg@2) 60 24K 91.7 [80.0, 100.0] 6
GPQA Diamond 198 24K 83.8 [78.1, 88.3] 19
MMLU-Pro (seeded subset) 150 24K 74.7 [67.2, 81.0] 4
BFCL v3, 4 exec_* categories, AST 240 2K 86.2 [81.3, 90.0] 3
BFCL v3, live_* categories, AST 508 2K 78.7 [75.0, 82.1] 5
LiveCodeBench test6 window 175 8K 46.9 [39.6, 54.2] 104 (59%): budget-limited, not a capability number
SciCode, subproblem pass rate 291 8K 30.2 [24.1, 36.9] 54 (19%): budget-limited
  • GPQA Diamond missed its pre-registered prediction. Before the 24K run, the project predicted 87–93% and set 85% as the falsification line. The run scored 83.8%. Of the 51 items that had hit the 8K cap, 19 still had not terminated at 24K, and the 32 that did terminate scored 78.1%. Items that finished inside 8K scored 93.9% (dev/archive/EVAL_NEXT_STEPS.md). No matched-harness run of the unpruned model exists, so the effect has not been attributed to pruning.
  • Budget changes results. At 8K, GPQA Diamond reads 72.7% with 51 items truncated.
  • AIME 2025 at 24K is incomplete (54 of 60 items) and is not reported.
  • Not measured: the unpruned model under the same harness, long-context benchmarks, multi-turn agentic benchmarks, and any perplexity or agreement comparison against the unpruned model.

Hardware and memory

Everything was measured on one NVIDIA Jetson AGX Thor (sm_110a, 20 SMs, 122.8 GiB unified memory, CUDA 13.0, 240 GB/s measured streaming bandwidth). The server processes one request at a time.

Measured
Weights resident 100.4 GiB, mapped zero-copy
Server ready, 32,768 context, FP32 KV rows (default) 119.9 / 122.8 GiB
Server ready, 131,072 context, packed KV (FP8 + UE8M0, 720 B/row instead of 2,048) 117.0 / 122.8 GiB
262,144 context, packed Allocates at 121.8 / 122.8 GiB. That leaves too little headroom to serve, so it is not supported.
Prefill 108.1 tok/s (bit-exact against the earlier path)
KV packing cost About 12% of prefill (107.9 to 94.8 tok/s). Tokens are identical; packing turns on automatically above 32,768.

The decode budget is dominated by bytes per token. The engine's roofline computes 11.20 GB read per decoded token. MLA projections account for 41.1% of that and routed experts for 30.8%. At 200 GB/s this gives a computed AR ceiling of 17.85 tok/s (docs/ROOFLINE.md). That ceiling is an estimate, not a measurement. Measured AR decode is 14.61 tok/s on the suite.

How it was made

  1. Pruning, by 0xSero. REAP expert rankings were transferred from an observation run on the earlier deepseek-ai/DeepSeek-V4-Flash revision 60d8d707 (21,289 rows across coding, math, science, tool calling and agentic trajectories). Router alignment was checked against 0731: every old expert's nearest current row had the same ID, and the minimum layer-mean cosine was 0.9924. Then 160 of 256 experts were kept in all 46 MoE scopes, including the three MTP blocks. Details are in reap_plan.json, REAP_MANIFEST.json and the base card.
  2. Draft-head fine-tune, by the author. Training covered the non-expert tensors of the three MTP blocks: 72 tensors, 517,318,215 parameters, covering attention, router, shared expert, hyper-connections, main_proj, Markov and confidence heads. Routed experts were copied untouched. The data was one epoch over 1,472 sequences, with 64 held out, generated by the base model itself from 1,536 prompts balanced over 8 categories; the agentic_format prompts came from BFCL. The loss mixed cross-entropy and total variation at 0.1/0.9, deficit weighting was clamped at 3.0, a KL anchor with β = 0.1 pulled toward the pre-training head, and the peak learning rate was 5e-5. Full recipe: docs/BEST_SETUP.md.
  3. Install. The BF16 master was re-quantised to the checkpoint's own formats, FP8 E4M3 with freshly computed power-of-two E8M0 block scales, by tools/build_trained_head.py. That script refuses any tensor whose round-trip error exceeds 0.10. The result was written into shards 46–48: 2,977 MTP tensors, 7,010,423,612 bytes (6.53 GiB).

Limitations

  • One machine, one engine. All speed and accuracy numbers come from one Jetson AGX Thor running the custom CUDA engine. No other runtime has been tested.
  • Single stream. The engine has no batching across clients.
  • Head is checkpoint-specific. It was trained against this 160-expert REAP and is not expected to transfer to another pruning or quantisation.
  • Held-out drafting. Trained heads in this programme win the frozen suite but have lost to the untrained head on held-out continuation drafting against a paired control (wiki/draft-head-finetuning.md). The β anchor was added to address this. The effect is not shown to be eliminated.
  • Pruned model. 37.5% of routed experts are removed and rankings were transferred, not re-observed on 0731. Accuracy versus the unpruned model is unmeasured under a matched harness.
  • SHA256SUMS was replaced on 2026-11-06. The earlier file was the base repo's: it listed 21 files this repo does not contain and base-checkpoint hashes for shards 46–48. The new file covers every file in this repo except README.md (which changes with card edits), using the Hub's LFS SHA-256 for large files. Shards 46–48 match the engine's head_card.json (790139019c60…, 335d72f9624f…, 80f11aa77095…). Check a download with sha256sum -c SHA256SUMS.
  • No chat-template code here. This repo omits the base repo's encoding/ directory. The CUDA engine has its own encoder; other runtimes should take encoding/ from the base repo.

Changelog and corrections

  • 2026-08-27: first upload, with the full 48-shard checkpoint and the s3recap-p25-b0.1 head in shards 46–48.
  • 2026-11-06: card rewritten and verified against repo files and build artifacts; a 2026-10-26 automated rewrite contained errors.

License and attribution

MIT, following the upstream and base repos. The DeepSeek copyright notice is retained in LICENSE.

Citation

@misc{devaney2026dsv4dspark,
  title  = {DeepSeek-V4-Flash-0731-REAP-DSpark: a fine-tuned DSpark draft head for lossless speculative decoding on Jetson AGX Thor},
  author = {Devaney, Patrick},
  year   = {2026},
  url    = {https://huggingface.co/patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark}
}
Downloads last month
604
Safetensors
Model size
193B params
Tensor type
BF16
·
F32
·
I64
·
F8_E4M3
·
I8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark

Finetuned
(1)
this model

Paper for patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark