Kimi K3 REAP-720 MXFP4

Repository: anuppillai/Kimi-K3-REAP-720-MXFP4.

This is an unofficial community derivative of Moonshot AI's Kimi K3 MXFP4 checkpoint. It removes 176 of the 896 routed experts in every MoE layer using router-weighted expert activation pruning (REAP), retaining 720 experts per layer. The checkpoint remains natively multimodal: the vision tower, multimodal projector, attention layers, shared experts, tokenizer, and processor were not pruned or requantized.

Documentation-only update. This revision corrects the model card and evaluation metadata. No model weights or configuration files are changed: all 96 safetensors shards (1,276,563,666,168 bytes total; 1,188.89 GiB) remain as previously published.

Evaluation verdict (current)

Quality gate: NOT VALIDATED.

The requested quality gate requires ≥90% of Full K3, measured separately on each of four dimensions — reasoning, vision, coding, and tool use. As of 2026-08-01, this checkpoint cannot be claimed to meet that gate: only the reasoning dimension has a verified score, and its comparison is unpaired (Full K3 was not run through the same prompt/choice/seed/serving harness, so it is not a causal paired pruning estimate). The vision dimension has an invalidated run with no valid final score; coding and tool use have no protocol-matched scores, only compatibility evidence.

Do not interpret publication of this repository as a claim that the model meets the four-dimension floor. Evaluation is currently paused and incomplete; this card should be revised if valid additional evidence is produced.

What changed

REAP saliency was accumulated per expert as sum(router_probability * l2_norm(expert_output)). Calibration streamed one MoE layer at a time as balanced multidomain text plus 16 image-text samples. Each of the 92 scored MoE layers retained its 720 highest-saliency routed experts (a 0.1964 removed fraction). Router rows and expert-correction-bias entries were pruned and renumbered consistently. No retained expert was missing activation coverage in the completed saliency run (retained_experts_never_activated = 0); retained weights remain in mxfp4-pack-quantized format.

This is structural expert pruning, not a newly trained or distilled checkpoint. It should reduce resident weight memory; because K3 still routes to 16 experts per token, it does not by itself reduce the number of active experts used for each token.

Evaluation

Reasoning — verified result, unpaired comparison

The GPQA-Diamond run used the full 198-item test split and scored 155/198 = 78.28%, with 0 infrastructure errors and 0 unscorable outputs; 3 items required a retry. This is a single run (runs = 1), not an average over repeated runs.

Protocol (from evaluation/gpqa_diamond_full.json):

• Dataset: hendrydong/gpqa_diamond_mc, config default, split test, expected items 198; completed items 198, resumed items 0.
• "GPQA-Diamond; source order preserved; structural-first schemas shuffled".
• Settings: temperature 1.0, top-p 0.95, reasoning_effort="max", max_tokens 32,768, timeout 900 s.
• official: true; protocol fingerprint 1654304ead05affa3c6ee2549ebebf100470c0f5c9c3481dc46790f241cb049f.

The published Full-K3 GPQA-Diamond reference is 93.5%, giving 83.72% retention. That is below the requested ≥90% retention floor. However, the comparison is unpaired: Full K3 was not run through the same prompt/choice/seed/serving harness, so this is not a causal paired pruning estimate — it is a verified REAP-720 measurement compared against a published number, not a controlled A/B. Machine-readable aggregate and per-item records are included as evaluation/gpqa_diamond_full.json and evaluation/gpqa_diamond_full_records.jsonl.

Vision — invalidated run; no valid final score

There is no valid vision score. A full MMMU-Pro attempt was run but invalidated and excluded from baseline comparison; do not report its 195/246 raw count as a benchmark score.

Invalidated run details (operator-verified, from the correction ledger):

Machine-readable audit disclosure: evaluation/mmmu_pro_invalidated_run.json.

A separate five-item evaluator smoke at 80% exists, but it is diagnostic only — it validates the evaluator and end-to-end generation path, not a floor estimate. It is too small to establish the vision floor. Published MMMU-Pro Full-K3 reference: 81.6%.

Coding — no protocol-matched score

There is no DeepSWE score (final_deepswe_score = null; status: NO PROTOCOL-MATCHED SCORE). An earlier compatibility check found that three generated-code tasks passed the same isolated graders for both REAP-720 and the existing Q2 K3 deployment. A newer same-prompt comparison against K3 via the Moonshot API is summarized below. These are qualitative generation checks, not DeepSWE retention. Published Full-K3 DeepSWE reference: 67.5 (run on the Kimi Code harness, whose exact historical harness revision, task snapshot, seeds, and trajectories are not all public).

Tool use — no terminal score

There is no Toolathlon-Verified score (final_toolathlon_score = null; status: NO TERMINAL SCORE). An official private-mode Toolathlon client (repository hkust-nlp/Toolathlon, commit 2aed2468858f15818acafa178518390cc4b0f5cb) connected its remote execution service to the local OpenAI-compatible endpoint. The selected find-alita-paper task did not reach a terminal score: two long model turns exhausted their output limit, and the job was cancelled after approximately 20 minutes (to free the single-sequence server for GPQA and MMMU-Pro runs). This is compatibility evidence only, not a Toolathlon score or a Full-K3 comparison. Published Full-K3 Toolathlon-Verified reference: 76.5 (full run needs 108 stateful task environments plus external service accounts; the historical K3 trajectories are gated and the immutable configuration behind 76.5 is not public).

Attempt record (evaluation/toolathlon_compatibility.json):

Accordingly, no coding/tool retention ratio is reported and no four-dimension pass claim is made.

Benchmark comparability limits

The published Full-K3 references used for the requested floors are 93.5 (GPQA-Diamond), 81.6 (MMMU-Pro), 67.5 (DeepSWE), and 76.5 (Toolathlon-Verified). The coding and tool-use references cannot honestly be compared with the lightweight evidence above: the historical harnesses, task snapshots, seeds, and trajectories behind the DeepSWE and Toolathlon-Verified numbers are not fully public.

Qualitative paired generation spot checks — not formal benchmarks

The same strict prompts were run once through the live REAP-720 deployment and K3 via the Moonshot API. Outputs were saved and tested locally. These single-generation comparisons are useful for examining practical strengths and failure modes, but they are not statistically powered benchmarks, protocol-matched retention measurements, or evidence that the four-dimension quality gate passed.

2D aquarium comparison

_{Figure: Static matched aftermath frame at approximately 10 seconds. Left animation: generated by K3 via the Moonshot API. Right animation: generated by Kimi-K3-REAP-720-MXFP4 (720/896 routed experts). Both WebGL animations were generated from the same prompt.}

What was encouraging about REAP-720 in these checks:

• It retained fully runnable Python generation across data structures, concurrency, edge cases, and strict output formatting after 19.64% routed-expert removal.
• Its aquarium was the more ambitious desktop illustration, with notably better fish, plants, water treatment, rotating triangular glass fragments, and settled debris.
• Its aquarium implementation was cleaner at runtime than the API sample and narrowly won the combined review despite losing the visual-fidelity subscore.

Known REAP aquarium limitations remain: Reset does not fully restore the ambient clock, fish, and bubbles; a live resize does not refresh the renderer dimensions/pixel ratio; the crack can escape the glass; and the mobile composition is weaker. These observations do not change the QUALITY GATE NOT VALIDATED verdict above.

Verified vLLM serving configuration

The checkpoint was loaded successfully on one 8× NVIDIA B200 node with the official vllm/vllm-openai:kimi-k3 image. SGLang is not validated for this derivative; only vLLM is claimed.

Short-context configuration (verified boot)

VLLM_ALLREDUCE_USE_FLASHINFER=0 vllm serve /model \
  --served-model-name kimi-k3-reap720 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --disable-custom-all-reduce \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3 \
  --max-model-len 8192 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.92 \
  --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[1],"pass_config":{"fuse_allreduce_rms":false}}'

Observed steady memory was about 168.99 GiB per GPU. A coherent arithmetic probe and multimodal preprocessing path passed. Standard safetensors loading was used; the fastsafetensors loader exceeded available staging memory on this single-node configuration.

Concurrent throughput profile (32K context, 8 sequences — verified)

A separate concurrent-throughput profile was verified with eight concurrent real MMMU image requests. Relative to the short-context command above, the differing settings were:

  --max-model-len 32768 \
  --max-num-seqs 8 \
  --kv-cache-dtype fp8_e4m3 \
  --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[1,2,4,8],"pass_config":{"fuse_allreduce_rms":false}}'

Measured result over a 20-second window: 9,420 generation tokens → 471.0 aggregate generation tokens/second, with 8 requests running, 0 waiting, and 0 server errors. This is aggregate concurrent throughput, not single-request decode speed.

Single-sequence mixed workload (verified)

With --max-num-seqs 1, the live vLLM Prometheus counter vllm:inter_token_latency_seconds recorded 124,017 inter-token intervals totaling 1,585.267 s across the 262K validation and GPQA workload: 12.783 ms mean inter-token latency, or 78.23 decode tokens/s. This is a Prometheus aggregate over mixed successful requests (including the 260K-context validation and the official-protocol GPQA evaluation), not a synthetic fixed-length throughput suite. The machine-readable counter snapshot is evaluation/throughput_262k.json.

Serving configuration for this measurement: tensor-parallel-size 8, max_model_len 262144, max_num_seqs 1, max_num_batched_tokens 2048, gpu_memory_utilization 0.95, kv_cache_dtype fp8_e4m3, mla_prefill_backend FLASHINFER, prefill-query quantization enabled, cudagraph_mode FULL_DECODE_ONLY.

Long context — 262,144 tokens (verified)

The same checkpoint passed a 262,144-token FP8-KV boot and a near-limit end-to-end request. Relative to the short-context command, the verified long-context settings are:

  --max-model-len 262144 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.95 \
  --kv-cache-dtype fp8_e4m3 \
  --attention-config '{"use_prefill_query_quantization":true,"mla_prefill_backend":"FLASHINFER"}'

The request contained 260,095 API-counted prompt tokens (260,094 locally counted) plus 53 completion tokens and finished normally in 54.78 s with finish_reason="stop", returning READY. Machine-readable evidence: evaluation/long_context_262k.json.

For this B200 image, forcing the FLASHINFER MLA prefill backend is required for FP8 prefill-query quantization; the auto-selected FlashAttention MLA prefill backend does not support that path.

Provenance and reproducibility

Revision caveat. The derivative config contains no embedded commit hash or source revision. Do not claim the build source is pinned to the observed public parent revision above; source_revision_verified is false. That revision is recorded only as the public parent state observed on 2026-08-01.

The pruning procedure is recorded in REAP_PRUNING.json (schema version 1, created_utc 2026-08-01), including SHA-256 hashes of the scripts and reference baselines for reproducibility:

The published reap_tools/eval_reap.py bytes hash to the value shown above, and that value matches REAP_PRUNING.json. A later unpublished local experiment increased the vision output budget to 32,768 tokens, so it has different evaluator bytes; no result from that modified evaluator is represented as a completed benchmark in this repository.

Recovered expert keep map

The exact retained-expert map is published as:

• provenance/kimi_k3_reap720_keep_map.json — human-readable artifact with per-layer new→old and old→new mappings, layer hashes, shard names, and validation statistics.
• provenance/kimi_k3_reap720_keep_map.bin — canonical row-major little-endian uint16 map with shape [92, 720].
• provenance/keepmap_validation.json — independent structural validation summary.

Recovery method: for each of the 92 MoE layers, every retained BF16 router row in gate.weight was matched byte-exactly against the 896 parent rows at the observed public parent revision. All 66,240 retained rows had exactly one parent match, were strictly ascending, and were injective. The corresponding mapped F32 e_score_correction_bias entries were then verified byte-for-byte for all 66,240 mappings. This establishes an empirical match to public parent revision 9f62e4e9fffbd0a83ddd60e1c209d828994b3569; it does not establish build-time source attestation, so source_revision_verified remains false.

Intended use and limitations

Use this checkpoint for research and deployment experiments where preserving K3's multimodal architecture while reducing resident weight storage is useful. Expert removal can change routing behavior and may degrade particular languages, domains, visual tasks, long-context behavior, or agent trajectories even when short tests look normal. Run task-specific evaluation before production use.

This repository does not claim safety improvements, alignment changes, or a new training procedure. The upstream Kimi K3 model limitations continue to apply.

License and attribution

The checkpoint and copied upstream code are provided under the Kimi K3 License. This derivative retains Moonshot AI's copyright and license notice. Users are responsible for reviewing the license, including its conditions for certain large commercial products and Model-as-a-Service businesses.

Upstream project and model authors: MoonshotAI/Kimi-K3 and moonshotai on Hugging Face.