🐳 DeepSeek-V4-Flash-Vision-Exp is 305B, and I packed it into a ~95 GiB mixed-precision EXL3 release that now SERVES on one DGX Spark. Receipts + Recipe below. 🚀
K2 across the routed-expert stack. K3 only where my layer sensitivity scan ranked the extra precision highest. MTP/DSpark drafter tensors preserved from source.
𝗠𝗜𝗫𝗘𝗗 𝗞, 𝗡𝗢𝗧 𝗙𝗟𝗔𝗧 𝟮-𝗕𝗜𝗧
The source checkpoint is already mixed format:
MXFP4 routed experts
FP8 attention
MXFP4 MTP/DSpark drafter
I dequantized each routed-expert tensor to BF16 before trellis encoding, then rebuilt the expert bank as:
43 MoE layers on a K2 base
K3 on layers 3, 13, 21, 22, 28, 41
attention / shared experts / embeddings kept at high precision (BF16)
48 output shards, ~95 GiB total
Those six layers came from a complete K2 vs K3 proxy-error scan, not guesses. The ranking was nearly flat across all 43 layers, so I am not pretending these are six magical outliers. The real question is whether the extra bits there produce a measurable fidelity gain over uniform K2 that comparison is still open.
𝗦𝗧𝗜𝗟𝗟 𝗢𝗣𝗘𝗡
prefill/decode numbers + max ctx
PPL/KLD vs uniform K2
MTP/DSpark speculative serving
CUDA-graph decode numbers
sixcat battery (first full run is in; publishing after a chat-template fix that currently understates tool use)
Model:
Serve it (one Spark, open source):