Thanks for the shout-out,
@ashxhart! Seeing my name in the 1.0.3 notes, and Mia's shout-out on top, really keeps the motivation up :D
The 1.0.3 release binary has no kernels for the RTX 5090. Building from source with one flag fixes it: -Dsm=120, no patch.
Same card, same day, Nemotron 3.5 Lightning, llama.cpp with MTP → TensorFold 1.0.3:
Code: 458 → 731 tok/s (1.6×)
Prose: 354 → 506 tok/s (1.4×)
• TensorFold's drafted output matches plain byte for byte, 10 of 10. llama.cpp with MTP changes tokens.
• vs 1.0.2: +8% code, +11% prose.
• One RTX 5090, Linux, 450 W. Ash's bench_openai.py, greedy, 256 tokens, median of 3 rounds.
• Different quants: MLX 4-bit vs GGUF Q4_K_M, quality not compared. llama.cpp MTP not tuned.
Build:
zig build -Dsm=120 -Doptimize=fast fatbins install
zig build -Dsm=120 -Doptimize=fast native
TensorFold 1.0.3 lands with a few cool things.
- Living Weights (beta with Nemotron on Metal) and coming to CUDA in 1.0.4
- Nvidia Ampere (beta) for Nemotron, and native Qwen3.8-27B with DFlash2
- Conversation prefix caching
- Various bugs fixed and improvements
Twelve contributor PRs helped make this happen, so a massive thank you to
@niklaslenz_ai and GitHub contributors akol1, jschmied, chaog992, AdrianBinDC and BobClawblaw, and everyone testing, reporting bugs and helping us improve TensorFold.
FYI, Living Weights is in very early beta and has sharp edges, but I think I’m on to something.
I’d love community feedback to make it better:
@plotarmordev @petruspennanen @MiaAI_lab @volatilemarkts
Show more