注册并分享邀请链接,可获得视频播放与邀请奖励。

Niklas
@niklaslenz_ai
I run local AI at home on 4 DGX Sparks, 1 M5U 256GB, 1 RTX 5090, 2 RTX 5060 Ti 16GB, and rank what runs best on the machine you own. LLMs, image/video gen. etc
加入 June 2026
187 正在关注    1.7K 粉丝
Thanks for the shout-out, @ashxhart! Seeing my name in the 1.0.3 notes, and Mia's shout-out on top, really keeps the motivation up :D The 1.0.3 release binary has no kernels for the RTX 5090. Building from source with one flag fixes it: -Dsm=120, no patch. Same card, same day, Nemotron 3.5 Lightning, llama.cpp with MTP → TensorFold 1.0.3: Code: 458 → 731 tok/s (1.6×) Prose: 354 → 506 tok/s (1.4×) • TensorFold's drafted output matches plain byte for byte, 10 of 10. llama.cpp with MTP changes tokens. • vs 1.0.2: +8% code, +11% prose. • One RTX 5090, Linux, 450 W. Ash's bench_openai.py, greedy, 256 tokens, median of 3 rounds. • Different quants: MLX 4-bit vs GGUF Q4_K_M, quality not compared. llama.cpp MTP not tuned. Build: zig build -Dsm=120 -Doptimize=fast fatbins install zig build -Dsm=120 -Doptimize=fast native
显示更多
TensorFold 1.0.3 lands with a few cool things. - Living Weights (beta with Nemotron on Metal) and coming to CUDA in 1.0.4 - Nvidia Ampere (beta) for Nemotron, and native Qwen3.8-27B with DFlash2 - Conversation prefix caching - Various bugs fixed and improvements Twelve contributor PRs helped make this happen, so a massive thank you to @niklaslenz_ai and GitHub contributors akol1, jschmied, chaog992, AdrianBinDC and BobClawblaw, and everyone testing, reporting bugs and helping us improve TensorFold. FYI, Living Weights is in very early beta and has sharp edges, but I think I’m on to something. I’d love community feedback to make it better: @plotarmordev @petruspennanen @MiaAI_lab @volatilemarkts
显示更多