TensorFold 1.2.8 is out, and this was a monster landing.
92 PRs merged, and 19 issues closed since 1.2.7. 🚀
There’s new support for Recursive’s Flint 40B-A2.5B on M5 Macs, experimental DeepSeek-V4.1 on Metal, Qwen3 embeddings and rerankers on CUDA, and OCR for page images on HIP, alongside request cancellation by ID and tighter server limits.
Among the measured gains;
- Qwen3.8-27B MLX 8-bit cold prompts are about 33% faster on an M5 Ultra.
- Flash Next CUDA prefill at TP=2 on 2K to 32K prompts is 26.5% to 26.8% faster on two GB10s and 30.9% to 33.9% faster on two RTX PRO 6000s, with the same output tokens in those tests.
Massive thanks to everyone contributing code, testing on their hardware and reporting the things that need fixing, especially
@3utsu @arminxnova @Bizuayeu @defilan @devopsfortunato @unigilby @javierbits @teleogen @petruspennanen @Shoestring_Dev @woolsey, and on GitHub AbrahamCete, BobClawblaw, dinadur, EugeneClaw, jschmied, kotevcode, Mirrdhyn, ppardus, SergiioB and shanedertrain; I really appreciate the time you all put into this.
@petruspennanen @volatilemarkts @MiaAI_lab @plotarmordev @jayleaton