TensorFold 1.3.3 is out 🚀
124 PR's merged and 34 Issues resolved in 24h
GLM-5.3-Flash went from 151.51 to 172.39 tok/s in median drafted decode after a 1,040,384-token prompt on four GB10s at TP=4. That’s 13.8% faster on CUDA with the same output tokens in that test. Thanks to
@MiaAI_lab for the kernel behind that gain.
- On Metal, GLM-5.3-Flash gets DFlash2 drafting. On a Mac without tensor units, GLM cold prompts from 2K to 64K took about 5–6% less time in the release tests.
- MOSS-Image generates pictures on CUDA through /v1/images/generations. GLM-OCR runs on HIP, and Idol gets its image encoder on Metal.
- DeepSeek-V4.1 gets structured output in the dsv41-cuda source build. Long CUDA prompts now prefill in chunks alongside running replies by default.
- LFM2-MoE, LFM3, MiniMax-M3 and Qwen3.5-2B on Metal can fill new prompts between other streams’ decode rounds.
- The server adds per-request deadlines, a prompt-cache flush endpoint, and deferred and namespaced tools.
Massive thanks to everyone who sent patches, tested on their own hardware, and filed repros.
@AIFlow_ML @bikepresentball @Bizuayeu @devopsfortunato @unigilby @javierbits @jayleaton @teleogen @niklaslenz_ai @plotarmordev @woolsey @petruspennanen @andersjw_ @shantanugoel @dennis_rye, and on GitHub AbrahamCete, BadAd84, Ch0mpaa, christopherowen, CodingEmil, eXsoR65, Flyinpiz-dev, garvrao80, hobbesd, jschmied, kotevcode, MaverickKB, millaguie, nuk3s, SergiioB, supernovae, tacosta and ultradaoto. I really appreciate the time you put into this.
@plotarmordev @jayleaton @petruspennanen @MiaAI_lab @volatilemarkts