Ash Hart(@ashxhart):TensorFold Inference Engine is here 🚀 I spent six months making one weight read count for more than one token on Apple Silicon. Draft tokens run through parallel lanes; the model verifies them together and keeps only what passes. Qwen 3.8 27B MLX 4Bit - 120-124tks Nemotron Lightning MLX 4Bit - 188-206tks Qwen3.8 Flash Next MLX 4Bit - 88-92tks CUDA Implementation is in Alpha showing strong gains. The Repo is in the comments 👇🏼

Ash Hart

@ashxhart

Doing my bit to bring local AI to everyone | | | | Thoughts are my own.

参加 October 2014

375 フォロー中 5.2K ファン

Ash Hart@ashxhart

2026.09.26 00:31

TensorFold Inference Engine is here 🚀 I spent six months making one weight read count for more than one token on Apple Silicon. Draft tokens run through parallel lanes; the model verifies them together and keeps only what passes. Qwen 3.8 27B MLX 4Bit - 120-124tks Nemotron Lightning MLX 4Bit - 188-206tks Qwen3.8 Flash Next MLX 4Bit - 88-92tks CUDA Implementation is in Alpha showing strong gains. The Repo is in the comments 👇🏼