Mia
@MiaAI_lab
Building with AI & LLMs | Insights, recipes, tools & honest experiments.
加入 July 2022
413 正在關注 36.1K 粉絲
TensorFold is going to change everything 🚀
Exciting times ahead! ⏳
TensorFold Inference Engine is here 🚀
I spent six months making one weight read count for more than one token on Apple Silicon.
Draft tokens run through parallel lanes; the model verifies them together and keeps only what passes.
Qwen 3.8 27B MLX 4Bit - 120-124tks
Nemotron Lightning MLX 4Bit - 188-206tks
Qwen3.8 Flash Next MLX 4Bit - 88-92tks
CUDA Implementation is in Alpha showing strong gains.
The Repo is in the comments 👇🏼
顯示更多
