Sebastian Christiansen(@CerebralCoding_):Drafter speedups are not a 256 token demo trick. M5 Max, TensorFold, Qwen3.8 27B: 129 tok/s across 162K tokens of long output, against 24 tok/s with drafting off. 30 runs, 2.7K to 8.2K tokens each, three sampling settings. 5x holds when the answers get long.

Sebastian Christiansen

@CerebralCoding_

Senior software engineer Automation + applied AI (pharma) Contributor: MLX-Serve, TensorFold, ds4 Local inference · Real hardware · Builders

加入 September 2026

514 正在关注 2.3K 粉丝

Sebastian Christiansen@CerebralCoding_

2026.11.10 13:12

Drafter speedups are not a 256 token demo trick. M5 Max, TensorFold, Qwen3.8 27B: 129 tok/s across 162K tokens of long output, against 24 tok/s with drafting off. 30 runs, 2.7K to 8.2K tokens each, three sampling settings. 5x holds when the answers get long.

显示更多