Tom Mann(@TomasMann1878):Day 71 of building in public! I've been testing out EXL3 as per @0xSero's suggestion yesterday using https://t.co/UfVnM8W3Fz Quick RTX 3090 test: same Qwen2.5-1.5B checkpoint, BF16 vs EXL3. BF16: 27 tok/s, 2,945 MiB EXL3 4.0 BPW: 157 tok/s, 929 MiB EXL3 3.0 BPW: 141 tok/s, 764 MiB 4.0 BPW decoded 5.8x faster. https://t.co/miKODANqzR https://t.co/aZb9xkMsMU https://t.co/spNKiHrrkB https://t.co/miKODANqzR Deep dive coming soon

Tom Mann

@TomasMann1878

Research agents @meta 🇬🇧 3x hack winner + Codex ambassador

加入 June 2025

2.1K 正在关注 2.7K 粉丝

Tom Mann@TomasMann1878

2026.08.28 07:30

Day 71 of building in public! I've been testing out EXL3 as per @0xSero's suggestion yesterday using Quick RTX 3090 test: same Qwen2.5-1.5B checkpoint, BF16 vs EXL3. BF16: 27 tok/s, 2,945 MiB EXL3 4.0 BPW: 157 tok/s, 929 MiB EXL3 3.0 BPW: 141 tok/s, 764 MiB 4.0 BPW decoded 5.8x faster. Deep dive coming soon

显示更多