Wei Wang(@weiwang_dd):Running a 763B model at 192K context usually demands a multi-node cluster. Meet Abyss — running DeepSeek-V4.1-Flash locally on a single dual-GPU workstation (2× RTX PRO 6000 96GB) by intelligently tiering native MXFP4 experts across VRAM, RAM, and NVMe. Latest benchmarks: ⚡ ~52 tok/s decode at 96K context 📖 ~14s prefill for 48K tokens (~3,400+ tok/s) In earlier matched tests against upstream llama.cpp, Abyss already delivered 5.2× faster prefill and 25% higher decode throughput—and our latest kernel & tiering optimizations push it even further. Code & details: https://t.co/yhTqjEsN2i #LocalLLM #DeepSeek #AIInfra #CUDA #Blackwell #OpenSource

2026.11.11 15:18

Running a 763B model at 192K context usually demands a multi-node cluster. Meet Abyss — running DeepSeek-V4.1-Flash locally on a single dual-GPU workstation (2× RTX PRO 6000 96GB) by intelligently tiering native MXFP4 experts across VRAM, RAM, and NVMe. Latest benchmarks: ⚡ ~52 tok/s decode at 96K context 📖 ~14s prefill for 48K tokens (~3,400+ tok/s) In earlier matched tests against upstream llama.cpp, Abyss already delivered 5.2× faster prefill and 25% higher decode throughput—and our latest kernel & tiering optimizations push it even further. Code & details: #LocalLLM# #DeepSeek# #AIInfra# #CUDA# #Blackwell# #OpenSource#

コミュニティへ転送