Need More VRAM
@needmorevram
AI Engineer • running models locally until something catches fire
llama.cpp • vLLM • GGUF • homelab
加入 January 2026
65 正在关注 718 粉丝
The time has come for low RAM users, this time v0.1.26 on a single RTX 3090 with 256K context and the new Low-RAM mode enabled.
I’m seeing ~94 tok/s live decode, while using just 15GB of system RAM.
This is getting ridiculous!
显示更多
