Need More VRAM
@needmorevram
AI Engineer • running models locally until something catches fire
llama.cpp • vLLM • GGUF • homelab
Joined January 2026
65 Following 718 Followers
The time has come for low RAM users, this time v0.1.26 on a single RTX 3090 with 256K context and the new Low-RAM mode enabled.
I’m seeing ~94 tok/s live decode, while using just 15GB of system RAM.
This is getting ridiculous!
Show more
