Need More VRAM
@needmorevram
AI Engineer • running models locally until something catches fire
llama.cpp • vLLM • GGUF • homelab
가입 January 2026
65 팔로잉 중 718 팬
The time has come for low RAM users, this time v0.1.26 on a single RTX 3090 with 256K context and the new Low-RAM mode enabled.
I’m seeing ~94 tok/s live decode, while using just 15GB of system RAM.
This is getting ridiculous!
더 보기
