GLM-5.3 Flash on 4x RTX 3090. Vision. 256K context. Good speed. Good quality.
Is that even possible?
Thats my next milestone.
I collected every weight layout, cache budget and runtime blocker I could find. The BF16 source and selective EXL3 Q4 weights are downloading right now.
My best bet is hybrid TP4 + EP4. Tensor parallel for attention, dense layers and Vision. Expert parallel for the routed MoE. Then REAP to remove the least useful experts instead of crushing the entire 320B model into one tiny uniform quant.
The first candidate is Q4 + REAP60. The quality target is BF16-derived Q3 + REAP50. CPU expert offload stays a fallback because loading the model means nothing if decode is unusable.
The final gate is one real image-bearing 256K request with CUDA Graphs, no eager, good retrieval, stable reasoning and usable decode speed.
I give the full target a 35-50% chance right now.
Lets see how far 96 GB of VRAM can go.