watch! i ran ling 3.1 flash on the same prompt twice with hermes agent on my floating tree spec
once as a q4 gguf across 4x dgx sparks and once on the api, with the same agent settings both times
> q4 over llama[.]cpp rpc: 12h 26m
> api, novita on openrouter: 57 min
> 7.5 vs 98.6 tok/s decode
> 325k vs 228k tokens out
> 12/12 tests + visual gate on both
> $0.27 for the whole api run
both builds pass everything, the api build has the better tree with real branches and 8,270 veined leaves, the q4 build kept the better cliff
the video is one camera path on both builds, frame for frame, q4 on the left and the api on the right
my verdict:
ling 3.1 flash is a strong model and the speed gap is the setup, q4 over rpc proves it fits on 4 sparks, for agent work you want it on a real serving stack, and an official 4bit build would make it practical locally
the long turns question from the 12 hour build is answered in the thread below 👇