And one more thing… My calibration data set for making quants! I haven’t been active making quants for a bit since many folks have kind of taken over the space of making nice EXL3 and now interesting derivative quants…. I will likely make more in the future to pack at specific quality levels I want. There is some cool adaptable bitrate things folks are exploring now but that will have some quality tradeoff versus an upfront commit. Since CuteAFD lets me gang my hardware together the way I want, I can target very specific sizing that I want… and if we get some huge new models, EXL3+ creation will come back in force! The point of sharing this is (1) let others use it for whatever they want, it seems to be a pretty decent calibration set (2) get any feedback about how it could be better.

CuteAFD v2 (let’s call it the first real release) CuteAFD turns consumer GPUs and DGX Sparks into a team that runs massive models with fast code generation, huge context windows, and image and audio input. It squeezes more out of the hardware you own while scoring quantized models against official full-precision originals, so you can see what you’re trading for speed and memory. V2 major items, in case you were daring and already tried it: - Multimodal: image and audio input across the families. - The 5090 platform: Many models can run with lower spark count and just a high end consumer GPU - Quality scoring: quantized models are scored against the official full-precision model. - Loading checkpoints: sliced checkpoints, plus cuteafd plan --files and fetch. - GLM Flash defaults: compact index cache, shared replay, tensor head and W8A8 drafter. Much thanks to @dangerm00se for collaborating on this release… our bots have fused into a ghetto swarm to bring this to you. Lots of model support just 3 release measurement cards attached.. find more on the landing page for the project!