CuteAFD v2 (let’s call it the first real release) CuteAFD turns consumer GPUs and DGX Sparks into a team that runs massive models with fast code generation, huge context windows, and image and audio input. It squeezes more out of the hardware you own while scoring quantized models against official full-precision originals, so you can see what you’re trading for speed and memory. V2 major items, in case you were daring and already tried it: - Multimodal: image and audio input across the families. - The 5090 platform: Many models can run with lower spark count and just a high end consumer GPU - Quality scoring: quantized models are scored against the official full-precision model. - Loading checkpoints: sliced checkpoints, plus cuteafd plan --files and fetch. - GLM Flash defaults: compact index cache, shared replay, tensor head and W8A8 drafter. Much thanks to @dangerm00se for collaborating on this release… our bots have fused into a ghetto swarm to bring this to you. Lots of model support just 3 release measurement cards attached.. find more on the landing page for the project!

And one more thing… My calibration data set for making quants! I haven’t been active making quants for a bit since many folks have kind of taken over the space of making nice EXL3 and now interesting derivative quants…. I will likely make more in the future to pack at specific quality levels I want. There is some cool adaptable bitrate things folks are exploring now but that will have some quality tradeoff versus an upfront commit. Since CuteAFD lets me gang my hardware together the way I want, I can target very specific sizing that I want… and if we get some huge new models, EXL3+ creation will come back in force! The point of sharing this is (1) let others use it for whatever they want, it seems to be a pretty decent calibration set (2) get any feedback about how it could be better.

For all the DGX Spark owners who are interested in a cooling solution: 3D printable "forced induction" fan shroud for the sparks. This allows for a 120mm Noctua PPC 3000rpm fan to force air through the intake of the unit, drastically increasing airflow through the internal heatsinks. Asus GX10s ran cooler by ~19°C under heavy load. Nvidia DGX Spark ran cooler by ~10°C under heavy load. This eliminates the overheating issue, as long as you don't mind the fan noise. 3D files are on my repo

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models.

WOW: Kimi K3 just killed the AI subscription era 🇨🇳 On July 27, Moonshot publishes the full K3 weights under a modified MIT license. ▫️ 2.8 trillion parameters ▫️ 1M token context ▫️ Open weights, yours to run "The frontier becomes a file." The closed labs charge you monthly for access. Moonshot just made the frontier a download. Does open weight end the subscription model, or do the big labs still win on polish?