Uh oh. If your AI benchmark uses an LLM-as-judge, you might want to read this. AllenAI tested six frontier models to see whether research ideas were actually novel. And tiny changes to the judge prompt caused crazy swings. GPT-5.4, same basic idea pairs “Which was judged novel?” → 93.5% “Which is novel?” → 40.9% “Which was judged more novel?” → 59.7% Across the study, changing the prompt moved accuracy by 👉50+ points 👈 in some cases and occasionally pushed judges below chance. So naturally they tried giving the judge more intelligence and tools. Didn't fix it. A GPT-5.6 Sol agent running xhigh reasoning with iterative literature search and no search cap scored → 0.78 macro-F1 Claude Opus 4.6 with only 5 pre-retrieved abstracts did about → 0.89 And the big agentic setup cost ~10× more per judgment. Two purpose-built novelty-evaluation systems lost to the researchers' cheapest simple prompted judge by → 8–29 macro-F1 points while costing up to → 32× MORE 🎯 Lesson Learned 👉 More model, more reasoning more RAG, and more money, doesn't automatically mean a better judge. AllenAI open-sourced the benchmark + code too, so now I want to see Qwen, DeepSeek and other local models run as the judges. The safer benchmark recipe is 👨‍🔬 build a human-rated calibration set 🤖 test the LLM judge against it 🔄 test multiple prompt/order variations 🗳️ aggregate multiple judgments 👀 send close calls back to humans Link in ALT

2 x ~$400 Snapdragon mini PCs. No GPU used and 64GB of RAM combined. And they're running a 95GB Qwen3.8-Flash-Next at 20.3 tps. 👀 The model is 31GB larger than their combined RAM. I have to build this. The setup 🖥️ 2× Lenovo IdeaCentre Mini ⚙️ Snapdragon X X1-26-100 💾 32GB LPDDR5 each 💽 512GB NVMe each 🔌 connected with one USB4 cable 💰 ~$800 total Here's the model: 🧠 Qwen3.8-Flash-Next 📦 IQ4_NL → 95GB Reported with the author's ToRc inference engine 📥 Read → 46.2 tps ✍️ Generation → 20.3 tps 💻 Coding generation → 19.2 tps And the Adreno GPUs aren't being used. ToRc uses the Snapdragon CPUs + Hexagon NPUs for some of the dense math, while the model's experts are streamed from NVMe as they're needed because the weights don't fit in RAM. The second box also isn't simply splitting normal token generation in half. For Flash-Next, ToRc pipelines MTP across the two machines over USB4. ⚠️ ToRc isn't publicly released yet and these are the author's results, not independently reproduced benchmarks. Link in ALT

This may be one of the more important Healthcare specialized Local AI weight releases. Ant open-sourced the weights for Ling-3.0-flash-Sante. The model itself isn't new as Sante has been available through APIs since September but ... What's new is 👉 you can now bring the healthcare model home. Stats 🧠 124B total parameters ⚡ only ~5.1B active/token 📚 256K context 🩺 healthcare-specialized 🔎 deep research 🤖 multi-agent collaboration 🔧 tool use It's built on the same Ling-3.0-flash architecture that already runs in llama.cpp. For reference, base Ling-3.0-flash GGUFs already exist around Q4_K_M → ~77GB IQ3 → ~54GB IQ2 → ~40–46GB IQ1 → ~28–31GB ⚠️ Those are BASE Ling-3.0-flash quants. I haven't found an official Sante GGUF yet. But now that the Sante weights are downloadable, this becomes a home model for 🖥️ 96/128GB workstations 🍎 high-memory Apple Silicon ⚡ DGX/RTX Spark-class unified memory 🔗 multi-GPU local systems And healthcare is one of the places where Local AI can be useful for private documents + research + agent workflows can stay on hardware you control instead of being sent to a cloud API. Ant also reports substantial healthcare gains in the new release, including HealthBench Professional 45.73 → 54.29 AFUMED ComplexReasoning 65.22 → 76.92 Link in ALT

Wow, that's a pretty nice bump for Ling-3.1-flash. Ant introduced the new Ling-3.1-flash-1019 checkpoint, and its Artificial Analysis score jumped Ling-3.1-flash → 41 Ling-3.1-flash-1019 → 44 The highlighted gains come from its updated agentic performance AA-Briefcase ~1400 → 1524 Elo AutomationBench-AA 62% → 68% The existing Ling-3.1-flash is already open-weight under MIT and it's big. Stats 🧠 ~560B total ⚡ ~25B active/token 📚 up to 1M context Community GGUFs already go down to IQ1_S → 142GB IQ2_XXS → 174GB IQ3_XXS → 256GB Q4_K_M → 354GB The existing Q4 has even been loaded across 4× DGX Sparks using llama.cpp RPC. ⚠️ 1019 is a new checkpoint/update, not a new Ling architecture announcement. Ant hasn't released the updated 1019 weights yet. It says open weights + API are coming soon. Link in ALT

Apparently, private Local AI roleplaying AI is a pretty serious use case. ReadyArt's new Wicked-Tutu family is already pulling serious downloads. And this is not a tiny model Stats 🧠 31B dense Gemma 4 💾 Q4_K_M → ~18.7GB 🎮 fits in 24GB-GPU territory 📚 up to 256K context 🏠 completely local Runs directly with ⚙️ llama.cpp 🦙 Ollama 🖥️ LM Studio ReadyArt trained it specifically as a private local roleplay model, with a purpose-built dataset and thousands of scenario categories. The earlier v0.1 GGUF has already passed 17K downloads, and they're iterating fast enough that a Final 31B build is already out. Local AI isn't only competing with ChatGPT but it's enabling categories where privacy + customization + no cloud at all are the product. ⚠️ The HF metadata says Apache 2.0, but the model card also adds personal/non-commercial terms, so I'd treat the licensing as unclear for now. Link in ALT

This small Local AI model could replace an entire document-processing pipeline. ATH-MaaS released OvisOCR3. It’s only a 0.8B-class multimodal model. But one local model can now do all these things 📄 document → Markdown 🧾 document → structured JSON 🔎 OCR + bounding boxes 🧮 formulas → LaTeX 📊 charts → structured data 🔑 key-value extraction 🖼️ figure captioning 🔀 flowcharts → Mermaid 📚 cross-page table reconstruction That last one is interesting. Give it consecutive document pages and OvisOCR3 can determine whether a table continues onto the next page and reconstruct the complete table. Team-reported results OCRBench → 88.8 OCRBench KIE → 90.5 Cross-page table Relation F1 OvisOCR3 → 97.07 Qwen3.8-Max → 94.47 And it’s 🧠 built on Qwen3.5-0.8B 🔓 Apache 2.0 🏠 downloadable/local ⚙️ Transformers + vLLM + SGLang 🔌 OpenAI-compatible local server through vLLM No official GGUF that I can find yet. Link in ALT

The company behind PUBG just trained its own 29B Local AI model from scratch. And the architecture is interesting. KRAFTON’s new Raon-1 mini is … 🧠 28.8B total parameters ⚡ only 4.3B active/token 💾 official Q4_K_M → 19.7GB 📚 262K context 🔓 Apache 2.0 🤖 reasoning + agent/tool use 18 of its 25 layers use fixed-state linear attention. Only 7 layers maintain a growing KV cache. KRAFTON says those layers use only ~8 KiB of BF16 KV per token. So even at the full 262K context, that’s roughly: 👉 ~2GB of growing KV cache before other runtime/state overhead. And in KRAFTON’s reported comparisons, this new 30B-A4B model beats Qwen3.6-35B-A3B on 11 of 18 listed benchmarks. Lots of new models coming for Qwen3.6-35B recently. ⚠️ Preview release. KRAFTON’s benchmark results. Upstream llama.cpp support is still pending and KRAFTON provides its own branch today. Link in ALT

Nvidia lets you build a 1TB Local AI cluster out of DGX Sparks. & Nvidia has a playbook for AI clustering them. 8 × DGX Spark 🧠 128GB each ↓ 🔥 1TB unified memory across the cluster Nvidia now officially supports manually configured clusters of up to 8 x DGX Sparks using their built-in ConnectX-7 networking. And Nvidia says that 1TB is enough to run frontier-class models like GLM-5.3 locally. They specifically call out 🧠 larger models 📚 longer contexts 🤖 multiple concurrent agents 👨‍💻 multiple developers sharing one local backend ⚙️ distributed development They list models such as → GLM-5.3 Flash → DeepSeek V4 Flash → Qwen3.8 Flash Next And later this month Nvidia Sync Model Launcher is supposed to make it easier. Link in ALT

AMD and Intel agreed on something pretty unusual. A local PyTorch AI app shouldn't have to care whether the CPU underneath it is AMD or Intel. AMD is folding its ZenDNN AI optimizations into oneDNN, the library originally developed by Intel. AMD + Intel have now aligned on oneDNN as the default open-source x86 acceleration path in PyTorch. ZenDNN has been working on refining for ... 🧠 LLM inference ⚡ INT4 / INT8 / BF16 / FP16 🧩 fused MoE operations 🚀 vLLM CPU acceleration 🔧 optimized attention/matmul Now that work is being brought into the common oneDNN ecosystem AMD + Intel are also jointly developing ACE, new x86 AI matrix instructions. The plan is for future AMD/Intel AI instructions → oneDNN → PyTorch So local PyTorch software gets one x86 target instead of separate vendor paths. ⚠️ This does NOT make llama.cpp/GGUF faster, and AMD/Intel haven't published new performance numbers from the merger. 🎯But for CPU Local AI, the two companies that spent decades fighting over x86 are now cooperating on the AI software layer. Link in ALT

Here we go! Purpose built AI generated LLMs! Then AI helped optimize the code that runs it. Recursive released Flint, a new coding/agent model designed specifically for fast local inference. Beats Qwen3.5/3.6/3.8 in some coding benchmarks! Stats … 🧠 39.8B total parameters ⚡ only ~2.5B active/token 📚 262K documented context 🤖 reasoning + tool use 🔓 Apache 2.0 And there are official GGUFs: 🟢 NVIDIA NVFP4 → 22.9GB 🍎 Apple INT4 → 26.0GB Recursive recommends 48GB-class memory for these local builds, so think: 🍎 48GB+ Apple Silicon 🟢 48GB+ NVIDIA ⚡ 64GB RTX Spark Recursive reports its optimized runtime at 🍎 M4 Pro → >125 TPs 🟢 64GB RTX Spark → >150 TPs And WIRED’s Will Knight actually ran Flint locally on a 48GB Mac and reported it was fast and successfully handled the coding jobs he tried. Side Note: Recursive built Flint using its Eureka Machine, an automated AI research system that helped 🧩 design the architecture 🧪 develop training recipes ⚙️ optimize infrastructure 💻 write hardware-specific inference kernels Nvidia says those specialized kernels improved Flint’s RTX Spark generation performance by 55% wo changing the model weights. Recursive-reported coding results SWE-bench Verified → 74.2 SWE-bench Pro v2 → 89.7 Terminal-Bench 2.1 → 54.9 LiveCodeBench v6 → 80.3 So this isn’t merely AI writing more AI code. Link in ALT

A single 24GB GPU can now run a 35B scientific research agent that spawns sub-agents and checks its own evidence. Sage-A1.5 is 35B / ~3B active, and its official Q4_K_M is just 21.7GB. Here are the new Sage-A1.5 model stats 🧠 35B MoE / ~3B active 💾 official Q4_K_M → 21.7GB 🎮 fits a single 24GB GPU at moderate context 📚 256K native context 👁️ multimodal 🔓 Apache 2.0 And it's specifically trained as a long-horizon research agent. So it is designed to 🔎 use tools/search/code 🤖 spawn parallel sub-agents ✅ inspect its own evidence 🔄 change direction when research stalls No separate critic model required for its verification mode. These are the full-precision benchmark results PinchBench Sage-A1.5 → 90.0 Kimi K3 → 88.2 GLM-5.3 → 86.6 GPT-5.6 Sol → 87.3 FrontierScience-Olympiad Sage → 81.0 Kimi → 69.0 GLM → 67.0 GPT → 73.0 SWE-Pro Sage → 61.2 Kimi → 63.3 GLM → 64.6 GPT → 64.6 ⚠️ Those benchmark scores are from the BF16 checkpoint, not the Q4. But the same model now has an official 21.7GB GGUF with llama.cpp / Ollama / LM Studio support. Link in ALT

This might be my new favorite all-purpose local AI model. Fits nicely into 16GB of VRAM for an RTX 5060/5070TI. The new SparkX2.6-4B beats Qwen3.5-9B in 80% of shared benchmarks and 97% of Gemma4-12Bs. Stats 💾 official Q4_K_M → 3.54GB 🧠 only 4B parameters 📚 native 1M context 💭 none / low / high thinking budgets 🤖 trained for local agents 🔓 Apache 2.0 It already runs with: ⚙️ llama.cpp 🦙 Ollama 🖥️ LM Studio 🍎 MLX 🚀 vLLM / SGLang In the team's reported benchmark suite Averages: Spark-X2.6-4B → 63.3 Spark-X2.5-4B → 52.1 Qwen3.5-9B → 49.1 So they're claiming a 4B model beats their tested 9B comparison overall. For normal local-agent contexts, though, a ~3.5GB reasoning/agent model with this runtime support is very interesting. Link in ALT

Underdog released Bark 3.0, and it's only a 163MB download. Runs locally w/Pytorch+CUDA or MLX On their Open ASR Leaderboard evaluation, Bark wins 🐶 Bark 3.0 → 4.94% WER → 163MB 🟢 NVIDIA Parakeet → 4.94% → 2.51GB 🌀 Whisper large-v3 → 6.04% → 9.26GB So Bark matches Parakeet's average accuracy while being 👉 ~15× smaller And beats Whisper while being 👉 ~57× smaller On recorded meetings check out these stats Bark → 7.82% WER Parakeet → 9.42% Whisper large-v3 → 13.63% And this isn't CUDA-server-only. 🏠 fully local 📱 phone-class 🍎 MLX + Core ML 🧠 Apple GPU / ANE 🪟 CPU + CUDA paths 🔓 CC-BY-4.0 I've checked and no GGUF. It uses its own compressed Bark format plus MLX/Core ML/Transformers runtimes. Bark 3.0 Plus is only 180MB and improves average WER to 4.76%. ⚠️ English only Link in ALT

DeepSeek V4.1 Flash 1017 is now the #1# open model on Agent Arena leaderboard. Hosted 🏆 #1# open Agent Arena 💰 ~$0.19 median/task 🔓 MIT weights Localmaxxers are already running the same 1017 checkpoint on 2× RTX 3090s + 128GB DDR4. And getting ⚡ ~30 tps decode 🚀 301–470 tps prefill This is a monster model w/552B MoE backbone 💾 + ~196B Engram memory 📚 1M native context The local build squeezes the experts to ~2.03 bpw and uses exact routing, with the big Engram tables on storage. ⚠️ Those 2x3090s use an aggressive quant. It does not mean the local 2-bit version retains the exact Arena score. But that's still kind of nuts. Link to the 2x 3090's test in the ALT

The RTX Spark laptop configuration I thought would signal real demand is the expensive one SKU. Microsoft Surface Laptop Ultra w/💾 128GB unified memory, full RTX Spark N1X, $5,899.99. Microsoft currently lists it 👉 Out of Stock ASUS also said demand for its RTX Spark systems “greatly exceeded” expectations and its initial planned volume had already been fully preordered by channel partners. MSI said its first N1X batch was nearly sold out and was asking Nvidia for more supply. People may be buying RTX Spark for the thing normal Windows laptops can't do. 1) 128GB unified memory 2) CUDA

Tencent has released its own official GGUFs of Hy-Memory-30B-A3B. And they didn't just throw up some quants. They benchmarked them too 💾 Q4_K_M → 18.24GB → 56.49% 💾 Q6_K → 24.71GB → 58.78% 💾 Q8_0 → 31.99GB → 57.63% All tested with llama.cpp on the same LoCoMo memory benchmark setup. 👉 Q6 actually scored highest. And the Q4 means this specialized long-term memory writer is now officially an ~18GB local model. Tencent even documents running it directly with llama.cpp: llama serve -hf tencent/Hy-Memory-30B-A3B-GGUF This is the sort of Local AI architecture I want to experiment with. Link in ALT

I think Sipeed may be skipping the MaixStation DGX-Spark-Killer configuration we would want most. They're planning these memory configs 💾 32GB 💾 128GB 💾 512GB But no standard 64GB model. Sipeed's reasoning is 32GB → dense 27B models 128GB → 125B-class MoE models 64GB → “would run neither better” A ~$2,000 64GB box with 273 GB/s bandwidth would be more interesting to me than a ~$2,000 32GB box. 64GB opens up a very different Local-AI options including, larger quants, larger context/KV cache, and more room for agents. Especially when Nvidia's new 64GB DGX Spark starts at $4,999. Sipeed says enterprise customers can custom-order 64GB. Am I wrong? Would you buy 32GB, 64GB or jump straight to 128GB?

Okay, now people are benchmarking Nvidia's 24GB Bolt on consumer hardware. Someone on Reddit put the official Nemotron 3.5 Bolt 74B-A9B IQ2 on an RTX 5090 💪 with llama.cpp. Without speculative decoding ⚡ 166 tps Add NVIDIA's LiLiCorr DFlash drafter 🚀 343 tps median That's about a 2.07× jump. And prompt processing 🔥 ~5,200 tps VRAM is interesting too 🧠 Bolt only → 21.0 GiB 👁️ + vision projector → 22.7 GiB 🚀 + projector + LiLiCorr → 25.0 GiB So the regular multimodal model really can squeeze onto a 24GB GPU. Link in ALT.

A new Mac Inference engine brings a 48GB M4 Pro MacBook running Qwen3.6-35B to 154 tps. The same benchmark with llama.cpp + MTP, 69 tps. Someone built another Local AI inference engine for Mac. It's called Kapitsa, and instead of trying to support every model, it specializes using🍎 Apple Silicon for either 🧠 Qwen 🟢 NVIDIA Nemotron On a 2024 M4 Pro MacBook Pro with 48GB RAM, running Qwen3.6-35B-A3B at 4-bit: ⚡ Kapitsa + DFlash2 → 154 tps ⚡ oMLX + DFlash2 → 127 tps ⚡ Ollama + MTP → 89 tps ⚡ LM Studio → 86 tps ⚡ MTPLX → 80 tps 🐌 llama.cpp + MTP → 69 tps That's about 2.2× llama.cpp's decode speed in their test. The strategy being, instead of building one generic runtime, Kapitsa hand-optimizes Metal kernels + execution paths around specific architectures, then stacks tweaks like 🚀 DFlash2 speculative decoding 🧠 model-aware caching ⚙️ optimized MoE execution 🔀 concurrent request batching And on an M5 Max 128GB, they report 🚀 Qwen3.6-35B → 281 tps 👁️ Nemotron 3.5 Super-VL 120B → 85 tps ⚠️ These are still developer benchmarks, not independent results. Links in ALT.

A new open LLM can remember an old conversation without putting that conversation back into its context. And at 128K history, its authors claim the persistent memory is ~66× smaller than keeping the full KV cache. MemT-1.0 Nano dropped today. Stats 🧠 36B total / 3B active ⚙️ built from Qwen3.6-35B-A3B 💾 10M dynamic memory state 🧠 451M trainable memory parameters 🔓 MIT-licensed weights But it does things a little differently. Normally, a long-running LLM needs old conversation + new message. So as history grows, you keep dragging more context/KV around, or retrieving old text and stuffing it back into the prompt. MemT puts memory inside the model itself. Its memory blocks run alongside the normal LLM attention. As you interact with it, the model learns to: 📝 write information 🗑️ forget information 🔄 update information 🧠 read it back later So you can: 💬 have a conversation 💾 save its latent memory state 🗑️ discard the plaintext conversation 🔄 start a new session with zero previous messages 🧠 restore the memory ➡️ and ask about what happened before Their demo does this. At 128K history, the authors report for MemT-Nano 📦 persistent memory → ~66× smaller than full-context KV cache ⚡ end-to-end inference → ~1.86× faster And the memory state stays a fixed size as history grows. MemT isn't a little memory model sitting next to Qwen. It takes a normal generative LLM and makes persistent memory part of the LLM's computation itself. They also released MemT-Flash: 🧠 288B total / 17B active 💾 336M dynamic memory state 🧠 3.61B trainable memory parameters 🎯 This could be especially interesting for long-running local agents. ⚠️ The 66× / 1.86× results are author-reported, and strong text-retrieval systems still beat MemT on some tasks where exact historical details matter. I can't find consumer GGUF benchmarks yet. Link in ALT.