holy shit, palmer luckey's anduril is building laptops in america and it runs crysis > bluechip, anduril and qualcomm > snapdragon x, arm > windows and linux > 90% us and allied parts, by cost > boards to be assembled in ohio > preorders open in 2027 he puts the price at about 20% over a high end pc, and in his july 2025 poll most voters said they'd pay that no ram or npu numbers yet, and those decide whether it can run local models

i used to think i was using a lot of ai, then i let agents handle my personal stuff and a completely different dimension unlocked i think every now and then everyone should try to let the models into their life to improve and fix and remove the junk when needed, because i've noticed sometimes things slip from your mind, but the machine will always be firm on the command and doesn't compromise i've seen so much change after i started integrating agents into my personal life, and only local models get that i appreciate that such technology exists and is available to everyone, this is the age of acceleration and i hope you're getting the best out of it and creating abundance

tell me i'm wrong every time an ai writes one word it reads the whole model out of memory again, so a 70gb model moves 70gb just to pick the next word, then starts over for the one after is that really how this works?

for anyone starting in local ai, the first thing you do is buy hardware and install ollama or lm studio without thinking about what powers it underneath and what actually runs it the first instinct should be learning the stack, use them but transition to the layer beneath with llama.cpp and try to understand it, then keep buying gpus, move to vllm and unlock your tp= do not settle there, even try to tweak the model and use mtp and all to speed it up, and try kernels along the way once you understand inference from the foundational level you will never have to worry about getting a job ever again learn inference

people keep asking what my main driver is, and the answer is not light an asus rog strix scar 18 with an rtx 5090 inside, 3.3 kg, i call it beast and it runs my local models and my video studio, the best laptop i've ever used > rtx 5090 laptop gpu, 24gb gddr7 > 896 gb/s memory, 175w max > 8gb more vram than a desktop 5080 > core ultra 9 275hx, 24 cores > 64gb ddr5 system ram > 2tb nvme, 18 inch 240hz mini led > ubuntu 24.04 and sometimes it encodes models, when both my dgx sparks were busy building deepseek v4.1 flash, beast quantized the drafter's 384 experts and checked the kernels bit for bit before the sparks timed them for local ai that fits in 24gb no other laptop beats it, no mobile gpu ships with more than 24gb of vram and none reads it faster than 896 gb/s the weak side is size, past 24gb a model spills into system ram and slows down, a 128gb mac or strix halo holds gpt-oss 120b in one pool, so my big models live on 2x dgx sparks and if your rtx laptop feels slow on linux, check nvidia-powerd first, mine was disabled and the gpu sat at 94w of its 175w, qwen 3.6 27b q4 went from 17.97 to 35.28 tok/s after systemctl enable --now nvidia-powerd

i almost quoted this 96gb 5090 with "holy shit" and then i zoomed into the board that photo is chiphell's december 2024 leak of a bare pny rtx 5090 board, shot about five weeks before the 5090 launched, with its handwritten stickers in the same spots, just recolored with "RTX 5090" written on the side > 16 memory pads around the gpu > 16 x 2gb = 32gb, a stock 5090 > 96gb needs 32 x 3gb, both sides and the die can do it, gb202 already runs 96gb inside the rtx pro 6000 that nvidia sells for $16,000, so a real 96gb 5090 is possible and it would put pro 6000 memory on a gaming card, this photo just isn't one before you trust a modded card, check > chips on both sides of the board > nvidia-smi showing the new size > priced above the stock card so far the 96gb 5090 is an alibaba listing priced under a stock 5090 and a photo from 2024, show me nvidia-smi and i'll be the first one shocked

i can't stop looking at this chart MiniCPM-OCR 1.0 is a 1.3B model built only for reading documents, and it's the one dot in the top right, the highest average score across four document benchmarks and the most pages per second their numbers: > 93.68 on Real5-OmniDocBench > 92.39 on Wild-OmniDocBench > 54.7 on OCRBench v2 > 1.217 pages/s on one rtx 5090 > 1.49x PaddleOCR-VL-1.6 throughput > 3.62x MinerU2.5 Pro throughput the real-world scores are the ones i care about, real documents reach a model as a phone photo of a contract at an angle or a curved page out of a book, and that's where it leads, on the standard OmniDocBench v1.6 it sits 0.4 behind the top ocr is the input for everything after it, agents and rag pipelines are only as good as the text they pull out of a pdf, and one wrong digit on an invoice makes the whole chain confidently wrong that's why the companion model got my attention, MiniCPM-OCR Jev is a 2B checker that reads the page next to any ocr model's output and scores each region, so you know which parts a human still has to check > AUROC 0.936 on JevOCRBench > Qwen3.5-397B-A17B gets 0.938 the parser and the checker are small enough to run on your own gpu, so contracts and receipts never leave your machine, with llama.cpp, vllm, sglang and ollama supported on day one weights and code from @OpenBMB: Hugging Face: MiniCPM-OCR 1.0: MiniCPM-OCR Jev: GitHub:

watch! i ran ling 3.1 flash on the same prompt twice with hermes agent on my floating tree spec once as a q4 gguf across 4x dgx sparks and once on the api, with the same agent settings both times > q4 over llama[.]cpp rpc: 12h 26m > api, novita on openrouter: 57 min > 7.5 vs 98.6 tok/s decode > 325k vs 228k tokens out > 12/12 tests + visual gate on both > $0.27 for the whole api run both builds pass everything, the api build has the better tree with real branches and 8,270 veined leaves, the q4 build kept the better cliff the video is one camera path on both builds, frame for frame, q4 on the left and the api on the right my verdict: ling 3.1 flash is a strong model and the speed gap is the setup, q4 over rpc proves it fits on 4 sparks, for agent work you want it on a real serving stack, and an official 4bit build would make it practical locally the long turns question from the 12 hour build is answered in the thread below 👇

watch! i ran ling 3.1 flash on the same prompt twice with hermes agent on my floating tree spec once as a q4 gguf across 4x dgx sparks and once on the api, with the same agent settings both times > q4 over llama[.]cpp rpc: 12h 26m > api, novita on openrouter: 57 min > 7.5 vs 98.6 tok/s decode > 325k vs 228k tokens out > 12/12 tests + visual gate on both > $0.27 for the whole api run both builds pass everything, the api build has the better tree with real branches and 8,270 veined leaves, the q4 build kept the better cliff the video is one camera path on both builds, frame for frame, q4 on the left and the api on the right my verdict: ling 3.1 flash is a strong model and the speed gap is the setup, q4 over rpc proves it fits on 4 sparks, for agent work you want it on a real serving stack, and an official 4bit build would make it practical locally the long turns question from the 12 hour build is answered in the thread below 👇

holy shit! typesafe is hacked and is not safe anymore i opened the dump and every output they ever gave a customer is in there, 0.00 to 1.00, mine was 0.67 on line 69 and now the whole internet has it a company called typesafe leaked its outputs as 101 floats and 0.30000000000000004 is missing from the dump, so the real leak is still out there the entire typesafe leak is 507 tokens of counting 0.00 to 1.00, my 2x dgx sparks do counting at 147.6 tok/s, so i can regenerate their whole customer history locally in under 4 s and nobody leaks mine but me

holy shit! typesafe is hacked and is not safe anymore i opened the dump and every output they ever gave a customer is in there, 0.00 to 1.00, mine was 0.67 on line 69 and now the whole internet has it a company called typesafe leaked its outputs as 101 floats and 0.30000000000000004 is missing from the dump, so the real leak is still out there the entire typesafe leak is 507 tokens of counting 0.00 to 1.00, my 2x dgx sparks do counting at 147.6 tok/s, so i can regenerate their whole customer history locally in under 4 s and nobody leaks mine but me

holy shit, opencode got hacked and it's officially fu*ked 'a large scale distillation attack, 1.4% of console users hit' and people got drained, one user says his openai and anthropic keys, used only in opencode, ran up over $400 overnight, another replied $1,700. who compensates that? every one of them had external providers linked, and it's still ongoing this is the warning for all of us, build your own harness and keep your keys on your own machine, don't park them in anyone's console anymore