"in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet" Maybe, maybe not. If it does, the internet (really: software and operating systems) obviously need some improvement and we should do so. If the bugs are there, they will be found, one way or another regardless of some LLM. What a great opportunity for a renaissance. Anyone else remember the doomerism around Y2K? Int overflows because of epoch seconds? This is that.

The push for regulatory capture and AI cabalism is accelerating. This, in large part, is why I'm building parallax. True P2P unstoppable, uncensorable AI systems aren't just neat, they're mandatory. This little single DGX spark "datacenter" in my forge with parallax can train (with other nodes) an 80b parameter model. - two solar blankets - ecoflow battery - lapdock for kvm - starlink internet - dgx spark Total input from solar ~600w (800w max) Total power draw from spark/starlink/etc. < 200w No datacenters, no water supplies, no external power grid, no multi gigabit link. We must design AI systems that are as resilient as the humble cockroach. Why do we all appreciate BTC? Because it's a mathematically certain sovereign ledger that can't be inflated away or centrally controlled. Current AI represent the greatest concentration of wealth, power, and information (IP theft? blackmail? or simply ads and behavior control?) the world has ever seen... As BTC did to finances, we must do to AI. Chutes + parallax is a step forward towards tearing down this moat.

Super excited to share the results of my deep, dark research cave over the past couple months building out parallax at @ExploitSummit Someone asked me if we had a breakthrough - couldn't even condense a response. The benchmaxxing era is (or should be) over, efficiency-maxxing is next.

NVIDIA B300 servers retail around $350k min, crazy power/cooling requirements/etc. (bad maths) per GPU (8) is $43,750, if you can even find them (narrator: you can't) then pay for collocation etc... 9 PFLOPS dense FP4. DGX Spark: $4700, plug in anywhere, 1 PFLOP (fp4 sparse). B300 9ish PFLOP dense FP4 super expensive impossible to collocate anywhere etc. etc., $4861.11 per PFLOP. DGX Spark $4700 plug in any 15 amp outline in any house, $4700/PFLOP (but sparse). Memory is slower, so I guess use less of it/less manipulation of it... Yep, math checks out - pretraining on decentralized DGX Spark with native sparse FP4 (checks notes: yes we're using ternary weights which slot perfectly into sparse FP4 compute) should be a thing. Time to test it out...

So engy starts as explicitly non-TEE inference, but better for some reason because each instance requires 80 gpus that are flaky and slow over 25gbps interconnects. Then they realize TEE is necessary for any real use case, but TEE without open source/fixed/attested workloads is pointless because you could just change the stack to log whatever you want otherwise. So now it'll be... a chutes clone? But without any transparency, run by an anon and his many alts and lackies who are in a perpetual smear campaign against chutes? How many times to tao holders need to be rugged/deceived by anons with huge promises? Careful out there.

TL;DR: libp2p TCP transport with dual trainer/syncer roles = chef's kiss For model training, don't bottleneck and single-point-of -failure yourself with a blob store (S3/R2). Also, don't bash your forehead against a wall using listening sockets on nodes that may be behind firewalls/NAT/port mapped containers/etc. Just set up a separate backbone (sync only, non-GPU/training nodes) across the world with super fast WAN and use push only from training/GPU nodes to this layer. Easy peasy, works like a charm. And bonus, libp2p's TCP transports are reliably better across providers/networks/countries compared to default quic/udp. Skynet?