In case you forgot (we haven't). There is no universe in which chutes exists without bittensor. Expanding our OpCo's service and refocusing on a healthier subnet dynamic has some people confused, but it's very explicitly for strengthening the subnet. roadmap/plan ICYMI

The midterm results, regardless of your opinion on them, come with a fairly high degree of uncertainly around the future of new datacenter capacity (and/or costs) in the US. Parallax is no longer really optional IMO (though it was already inevitable). “You never change anything by fighting it; you change things by making them obsolete through superior technology.” - R. Buckminster Fuller Status updates: Almost done with dataset prep. It turns out ~10t tokens of high quality data tokenized and properly upsampled and so on is a lot of work and compute. Apple M5 and DGX spark support locked in, still waiting on my AMD Ryzen AI MAX+ PRO 495 from lucebox to get that supported as well. Still chugging away at the final touches on the code/agentic pipeline also (but that would be after the pretraining anyways, non-blocking). Settled on the openai o200k tokenizer, updated to always use single tokens for digits for better math capabilities (hopefully). (and more to come soon)

I accidentally forgot to pay the power bill and they turned it off. Happy little accident, because now I get to enjoy the wood stove while re-seating a few hundred 223 rounds (I loaded these originally for a precision bolt action which I traded a few years ago for a 556 and need shorter overall length rounds). Pure zen, memories of hunting trips past, deer season in a couple weeks, good day to zero in the scope too.

Here's a very crude simplified chutes roadmap/plan, for near term: - move the most unprofitable things off of the subnet, including today's announcement of some (not all) additional inference options (not paid via emissions) - this allows us to scale the things we want and need to avoid capacity issues/better direct incentive - this includes models we use to build data pipelines and relationships to other teams/networks/etc - add options that are completely revenue positive (hourly pricing/rentals/simple docker based jobs/etc), TEE, private, secure, priced above cost, more extensive version of private chutes we have today basically - expand the hourly compute offerings to also include CPU only (e.g. to run agents, web browsers, sandboxes, non-GPU work of any kind) - use some of the incentive for ideally non-datacenter based model training and serving - build a non-extractable product on chutes from the models we build, via dynamic edge nodes, possibly from the epigenesis work (smaller faster cheaper faster than lora customizations from the sparse sections of pretrained model, participant supplied) - models and pipeline etc. will be open source, the customization and ensemble thereof will live only in chutes, inc. the continuously evolving set of served variations/RL/etc. We'd like, to the extent we can, attract participants who want to build and have no obligation to pay datacenters or any particular hardware vendor, to keep the value inside the subnet at all times by attacking sell pressure at the source (DC commitments, chip purchases, etc). We will continue to have secure TEE inference and GPUs; not elimating anything wholesale, just focusing more on a commodity that doesn't have to flow out to DCs and single hardware vendors. NVIDIA reportedly has 75% profit margins, revenue for the sake of revenue fueling GPU reselling goes straight to one place, basically. Fine, I guess, but I'd rather work together as individuals with whatever hardware we can bring, to a reasonable degree.

True? No clue. The point of parallax is to put the network into as many hands as possible in a verifiable trustless way, and for the output (the models) to not require yet another source for the wealthiest companies on earth to monopolize (that is, a model that can run even on a phone and not require "now's" completely unaffordable mega GPU). The current AI marketplace is fueled by a massive debt market with very few beneficiaries at the world's expense. If you like K shaped economies here it is in plain sight; trickle up resources, consolidating into a few, spreading the gap at ever-increasing speed. Whether this is self serving for Mr. Musk to gather up more resources for pennies on the dollar or a true statement is irrelevant, the outcome is the same. Will it crash tomorrow, next week, never? No clue, but I would rather not build the foundation on this gamble. I'm possibly (probably?) doing it wrong, but the goal is and has always been, people first, which means accessibility first. Let's make "good enough" models with no kill switch and no hardware moat. Please and thank you. If we get superintelligence out into the universe without/at the cost of consciousness to witness it, we have accomplished nothing, and have in fact forgotten the whole point.

There are ~two economies: attention and revenue. Focus on whichever you like but if you only pick one, it probably eventually breaks. Free high intelligence = maximum attention (but probably financial/resource crisis), and conversely maximally expensive commodities restricting access rugs the attention eventually and likewise probably leads to collapse. Parallax and the new "living model" type system created out of it is perhaps a way to foster both by decreasing prices (run on phone, macbook, spark, whatever) rather than increasing them further (with larger and larger models), and ideally maintaining the intelligence (and boosting creativity/customization via this epigenesis project to prevent too much concentration into any single company/model). Ideally, everyone just brings their own resources to fund the training (sparks + apple + amd) then they own the output forever and can shape it any which way with baking in their own memory/knowledge/rules into the otherwise sparse weights after training.

Ledger itself ok, but fraudulent products less so of course. This is why I tell people I don't trust anything with a circuit board lol. These keystone tablets are pretty cool, handed a few out at exploit summit. Even better if you leave out some of the words and remember them. (and multi-sig of course)

Still chugging along great, EDA seems much better! Also 7 host failures so far with automatic expert ownership failover. I did experiment also with triadic linear attention adapted to EDA and it didn't seem to have much benefit (probably skill issue on my part):

My two cents on this "math is solved" and "maybe we lose cryptography" question (not that anyone asked of course). The questions are the important part, I think. What was the inspiration that led to the question which then ultimately created the answer? Having answers to old questions is very nice, but more importantly I think we just need to now use that knowledge to ask better questions. Don't be a doomer, ask bigger questions now that we have more powerful tools to help answer them. Don't let answers to old questions extinguish the creative spark, if you will.

There's another magic trick (aka cool math) I believe this parallax enforced sparsity enables by the way... <- prune weights, adapt the pruned weights, sharply cuts catastrophic forgetting risk. If you have a model with the enforced sparsity, 50% of the weights are locked in to particular patterns. What if you mask the other expert weights leaving only those sparse contiguous weights to learn new data? That's the intuition, perhaps it works, perhaps not. But... if it does, it would make the base model significantly more malleable for new domains/tasks/whatever. SFT/continued pretraining with less risk of catastrophic forgetting and without lots of magic tricks and RL and re-sampling in your own words etc.

EDA is superior to GDN-2 as it turns out, it avoids the failure mechanism entirely (so far), and the new run (lambda) is blowing past this one (kappa). Might add in triadic EDA next for better long context capability.

Kappa run is now stopped, not too bad overall! We passed llama 3.2 1b on ARC-C, OBQA, TQA, etc., nearly matched on ARC-E, SciQ, etc. Considering that was 9T tokens and we hit those numbers on the 576B checkpoint, AND we did so around 90% cheaper per token, I'd call it a solid preliminary win! Issues with the run: 1. In one of the Gated DeltaNet-2 heads, the per-channel decay gate underflowed, causing state wipe, and ourput RMSNorm amplified near-zero outputs ~1000x into fleet wide grad spikes. 2. Very late/slow nodes were so far behind their evidence was discarded, not a huge effect but we should have had better tolerance for stale data and partial updates. 3. We packed samples but didn't do masking/etc. which is sub-optimal for pretraining (not a huge diff but worth fixing anyways). 4. The first and last layers being GDN2 may be suboptimal also, vs. SWA, no clear evidence but might be worth switching up the order a bit for those two. We have a fix for this GDN2 issue, but while that was still running I also tested EDA ( and it actually addresses some of the root cause directly (and seems to have better loss than GDN2 anyways) so we'll probably switch to EDA once all tests come back clean and pass the kappa's instability regime (and waiting until we have highly optimized tilelang kernels for EDA to keep MFU/throughput high). Onwards and upwards 🚀

"in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet" Maybe, maybe not. If it does, the internet (really: software and operating systems) obviously need some improvement and we should do so. If the bugs are there, they will be found, one way or another regardless of some LLM. What a great opportunity for a renaissance. Anyone else remember the doomerism around Y2K? Int overflows because of epoch seconds? This is that.

The push for regulatory capture and AI cabalism is accelerating. This, in large part, is why I'm building parallax. True P2P unstoppable, uncensorable AI systems aren't just neat, they're mandatory. This little single DGX spark "datacenter" in my forge with parallax can train (with other nodes) an 80b parameter model. - two solar blankets - ecoflow battery - lapdock for kvm - starlink internet - dgx spark Total input from solar ~600w (800w max) Total power draw from spark/starlink/etc. < 200w No datacenters, no water supplies, no external power grid, no multi gigabit link. We must design AI systems that are as resilient as the humble cockroach. Why do we all appreciate BTC? Because it's a mathematically certain sovereign ledger that can't be inflated away or centrally controlled. Current AI represent the greatest concentration of wealth, power, and information (IP theft? blackmail? or simply ads and behavior control?) the world has ever seen... As BTC did to finances, we must do to AI. Chutes + parallax is a step forward towards tearing down this moat.