The midterm results, regardless of your opinion on them, come with a fairly high degree of uncertainly around the future of new datacenter capacity (and/or costs) in the US. Parallax is no longer really optional IMO (though it was already inevitable). “You never change anything by fighting it; you change things by making them obsolete through superior technology.” - R. Buckminster Fuller Status updates: Almost done with dataset prep. It turns out ~10t tokens of high quality data tokenized and properly upsampled and so on is a lot of work and compute. Apple M5 and DGX spark support locked in, still waiting on my AMD Ryzen AI MAX+ PRO 495 from lucebox to get that supported as well. Still chugging away at the final touches on the code/agentic pipeline also (but that would be after the pretraining anyways, non-blocking). Settled on the openai o200k tokenizer, updated to always use single tokens for digits for better math capabilities (hopefully). (and more to come soon)