Training a 100B parameter model normally means renting a block of matched, high-spec hardware and holding it unbroken for the length of the run. Orion-100B ran across five data centres instead, on single A100s at around $1.25 an hour each. That puts a full replica at roughly $20 an hour, and the entry cost about 2.5x below approaches that need high-spec nodes throughout. It ran as a viability proof rather than to a finished model, and it held. We're in Montreal on Monday talking about what that changes. @ExploitSummit

Training and rollout logprobs matched bit for bit on ROCm. The @RLKernel team integrated RL-Align/RL-Kernel with vllm-project/vime. A 200-step Qwen3-8B GRPO run on 8× @AMD MI300X recorded zero logprob mismatches between Megatron training and vLLM rollout. The strict path aligns reduction order, intermediate precision, rounding points, and math primitives across both sides. Deep dive:

Training and teamwork matter when every minute counts. CAL FIRE/San Mateo County Fire responded to a remote rescue at Sam McDonald Park after a hiker fell from a trail and needed medical assistance. The response brought together ground crews, CAL FIRE Santa Clara Unit Alma Helitack, Stanford Life Flight and local mutual aid partners. Crews established a landing zone, reached the patient on foot and coordinated a successful air transport. Remote rescues like this require agencies to train, communicate and operate together before an emergency ever happens. California’s mutual aid system helps bring the right resources together when and where they are needed. Assisting agencies included CAL FIRE/San Mateo County Fire, La Honda Fire Brigade, Loma Mar Volunteer Fire, AMR, San Mateo County Parks and the San Mateo County Sheriff’s Office. #CALFIRE# #MutualAid#

Training a model to predict the next concept, not just the next token, made 8.9B-scale pretraining converge 1.95x faster. NCP-ArchPreview: Moving towards Latent Space Language Models through Next Concept Prediction The model learns discrete concepts spanning multiple tokens via vector quantization, and jointly trains token-level and concept-level prediction in one latent-space architecture. 🚀 Highlight 1: Strikingly faster convergence Trained on the exact same 5.73T tokens as OLMo-3-7B, it matches the baseline's final loss after consuming only 51.3% of the tokens, while beating it by 2.45 points on the downstream macro-average. 🧩 Highlight 2: Concept prediction genuinely drives the gains Ablations that add the Concept Module, hierarchical residual connections, and the NCP loss one at a time each independently improve the loss, and the advantage holds even under parameter- and compute-matched comparisons. ⚡ Highlight 3: The learned concept space stays useful after pretraining Domain adaptation that updates only 17M parameters gains more capability with less forgetting than full fine-tuning, and injecting concept states into a speculative drafter improves accepted length by 4.17%. Treating concepts as a first-class training target, rather than a side effect, looks like a genuinely practical blueprint for next-generation model design. #LLM# #LanguageModels#

training models from scratch gives you so much control on core competencies of the final models. Say you wanna deploy a fast model for cybersecurity applications, you can verifiably steer the model during “pre”-training towards fundamental behavior that during “post”-training” allows for the strongest guardrails reliability & resiliency. This you cannot do with a checkpoint you don’t know its pre-training data, and initial conditions.