loved this, especially how thinky approached pre-training + midtraining. notes from blog post: 1. initial pretraining phase (H1) covered 8.4 trillion tokens of a broad mix of natural language, code, and image data. 2. a second phase (H2) of 3.8 trillion tokens of higher-quality, newer data the model hadn’t seen, at 512K context. 3. the third and fourth phases mixed in data in the deployment format. The third phase was about 320 billion tokens of reasoning and agent trajectories at 1M context, alongside replayed pretraining data. 4. the fourth phase (H4) kept only the highest-quality data: about 34 billion tokens of agent trajectories. the H3 and H4 mixtures consisted mostly of trajectories sampled from earlier models including Kimi K3 and our own models, filtered for correctness and alignment.