Alexander Whedon(@alex_whedon):One of the key insights from our SubQ 1.1 Small technical report (https://t.co/Y6kBYgPXv6 ) was that super long-context pre-training decreased the reliance on super-long-context post-training to enable super-long-context modeling capabilities. Million-token-plus pre-training enabled the model the extrapolate post-trained capabilities to larger-than-trained lengths. For example, we pre-trained a model with one-million-token inputs and then did post-training at or below one million tokens, and it was able to perform NIAH with high accuracy at multi-million-token lengths. Read more in our technical report!

2026.06.26 14:07

One of the key insights from our SubQ 1.1 Small technical report ( ) was that super long-context pre-training decreased the reliance on super-long-context post-training to enable super-long-context modeling capabilities. Million-token-plus pre-training enabled the model the extrapolate post-trained capabilities to larger-than-trained lengths. For example, we pre-trained a model with one-million-token inputs and then did post-training at or below one million tokens, and it was able to perform NIAH with high accuracy at multi-million-token lengths. Read more in our technical report!