LLM inference produces one token at a time. An 8B flow language model challenges that assumption. Title: Large Flow Language Models URL: 🌊 This work adapts pretrained autoregressive models into Lafa, a blockwise flow language model. It trains on fewer than 30B tokens and reaches 2.54x the throughput of AR at 7B with comparable quality. ⚡ Highlight 1: Asynchronous flows Each token in a block gets its own time, with the denoising schedule learned from the loss. A left-to-right tendency emerges on its own, without being imposed. 🧠 Highlight 2: Self-speculative decoding A one-step flow drafts tokens, and the AR head kept inside the same model verifies them. The output matches greedy AR decoding while producing roughly 2.5 to 5 tokens per forward pass. 🪶 Highlight 3: Less training, better quality Built on Qwen2.5 and Qwen3, it uses at least 18x fewer training tokens than Dream, LLaDA, and Efficient-DLM. The 7B average accuracy is 60.9 versus 58.4 for AR SFT. A strong sign that continuous flows can be a practical foundation for parallel generation. #LanguageModels# #FastInference#

Training a model to predict the next concept, not just the next token, made 8.9B-scale pretraining converge 1.95x faster. NCP-ArchPreview: Moving towards Latent Space Language Models through Next Concept Prediction The model learns discrete concepts spanning multiple tokens via vector quantization, and jointly trains token-level and concept-level prediction in one latent-space architecture. 🚀 Highlight 1: Strikingly faster convergence Trained on the exact same 5.73T tokens as OLMo-3-7B, it matches the baseline's final loss after consuming only 51.3% of the tokens, while beating it by 2.45 points on the downstream macro-average. 🧩 Highlight 2: Concept prediction genuinely drives the gains Ablations that add the Concept Module, hierarchical residual connections, and the NCP loss one at a time each independently improve the loss, and the advantage holds even under parameter- and compute-matched comparisons. ⚡ Highlight 3: The learned concept space stays useful after pretraining Domain adaptation that updates only 17M parameters gains more capability with less forgetting than full fine-tuning, and injecting concept states into a speculative drafter improves accepted length by 4.17%. Treating concepts as a first-class training target, rather than a side effect, looks like a genuinely practical blueprint for next-generation model design. #LLM# #LanguageModels#

"Context Language Models" Most agents rely on an external harness to manage context. This paper instead lets the model directly rewrite its own context as a file, learning when to compress, delete, preserve, or reorganize information. At zero-shot, it gets 11.4% higher BrowseComp-Plus accuracy with 21.5% fewer FLOPs, and RL further improves the context-management policy.

Recursive Language Models: Claude Code, Agent Swarms, Big Research Bets, & the $40M AI Problem Solver MIT’s @a1zhang explains why Claude Code, Codex, and Pi are basically the same, how RLMs use code, context offloading, and recursive subagents to generalize across tasks, why one expert can sometimes replace a trillion-token brute-force search, what OpenAI’s 10,000-agent, 130B-output-token experiment reveals about the future of language models, and why academia’s biggest advantage is the freedom to take weird, ambitious research bets.

Large language models #LLMs# hallucinate websites and domains that sound plausible, but aren’t useful. Criminals are setting up malicious websites under those domains, in a new scam called “slop squatting.”

When language models first started using tools well, I was sympathetic to the narrative that instead of scaling up language models, all we needed was a strong enough "cognitive core", say 1B parameters, and anything else could be done with tool use, like browsing the internet or executing code. I think a lot of people were sympathetic to this argument, and indeed it is pretty hard to come up with a meaningful task that cannot be in principle achieved by a 1B model with adequate access to tools. For example, any esoteric fact that a large language model would know can be, in principle, retrieved from the internet and reasoned over by a 1B language model. However I now think this is totally wrong for one simple reason: doing tasks quickly and naturally without tool use matters a lot. The way that I internalized this reason was actually in my personal journey learning badminton this year. In badminton I am very much like a "1B cognitive core". While I can physically do every movement in a badminton shot that my coach teaches me, it requires a lot of work to mentally remember every cue and put it together. In practice I can do a shot almost perfectly, but I struggle to do it across a point and I definitely can't do it consistently in a game. This is obviously different from someone who has practiced a shot ten-thousand times and effortlessly executes it as a natural instinct. In the same way, language models knowing a fact internally, without tool calls, is meaningful. The first reason is that we obviously care about speed; you'd much rather get an answer immediately than have the model think a long time to be sure of its answer or browse the web. A second reason is that there are some things that are simply best learned via backpropagation over lots of data. If you ask about how people generally think of the Shambhala music festival, you'd rather a large language model give you an aggregate opinion based on all the data on the internet, than get a regurgitation of the first three reviews that show up in a web search. A third reason is that having to do a lot of work to find an answer is not as reliable as already knowing the answer. While this does not have to be true in theory, it is probably true in practice, at least for now. If you have to re-look up facts or redo a mathematical derivation all the time there is a higher chance of mistakes, which can compound in a long-horizon task. Once you buy that it is valuable to do things parametrically without tool use, then you must buy the argument that a 1B cognitive core is not sufficient. There is an information limit to how much knowledge can be internalized by a 1B model, and we will surely want AI to know more than that. Even 1T probably won't be enough. We will want the AI to know as much about our world as possible, we will want it to be updated with new information, and our expectations of what AI can do for us will continue to grow. In summary, tool use enables small models to do a lot more, but those who demand the highest quality intelligence will always want larger models. Bitter lesson strikes again.

Frontier language models can solve advanced reasoning tasks yet still fail to copy long, repetitive strings exactly. The paper finds that copying becomes less reliable as inputs grow longer or repeat the same patterns. The failure is partly architectural. With standard 1D RoPE, producing output token y_k requires retrieving x_k from a relative offset that changes with input length. The choice of positional encoding can determine whether a model copies precisely or follows nearby patterns. 2D-RoPE changes the layout. It treats the source and output as separate rows, so corresponding tokens line up in the same column. In synthetic tests, one-layer models trained on lengths 1–100 copied perfectly at lengths up to 1,000 times longer. The advantage also appeared in pretrained models up to 1.4B parameters, with comparable common-sense performance at the tested scales. The current design still leans on line breaks, so it is not a universal fix. – arxiv. org/abs/2607.16072 Title: "Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D"