Drew Breunig(@dbreunig):The return of “neuralese” makes me wonder: 1. Is this a result of the reward function encouraging shorter reasoning? If so, are models developing their own steno-style shorthand to achieve this goal? 2. @mlpowered once discussed how models, when they see a problem a sufficient number of times, go through a “phase transition” from rote memorization to building an algorithm to generally represent the pattern. Is neuralese in reasoning an external manifestation of this?

2026.08.11 20:48

The return of “neuralese” makes me wonder: 1. Is this a result of the reward function encouraging shorter reasoning? If so, are models developing their own steno-style shorthand to achieve this goal? 2. @mlpowered once discussed how models, when they see a problem a sufficient number of times, go through a “phase transition” from rote memorization to building an algorithm to generally represent the pattern. Is neuralese in reasoning an external manifestation of this?

Alexander Panfilov@kotekjedi_ml

2026.08.11 12:00

2) Illegible reasoning: We confirm prior reports by @ApolloResearch: OpenAI models sometimes reason in alien-like language, referring to themselves as “we” or “it,” or spiraling into cursed loops of “vantages,” “marinades,” and “watchers.” CoT-monitoring people are doing God’s work, as in many traces, even with the prompt, it’s just impossible to tell what the model is up to. We show more examples at