Drew Breunig(@dbreunig):The most recent batch of models start to respond in a limited and localized vocabulary as you go over ~200k context. Unique terms are adopted, more like pointers (symbolic) than jargon (language). At least it feels that way to me. I end up spending significant effort getting them to “decompile” this dialect when writing reports, or even product copy.

Drew Breunig

@dbreunig

Writing about and working on AI, DSPy, geo, and data. Trying to prevent prompt debt and help others build maintainable intelligent applications.

加入 March 2008

1.2K 正在關注 10.4K 粉絲

Drew Breunig@dbreunig

2026.09.28 02:09

The most recent batch of models start to respond in a limited and localized vocabulary as you go over ~200k context. Unique terms are adopted, more like pointers (symbolic) than jargon (language). At least it feels that way to me. I end up spending significant effort getting them to “decompile” this dialect when writing reports, or even product copy.

roon@tszzl

2026.09.27 00:45

I bet we figure out that neural networks can be decomposed into evolved symbolic systems of sorts and that it will be achieved before capital S Superintelligence but everyone has to lock in