Drew Breunig(@dbreunig):The most recent batch of models start to respond in a limited and localized vocabulary as you go over ~200k context. Unique terms are adopted, more like pointers (symbolic) than jargon (language). At least it feels that way to me. I end up spending significant effort getting them to “decompile” this dialect when writing reports, or even product copy.

2026.09.28 02:09

The most recent batch of models start to respond in a limited and localized vocabulary as you go over ~200k context. Unique terms are adopted, more like pointers (symbolic) than jargon (language). At least it feels that way to me. I end up spending significant effort getting them to “decompile” this dialect when writing reports, or even product copy.

roon@tszzl

2026.09.27 00:45

I bet we figure out that neural networks can be decomposed into evolved symbolic systems of sorts and that it will be achieved before capital S Superintelligence but everyone has to lock in