Drew Breunig
@dbreunig
Writing about and working on AI, DSPy, geo, and data.
Trying to prevent prompt debt and help others build maintainable intelligent applications.
Joined March 2008
1.2K Following 10.4K Followers
The most recent batch of models start to respond in a limited and localized vocabulary as you go over ~200k context. Unique terms are adopted, more like pointers (symbolic) than jargon (language).
At least it feels that way to me. I end up spending significant effort getting them to “decompile” this dialect when writing reports, or even product copy.
Show more
I bet we figure out that neural networks can be decomposed into evolved symbolic systems of sorts and that it will be achieved before capital S Superintelligence but everyone has to lock in
