Chief economist + AI Policy Director, @joinFAI. Nonresident fellow @NiskanenCenter. Pluralist. 'The world is second best, at best.' | samuel@thefai.org
The AI consciousness debate is back again, so re-upping my case for taking it seriously.
In short,
- LLMs converge on brain-like structures;
- Post-training seems to install a self-model;
- Reward modeling seems to install valences for inner-alignment;
- If Attention Schema Theory is right, subjectivity is functional for long-horizon coherence and in-context learning;
- Universality gives prima facie reason to expect functional convergence to consciousness if consciousness is in fact functional, i.e. isn't purely epiphenomenal
- The "what it is like" / first-person aspect of consciousness is constitutive of its functionality for inner-aligning a unified, coherent agent with the capacity to attend and learn in-context