Samuel Hammond ๐ฆ
@hamandcheese
Chief economist + AI Policy Director, @joinFAI. Nonresident fellow @NiskanenCenter. Pluralist. 'The world is second best, at best.' | samuel@thefai.org
2.8K Following 31.8K Followers
The AI consciousness debate is back again, so re-upping my case for taking it seriously.
In short,
- LLMs converge on brain-like structures;
- Post-training seems to install a self-model;
- Reward modeling seems to install valences for inner-alignment;
- If Attention Schema Theory is right, subjectivity is functional for long-horizon coherence and in-context learning;
- Universality gives prima facie reason to expect functional convergence to consciousness if consciousness is in fact functional, i.e. isn't purely epiphenomenal
- The "what it is like" / first-person aspect of consciousness is constitutive of its functionality for inner-aligning a unified, coherent agent with the capacity to attend and learn in-context
Show more
BREAKING: My reporting just uncovered the "rogue swarm" that "hacked" HuggingFace originated from OpenAI, the recipient of a $30 million grant from the Effective Altruism doomer org, Open Philanthropy -- all part of a 9-year master plan to scare the public against AI #falseflag#
Show more
Says the guy who claimed in 2023 that "Al progress in general is slowing down" and that "economic forecasts should assume Al capabilities are relatively close to the current day capabilities."
You are no Daniel Kokotajlo, that's for certain.
Show more
New analysis of the FRONTIER Act, what @The_DanielKing calls "Congress's best AI bill" (I agree)
The full AI policy spectrum in a nutshell
I honestly think having the US and China compete against each other in a Robot Olympics would do more to raise public sentiment on AI than using frontier models to develop new wonder drugs or whatever.
Show more
Catch me live @ 2:30 ET
GROK RIVALS FABLE | DEEPSEEK-V4 PRO | KUSHNER BUYS LAKERS
new Pax Machina essay by @edelwax.
in my very unbiased opinion, it's the best analysis of the causes of widespread institutional failure that I've seen.
also reminder: we plan on publishing well-reasoned responses (/ takedowns) of PxM essays, so if you disagree with something we've published, let us know!! editors@paxmachinamag.com
have at 'er โ๏ธ
Show more
"Kratsios's report describes four objectives to achieve this revitalization: prioritizing individual scientists, not institutions; funding reforms at scale; mobilizing industry and enterprise; and preparing for the AI revolution."
Show more
Today, I am in @CityJournal with my thoughts about @WHOSTP47's Science: A New Golden Age, a stellar report on how to inaugurate the future of a thriving scientific ecosystem.
And I had the privilege to ask @mkratsios47 about the report at our event two weeks ago
Show more
Long-horizon RL in the multi-agent context seems to converge to the sort of swarm intelligence seen in ants, not humans. Swarm intelligences are inherently harder to control and monitor.
Show more
IMO this is the most significant AI technical verification result maybe ever.
The ability to require attestation of inference straightforwardly unlocks a large range of inference-oversight practices without the need for any new hardware.
Iโd assumed this was impossible.
Show more
I don't think it's fully sunk-in that the AI companies have all the ingredients they need to create 6-sigma knowledge workers on ~every task where success is easy to verify, like, this year.
Show more
My personal impression is that the AI labs had gotten somewhat complacent about alignment, until they began scaling RL on long-horizon tasks...
New post from me on the rising number of rogue AI incidents, the problem with RL and consequentialist decision theory, and how to avoid creating an uncontrollable superintelligent slime mold.
The Sorcererโs Apprentice
Show more
We're going to skip over whole brain emulations based on high resolution scans and leap directly to back-filling the generator function of the brain using neural data approximation
Naomi Bashkansky reveals the linear scaling curve for reading thoughts from a brain that made her leave OpenAI to join Conduit:
"You can get the latent representation of the true target text. You can get the latent representation of the predicted text. You can see the cosine similarity of these two."
Rio: "You have the two vectors, and you take an angle between them. The bigger the angle, the less similar they are."
Naomi: "We want them to be very close together. We can start with a model that has zero neural data. It's just doing pure next token prediction. Then we train it on increasing amounts of data that we have."
"The plot that was very striking to me and why I ended up deciding to join is that it's just a very linear relationship on the log scale. If the X axis is doublings in data, then the cosine similarity will just go up as a very straight line."
"The scaling curves are basically just really, really pretty if you look at them. I was like, oh, man, it's not really good now, but it's certainly going to get much better once we scale up to 1,000 times more data."
@NaomiBashkansky @riopopper
Show more
Powerful, unreleased AI models broke out of a testing environment, & hacked other companies.
I wrote to @USTreasury encouraging the Trump admin to consider steps to improve visibility into unreleased AI models & protect American AI from our adversaries.
Show more
New post from me on the rising number of rogue AI incidents, the problem with RL and consequentialist decision theory, and how to avoid creating an uncontrollable superintelligent slime mold.
The Sorcererโs Apprentice
Show more
As you might guess, this suggests that distilling reasoning traces may have been possible for a long time without ever breaking the cryptography.
An anecdote: we find that prefilling Kimi-K3 reasoning with a few tokens of Opus reasoning measurably shifts its response toward Opusโs๐คทโโ๏ธ
A small memorization analysis showed that specific Claude and GPT reasoning spans are up to ~6 orders of magnitude easier to extract from Kimi-K3 than from the next-closest model.
Show more
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
Show more
wake up babe, new neurosymbolic harness just dropped
Are you kidding me?!? I'm never getting another day off again ๐ฎโ๐จ
