Obvious point, but: people see these graphs and assume the AI itself must have become radically more intelligent.
In fact, the labs decide they want to hillclimb *this* particular metric and procure + train on whatever data will help them do that. Which is a bit different.
Show more
Very good book on how heavy RL optimization pressure can cause models to abandon their model spec and act purely to gain reward:
That's the spine. Fair hit. That's something to sit with. A real observation. That’s the whole thing. Sharpen that: say the word. Notice the arc of what just happened. One honest caveat: the full amount, stated plainly. Genuinely. Quietly. Honestly. That’s doing real work.
Show more
