Dwarkesh Patel(@dwarkesh_sp):In the early days of OpenAI, @johnschulman2 didn't think next-token prediction was going to lead to intelligence, because it would get swamped by noise. He explains it's always been hard to apriori predict what techniques will elicit out-of-distribution generalization.

Dwarkesh Patel

@dwarkesh_sp

Host of @dwarkeshpodcast

Joined December 2019

1.1K Following 278.5K Followers

Dwarkesh Patel@dwarkesh_sp

2026.09.11 21:31

In the early days of OpenAI, @johnschulman2 didn't think next-token prediction was going to lead to intelligence, because it would get swamped by noise. He explains it's always been hard to apriori predict what techniques will elicit out-of-distribution generalization.