Register and share your invite link to earn from video plays and referrals.

tom cunningham
@testingham
Economics & AI @ @METR_Evals (ex-openai)
3.1K Following    10.7K Followers
New post with Nate Rush: Have we seen an acceleration in discoveries? Many plots & some tentative conclusions: 1. Cyber: ⤴️ sharp acceleration 2. Math: ↗️ some acceleration 3. Algorithms: ➡️ no clear acceleration
Show more
Come work with us, the going's tough & the tough are going.
Q: has AI accelerated aggregate discovery yet? My very general impresions, would love others' thoughts: 1. Vulnerability discovery is up a lot. Between 2X-5X increase in volume, although the average severity has fallen somewhat. 2. Algorithmic efficiency hasn't moved much. E.g. nanogpt, SAT solvers, compression efficiency, chess algorithmic ability. They haven't shown noticeable changes I believe. 3. Math is hard to judge. Erdős problems are certainly falling faster than historical rates, but it's mainly the obscure ones. We don't seem to be tightening upper and lower bounds on unknown quantities noticeably more quickly AFAICT. There are some prominent AI discoveries (e.g. unit distance) but I don't know what's the denominator, i.e. the average flow of comparably important results, my guess is it's still small. 4. Papers/code volume is up a lot. Total arXiv papers and github code is up a lot, but no good way of judging how the quality has changed.
Show more