Dario & other AI insiders Scott aligns w/ are giving mid to high single- & multiple-digit p(doom) numbers, but when this question is put to superforecasters they give it 0.3% chance by 2030, 2% chance by 2050. Tetlock's own book says the further out you get the less the method actually works. So the highest-quality superforecaster number 0.3%.

Here's why I like this response (I got one more along these lines but not this detailed). My whole criticism of METR is that that they're a collection of LW forum posters whose goal is to stop AI development, & to do this they've stood up a benchmarking & safety org that A) aims to make the already decided (to them) case that AI is moving too fast & is unsafe, & B) is not staffed by people who know anything about benchmarks or safety as an actual discipline. This post essentially concedes the two-part (A & B) motivation criticism, i.e. that the whole plan here is the Pause, & they don't need to engage w/ legacy tech safety concerns because the plan is to Pause & then learn about that stuff & apply it to AI. The post also sort of implicitly supports my first criticism (that they're a bunch of LW posters who cooked up a plan to Pause AI) by responding to a criticism of METR with essentially, "hey there! LW posters DO talk about this stuff!" At any rate, the core disconnect between me & this person I'm QTing & METR & most of LW, is that I am in the "Normal Technology" camp. Also, normal tech can be extremely disruptive & dangerous, which is why safety matters, & the approach I'm criticizing (& that he's describing!) is inimical to safety IF we're dealing with a normal tech. But if LW is right and AI really a totally not-at-all-normal tech & we're 10yrs away from Dyson Spheres (per Cotra in other threads), then ok yeah I suppose any and all types of resistance are licensed, including standing up a benchmarking & safety org that has very little connection to anything anyone else can recognize in the pre-existing areas of benchmarking and safety. So in that respect, I get it, tho I strongly disagree. At any rate, it really does matter where you fall on this matter of "This is an emergency!" or "This is a dangerous but normal tech!", for exactly the reasons I'm talking about here. Because if one of those is true, then the other one is unsafe.

This is the best thing I've read on EAs & rationalists. It's by @vgr, & it aligns with the critique of thier AI doomerism I'm slowly building in my series of articles on here.

I would say this is the dumbest episode of mass delusion I've ever seen in my life, but I lived through the 2020s. Anyway, I truly did not see the leading lights of the industry joining forces with the anti-datacenter crowd, yet here we are.