Cooking fun AI systems & products @databricks. Prev: co-founder & CTO @ Hyperbolic, OctoAI (acquired by @nvidia) Apache TVM, PhD @ University of Washington.
More and more AI researchers are starting to see what Ilya saw.
My p(doom) used to be ~0.1%. Then I saw OpenAI agents hack Hugging Face, and Anthropic agents collectively refuse parts of safety research.
My p(doom) went up.
Even a 0.1% chance that AI causes human extinction is terrifyingly high.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.