Incoming UC Berkeley prof. @sayashk reveals AI reliability is improving 4–10X slower than capability and why 90% accuracy can still be catastrophic: "We came up with a list of metrics that other industries had used, industries like aviation and nuclear safety and so on. We tried to port these metrics to the AI agent space." "Reliability improving much more slowly than capability. Based on these early results, it was between four and 10 times as slow as capability improvements have been in terms of accuracy numbers." "That's why AI's effect on the job market has been augmentation not automation. You can't accept an error rate of even 1% or 5%." "Rabbit R1 and Humane Tech Pin were personal agent assistants. If your product orders your DoorDash food to the correct address 90% of the times, it's a catastrophic product failure because 10% of the times your users are mad at you." @UCBerkeley