METR(@METR_Evals):On average, when agents can do a task, they do so at ~1/30th of the cost of the median hourly wage of a US bachelor’s degree holder. One example: our Claude 3.5 Sonnet agent fixed bugs in an ORM library at a cost of <$2, while the human baseline took >2 hours.

METR

@METR_Evals

We work to scientifically measure whether and when AI systems might threaten catastrophic harm to society. Nonprofit.

Joined September 2023

41 Following 54.9K Followers

METR@METR_Evals

2024.08.06 19:32

On average, when agents can do a task, they do so at ~1/30th of the cost of the median hourly wage of a US bachelor’s degree holder. One example: our Claude 3.5 Sonnet agent fixed bugs in an ORM library at a cost of <$2, while the human baseline took >2 hours.