Register and share your invite link to earn from video plays and referrals.

Alex Veremeyenko
@alex_verem
open source, local ai, ai for good also know as: @alex_prompter
Joined May 2023
626 Following    103.2K Followers
A lab used AI agents to help build its AI model. When agents got stuck, humans took over 0.7% of the time. The team behind a new open model called Atria Dawn studied their own work while they built it, and they just published the results. They analyzed 769 task records from 56 people, along with the logs of what the AI agents did. AI was involved in 96.5% of the tasks where people recorded whether they used it. The number that got my attention is a different one. For one in three finished AI-assisted tasks, the researchers said the work would not have been possible without AI. They didn't mean it would have been slower. They meant nobody would have attempted it. The agents also came up with a lot of the ideas. In more than half of the decisions about methods, the AI proposed the option and a human picked it. Humans still kept the final call. People made the final choice in 85.5% of method decisions and in 93.4% of decisions about what the goal was. When an agent got stuck, a human stepped in three times out of four. But the human almost never took over the work. That happened in 0.7% of cases. Most of the time the person explained what was missing, and the agent went back and fixed it. One number is climbing fast. In four weeks, the typical number of actions an agent took per human instruction went from 11 to 28.5. The authors are direct about what that means. Each human decision now depends on more AI work than one person can check. They warn that people could end up as reviewers who can only say yes. They also admit that many of their own researchers ran agents with the permission checks turned off, because approvals slowed the work down. Convenience decided how much authority the AI got, and nobody planned it that way. The model itself took the top score on 5 of the 16 benchmarks the team reported. The record of how they built it is the more useful half of the paper.
Show more