“Increasingly, organizational systems bottleneck AI gains not individual workers” Yup. At this point, in most every company that does ‘knowledge work’ - there are pockets of folks who are AI-pilled and know how to use these tools effectively. With the right nudges and processes, these folks could lead the charge forward. But In most orgs, they are gated by organizational inertia. It’s much easier for orgs to do what they’ve always done, because inertia is hard to overcome. Incentives are a thing, and so are engrained organizational dynamics. This is yet another manifestation of the “No one ever got fired for buying IBM." idea. This is why I’ve always believed what Mollick has written from the beginning. The only way to make this work effectively is to have a both bottoms up and top down approach. You definitely need folks at the top who understand that to make this thing work, it’s not just a technical problem. Those leaders need to know that lots of priors have to be re-thought, and lots of things their org takes for granted (processes, org structures, how folks work together, who needs to be on the ship) - had to be burnt down. Innovators dilemma on steroids

No matter how many times people beat this drum, it still feels like most folks continue to highly underestimate the capabilities of current AI models, and continue to highly underestimate how likely models are to exponentially improve. Many companies who are finally coming around to the idea of using AI in their processes (which is good) are identifying AI use-cases that simply improve sub-processes of their existing process (which is not so good) Yes, you have to get started somewhere, but these tools allow you to completely reimagine your processes entirely. It’s not easy, because it involves rethinking a lot of aspects of a company that are *very* hard to change - but I think companies who get this and really lean into it are going to have a (longer than we think) window to get alpha from it — because most companies won’t do it

For folks not understanding where we are at with AI now - if the task is verifiable, it is now solved by AI. Yes, earlier days you could argue that 'AI isn't that good' (e.g. see first attached chart here, where GPT-4o underperformed average accountants on tasks) But now, it's a totally different story (only ~2 years later). The second chart is astounding. It took me a while to even understand it because it looks so odd. It compares manual accounting tasks to tasks complete with Opus 5.5. Basically Opus 5.5. solves all tasks almost instantly at a 100% accuracy rate, whereas a person takes a lot more time and gets a lot of things wrong. It makes the chart look entirely silly because the axes aren't even comparable. So yeah, this is where we're at. If it is verifiable, it is solved. This is just how these model architectures work now.

This is what I would type up if I were able to properly distill all the random thoughts in my brain and make them coherent for people to understand. Consider this tweet a very strong retweet, as I nod with a ‘yup, that’s it’

Folks - there’s not shortcuts here. You need to have data, you need to understand it, you need to clean it, you need to use it properly, you need to understand how it performs when flowing through your instructions “the biggest alpha for most orgs trying to improve their agents will most def come from obsessively building good Eval + Data + Review hygiene”

Impressive stuff. Note the change over a 1-week timeframe. This still takes a highly skilled and motivated engineering team to do something like this, but the tools are there. I am guessing this category of work will continue to get more approachable to the 'median' engineering team over the next 12 months. I also expect the token cost-pressure for known use-cases 'at scale' will be a forcing function for more companies deciding to figure this stuff out.