It's felt like harness month on Twitter. We're seeing much faster and cheaper gains on a bunch of benchmarks by focusing on harness improvement rather than model improvement. Allowing the harness to be updated during use is just another form of continual learning. The common issue with continual learning is catastrophic forgetting. But a lot of us have an 80/20 situation where most of our agent usage is in the same domain, so it's kind of wasteful to not specialize the harness for the environment it's being used in. I like the implicit bet the Exo project makes -- allowing specialization beyond just memories and skills may lower the floor, but I think it raises the ceiling more. Excited to have it in the next (not yet announced) @LaudeInstitute Slingshots batch. @AlexKrentsel is also an absurdly natural podcast guest. The motivations for the project are well explained on @swyx's recent Latent Space podcast:

The guidance to AI to "talk to me like I'm a sharp CEO who has no project context" feels particularly appropriate given the "ICs now act like managers of agents" trend we're seeing. It's a very parallel situation, actually. In a big company, you often need to prep briefs for execs who (if they're good) have strong instincts on the shapes of solutions that do and don't work and a deep understanding of org or company priorities so they can decide which proposed solution has the most favorable tradeoffs. The briefs need to be very info dense (not novels) but also super clear and standalone (not relying on context from past meetings that they either weren't in, or were dozens or hundreds of meetings ago for them). It takes a lot of work to make a good brief with those properties, a la "it's easier to write a long letter than a short one." But...how sweet if you can get to the point where that's just how your agent responds to you. You have instincts and understand your priorities better than it does, and it does the legwork to lay out the tradeoff space succinctly and clearly so you can weigh in as efficiently as possible.

The simpler the algorithm, the more willing I am to believe that it's a real breakthrough and not just overfitting to a niche domain or a few datasets (a sort of meta "bias vs variance" tradeoff in action). What got me excited about explorative modeling is that this algorithmic insight (1) is just "add a for loop here" (doesn't get much simpler than that), (2) shows gains across video, image, and text, (3) shows even larger advantage at larger scale, and (4) results in pretty substantial improvements ("X times better", not "X percent better"). Congrats to @AlexiGlad, @hengjinlp, and @du_yilun on exciting results! It's been fun watching this come together -- @LaudeInstitute Slingshot time & resources well spent. With more results to come...

The bitter lesson rejects domain-specific structure. Problem decomposition & recombination (what RLMs make first-class) is about as general-purpose of a problem-solving tool as it comes. Reminds me of the march of processor speeds: CPUs got faster every year until ~2004, when clock speeds hit a heat wall around 3 GHz. Transistors kept getting cheaper, but the only way to keep getting gains was to decompose workloads into pieces and go parallel. As the tasks we give LLMs get longer (context length/time horizon) and more complex, scaling up to 100T parameters will probably help...but my money is on big-O-improving innovations like this improving generalization faster.