the other part of this is how almost everything with traction in the AI sphere is increasingly a lightly-veiled quasi-technical meme or a shallow dunk of some kind by surprisingly many bitter people; this is X of all places but there really used to be substance not long ago

Introducing Prime Agent: A self-improving RLM harness for coding and long-running autonomous tasks. Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.

whenever something is crazy hype and is followed up by an armada of PR, you gotta bring out the bullshit detectors so i cloned the repo and did a little digging and found a lot of sound and fury. please enlighten me if any of these facts are untrue, would love to be educated RLM claim either overblown or false this doesn't even appear to be RLM ( RLM is "cool" as a concept because there's a premise of going infinitely deep to do arbitrary decomposition of a problem. in practice though it's not really hard, you just make subagents callable as functions and wrap that in a code execution tool. the reason RLMs have not been productionized and is just seen as a research ditty is that 1) going arbitrarily deep down the stack is a shitty thing to do in production without guardrails, 2) most problems do not need more than 2 layers of decomposition, and 3) agents are bad at banana phoning each other and will fail to preserve the subtleties of human requests, leading to a lot of inefficiencies and bad behavior. to solve these problems, there needs to be a fair bit of innovation, either on model training end or on harness end, to mitigate these issues. so what were the innovations in prime agent? they set RLM_MAX_DEPTH = 1? ... ok what? so theres no innovations, it's just calling subagents like every other harness in existence. if RLM MAX DEPTH is 1, it's not even RLM? it's just... a harness? so now i try to look up to what depth the ARC AGI 3 hillclimbing was done at. not surprisingly this doesnt seem to be disclosed on the blogs / posts. if it's just 1 layer deep, that's just shady marketing. a real innovation or contribution to harness engineering would be to detail the things that were improved to get a max depth of >3 to work without blowing up your computer, wasting a bajillion tokens, taking forever, or having agents go off the rails. i didnt find any 97% is just overfitting after hillclimbing on a public eval ok folks. it's nice that opus5 does better than sol than terra, but at the end of the day, we're looking at a public eval in which problems and solutions are open access, so any agent harness can arbitrarily overfit to the task set however much it wants to. if you do a codex or claude loop with some semi shady prompting to just get 100%, im pretty sure it will too. now obviously this is almost certainly not what the prime intellect team instructed their agents to do (otherwise you'd see that terra get 100% after a couple iterations), let's just all remember that there is no train / test split, and that ARC explicitly says - Public-set scores are vulnerable to task-specific overfitting. - They are “emphatically not” valid evidence of progress toward AGI. - Real generalization should be tested on the 55 semi-private or 55 fully private games. It's difficult to assess the degree of overfitting in Prime Agent's self-improving harness (which, btw, is not terribly different from nous agent or really any self-improving system that just looks at old JSONL agent trajectories and suggests skills / memories / system prompt appendments / tools / extensions). the more overfitting, the less impressive the 97% performance is. a counter-example that i found to be legitimately impressive was -- minimal / almost no overfitting, stupidly simple and general solution, and 95% on ARC AGI 3. i think the fairest thing to say about this system, given it's independent daemon system for managing subagent lifecycles, is that Prime Agent contributes a fairly robust persistent asynchronous agent-process tree. It does not demonstrate a solution to scalable deep recursive agency

Prime Agent was designed as a coding agent, but also can be used for general agentic tasks in any domain. Prime Agent can autonomously run on challenging long-horizon settings like Factorio and MazeBench and yield strong results relative to existing harnesses.

Introducing Prime Agent: A self-improving RLM harness for coding and long-running autonomous tasks. Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.