가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Mike Knoop
@mikeknoop
co-founder @ndea and @zapier @arcprize
가입 July 2009
368 팔로잉 중    25K 팬
On the limits of RSI / intelligence -- The below chart shows that intelligence is not some unbounded scalar stat. For any given situation, there is an optimal decision. You cannot be smarter than optimal. Therefore intelligence can be measured as a ratio of how good your decision is against optimality (over all decisions). It's capped at 100%. And Astra is already 80% optimal on ARC v3 speedruns. The most plausible areas for RSI to play a meaningful role near-term are: 1. Horizontal data acquisition. Models are limited in generality by the knowledge in the weights. RSI could automate data generation or gathering which enables them to in-paint knowledge gaps faster and faster. 2. Efficiency / cost. We are very far away from optimality. This is a good candidate for autoresearch. And cheaper models means even more autoresearch.
더 보기
On the 25 ARC-AGI-3 public games, GPT-6 Astra scored 100% using the new provider adapter harness. But that score alone doesn't show just how efficiently it solved them - or how close it came to the fewest-action solutions anyone has found, including runs produced through extensive software-assisted search. The chart below compares three data points for each Public game: 1. The human baseline, calculated by summing the upper-median human action count for each level 2. The number of actions Astra used at max reasoning with the new provider adapter harness, which preserves opaque reasoning across requests and enables auto-compaction 3. The best known valid fewest-action run on each game - aka the speedrun I asked Peter Findley, an engineer who holds or ties the speedrun record on all 25 games, how he produced his runs. He said he used Codex CLI and a custom harness combining object-oriented memory, layered metacognition, skills, and hooks. He used extensive breadth-first search on a 256 GB RAM machine, but says Monte Carlo tree search with negative perturbations found what he described as the global minima. In short, he put a lot of work into programmatically achieving speedruns. So how did Astra do? It crushed the human baseline, using 6,485 actions compared with 17,135. That efficiency, combined with completing every level, is how it achieved 100%. Almost as remarkable is how close it came to several speedrun records. It tied three games (ar25, ft09, and sb26) and came within 5% on three more (s5i5, tu93, and dc22). Summed across all 25 games, the speedruns used 5,502 actions. Astra used 6,485 - 62% fewer than the human baseline and just 18% more than the speedruns. The fact that it came this close in a single recorded run per game, without executing code or repeatedly replaying the games, is incredible. It speaks to how capable Astra is, at least on ARC-AGI-3-like tasks requiring world modeling, spatial reasoning, rule induction, state tracking, planning, exploration, adaptation, and efficient problem-solving.
더 보기