so like it starts as a general-purpose harness but technically by the end of the task it has self-evolved into an ARC-AGI-3 specific harness
did some yapping about how we’re approaching continual learning :)
it's worth noting that due to rising costs of pretraining, the frontier isn't *that* accessible. training a 5T-A100B model on 50T tokens on GB300s costs like $10 million for the hero run. double it for RL. there's only a few thousand companies in the world that could justify it
Show more
wow double fallback. asked fable to help with some slides. they’re gonna route me to haiku 3 next
if you work at anthropic and want a prime intellect hoodie DM me
offer is valid only for members of technical or non-technical staff that are *not* about to leave and start a neolab (honor system)
you can now train:
- a 100B-parameter reasoning model
- for 40-turn SWE agent tasks
- in your own coding harness
- for 1000 RL steps
- on just 6 H200 nodes
- in under 2 days
infra co-design is magical
Show more
this has been a labor of love for months with @mikasenghaas and @xeophon driving incredible progress, and `v1` is now finally ready for prime time
we set out to fully modernize `verifiers` for the agent harness era, and unlocked some insane efficiency gains along the way
Show more
are you starting to see why called it that
unless it is *consensus* at the frontier that openai + anthropic are the only players, the talent will flow, and capabilities trend towards common knowledge
and there isn’t even consensus about whether msl/deepmind/spacexai are real players, not to mention the dozens of neolabs
Show more
new algorithm called "Dr. OPSD", it stands for "on-policy self-distillation done right"
we raised $130M @ $1B for our series A
>$100M run rate
we’re just getting started
there will soon come a time, perhaps next year, when voice interaction + multimodal reasoning models are good enough and fast enough that you can actually just program like this
Show more
just heard about this new ai + human data company called omegle where it's like chatgpt but also the new poke human feature where you get to chat with a real human and have them answer your questions. seems kinda like AGI
Show more
yeah dude in-context learning is all you need don't worry. btw you gotta check out the new model it's better because they trained it a lot more on a bunch of stuff
the 4 types of bars in san francisco
wow fable just completely one-shotted this single-page html artifact
