BREAKING:
@GeneralistAI just shipped a one-shot model for robots!
Called GEN-1.5, it is a robot foundation model from Generalist AI that takes in video (a 30-second memory window) plus language, proprioception and other sensors, and outputs 100 Hz action trajectories for manipulation.
It was pretrained for over eight months on the company's proprietary data engine of real activities captured in homes, warehouses and factories (1,891,392 scenes), with no simulation data.
It is the successor to GEN-0 and GEN-1, and the blog's claim is that GEN-1.5 shows one-shot and few-shot adaptation, compositional chaining, human-to-robot imitation and zero-shot sim transfer, none of which it was explicitly trained to do.
Unfortunately, model size, robot hardware, inference latency, model weights, and pretraining data volume are not disclosed.
Generalist states plainly that it made no architectural changes to promote in-context learning, no inner or outer meta-learning loop, and yet the model adapts from a single 3-to-12-second demonstration (59% one-shot average across 10 tasks).
That is the actual GPT-3 moment ported to control: rather than engineering few-shot adaptation (the entire meta-learning literature), you scale pretraining and the adaptation falls out.
It is the maximalist counter to the Patch Policy /
@LerrelPinto /
@ylecun "engineer the representation, keep it small" camp -> Generalist bets on emergence from scale, not architectural cleverness.
Few-shot adaptation changes the weights by less than 0.15%, which means the model already contains the skill.
Ten gradient steps on five minutes of data lift success to 83%, but move the weights under 0.15%, so adaptation is retrieving a capability the prior already holds, not learning a new one.
That is the strongest version of the thesis: the prior is the moat and real robot data barely addresses it.
Generalist's twist is that the nudge needed is almost nothing, which inverts the "we need robot data to adapt the model" premise under the VLA stacks.
It pretrains on real data only, with zero simulation, yet claims zero-shot transfer to sim, the reverse of the usual direction!
The data-strategy claim underneath is that broad-enough real pretraining subsumes sim as just another domain, rather than sim bootstrapping real, real-to-sim instead of sim-to-real.
I wonder how did they built their data pipeline, about which they unfortunately do not provide details.