Over the past few months, we’ve been thinking a lot about what it would actually take to build agents that continuously improve from their own experience. Today, we’re open-sourcing our continual learning infra, Reef. The idea is simple: instead of treating inference as the end of the pipeline, Reef turns live agent interactions into a continuous learning loop. It serves real applications, captures trajectories and feedback as structured experience, and lets different learning recipes use that experience to improve the system. What evolves isn’t just the model. Reef is designed to evolve the whole agent — model weights and the harness — then evaluate, version, and safely deploy those updates back into serving. Really excited to finally share Reef we’ve been building toward continual self-improvement! Come and check it out: And join the Discord group for more updates: