some cool things i liked about verifiers v1: - my favorite shift from v0 to v1 is the move from an environment-centric abstraction to a rollout-centric one. v1 has this clean abstraction: taskset × harness × runtime → trace tasksets owning both data and scoring is much cleaner. - making the trace a first-class artifact is the biggest upgrade. also liked how graph-based trace storage avoids duplicating shared prefixes and makes long-horizon trace analysis much more practical. - being able to run the same taskset under different harnesses (kimi-code, rlm, codex, etc.) makes evaluations much more useful.

absolutely loving this. for the past few months i’ve also been exploring synthetic and procedural environment generation for agents (though at a much smaller and earlier-stage scale). the projects i’ve been working on (synthetic workspace gym and rlvr gym) include a small set of environment families such as python script repair, pipeline repair, tabular tasks, scheduling, graph planning and retrieval-style workspace tasks, with workspace artifact logging, hidden evaluators, trajectory traces and final diffs. the motivation was mostly curiosity...prompts alone feel too weak as a substrate for studying long-horizon agents. what we really need are executable worlds where agents can inspect state, take actions, receive grounded feedback and fail in ways that are actually analyzable. that is why i’m especially excited by what the prime intellect team has done here. we need thousands of diverse, verifiable, stateful tasks across domains not just static prompt datasets or a handful of handcrafted benchmarks. imo scaling this lets us study curriculum, tool use, verifier design, task difficulty, trajectory quality, reward hacking and long-horizon generalization in a much more systematic way. my own versions are still early and small but it’s really exciting to see PI push this direction seriously: evolving environments, calibrating difficulty and generating broad task distributions across thousands of domains. also i really loved the clean and neat blog post!

i really like the way @sainingxie articulates his thoughts and builds elegant analogies around them. i’d strongly recommend watching the section of the podcast where he talks about research taste. one idea that stood out to me was his view that good research taste means being able to see through the illusions or superficial framing of a problem and uncover its true underlying substance. he connects this beautifully with philosophical ideas which makes the discussion even richer. i also really liked his analogy that great research is a lot like filmmaking or storytelling. btw thanks @eliebakouch for the recommendation!