Register and share your invite link to earn from video plays and referrals.

mashrur haider
@Mhr1036
product | post training @nebius | opinions are my own
Joined January 2025
263 Following    653 Followers
To train an AI agent on a real job, you have to define the job twice. First, as work the agent can attempt. Second, as a test of whether it succeeded. The gap between those definitions is where a lot of reinforcement learning engineering lives. DeepSeek-V4.1-Flash and GLM-5.3 describe complementary parts of this problem. DeepSeek organizes what to teach. Its task-generation pipeline uses a growing knowledge graph to connect broad domains with specific concepts. Agents retrieve relevant papers, articles, and code, then use those materials to generate tasks. This gives the pipeline control over coverage, including specialized knowledge that ordinary sampling might miss. DeepSeek, §4.2.2 GLM makes tasks executable and checks their rewards. Research agents turn patterns from professional work into environments with dependencies and hidden state. A judge agent attempts the tasks to check solvability. For some tasks, the pipeline generates verifiers without access to the reference solution. Solver attempts help expose reward shortcuts. The verifier must pass checks covering a known solution, doing nothing, and leaving the task unfinished. GLM-5.3 Consider an assignment to make a training job faster. The first definition includes the code, workload, dependencies, hardware, and constraints. The second must determine whether the change produced a genuine speedup while preserving correctness. A model might improve the measured runtime by skipping computation or exploiting a weakness in the test workload. The score rises, even though the intended job remains unfinished. That is why an executable environment is only the beginning. It also needs to teach something relevant, distinguish useful progress from shortcuts, and remain informative as the model improves. Replaying an easy task can produce more trajectories with little additional learning. Generating diverse tasks can broaden practice without guaranteeing that it matches the work users need done. And a verifier can be perfectly consistent while checking the wrong thing. My takeaway is that environment quality should be judged by transfer: does training here improve performance on unfamiliar work? I would test that on tasks kept separate from environment development, compare against a baseline under a fixed training budget, and include the cost of building and validating the environments. GLM explicitly acknowledges that its pipelines still require meaningful human involvement. That work belongs in the economics. GLM-5.3 Working on post-training at Nebius, this sharpens the infrastructure opportunity for me. Teams need reproducible environments, efficient rollouts, inspectable rewards, and evaluations that reveal whether the learning transfers. Domain expertise becomes valuable when a team can encode realistic constraints and recognize failures that a generic checker would miss. My bet is that this capability will become an important advantage in post-training. A training environment earns its value through the work the model can do after training, including work that environment never contained.
Show more