New on Dedicated Model Inference: canary rollouts. Upgrade the model behind a live endpoint without downtime. Traffic moves from your current deployment to the new checkpoint in gated steps (default 5% → 25% → 50% → 100%). Health checks run before any traffic shifts. After every step, metric gates compare the new model's p95 latency and error rate against the old one. If a gate trips, the rollout pauses at the canary share and waits for you: resume, promote to 100%, or roll back. Three strategies: canary, blue-green, and rolling. Available now via the tg CLI, REST API, and Python SDK. Learn how to start a rollout:

When you add annotations as rollouts in RL training, something counterintuitive happens — good policy actions start receiving negative advantages. OraRL names the problem and fixes it. Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs ❓ What goes wrong when you add oracle rollouts to GRPO training? 💡 "Advantage Inversion" occurs. Standard GRPO normalizes advantages across the rollout group using group mean and variance. Injecting a high-reward oracle rollout raises the group mean, which in turn causes policy rollouts that outperform the current average to receive negative advantages — a sign flip. Measuring 92,024 rollouts, naive oracle mixing inverted advantages in 42.5% of groups and 22.4% of rollouts. ❓ How does OraRL solve this? 💡 Five decoupled components. First, an oracle-free on-policy baseline is constructed without variance normalization (inversion is impossible by construction). A directional gain from the oracle-policy gap amplifies above-mean rollouts. The oracle advantage itself is calibrated against the strongest on-policy signal so it can't dominate the update. Sign-balanced advantage pruning retains equal numbers of positive and negative rollouts, compressing the residual flip rate to 0.3% and delivering a 1.48× speedup. ❓ How does it perform? 💡 Video-ORA-9B scores 73.1 on VSI-Bench, surpassing GPT-5 (55.0) and Gemini-3-Pro (55.1) by 18+ points. ReasonVOS segmentation improves +42.2 J&F and VideoHolmes gains +15.2 points over backbone. Training cost is just 2.2× SFT — under half the 4.9× overhead of GRPO with chain-of-thought. ❓ What about inference efficiency? 💡 No chain-of-thought is required, so P90 inference latency is 25.15 seconds vs 62.67 seconds for the CoT backbone. On data scaling, OraRL gains +5.2 points at 100k prompts versus +2.8 for GRPO, showing better sample efficiency throughout. #VideoMLLM# #ReinforcementLearning#

Paper Instruments benchmarked sandbox providers for their RL rollouts, and E2B's tool execution came in up to 3x faster than every other sandbox provider they tested, reducing idle GPU time and training cost. @paperinstr trains frontier models for knowledge work (consulting, finance, banking, law), running thousands of concurrent GRPO rollouts, each needing the solver and grader fully isolated from one another to prevent reward hacking. Every rollout also needs to start from the exact same state in order to reflect the policy being trained accurately. It also needs to boot and execute fast, since the GPU sits idle waiting on the rollout to finish. With @e2b, Paper Instruments isolates each rollout in a microVM, builds the initial state from a template, then snapshots the running sandbox, letting thousands of rollouts launch straight from the snapshot instead of booting cold, so they get consistency and speed while reducing training cost. Read the full case study:

The hard part of training agents is the infrastructure: long rollouts, slow environments, tool calls, API delays, and keeping training compute busy while all of that happens. Berkeley CS294-196

On-policy distillation has the same systems bottleneck as RL: rollouts dominate training time on reasoning workloads. Going async fixes throughput but feeds the learner stale-policy data, and what staleness does to OPD specifically was unstudied. The clean finding is that KL direction decides robustness. Teacher-weighted forward KL shrugs off stale rollouts, student-weighted reverse KL breaks under them, and for the reverse-KL case nothing from async RL beats just recomputing the signal under the current student. Finite teacher-score caches then turn the estimator into a bias-variance tradeoff, which is the case for multi-sample Monte Carlo. AsyncOPD: How Stale Can On-Policy Distillation Be? Paper:

“World in World: Explore the World with World Models” Video world models can generate long rollouts, but controlling them from new viewpoints usually needs task-specific training or adapters. This paper instead turns source frames, geometry, and past generated states into visual evidence that a frozen world model can directly read through self-attention. This then gives training-free camera-controlled rerendering with better long-horizon consistency, unseen-view completion, and lower camera error than prior methods.

Schmidhubers argument for consciousness in LLMs: - the one constant in training and during RL rollouts that influences the next token prediction is the model itself, so models should learn to model themselves to better predict what comes next claiming that this results in conscious AI is silly I think at best this gives you self-awareness/some metacognition which models already have, but not consciousness

my friend is asking why so much demand for sandbox as service startup and we were able to explain because of rl rollouts but why labs may take their service instead of doing it themselves. how can they sustain in long term.