In RL training, a vLLM rollout engine and a Megatron trainer can run the same policy yet disagree on a token's logprob due to floating-point non-associativity.
SkyRL's IsoExec combines an execution contract with a unified model, aligning rounding-sensitive execution choices across rollout and training.
Bitwise parity holds across different TP, EP, and SP layouts. For Gated DeltaNet, the chunkwise-parallel recurrent algorithm makes parallel training and prefill bitwise identical to recurrent decode.
Qwen3.5-35B-A3B, DAPO, 8xH100, 50 steps: logprob diff 1.6e-2 to 6.7e-7, full-step overhead 25.3% ✅ vLLM's scheduler and CUDA graphs still apply.
Thanks to
@JiangAlexander1 and the SkyRL team at
@NovaSkyAI.
🔗