Xinyu Zhang(@xinyzng):We stress tested Ray-2.58.0 for improvements on fault tolerance and reliability by scaling video data processing pipeline to caption 0.6 Petabytes of video data on 1600 Nvidia Blackwell GPUs in collaboration with @CoreWeave With the new release Ray Data shows: • 𝟲𝟬％ higher throughput for streaming data processing • 𝟮𝟯％ faster for batch inference • 𝟮𝟰％ faster for Ray Data shuffle This will significantly improve how world model and physical AI companies handle pre-training data. Blog:

Xinyu Zhang

@xinyzng

Scaling RL/Inference infra with @raydistributed backed by @anyscalecompute | Prev @LTIatCMU @Meta

参加 November 2023

1.1K フォロー中 903 ファン

Xinyu Zhang@xinyzng

2026.08.31 17:52

We stress tested Ray-2.58.0 for improvements on fault tolerance and reliability by scaling video data processing pipeline to caption 0.6 Petabytes of video data on 1600 Nvidia Blackwell GPUs in collaboration with @CoreWeave With the new release Ray Data shows: • 𝟲𝟬％ higher throughput for streaming data processing • 𝟮𝟯％ faster for batch inference • 𝟮𝟰％ faster for Ray Data shuffle This will significantly improve how world model and physical AI companies handle pre-training data. Blog: