dots studio(@dotsstudioai):Under the hood of dots3-note Preview: UltraEP ⚡ Another piece of the infra behind dots3-note Preview is UltraEP, our software–hardware co-designed runtime load-balancing system for fine-grained MoE models. UltraEP dynamically replicates hot experts and redistributes tokens based on real-time routing loads, leveraging high-bandwidth NVLink interconnects to keep expert workloads close to ideally balanced. The result: <300 μs critical-path overhead and 94.3% of ideal balanced performance on average. We’ve now open-sourced it.

dots studio

@dotsstudioai

rednote's AI lab, creating frontier intelligence for daily life.

加入 August 2026

14 正在关注 2.6K 粉丝

dots studio@dotsstudioai

2026.08.20 08:38

Under the hood of dots3-note Preview: UltraEP ⚡ Another piece of the infra behind dots3-note Preview is UltraEP, our software–hardware co-designed runtime load-balancing system for fine-grained MoE models. UltraEP dynamically replicates hot experts and redistributes tokens based on real-time routing loads, leveraging high-bandwidth NVLink interconnects to keep expert workloads close to ideally balanced. The result: <300 μs critical-path overhead and 94.3% of ideal balanced performance on average. We’ve now open-sourced it.

显示更多