dots studio(@dotsstudioai):Under the hood of dots3-note Preview: UltraEP ⚡ Another piece of the infra behind dots3-note Preview is UltraEP, our software–hardware co-designed runtime load-balancing system for fine-grained MoE models. UltraEP dynamically replicates hot experts and redistributes tokens based on real-time routing loads, leveraging high-bandwidth NVLink interconnects to keep expert workloads close to ideally balanced. The result: <300 μs critical-path overhead and 94.3% of ideal balanced performance on average. We’ve now open-sourced it.

2026.08.20 08:38

Under the hood of dots3-note Preview: UltraEP ⚡ Another piece of the infra behind dots3-note Preview is UltraEP, our software–hardware co-designed runtime load-balancing system for fine-grained MoE models. UltraEP dynamically replicates hot experts and redistributes tokens based on real-time routing loads, leveraging high-bandwidth NVLink interconnects to keep expert workloads close to ideally balanced. The result: <300 μs critical-path overhead and 94.3% of ideal balanced performance on average. We’ve now open-sourced it.