Macrocosmos(@MacrocosmosAI):We trained Orion-16B without a reserved cluster anywhere in it. The numbers underneath that: It sustained around 80k tokens per second across 4090s, 5090s, A100s, L40s, and A6000s, at roughly 20% MFU. The accessible pool came to about 18 B200s equivalent at FP16. Nodes dropped, throughput swung, and machines ran inconsistently throughout. Training carried on without direct intervention. Imperfect compute, one finished model.

Macrocosmos

@MacrocosmosAI

Building distributed intelligence on Bittensor. SN1, 9, 13

加入 March 2024

22 正在关注 7.6K 粉丝

Macrocosmos@MacrocosmosAI

2026.09.23 17:15

We trained Orion-16B without a reserved cluster anywhere in it. The numbers underneath that: It sustained around 80k tokens per second across 4090s, 5090s, A100s, L40s, and A6000s, at roughly 20% MFU. The accessible pool came to about 18 B200s equivalent at FP16. Nodes dropped, throughput swung, and machines ran inconsistently throughout. Training carried on without direct intervention. Imperfect compute, one finished model.

显示更多