Horace He(@cHHillee):Also, their cache hits are nearly as expensive as their cache misses! From a GPU efficiency perspective, this makes no sense. Prefill is more efficient than decode, but it's not 60x more efficient! I suspect the reason is due to two subtleties in openrouter routing. (2/5)

Horace He

@cHHillee

@thinkymachines Formerly @PyTorch "My learning style is Horace twitter threads" - @typedfemale

加入 February 2010

633 正在關注 55.4K 粉絲

Horace He@cHHillee

2026.10.05 00:32

Also, their cache hits are nearly as expensive as their cache misses! From a GPU efficiency perspective, this makes no sense. Prefill is more efficient than decode, but it's not 60x more efficient! I suspect the reason is due to two subtleties in openrouter routing. (2/5)