TPUSA has refused to punish a Florida college chapter for hosting James Fishback, instead supporting free speech and claiming that chapters are independent of the national leadership. Follow: @AFpost

TPUv7 has a specialized hardware-accelerated unit called the SparseCore. TPU's specialized SparseCore handles data movement, gathering each expert’s tokens into contiguous groups, while the TensorCore is left to run the expert matrix multiplications. When using SparseCore for rearranging expert inputs into the MoE kernel, it results in 12% better throughput. Combined with other optimizations & TPU's lower TCO, TPU can achieve up to 50% better perf per dollar than Blackwell Ultra, as seen on InferenceX.

TPU Inference Externalization Full Steam Ahead - InferenceX, Up to 50% Better Performance per Dollar, Rapid Externalization of TPU stack, Growing Customer Base, Ironwood, TPUv8i, Reducing CUDA Moat

TPU🚨 is working with the popular OSS inference optimization library Mooncake on integrating TPU with Mooncake Store. Similar to NVL72, KVCache DRAM P2P pooling will initially happen on the scale-out network via TENT instead of using ICI/NVLink. 🔥 Mooncake basically improves performance per TCO of production inference!

TPU 8i and TPU 8t deliver sustainable economics—offering up to 80% better performance-per-dollar for low-latency serving while seamlessly scaling to 1M+ chips! Learn how our 8th-generation TPU system drives the next frontier of AI ↓

TPU 8 will be a massive leap forward in energy efficiency—with up to 2x better PPW than Ironwood. But for us, sustainability isn’t about waiting for the next best chip; it’s a daily effort to make our entire production fleet more efficient → #8daysofTPU8#

tpuf quantizes vectors to improve perf (RaBitQ) the algo randomly rotates vectors, and we were using matmul at O(d²) space & time, brutal at high dims. 10k = 400MB in RAM! we rebuilt the rotation using FWHT at O(d) space & O(d log d) time. ~no recall loss, 10k = only 5kB in RAM