TPUv7 has a specialized hardware-accelerated unit called the SparseCore. TPU's specialized SparseCore handles data movement, gathering each expert’s tokens into contiguous groups, while the TensorCore is left to run the expert matrix multiplications. When using SparseCore for rearranging expert inputs into the MoE kernel, it results in 12% better throughput.

Combined with other optimizations & TPU's lower TCO, TPU can achieve up to 50% better perf per dollar than Blackwell Ultra, as seen on InferenceX.