Google announced two additions to its eighth-generation Tensor Processing Unit family on November 9: TPU 8i Max and TPU 8tx. They join TPU 8t and TPU 8i, which Google announced at Cloud Next in April, making this the first year the company has introduced four TPU systems.
TPU 8i Max places two TPU 8i chips in a single package. Google says it doubles compute, SRAM and HBM bandwidth relative to TPU 8i and fits twice the silicon in the same rack footprint, aimed at serving models with very large context windows at the highest compute density per rack.
TPU 8tx is designed for confidential inference and small-scale training. It uses the same silicon as the training-focused TPU 8t, paired with 5th Gen AMD EPYC host CPUs, and adds hardware-level isolation to protect data and model weights while they are processed. Google says it delivers up to 2x lower latency than TPU 8t using its Boardfly network.
For context, Google says TPU 8t offers up to 2.7 times better training performance than the previous generation, and TPU 8i offers 80% better inference price-performance. The post did not give availability dates for the two new systems. Google also noted that Cloud Pathways and GKE dynamic slicing are now generally available on Google Kubernetes Engine, and that its open-source TPU Sync layer cuts KV transfer latency and time to first token by more than 90%, according to the company.
The new systems target two pressures on inference buyers: long-context workloads that are limited by memory bandwidth, and regulated customers who need guarantees that prompts and weights stay protected during processing. Google says TPUs now serve nearly 100 customers.