Initial results for NVIDIA's Vera Rubin NVL72 are now live on AA-AgentPerf, with up to ~5.7x the total throughput per megawatt of its predecessor GB300 NVL72 on DeepSeek V4 Pro, ~5.4x on DeepSeek V4.1 Flash, ~3.0x on GLM-5.2 and ~2.8x on Kimi K3

Vera Rubin is NVIDIA's successor to Blackwell, and Vera Rubin NVL72 is its rack-scale system, with 72 Rubin GPUs and 36 Vera CPUs. Each Rubin GPU has the same 288 GB of memory as Blackwell Ultra, but 2.4x the memory bandwidth, 1.7x the NVLink bandwidth and 2.3x the dense NVFP4 compute. NVIDIA submitted every Vera Rubin result, running TensorRT-LLM and vLLM with NVIDIA Dynamo

AA-AgentPerf is our agentic inference performance benchmark. It replays real agentic coding sessions, including long cached contexts and tool-call delays, to give a realistic measure of inference performance in day-to-day agentic workloads. We measure total system throughput per megawatt of measured GPU power, and per-agent output speed (p25 across requests) at varying concurrency levels, to show the trade-off between system utilization and individual user experience

Initial results:

➤ Vera Rubin's lead is largest at faster per-user speeds, where GB300's throughput falls away. On DeepSeek V4 Pro, going from ~120 to ~165 tok/s per user costs GB300 81% of its throughput per MW, compared to 16% for Vera Rubin

➤ At peak throughput the two are closer, with Vera Rubin at 272M tokens/s per MW on DeepSeek V4 Pro vs 239M for GB300 (1.14x), 571M vs 395M on DeepSeek V4.1 Flash (1.45x), 272M vs 201M on GLM-5.2 (1.35x) and 70M vs 56M on Kimi K3 (1.26x). We expect the gap between the systems to widen as Vera Rubin kernels and other serving software are optimized

➤ Vera Rubin sets the fastest per-user speeds on our leaderboard for DeepSeek V4.1 Flash (906 tok/s, the fastest result for any model), GLM-5.2 (408 tok/s) and Kimi K3 (255 tok/s)

Congratulations @nvidia on the strong results!

Read the full analysis: https://artificialanalysis.ai/articles/vera-rubin-nvl72-agentperf