NVIDIA Blackwell Dominates InferenceMAX Benchmarks with Unmatched AI Efficiency

NVIDIA’s Blackwell platform has achieved a remarkable feat by dominating the new SemiAnalysis InferenceMAX v1 benchmarks, delivering superior performance and efficiency across diverse AI models and real-world scenarios. This independent benchmark measures the total cost of compute, providing invaluable insights into the economics of AI inference, according to NVIDIA’s blog.

Unmatched Return on Investment

The NVIDIA GB200 NVL72 system stands out with its exceptional return on investment (ROI). A $5 million investment in this system can yield $75 million in DSR1 token revenue, marking a 15x ROI. This impressive economic model underscores the potential of NVIDIA’s AI solutions in delivering substantial financial returns.

Efficiency and Performance

NVIDIA’s B200 software optimizations have achieved an impressive reduction in cost per token, reaching two cents per million tokens on gpt-oss. This results in a 5x lower cost per token within just two months. The platform further excels in throughput and interactivity, with the NVIDIA B200 achieving 60,000 tokens per second per GPU and 1,000 tokens per second per user on gpt-oss, thanks to the latest NVIDIA TensorRT-LLM stack.

Advanced Benchmarking with InferenceMAX v1

The InferenceMAX v1 benchmark highlights Blackwell’s leadership in AI inference by running popular models across various platforms and measuring performance for a wide range of use cases. This benchmark is crucial as it emphasizes efficiency and economic scale, essential for modern AI applications that require multistep reasoning and tool use.

NVIDIA’s collaborations with major AI developers such as OpenAI and Meta have propelled advancements in state-of-the-art reasoning and efficiency. These partnerships ensure the optimization of the latest models for the world’s largest AI inference infrastructure.

Continued Software Optimizations

NVIDIA continues to enhance performance through hardware-software codesign optimizations. The TensorRT LLM v1.0 release marks a significant breakthrough, making large AI models faster and more responsive. By leveraging NVIDIA NVLink Switch’s bandwidth, the performance of the gpt-oss-120b model has seen dramatic improvements.

Economic and Environmental Impact

Metrics such as tokens per watt and cost per million tokens are crucial in evaluating AI model efficiency. The NVIDIA Blackwell architecture has lowered the cost per million tokens by 15x compared to previous generations, enabling substantial cost savings and fostering broader AI deployment.

The InferenceMAX benchmarks use the Pareto frontier to map performance, reflecting how NVIDIA Blackwell balances cost, energy efficiency, throughput, and responsiveness. This balance ensures the highest ROI across real-world workloads, underscoring the platform’s capability to deliver efficiency and value.

Conclusion

NVIDIA’s Blackwell platform, through its full-stack architecture and continuous optimizations, sets a new standard in AI performance and efficiency. As AI transitions into larger-scale deployments, NVIDIA’s solutions promise to deliver significant economic returns, reshaping the landscape of AI factories.

Image source: Shutterstock