AMD Runs Top AI Models at Half the Cost of NVIDIA Blackwell
The AMD MI355X inference cost advantage over NVIDIA Blackwell just got its clearest data point yet. Wafer.ai, a Y Combinator-backed AI inference startup, published benchmark data showing it served GLM-5.2 — a 52-billion-parameter open model — on AMD MI355X GPUs at over 2× lower cost than equivalent NVIDIA Blackwell (B300) deployments. The result, following Artificial Analysis measurement standards, challenges the assumption that NVIDIA’s premium hardware is the only viable option for production AI inference.
This AMD MI355X versus NVIDIA Blackwell cost benchmark landed as credible with developers: the company published its findings on July 3, 2026, generating 264 points and 92 comments on Hacker News — the strongest signal of developer interest in any AMD inference story this year. The benchmark used a 10,000-input / 1,500-output token workload, the same standard Artificial Analysis uses for its provider leaderboard, and achieved 213 tokens per second single-stream and 2,626 tokens per second per node.
The AMD MI355X Inference Cost Breakdown: 2.75× Cheaper Per GPU
AMD’s MI355X costs roughly 2.75× less than the NVIDIA B300 on average, according to Wafer’s analysis. At comparable throughput, that translates to a 2× cost advantage on a per-token basis for this workload — the gap between a compelling business case and just an interesting benchmark result.
The key enabling technique was quantization: Wafer converted GLM-5.2 from bf16 to MXFP4 format using AMD’s Quark toolkit, and claimed the quantization was lossless relative to the FP8 baseline. That’s the crucial caveat — MXFP4 can degrade output quality on complex reasoning tasks, and Wafer’s “lossless” claim applies to this specific model and workload configuration. Independent verification of that claim hasn’t been published yet.
GLM-5.2’s capabilities make it an ideal model for this test. Interconnects.ai called it “the step change for open agents” — read more in the GLM-5.2 launch analysis. A 52B-parameter open model running at production throughput on AMD hardware is a different proposition from running a smaller model that never stressed the GPU’s memory bandwidth in the first place.
Why This Benchmark Has Real Weight
Wafer is not a neutral party — they sell AMD-based inference and have an obvious incentive to publish favorable numbers. But two things give this more credibility than a typical vendor claim. First, they used Artificial Analysis’s methodology, which is the same framework developers use to compare providers on NVIDIA Nemotron and other benchmarks. Second, they published their numbers in enough detail that other providers could replicate or refute the test.
Wafer’s prior track record helps. Their “The Overlooked GPU” post on AMD MI300X inference established a reputation for rigorous optimization work before this piece. The HN community — which is deeply skeptical of vendor benchmarks — gave it 264 points, suggesting the technical readership found the methodology defensible.
The broader AI infrastructure cost trend is also moving in AMD’s direction. Inference cost per token has fallen more than 70% over 18 months as hardware competition increases and quantization improves. Each cost reduction makes the margin between NVIDIA’s CUDA ecosystem premium and AMD’s ROCm alternative more relevant to purchasing decisions.
The CUDA Moat Is Softening, but It Isn’t Gone
AMD’s longstanding problem has been ROCm — its GPU software stack — which lagged behind NVIDIA’s CUDA in maturity, compatibility, and developer tooling. The fact that Wafer achieved production-grade throughput using MXFP4 quantization via AMD Quark suggests the software gap is closing for inference workloads specifically, even if training workloads still heavily favor CUDA.
NVIDIA still leads on raw throughput. The MI355X doesn’t match the B300 on tokens per second at full precision — this benchmark wins on tokens-per-dollar, not tokens-per-second. For hyperscale inference where throughput-per-dollar is the optimization target, that distinction is meaningful. For real-time applications where raw speed matters more than cost, NVIDIA’s position is still comfortable.
The competitive pressure is accumulating, though. Developers building on AI today care about AI model pricing as a first-order concern, and every benchmark that shows AMD offering comparable quality at lower cost makes it harder for NVIDIA to maintain its current pricing premium. The GPU war isn’t over — but it’s no longer a foregone conclusion.
Frequently Asked Questions
How does the AMD MI355X compare to NVIDIA Blackwell for AI inference?
According to Wafer.ai’s July 2026 benchmark, the AMD MI355X delivers comparable throughput to NVIDIA’s Blackwell B300 on LLM inference at roughly 2× lower cost per token. The MI355X achieves this by costing approximately 2.75× less per GPU — though NVIDIA still leads on raw tokens-per-second at full precision.
What is GLM-5.2 and why was it used in this benchmark?
GLM-5.2 is a 52-billion-parameter open-weight AI model developed by Zhipu AI, a Chinese AI research lab. It’s designed for agentic tasks and is one of the most capable open models available for download and self-hosting. Wafer chose it because its scale genuinely stresses GPU memory bandwidth, making the benchmark representative of real production workloads.
What is MXFP4 quantization and does it reduce quality?
MXFP4 is a 4-bit floating-point format that reduces model memory requirements and increases throughput compared to standard bf16 or FP8 formats. Wafer claims their MXFP4 quantization of GLM-5.2 is lossless relative to the FP8 baseline — meaning no measurable quality degradation for this workload — though that claim has not been independently verified by a third party.
Is AMD’s ROCm software mature enough for production AI?
For inference workloads specifically, Wafer’s results suggest ROCm has reached production viability on the MI355X — at least for models that have been properly optimized for the hardware. Training workloads remain dominated by NVIDIA’s CUDA ecosystem, which has a deeper tooling and library advantage. The software gap is closing, but it hasn’t closed equally across all AI use cases.
Why do inference costs keep falling?
Inference costs have fallen more than 70% over the past 18 months due to a combination of factors: hardware competition between NVIDIA, AMD, and custom silicon (Google TPUs, AWS Trainium); improved quantization techniques that extract more performance from the same hardware; and software optimizations like continuous batching and speculative decoding. Competition between cloud providers and inference startups like Wafer is accelerating this trend.
The Wafer benchmark adds to a growing body of evidence that inference infrastructure is becoming a genuine competitive market — not a NVIDIA monopoly. For developers choosing where to run models, that competition is the best possible news.
Last Updated: July 2026
