Measured tier · third-party sourced · compiled 2026-09-10

Measured performance that already exists — and how independent it is

The rest of this site grades disclosure: what a seller writes down about its fabric. This page is different — it collects performance that has actually been measured and published, 23 sourced observations in all. But a measured number is only as good as who measured it, so every row here is classed by independence, most to least. None of this is our own measurement — that tier (Verified, §8) does not exist yet. This is the measured reality that exists today, sorted honestly.

The measured record, by independence

Same caveats as the $/M-token bridge: compare rows only within a mode (ceiling vs realistic differ up to ~8× on identical hardware); software moves faster than silicon, so every row is stack-and-date-stamped; FP4 tokens are faster but not identical to FP8 tokens (§4.3). Output tokens per second, aggregate over the stated GPU count.

Audited benchmark · 4 observations

MLPerf Inference — a neutral consortium suite with a closed, results-reviewed division. The gold standard for independence, even when the run is submitted by the hardware vendor: the rules and the review are not theirs.

Independent open benchmark · 3 observations

SemiAnalysis InferenceMAX / InferenceX — a continuously-run, open-source harness operated by a third party across best-of-breed serving stacks. Independent of the vendor; methodology public.

Independent third-party test · 4 observations

A party with no stake in the result (an infra vendor testing others’ hardware, a serving-framework-neutral lab) published the number with its method. Not audited, but not self-interested.

Vendor / framework self-published · 11 observations

The chipmaker, the serving-framework project, or the host published its own number to show its own hardware/software in the best light. Real measurements, but chosen configs — treat as a ceiling.

Marketing aggregator — indicative only · 1 observation

A GPU-cloud marketing page or aggregator with framework/context undisclosed or far from saturation. Directional at best; never load-bearing.

What's measured — and the hole where the real answer should be

Notice what every rung-2 number above actually is: inference throughput — tokens per second on a serving workload. That's the performance the public benchmark ecosystem measures, because it runs on one node and anyone can reproduce it. It is genuinely useful, and it feeds the cost-per-token math.

But it is not the number the futures market, the lenders, or a training buyer are exposed to. Their question is delivered collective bandwidth — DCB, §3.3 — measured all-reduce across a multi-node training allocation, under production tenancy. That is where "8×H100 on shared Ethernet" and "8×H100 in a non-blocking InfiniBand domain" stop being the same good. And there is almost no public measured DCB at all: it needs someone to run a probe across a real fleet under real load, and no neutral party owns the clusters. The gap even shows up inside inference: the AMD MI300X 70B row publishes a request-rate but withholds a tok/s figure — measured, not disclosed.

SemiAnalysis's ClusterMax rates neocloud operational quality from hands-on testing — the closest thing to third-party measured provider grading that exists, and worth reading; we point to it rather than republish it. But even ClusterMax is a provider rating, not a per-allocation delivered-bandwidth number a contract can cite. That row of the ladder is empty across the whole market.

That emptiness is the entire reason the harness exists. ASSAY-1 §8 is an open, MIT-licensed probe an operator runs on its own metal, signed and registered — turning a disclosed spec into a measured, source-classed operator-submitted row. When the first one lands it will be the first entry on rung 3, and it will appear here labeled for exactly what it is. Operators: the founding cohort is free.