We run a GPU spec catalog, and over a couple of years it grew into a database of 13,566 GPUs — from the GeForce 256 (1999) all the way to Blackwell and the MI355X (2025). At some point the interesting question stopped being "which card is faster" and became "how did the whole industry move?" — how much did FLOPS actually grow, where did the thermal envelope hit a wall, and who really led the NVIDIA-vs-AMD race in different years.

Here's a breakdown from our data. I'll give you two things up front that usually get hidden: the methodology (what I measured, where the data is noisy) and, at the end, an open dataset — grab it and dig in yourself.

TL;DR

• Peak FP32 of the flagship grew ~400× in 19 years: 0.3 TFLOPS (GeForce 8800 GTX, 2006) → 126 TFLOPS (Blackwell, 2025). On a semi-log axis it's almost a straight line.
• TDP crept up slowly (155 → 300 W over 2006–2020), then exploded in the datacenter: 700 W (H100), 1000 W (MI325X / B200), 1400 W (MI355X, 2025).
• Meanwhile performance per watt grew ~100× — they "draw more" but "do far more per watt." The main driver is the process node (90 nm → 3 nm) plus architecture.
• The NVIDIA/AMD duel by peak FP32 moved in waves: AMD led in the early 2010s (GCN era) and again in 2023–24 (Instinct MI300/MI325), NVIDIA in 2016–2020 (the AI pivot) and in 2025 (Blackwell). But "raw FP32" is a misleading metric, and I'll explain why below.

Methodology (please read before yelling at the numbers)

What these TFLOPS are and why they're "theoretical." Every FP32 figure here is the theoretical peak that vendors compute as:

FP32 TFLOPS = (shader ALUs / CUDA cores) × boost clock (Hz) × 2 / 10^12

The ×2 is because an FMA (fused multiply-add) does a multiply and an add in one cycle — two operations. This is a ceiling, not real-world throughput: in practice you hit noticeably less — typically 60–90% on well-optimized compute-bound kernels, and a fraction of that on memory-bound ones — because memory bandwidth, SM occupancy, instruction mix, and the fact that boost clocks don't hold under sustained load and thermal limits all get in the way. Theory diverging from practice is normal. The theoretical peak is valuable for a different reason: it's computed by one formula across every card and generation, so it's a fair comparable yardstick for a historical look — which is exactly what spec sheets list and what we use. Real performance is measured with benchmarks (those are a separate table in the dataset).

A few more honesty notes:

• "Flagship of the year" = the card with the maximum fp32_performance released that year, tracked separately for NVIDIA and AMD.
• For the TDP/efficiency curves I excluded dual-GPU cards (GTX 295, HD 6990, R9 295X2, etc.) — otherwise the power and FLOPS double up and wreck the trend.
• vendor is filled in for ~2,360 of 13,566 cards (the rest are mostly OEM/partner board variants). Medians use the labeled subset; flagship peaks are fully labeled.

1. FLOPS: an almost perfectly straight exponential

Peak FP32 of the single flagship by year (NVIDIA):

≈400× in 19 years is a CAGR of about 37% per year. On a semi-log axis the line is almost perfectly straight — a classic exponential that only recently started bending on the desktop segment and moved into the datacenter.

2. TDP: a quiet climb, then a datacenter explosion

For a decade and a half the flagship thermal envelope stayed in a 150–300 W band. The break comes after 2020, and it's entirely datacenter-driven: AI accelerators (SXM/OAM modules) jumped to 700–1400 W because they're cooled by liquid in a rack, not by a fan in a case. The desktop ceiling separately hit ~450–600 W (RTX 4090/5090).

There's a curious gap if you look only at NVIDIA's consumer flagships: the GeForce flagship sat at exactly 250 W for seven years (2013–2019) — GTX 780 Ti, Titan X, 1080 Ti, 2080 Ti — and only broke that ceiling with the RTX 3090 (350 W, 2020), then 4090 (450 W) and 5090 (575 W). Datacenter accelerators, by contrast, went to 700–1400 W almost immediately. It looks like what capped gaming TDP wasn't the silicon so much as the market — cases, PSUs, and buyer habits; in a rack none of those constraints apply, and watts grew without looking back. (That's interpretation, of course — the spec stores watts, not intentions — but a 250 W plateau across seven generations shows up clearly in the data.)

3. Performance per watt: this is where the real progress is

If you only look at TDP it feels like "everything's getting worse, cards guzzle power." But FP32 per watt tells the opposite story:

~100× in efficiency. Peak "classic" efficiency lands in 2022 (Ada/L40S); the 2024–25 datacenter cards sometimes lose on TFLOPS/W because they deliberately trade efficiency for absolute compute density in the rack. The main drivers of efficiency are the process node (90 nm → 3 nm) and architecture, not clock speed.

4. The NVIDIA vs AMD duel

If you mark, year by year, whose single flagship had the higher FP32:

The picture is wavy, and honestly I included it partly to give AMD its due — because on raw FP32, AMD led more often than people remember, both in the GCN era and again on recent Instinct parts. But raw FP32 is a deceptive metric for the modern world. The AI era isn't won on FP32; it's won on FP16/BF16/FP8 and on software.

And here's the catch that trips up most "raw FP16" comparisons:

FP16/tensor numbers are not directly comparable between vendors because of structured sparsity. Starting with Ampere (A100), NVIDIA's spec sheets quote tensor FP16/BF16 figures with sparsity already applied — that's 2× the dense value (the feature processes sparse matrices twice as fast). AMD has no equivalent spec line — those are dense. So NVIDIA's raw FP16 numbers (A100+) need to be halved to compare fairly with AMD: A100 = 624 (sparse) → 312 dense, H100 = 1979 → ~990 dense. With tensor cores (since V100, 2017) and the CUDA ecosystem, NVIDIA built a moat that raw FP32 simply doesn't show.

H100 isn't one card — it's a family

A good example of why "an H100" isn't yet a spec: several different SKUs of the same chip live under that name in the database. TDP swings 2× — from 350 W on the PCIe 80 GB version to 700 W on the SXM5 modules; there's also an NVL at 94 GB and 400 W. Memory is 80 / 94 / 96 GB, and the PCIe-80 ships on slower HBM2e while SXM and the higher PCIe parts are on HBM3. Even peak FP32 "floats" by ~30%: 51 TFLOPS on the PCIe-80 vs 67 on any SXM5 (same silicon, different clocks and power). So when a price list or an email just says "H100," it's worth asking which one — otherwise you'll compare a 350 W PCIe card to a 700 W SXM module as if they're the same thing. In the dataset, each is its own row.

Open dataset — take it

We published a cleaned dump of our GPU spec database for anyone who wants to dig in themselves — it's on the GPU Ark open dataset page (CSV, SQLite, and a single .tar.gz):

• 13,566 GPUs (fields: vendor, manufacturer, release date, architecture, process node, transistors, clocks, memory size and type, bus, FP16/FP32/FP64/BF16/TF32/INT8, TDP, NVLink, CUDA SM, and more) + 993 third-party benchmark results (join on gpu_id).
• Formats: CSV (Excel/pandas) and SQLite (ready-made SQL) — two tables, gpu_specs and benchmarks.
• License: CC BY 4.0 (attribution to gpuark.com).
• Only public specs — no competitor prices, no internal mappings, no editorial content.
• A KNOWN_ISSUES.md sits next to it with the same caveats as above (sparse vendor, VRAM units on old cards, FP16 noise), plus a README.md with a five-line pandas example.

If you'd rather explore interactively before downloading, the same data powers the GPU comparison tool on the site.

Takeaways

1. FLOPS grew as an almost perfect exponential (~37%/yr) — but the "free" growth is over; from here we pay with thermal envelope and a move into the rack.
2. Real progress is measured not in watts and not in raw FP32, but in performance per watt (×100) — and that rides on the process node.
3. AMD fought and led on "raw" numbers more often than the common narrative admits; but the AI era was defined by tensor + software, not FP32.

The data is open — if you find something in it we missed, tell me and I'll add it.