H100 vs H200 vs B200: which GPU to rent in 2026

The short answer

• H100 vs H200. Same compute on NVIDIA's spec sheets. The H200 adds memory: 141 GB of HBM3e at 4.8 TB/s against 80 GB at 3.35 TB/s on the H100 SXM (NVIDIA H200, NVIDIA H100). Pay the H200 premium when memory capacity or bandwidth is your bottleneck.
• H200 vs B200. The B200 is Blackwell. Per GPU it has about 180 GB of HBM3e, 8 TB/s, 4.5 PFLOPS of dense FP8, FP4 support and 1.8 TB/s of NVLink (derived from NVIDIA's 8-GPU HGX B200 and DGX B200 figures).
• Price. On 2026-11-09 the lowest published on-demand list prices were $2.45 for H100 SXM, $3.62 for H200 (NVL) and $5.43 for B200, per GPU-hour. Our GPU cloud pricing table has every provider.
• Our default. Train new large models on B200 if you can get it on a proper InfiniBand fabric. Serve on H200 or B200 depending on whether you are memory-bound or compute-bound. Keep H100 for fine-tuning, experiments and workloads already tuned for Hopper.

Spec table: H100, H200, B200 and GB200

All figures come from NVIDIA's product pages. NVIDIA prints Tensor Core numbers "with sparsity" (structured sparsity). Dense numbers are half of that, and dense is what most training and inference actually gets. We show both. B200 and GB200 per-GPU numbers are our division of NVIDIA's system figures, marked "derived".

Sources: H100, H200 (NVIDIA marks H200 figures preliminary), HGX B200 (8 GPUs: 72 PFLOPS FP8 sparse, 144 | 72 PFLOPS FP4 sparse | dense, 36 PFLOPS FP16/BF16 sparse, 1.8 TB/s GPU-to-GPU NVLink), DGX B200 (1,440 GB, 64 TB/s total), GB200 Superchip (2 GPUs: 40 | 20 PFLOPS NVFP4, 20 PFLOPS FP8 sparse, 372 GB, 16 TB/s), NVLink (domain sizes). BF16 dense for H100/H200 is half of NVIDIA's 1,979 TFLOPS sparse figure. NVIDIA's H100 page does not name the memory type.

PCIe variants

Some clouds list "H100 PCIe", "H100 NVL" or "H200 NVL". These are dual-slot PCIe cards, not SXM modules. NVIDIA lists the H100 NVL at 94 GB, 3.9 TB/s, 350 to 400 W and 600 GB/s of NVLink through a bridge. The H200 NVL keeps 141 GB and 4.8 TB/s but runs at up to 600 W, with 2-way or 4-way NVLink bridges (H100, H200). Both drop to 3,341 TFLOPS of sparse FP8. For single-GPU inference they are fine. For multi-GPU training, insist on SXM. The GPU cloud checklist covers form factor in more detail.

GB200 NVL72 in one paragraph

The GB200 NVL72 is a rack, not a server. It has 36 Grace CPUs and 72 Blackwell GPUs in one NVLink domain with 130 TB/s of NVLink bandwidth, 13.4 TB of HBM3e and 576 TB/s of memory bandwidth (NVIDIA GB200 NVL72). NVIDIA claims 30x the LLM inference of H100 and 4x the training speed, but its footnotes set specific conditions: a 1.8T-parameter MoE, 32,768 input tokens, and H100 clusters scaled over InfiniBand. Read those as best-case marketing numbers. The rack matters most for very large mixture-of-experts models, where expert parallelism across 72 GPUs avoids slower network hops.

AMD Instinct: MI300X, MI325X and MI355X

AMD's accelerators sell on memory. All three are OAM modules with 8 Infinity Fabric links (7 on the MI355X). Figures below are from AMD's product pages, dense first and sparse in brackets.

Sources: MI300X, MI325X, MI355X.

The MI325X has the same compute as the MI300X with more memory. The MI355X is the generational jump and lines up against the B200 on memory bandwidth and FP8. On 2026-11-09, DigitalOcean listed the MI300X at $2.59 and the MI325X at $3.80 per GPU-hour on demand, and Crusoe listed the MI300X at $3.45 (DigitalOcean, Crusoe). Vultr's public API listed the MI355X at $2.59 per GPU, but most top-end Vultr plans showed zero deployable locations when we checked.

Price comparison: lowest published on-demand rates

This table is the lowest and highest on-demand list price per GPU-hour that fixed-price clouds published on 2026-11-09. It excludes spot, preemptible, community-host, serverless and "from" prices. Where a provider prices per node, we divided by GPU count. Prices change often, so follow the link before you buy.

USD per GPU-hour, on-demand list prices seen 2026-11-09. "Providers publishing" counts fixed-price clouds in our dataset with a clear on-demand rate for that GPU.

Three things to know about these numbers:

• Prices are moving up. Crusoe's pricing page announces that from 2026-12-01 its H100 goes from $3.90 to $4.50, its H200 from $4.29 to $5.40, and its B200 gets an on-demand rate of $8.50 (Crusoe). Lambda's public rates, including B200 at $7.69 on an 8x instance, take effect 2026-11-16 (Lambda).
• Marketplaces go lower and swing more. On the same day Vast.ai showed live offers from $2.27 for H100 SXM (median $2.87), $3.43 for H200 (median $4.34), $5.50 for B200 (median $8.75) and $8.13 for GB200 (Vast.ai). See GPU cluster rental for how brokers and marketplaces work.
• The cheapest list price is not always available. The lowest H100 rate here, Denvr at $2.45, comes from a provider SemiAnalysis lists as "Not recommended: Unavailable" in ClusterMAX 3.0 (SemiAnalysis). Among providers SemiAnalysis rated Bronze or better, the lowest H100 SXM list price was Hyperstack at $3.20, and the lowest B200 was Hyperstack at $6.00.

Cost per useful work (illustration)

This section is an illustration, not a benchmark. It divides list prices by NVIDIA's peak specs. Real jobs reach a fraction of peak. The PaLM paper defines model FLOPs utilization (MFU) as observed throughput relative to the theoretical maximum at peak FLOPs, and reported 46.2% for PaLM 540B on TPU v4 (Chowdhery et al.). Your MFU will differ by model, code and cluster.

To keep it fair we compare prices inside the same provider. Hyperstack, Nebius and CoreWeave each published H100 SXM, H200 and B200 on-demand rates on 2026-11-09.

Inputs: on-demand list price per GPU-hour on 2026-11-09; dense FP8 of 1.979 PFLOPS (H100, H200) and 4.5 PFLOPS (B200, derived); bandwidth of 3.35, 4.8 and 8 TB/s. Lower is better in the first column, higher is better in the second.

What the arithmetic says:

• At all three providers the B200 buys the cheapest peak FP8 compute. At Hyperstack it is $1.33 per dense PFLOP-hour against $1.62 for H100. At CoreWeave it is $1.91 against $3.11.
• The B200 also buys the most memory bandwidth per dollar at all three. The H200 sits between the two.
• The H200 is the worst compute deal on paper, since it costs more than an H100 for the same FLOPS. You rent it for the extra 61 GB of memory, not for compute.

A worked memory example. A 70-billion-parameter model in FP8 needs about 70 GB for weights alone (one byte per parameter). On an 80 GB H100 that leaves almost nothing for the KV cache, so you shard across two GPUs or more. On a 141 GB H200 it fits on one GPU with about 70 GB left for cache. On a B200 with about 180 GB it fits with more room again. Fewer GPUs per replica means less cross-GPU traffic and simpler serving.

Which to rent for training

• Pre-training or large fine-tunes across many nodes: B200 on 8-GPU HGX nodes with InfiniBand or RoCE between nodes. The peak compute and bandwidth per list dollar are better, as the illustration above shows. Ask for the fabric speed per node in writing. NVIDIA's DGX B200 reference design has 8 ConnectX-7 ports at up to 400 Gb/s each (DGX B200).
• Very large mixture-of-experts models: GB200 NVL72, if you can get it. The 72-GPU NVLink domain is the point. Only CoreWeave published an on-demand GB200 NVL72 rate on 2026-11-09 ($10.50 per GPU-hour, derived from a $42.00 four-GPU instance). Everyone else sells it through sales.
• Fine-tuning, LoRA, research runs, code that is tuned for Hopper: H100 SXM. It is the most widely available and cheapest high-end part, with 13 fixed-price clouds publishing an on-demand rate in our dataset.
• AMD: MI300X or MI355X if your stack already runs on ROCm, or if memory per GPU matters more than ecosystem. Test before committing.

For multi-node training the GPU choice matters less than the cluster around it: fabric, storage, health checks and the provider's response when a node fails. Read how to rent a GPU cluster before you sign for more than a few nodes.

Which to rent for inference

• Memory-bound decoding, long contexts, large batch KV cache: H200 or B200. Compare GB of HBM per dollar and TB/s per dollar at your provider. At Hyperstack on 2026-11-09 the H200 gave 35 GB of HBM per dollar-hour, the B200 30 and the H100 25.
• Throughput at low precision: B200. FP4 support doubles peak throughput again over FP8 on NVIDIA's spec sheets, if your model quality holds at FP4.
• Small models and single-GPU endpoints: H100 PCIe or H100 NVL is often enough and costs less. Some clouds also sell serverless inference billed per second, priced above their on-demand pods. RunPod listed serverless H100 at $5.39 per hour against $3.99 for an on-demand H100 SXM pod (RunPod).

Before you rent

Run your own job on one node of each candidate for a few hours. Measure tokens per second or step time, divide by the hourly price, and pick on that number. Spec sheets and our arithmetic above only narrow the list. If you need more than one node, our 11-step checklist covers fabric, storage, contracts and exit terms. New to the category? Start with what a neocloud is, and keep the glossary open for terms like SXM, HBM3e and NVL72. Profiles: Hyperstack, Nebius, CoreWeave, Lambda, Massed Compute.