Home / GPU cloud glossary
GPU cloud glossary
68 terms you meet when renting GPUs, from SXM and NVL72 to MFU and data egress. Each definition is one to three sentences, with a source where a number or a standard is involved.
How to use this glossary
Jump to a letter, or link straight to a term: every entry has its own anchor, such as /glossary/#nvlink. Hardware numbers come from NVIDIA's and AMD's product pages. Prices are on-demand list prices seen on 2026-11-09. For how the terms fit together, read what a neocloud is, the GPU cloud checklist, H100 vs H200 vs B200 and GPU cluster rental, or check current rates on the pricing page.
Terms from A to Z
A
- Aggregator
- A service that lists GPU capacity from many clouds behind one account and API. Shadeform, for example, aggregates about 18 clouds and says you pay the same price as going direct. See GPU brokers. Source: Shadeform.
- All-reduce
- A collective operation in which every GPU contributes data and every GPU receives the combined result. Data-parallel training uses it to sum gradients, so its speed depends on NVLink and the inter-node fabric. Source: NVIDIA NCCL.
B
- B200
- NVIDIA's Blackwell data center GPU, usually sold in 8-GPU HGX servers. Per GPU it has about 180 GB of HBM3e, 8 TB/s and 4.5 PFLOPS of dense FP8, derived from NVIDIA's 8-GPU figures. See H100 vs H200 vs B200. Source: NVIDIA HGX, DGX B200.
- B300
- NVIDIA's Blackwell Ultra GPU. An HGX B300 board holds 8 GPUs with 2.1 TB of total memory and 1.6 TB/s of networking bandwidth, against 1.4 TB and 0.8 TB/s for HGX B200. Source: NVIDIA HGX.
- Bare metal
- Renting a whole physical server with no hypervisor between you and the hardware. You control the operating system, drivers and kernel, which matters for custom networking and performance tuning.
- BF16
- bfloat16, a 16-bit floating-point format used for mixed-precision training. NVIDIA lists 1,979 TFLOPS of BF16 Tensor Core throughput with sparsity for the H100 SXM, or about 989 TFLOPS dense. Source: NVIDIA H100.
- Blackwell
- The NVIDIA GPU architecture after Hopper. It includes B200, B300 and the GB200 and GB300 superchips, and adds FP4 Tensor Core math and fifth-generation NVLink at 1.8 TB/s per GPU. Source: NVIDIA HGX, NVLink.
- Burn-in
- Running stress, memory and communication tests on new nodes before you accept them. Our view: insist on a burn-in or acceptance period before billing starts on any reserved cluster. See Cluster quotes.
C
- ClusterMAX
- SemiAnalysis's third-party evaluation of GPU cloud clusters. Version 3.0 (September 2026) grades providers Platinum, Gold, Silver or Bronze, with Participation and Not recommended categories. It is SemiAnalysis's rating, not ours. Source: SemiAnalysis, Ratings.
- Colocation
- Renting space, power and cooling in a data center for hardware you own. Some GPU providers, such as Cirrascale and Firmus, also sell colocation alongside cloud capacity.
- Community cloud
- Runpod's name for GPUs hosted by third parties on its platform, priced below its own Secure Cloud. On 2026-11-09 a community H100 SXM listed at $2.69 per GPU-hour against $3.99 on Secure Cloud. Source: Runpod pricing.
- ConnectX
- NVIDIA's line of network adapters for InfiniBand and Ethernet. The DGX B200 uses 8 single-port ConnectX-7 adapters at up to 400 Gb/s each for its cluster network. Source: DGX B200.
- CUDA
- NVIDIA's programming platform for GPUs. The CUDA Toolkit provides the compiler, libraries and tools that most AI frameworks build on when they run on NVIDIA hardware. Source: NVIDIA CUDA Toolkit.
D
- Data egress
- Data transferred out of a provider's network, which some providers bill per GB. Terms vary: Cirrascale states no ingress or egress fees, and Latitude.sh includes 20 TB of free egress per server. Source: Cirrascale, Latitude.sh.
- Dense (FLOPS)
- Peak throughput without structured sparsity. NVIDIA prints most Tensor Core figures with sparsity and says dense is one-half of the sparse figure. See Sparsity. Source: NVIDIA HGX.
- DGX
- NVIDIA's complete AI server system. A DGX B200 has 8 Blackwell GPUs with 1,440 GB of total GPU memory, 64 TB/s of HBM3e bandwidth and about 14.3 kW maximum system power. Source: DGX B200.
F
- FP4
- 4-bit floating-point math for inference, supported on Blackwell and on AMD's MI355X. NVIDIA lists 9 PFLOPS of dense FP4 per B200 (derived), and AMD lists 10.1 PFLOPS of MXFP4 for the MI355X. Source: NVIDIA HGX, AMD MI355X.
- FP8
- 8-bit floating-point math, in E4M3 and E5M2 variants, used for training and inference on Hopper, Blackwell and AMD Instinct. H100 SXM and H200 SXM list 3,958 TFLOPS with sparsity (1,979 dense). Source: NVIDIA H100, AMD MI300X.
G
- GB200
- The Grace Blackwell Superchip: one Grace CPU and two Blackwell GPUs with 372 GB of HBM3E. It is the building block of the GB200 NVL72 rack. Source: NVIDIA GB200 NVL72.
- GB200 NVL72
- A rack of 36 Grace CPUs and 72 Blackwell GPUs in one NVLink domain, with 13.4 TB of HBM3E and 130 TB/s of NVLink bandwidth. Only CoreWeave published an on-demand rate on 2026-11-09: $10.50 per GPU-hour. Source: NVIDIA GB200 NVL72, CoreWeave pricing.
- GPU-hour
- One GPU for one hour, the standard billing unit for GPU clouds. When a provider prices an 8-GPU node, divide by 8 to compare per GPU-hour, as we do on our pricing page. See GPU cloud pricing.
H
- H100
- NVIDIA's Hopper data center GPU. The SXM version has 80 GB of memory at 3.35 TB/s, up to 700 W and 900 GB/s of NVLink; the PCIe-based H100 NVL has 94 GB at 3.9 TB/s. See H100 vs H200 vs B200. Source: NVIDIA H100.
- H200
- A Hopper GPU with the same compute as the H100 and more memory: 141 GB of HBM3e at 4.8 TB/s. It helps most on memory-bound inference and long contexts. Source: NVIDIA H200.
- HBM3e
- The generation of High Bandwidth Memory used on H200, B200, GB200, MI325X and MI355X. Capacity and bandwidth of HBM often decide how large a model fits on one GPU. Source: NVIDIA H200, AMD MI325X.
- HFU
- Hardware FLOPs utilization: all FLOPs the hardware executes, including rematerialization (recomputation), as a share of peak. The PaLM paper reported 57.8% HFU against 46.2% MFU for PaLM 540B. See MFU. Source: Chowdhery et al..
- HGX
- NVIDIA's 8-GPU baseboard platform that server makers build on. HGX B200 carries 8 Blackwell SXM GPUs linked by fifth-generation NVLink with 14.4 TB/s of total NVLink bandwidth. Source: NVIDIA HGX.
- Hopper
- The NVIDIA architecture behind H100 and H200, with fourth-generation NVLink at 900 GB/s per GPU and up to 8 GPUs per NVLink domain. Source: NVLink.
- Hyperscaler
- A very large general-purpose cloud: AWS, Microsoft Azure, Google Cloud and Oracle Cloud. They rent GPUs as one of many services. SemiAnalysis rates them in ClusterMAX alongside neoclouds. See Neoclouds vs hyperscalers. Source: SemiAnalysis.
I
- InfiniBand
- A switched network standard from the InfiniBand Trade Association, used to connect GPU servers in training clusters. NVIDIA sells it as the Quantum platform, with in-network computing such as SHARP. Source: IBTA, NVIDIA InfiniBand.
- Infinity Fabric
- AMD's GPU-to-GPU link on Instinct accelerators. The MI300X has 8 Infinity Fabric links at 128 GB/s peak each; the MI355X has 7 at 144 GB/s. Source: AMD MI300X, AMD MI355X.
- Instant cluster
- A self-serve multi-node GPU cluster you can launch without a sales call or term. Runpod Instant Clusters go up to 64 GPUs on InfiniBand or RoCE v2. Source: Runpod.
K
- Kubernetes
- An open source platform for managing containerized workloads and services. Many neoclouds offer a managed version. Source: Kubernetes docs.
- KV cache
- GPU memory that holds attention keys and values for every token in context during LLM inference. It grows with context length and batch size, which is why memory capacity drives inference cost.
L
- Lustre
- A parallel file system used for shared cluster storage. SemiAnalysis measured Vultr's Lustre storage tier at 91.7 GB/s in ClusterMAX 3.0. Source: SemiAnalysis.
M
- Marketplace
- A platform where independent hosts or capacity holders list GPUs at their own prices. On Vast.ai, host prices move with supply and demand; on 2026-11-09 the lowest H100 SXM offer was $2.27 per GPU-hour. See GPU brokers. Source: Vast.ai pricing.
- MFU
- Model FLOPs utilization: observed training throughput relative to the theoretical maximum at peak FLOPs. The PaLM paper introduced it and reported 46.2% for PaLM 540B. Source: Chowdhery et al..
- MI300X
- AMD's Instinct accelerator with 192 GB of HBM3 at 5.3 TB/s and 2.61 PFLOPS of dense FP8, at 750 W peak board power. DigitalOcean listed it at $2.59 per GPU-hour on 2026-11-09. Source: AMD MI300X, DigitalOcean.
- MI325X
- AMD's MI300X successor with the same compute and more memory: 256 GB of HBM3E at 6 TB/s, at 1,000 W peak board power. Source: AMD MI325X.
- MI355X
- AMD's CDNA 4 Instinct accelerator with 288 GB of HBM3E at 8 TB/s, 5 PFLOPS of dense FP8, MXFP4 support and 1,400 W board power. Source: AMD MI355X.
- MIG
- Multi-Instance GPU: splitting one NVIDIA GPU into isolated slices with their own memory. An H200 SXM supports up to 7 MIG instances of 18 GB each. Source: NVIDIA H200.
N
- NCCL
- The NVIDIA Collective Communications Library, which provides multi-GPU and multi-node primitives such as all-reduce, all-gather and broadcast. Distributed training on NVIDIA GPUs runs its gradient traffic through libraries like this. Source: NVIDIA NCCL.
- nccl-tests
- NVIDIA's open source benchmarks for NCCL operations, such as all_reduce_perf. Running them across every node is the quickest way to check a cluster's fabric before you accept it. Source: GitHub.
- Neocloud
- A cloud provider whose main product is GPU compute for AI, rather than a broad catalog of services. CoreWeave, Nebius, Lambda and Crusoe are well-known examples. See What is a neocloud.
- Node
- One server in a cluster. In GPU clouds a node usually means an 8-GPU HGX server, and many providers price per node-hour.
- NVL72
- NVIDIA's name for a rack with 72 GPUs in one NVLink domain, as in GB200 NVL72 and GB300 NVL72. Hopper systems top out at 8 GPUs per NVLink domain. Source: NVIDIA NVLink.
- NVLink
- NVIDIA's GPU-to-GPU interconnect. Fourth generation (Hopper) runs at 900 GB/s per GPU and fifth generation (Blackwell) at 1.8 TB/s per GPU. Source: NVIDIA NVLink.
- NVLink Switch
- The switch chip that extends NVLink beyond one server so every GPU in a rack can talk to every other at NVLink speed. It is what makes a 72-GPU NVLink domain possible. Source: NVIDIA NVLink.
O
- OAM
- The module form factor AMD uses for Instinct MI300X, MI325X and MI355X. It plays the same role as NVIDIA's SXM: a socketed module on a multi-GPU baseboard. Source: AMD MI300X.
- On-demand
- Pay-as-you-go GPU rental with no term, billed by the hour, minute or second. On 2026-11-09 published on-demand H100 SXM list prices ranged from $2.45 to $6.155 per GPU-hour. See GPU cloud pricing.
- Order book
- A market where buyers and sellers post bids and asks for capacity. SF Compute sells blocks of GPU time this way: you set a node count, a time window and a maximum price per node-hour. Source: SF Compute docs.
P
- PCIe
- The standard expansion-card form factor and bus. PCIe GPUs such as the H100 NVL draw less power and cost less per hour than SXM, with PCIe Gen5 at 128 GB/s to the host. Source: NVIDIA H100.
- PFLOPS
- Petaflops: one quadrillion (10^15) floating-point operations per second, or 1,000 TFLOPS. Blackwell-era spec sheets quote most figures in PFLOPS.
R
- RDMA
- Remote Direct Memory Access: moving data between machines' memory without involving the remote CPU. InfiniBand and RoCE both provide it, and GPU training clusters depend on it. Source: IBTA.
- Reserved
- Capacity committed for a fixed term in return for guaranteed access and usually a lower rate. Published minimums on 2026-11-09 ranged from 3 days at Together AI to 12 months at DigitalOcean. See GPU cluster rental.
- RoCE
- RDMA over Converged Ethernet, an industry standard transport that enables RDMA on Ethernet networks. It is the main alternative to InfiniBand for GPU cluster fabrics. Source: IBTA.
- ROCm
- AMD's open software stack for GPU-accelerated computing, the AMD counterpart to CUDA. Check that your framework and kernels run on ROCm before you rent AMD Instinct. Source: AMD ROCm docs.
S
- Serverless inference
- Running models on GPUs billed per second or per token without managing instances. Runpod listed serverless H100 at $5.39 per hour on 2026-11-09, above its $3.99 on-demand pod. Source: Runpod pricing.
- NVIDIA's Scalable Hierarchical Aggregation and Reduction Protocol. It offloads collective operations to InfiniBand switches, which cuts the data crossing the network. Source: NVIDIA InfiniBand.
- Slurm
- An open source, fault-tolerant cluster management and job scheduling system for Linux clusters. Many neoclouds offer it managed. Source: SchedMD.
- Slurm on Kubernetes
- Running Slurm as a workload inside Kubernetes so one cluster serves both batch jobs and services. Examples include CoreWeave SUNK, Nebius Soperator and Slinky, which Verda uses.
- SOC 2
- A CPA examination of a service organization's controls relevant to security, availability, processing integrity, confidentiality or privacy. Ask for the report and check its type and the period it covers. Source: AICPA.
- Sparsity
- NVIDIA and AMD quote many peak figures "with sparsity" (structured sparsity), and those are double the dense figures. Use dense figures for cost estimates unless you know your model runs sparse. Source: NVIDIA HGX, AMD MI300X.
- Spot
- Interruptible or preemptible capacity sold below on-demand prices that the provider can reclaim. DigitalOcean says spot Droplets can be reclaimed with about two hours' notice or less. Source: DigitalOcean.
- SXM
- NVIDIA's socketed GPU module form factor used on HGX boards. SXM GPUs get full NVLink and higher power limits: up to 700 W and 900 GB/s for H100 SXM. Source: NVIDIA H100.
T
- TDP and TBP
- Thermal design power (NVIDIA) and typical board power (AMD): the power a GPU is designed to draw. Examples: H100 SXM up to 700 W, MI300X 750 W peak, MI355X 1,400 W. Source: NVIDIA H100, AMD MI355X.
- Tensor Core
- The matrix-math units in NVIDIA GPUs that do most AI work. Spec sheets list FP8, FP16, BF16 and TF32 throughput as Tensor Core figures. Source: NVIDIA H100.
- TFLOPS
- Teraflops: one trillion (10^12) floating-point operations per second. Always check the precision (FP8, BF16) and whether the figure is sparse or dense.
X
- Xid error
- An error message reported by the NVIDIA driver for a GPU problem; NVIDIA documents what each Xid number means. Watching for Xid errors is a basic health check on any GPU node. Source: NVIDIA Xid docs.
Common questions
What is the difference between SXM and PCIe GPUs?
SXM is a socketed module on an 8-GPU HGX board with full NVLink and higher power, up to 700 W on H100. PCIe cards fit standard slots, draw less power and link to at most a few neighbors through bridges. Train on SXM; PCIe is often fine for single-GPU inference.
What is the difference between InfiniBand and RoCE?
Both provide RDMA between servers. InfiniBand is its own network standard; NVIDIA sells it as the Quantum platform. RoCE runs RDMA over Ethernet. Either can work for training clusters, so measure the actual fabric with nccl-tests.
What does NVL72 mean?
NVL72 is a rack with 72 GPUs in one NVLink domain, as in GB200 NVL72. All 72 GPUs talk at NVLink speed, against 8 GPUs per domain on Hopper systems.
What is a GPU-hour?
One GPU used for one hour. It is the standard unit for GPU cloud prices. An 8-GPU node at $24 per hour costs $3 per GPU-hour.
What is the difference between spot, on-demand and reserved?
On-demand has no term and a fixed list price. Spot is cheaper but can be reclaimed by the provider. Reserved commits you to a term in return for guaranteed capacity and usually a lower rate.
What is MFU?
Model FLOPs utilization is observed training throughput divided by the theoretical maximum at the hardware's peak FLOPs. The PaLM paper defined it and reported 46.2% for PaLM 540B.
What are sparse and dense TFLOPS?
Sparse figures assume structured sparsity and are double the dense figures. NVIDIA prints sparse numbers in most tables. Use dense numbers for cost estimates.
Sources
- NVIDIA H100 specifications
- NVIDIA H200 specifications
- NVIDIA HGX specifications
- NVIDIA DGX B200 specifications
- NVIDIA GB200 NVL72 specifications
- NVIDIA NVLink and NVLink Switch
- NVIDIA Quantum InfiniBand platform
- NVIDIA NCCL
- NVIDIA CUDA Toolkit
- NVIDIA Xid errors
- NVIDIA nccl-tests
- AMD Instinct MI300X specifications
- AMD Instinct MI325X specifications
- AMD Instinct MI355X specifications
- AMD ROCm documentation: what is ROCm
- InfiniBand Trade Association: specification and RoCE
- Slurm overview (SchedMD)
- Kubernetes overview
- Chowdhery et al., PaLM: Scaling Language Modeling with Pathways
- SemiAnalysis: ClusterMAX 3.0
- AICPA: SOC 2
- Runpod pricing (seen 2026-11-09)
- DigitalOcean GPU Droplet pricing (seen 2026-11-09)
- Vast.ai pricing (seen 2026-11-09)
- CoreWeave pricing (seen 2026-11-09)
- SF Compute docs: order book
- Shadeform