Hosting & CloudGlossaryPart 17 of 17
Running the Provider Glossary
78 terms from the Hosting & Cloud book's Running the Provider part, each defined in a line and linked to the chapter that explains it.
account_id,resource_id,sku_idand time ranges: the four keys every provider join runs through: the paying tenant and its standing, the thing that ran, its price, and the periods (exists_during,held_during,valid_during) that tie records to instants (Provider Architecture).- Annual revenue retention: the share of last year's revenue not lost to customers who left, the retention variant that isolates churn from expansion (Operating Metrics).
- Archive node: a blockchain node that keeps every historical state back to genesis, so calls can be answered at any past block; tens of terabytes in Ethereum's hash-based layout, a few in the newer path-based ones (Blockchain Nodes, RPC & Validators).
- ARR (annual recurring revenue): in this industry almost never a sum of contracts but a recent period's revenue annualized, so its definition is part of the number (Operating Metrics).
- Back-end (scale-out) fabric: the network that carries GPU-to-GPU traffic between servers, InfiniBand or RoCE, bundled into a cluster or placement construct rather than priced separately (GPU & AI Clouds).
- Batch pricing (AI APIs): discounted asynchronous processing on frontier model APIs, about half the standard price, which stacks with prompt caching (Where AI Runs: Home, Colo, Cloud or Frontier).
- Blast radius: the share of customers one failure can reach, bounded by design through regions, zones, cells and shuffle sharding rather than discovered in an outage (Provider Architecture).
- Break-even month: the month in a year-over-year model when cumulative savings from owning recover the up-front purchase (Total Cost of Ownership).
- Cached input: prompt tokens served from a model provider's cache at a fraction of the input price, one of the main levers on frontier API cost (Where AI Runs: Home, Colo, Cloud or Frontier).
- Capacity block (GPU): a fixed number of GPU instances for fixed dates, bought ahead at a price set at purchase, the way hyperscalers ration scarce large GPUs (GPU & AI Clouds).
- Capex versus cash view: owned hardware seen as straight-line depreciation over a useful life for the comparison, and as cash leaving at purchase (or financing payments) for whether the company can afford it (Total Cost of Ownership).
- Commissioned TCO study: a vendor-funded analysis, such as a Forrester Total Economic Impact study, that picks the composite customer, baseline and period; marketing, not a model of a workload (Total Cost of Ownership).
- Component status: the state of one product and region on a status page (Operational, Under Maintenance, Degraded Performance, Partial Outage, Major Outage), from which the top-level status is computed (Outages, Post-Mortems & Status Pages).
- Compute unit (RPC): an RPC provider's weight for one method call, set by what the call costs to serve, so a trace and a block-number lookup are different prices for the same request; also called credits or API credits (Blockchain Nodes, RPC & Validators).
- Constant work: a reliability pattern in which a system does the same amount of work whatever happens, such as health checkers pushing every result every time, so load never spikes during failure (Provider Architecture).
- Control-plane dependency: a hidden dependency of a control plane on one building, another cloud, a database or the monitoring itself, the cause of several large outages (Outages, Post-Mortems & Status Pages).
- Cost per unit: cost of revenue divided by what was delivered (per VM-hour, GB-month, GB delivered, message, GPU-hour, million tokens), the number pricing should be set from (Operating Metrics).
- Distributed validator: a proof-of-stake validator whose key is split across several operators' nodes so a quorum must sign, the way to add redundancy to a validator without a second copy of the key (Blockchain Nodes, RPC & Validators).
- Doppelganger protection: a validator client listening for its own key on the network for a couple of epochs before signing, a best-effort last line against running the same key twice (Blockchain Nodes, RPC & Validators).
- Downgrade to free: an account moving from paid to a free plan, counted separately from churn because the account still exists (Operating Metrics).
- Earning hours (GPU): the hours a GPU spends under paid use, its real utilization measure and what decides whether owning it beats renting (Operating Metrics).
- Front door: the control-plane layer that authenticates, authorizes, rate-limits and checks quota on every API call before anything is accepted (Provider Architecture).
- Front-end network (GPU): the network that carries everything except GPU-to-GPU traffic in an AI cluster: storage, checkpoints, the customer's other services and the internet (GPU & AI Clouds).
- Frontier API: a model lab's hosted API, where the customer owns nothing and pays per token (input, cached input, cache writes, output) under the lab's terms (Where AI Runs: Home, Colo, Cloud or Frontier).
- FTE fraction: a placement's recurring platform work in hours a month divided by productive hours, a stated and dated estimate rather than a fact (Total Cost of Ownership).
- Fully loaded cost: what an employee costs beyond salary; the chapter's public placeholder divides the BLS median systems administrator wage by the 70% wage share of compensation, about $141,600 a year (Total Cost of Ownership).
- Global-config failure pattern: the recurring outage shape: something that isn't code changes, propagates everywhere quickly by design, crashes a consumer that trusted it, and takes longer to recover from than to cause (Outages, Post-Mortems & Status Pages).
- Gross margin with pass-through: revenue less the cost of serving it over revenue, which pass-through fees lower without changing gross profit, so it is reported with and without them (Operating Metrics).
- Hosted open model: an inference provider serving an open-weight model the customer chooses, billed per token, with precision and speed varying by host (Where AI Runs: Home, Colo, Cloud or Frontier).
- Hosted status page: a SaaS status page product that models components, incidents and subscribers and runs on infrastructure independent of the provider's own (Outages, Post-Mortems & Status Pages).
- Incident lifecycle (status page): an incident's movement through Investigating, Identified, Monitoring and Resolved, with scheduled maintenance on its own lifecycle (Outages, Post-Mortems & Status Pages).
- Kinds of ending: the ways a product ends (free tier withdrawn, product shut down, version end of life, business sold, maintenance mode, location or endpoint moved), each owing customers something different (Migrations, Shutdowns & End of Life).
- Logo churn: customers active at the start of a period and not at its end, over customers at the start, meaningful only with the threshold that defines "active" written down (Operating Metrics).
- Magic Pocket: the storage system Dropbox built to move file content off Amazon S3 into its own infrastructure in colocation, the best-documented storage repatriation (Repatriation).
- Maintenance mode: a product kept running with fixes but no new features, which leaves customers to decide whether to stay (Migrations, Shutdowns & End of Life).
- Metering pipeline: the path from data-plane usage emission to rated usage, in which the data plane emits and never waits, every hop is idempotent and counts are reconciled against what is running (Provider Architecture).
- Miner-to-AI lease: a bitcoin miner leasing its powered, converted sites to an AI tenant for ten to twenty years, often with a hyperscaler backstopping the tenant's payments (GPU & AI Clouds).
- Multi-year take-or-pay: dedicated GPU capacity contracted for years, partly prepaid and paid whether used or not, the contract neoclouds finance their GPUs against (GPU & AI Clouds).
- Neocloud: a GPU-focused cloud provider that sells clusters on demand and under multi-year contracts, a utility financed like an infrastructure project with contracts as the collateral (GPU & AI Clouds).
- Net dollar retention: what last year's customers spend now, as a share of what they spent then; each provider computes it differently enough that the definition is part of the number (Operating Metrics).
- NVLink domain: the set of GPUs joined by NVLink and NVSwitch, the fastest network a model can use, inside one server or, in rack-scale systems such as GB200 NVL72, across a rack (GPU & AI Clouds).
- Off-bill costs: what no provider invoices: people and on-call, upgrade treadmills, observability and security tools, migration and exit, commitment risk, opportunity cost and depreciation (Total Cost of Ownership).
- On-call floor: the minimum rotation for 24-hour response, about eight engineers at one site or six per site across two, a headcount whatever the workload's size (Total Cost of Ownership).
- Operation (long-running): the record an API returns for anything slower than a few seconds, which the caller polls while the provisioning workflow runs (Provider Architecture).
- Opportunity cost (infrastructure): the product work an engineer doesn't do while running infrastructure, the reason small teams pay a managed service's premium (Total Cost of Ownership).
- Own the baseline, rent the burst: running steady load on owned hardware busy and keeping peaks and unknowns elastic in the cloud, at the cost of two platforms and the transfer between them (Repatriation).
- Owned annual cost: hardware over useful life plus colocation, network, support and spares, licenses, the change in staff and the cost of capital, compared with the rented side's effective rate and transfer costs (Repatriation).
ownership_costs: effective-dated rows of off-bill cost per workload and scenario, on a monthly, FTE, one-time or capex basis, compared with the bill in one query (Total Cost of Ownership).- Paid extended support: keeping old versions running for a fee before a forced upgrade, which turns a forced migration into a choice with a price (Migrations, Shutdowns & End of Life).
- People delta: the people a placement needs minus the people the company has regardless; zero when the existing team stays the same size, as 37signals reported (Total Cost of Ownership).
- Per-account status: status a provider computes for one customer's own resources and regions, telling each customer what is theirs where a public page can't (Outages, Post-Mortems & Status Pages).
- Per-output pricing: serverless model APIs charging per image, second of video or megapixel rather than per GPU-second (GPU & AI Clouds).
- Physalia: AWS's transactional key-value store for control-plane state, begun as the configuration master for EBS, highly available and strongly consistent by being available for only the keys each client needs (Provider Architecture).
- Post-mortem: a provider's public account of an incident, owed under published criteria, revised as understanding improves with both versions kept, and listing remediations as promises (Outages, Post-Mortems & Status Pages).
- Productive engineer-month: the 140 hours a month the chapter assumes an engineer spends on work after meetings and leave, the divisor for FTE fractions (Total Cost of Ownership).
- Provisioning: turning an accepted request into a running thing: authenticate, check standing and quota, accept idempotently, return an operation, place, run the workflow, converge and report (Provider Architecture).
- P/S/C (shared responsibility by task): marking each recurring task as absorbed by the provider, shared, or the customer's, per platform, the basis for counting people (Total Cost of Ownership).
- Reconciliation model: a provisioning style, made familiar by Kubernetes, that declares desired state and lets controllers converge current state toward it, instead of running an explicit workflow (Provider Architecture).
- Refresh (hardware): buying the fleet again at the end of its useful life, year six of a five-year model, which is why a five-year saving isn't permanent (Total Cost of Ownership).
- Renewal rate: the share of customers or names renewing at term end, reported by hosting and domain businesses and best broken out by term number for prepaid products (Operating Metrics).
- Repatriation: moving a workload from rented cloud capacity back to hardware the company owns or leases, usually in colocation, worth it when load is steady, the team exists and the cloud's discounts are already taken (Repatriation).
- Revenue churn: revenue lost from customers who left or shrank, over starting revenue, the complement of revenue retention (Operating Metrics).
- Router-level fallback: a model router trying models or providers in order and falling back on any error, billing at the model that served (Where AI Runs: Home, Colo, Cloud or Frontier).
- RPC provider (blockchain): a business that runs pools of blockchain nodes and sells access to them as an API, metered in weighted calls under plans with per-second limits (Blockchain Nodes, RPC & Validators).
- Serverless GPU: GPU compute billed per second or per output that scales to zero, with cold starts and a premium per hour at high utilization (GPU & AI Clouds).
- Slashing: the protocol penalty for a validator key that signs contradictory messages (two blocks for one slot, a double or surround vote), larger when many validators are slashed together; being offline is not slashable (Blockchain Nodes, RPC & Validators).
- Slashing protection: a validator client's local record of what a key has signed, checked before every signature and moved with the key in the EIP-3076 interchange format (Blockchain Nodes, RPC & Validators).
- Sold versus installed: capacity allocated to customers over capacity installed, by zone and instance type, the utilization measure that drives capacity orders and spot pricing (Operating Metrics).
- Steadiness: typical load over peak load; the closer the median is to the peak, the busier an owned fleet runs and the better owning compares with renting (Repatriation).
- Sticker price: the provider's invoice alone, often the smallest term in a small workload's total cost (Total Cost of Ownership).
- Sunset lifecycle: the ordered end of a product: announce every date at once, close to new customers, stop changes, go read-only, shut down, then delete after a stated window (Migrations, Shutdowns & End of Life).
- Third-party components: status page components that mirror a vendor's own status page, so customers see when the failure is in the underlay (Outages, Post-Mortems & Status Pages).
- Total cost of ownership (TCO): everything a workload costs to run for a period: the bill plus people, upgrades, off-bill tools, migration and exit, commitments and, for owned hardware, depreciated capital (Total Cost of Ownership).
- Used versus sold: what allocated capacity actually does; the gap between it and sold capacity is the provider's overcommit margin and the noisy neighbor's risk (Operating Metrics).
- Useful life: the period owned hardware is depreciated over; five years is the default for general compute, and Amazon moved its servers from five to six years in 2024 and a subset back to five in 2025 (Total Cost of Ownership).
- Utilization of the owned fleet: average used capacity over capacity owned, the ratio that decides whether owning beats renting (Repatriation).
- Vendor risk: outages caused by a privileged agent's content push, a provider's own tooling, a building or a dependency of a dependency, rather than by the provider's own code (Outages, Post-Mortems & Status Pages).
- Workflow model: a provisioning style that runs an explicit sequence of steps with compensation on failure, recording each step's result so a crashed workflow resumes rather than restarts (Provider Architecture).