GPU Investment Calculator

Does a GPU pay for itself selling tokens? Pick a GPU and an open-weight model. A roofline model built from each model's real architecture estimates tokens per second at a speed users will accept. Those tokens are priced at what providers charge today, then set against power, hardware price and lifetime. The matrix at the bottom shows which GPU × model pairs make money and which never will.

1 Choose a GPU

2 Choose a model

3 Workload & serving

Engine tuning

Throughput

Tokens per second vs concurrent users
Where the time and memory go

4 Costs & prices

Power draw assumptions

Return on investment

Cumulative cash, month by month USD, after purchase
Year by year

Which combinations make money?

Every GPU × model pair under your current workload, costs and price basis, each with its own best deployment. Colour is NPV relative to what the hardware costs — blue makes money, red loses it, grey is break-even. Click a cell to load that pair above.

Investment memo

The model gets the numbers on this page — the selected deal and the best and worst pairs in the matrix — and writes a short investment-committee note. It quotes the engine's numbers; it doesn't calculate them.

How the throughput is estimated

Each model's geometry comes from its config.json on Hugging Face: layers, KV heads, head size, experts, sliding windows, latent (MLA) or compressed attention, and the recurrent state of linear-attention and Mamba layers. Serving is modelled as one continuous-batching decode step with chunked prefill mixed in, as vLLM, SGLang and TensorRT-LLM do.

  • Bytes per step — weights actually touched (for mixture-of-experts, the expected share of experts hit by the batch: 1 − (1 − k/E)tokens), plus every sequence's KV cache at the average context, plus recurrent state read and written.
  • FLOPs per step — 2 × active parameters per token for decode and prefill tokens, plus attention over the cached entries (capped by sliding windows and sparse top-k).
  • Step time — the slower of memory time and compute time, plus tensor-parallel all-reduce time (2 per layer), plus scheduler and kernel-launch overhead.
  • Batch — the largest number of concurrent sequences that fits in memory and still gives every user your minimum speed. GPUs per replica is searched over 1, 2, 4… up to the NVLink domain; beyond it, multi-node at 400 Gb/s.

Calibration: with an MLPerf-style 300-in / 300-out load and no speed floor, Llama 70B gives about 3,000 output tok/s per H100 (MLPerf ≈ 3,000), 3,500 per MI300X (≈ 3,300) and 14,000 per B200 at FP4 (≈ 12,000). Treat results as ±25%. Speculative decoding and prefix caching, which real providers use, would raise throughput; heavy-tailed request lengths would lower it.

How the money is calculated
  • Revenue = (input tok/s × input price + output tok/s × output price) × load, month by month, with prices falling at your yearly decline rate.
  • Power = GPUs × (TDP × (idle + load × (serving − idle)) + host watts) × PUE × hours × price per kWh.
  • Capex = GPUs × (GPU price + server & networking). Rack-scale prices already include the rack. Resale value comes back in the last month.
  • NPV discounts monthly cash at your rate; IRR is annualised from monthly flows; payback is the month cumulative cash turns positive.
  • Break-even load is the average load at which NPV is zero; break-even price is the share of today's token price at which NPV is zero.

Not included: taxes, financing, staff, bandwidth, sales and payment fees, and the risk that newer models make yours unsellable before the hardware wears out.

Data sources & dates